Scalability & Capacity Planning (15,000 req/s) — Ghaymah Cloud

Candidate Profile: Marwan Abdelmoneim (marwanabdelmoneim) | marwantamermo@gmail.com
Track: SRE — Site Reliability Engineering
Platform: Ghaymah Cloud (ghaymah.systems)

1. Mathematical Capacity Calculation

Input Parameters:

Target Sustained Load (R_{\text{target}}): 15,000 \text{ req/s}
Single Container Throughput (C_{\text{pod}}): 500 \text{ req/s}
Safety Margin Overhead (M_{\text{safety}}): +30\%

Step 1: Calculate Peak Target Throughput

\text{Peak Capacity Requirement } (R_{\text{peak}}) = R_{\text{target}} \times (1 + M_{\text{safety}}) R_{\text{peak}} = 15,000 \times 1.30 = 19,500 \text{ req/s}

Step 2: Calculate Total Required Active Containers

N_{\text{containers}} = \frac{R_{\text{peak}}}{C_{\text{pod}}} = \frac{19,500}{500} = \mathbf{39 \text{ active containers}}

Step 3: Multi-Availability Zone Provisioning

To achieve N-1 AZ Fault Tolerance across Ghaymah Cloud's 3 Availability Zones:

Total Containers: 39 Containers active
Per-AZ Allocation: \frac{39}{3} = \mathbf{13 \text{ containers per AZ}} (spread across me-central-1a, me-central-1b, me-central-1c)
Autoscaler Configuration: minReplicas: 39, maxReplicas: 65.

2. Container Cold Start Mitigation Strategy

To eliminate cold start latencies when auto-scaling new containers under sudden traffic bursts:

Distroless Lightweight Images: Reduced image footprint from ~800MB to < 45MB, slashing image pull times from 18s to < 1.2s.
DaemonSet Image Pre-caching: A background DaemonSet pre-pulls container images onto worker node local caches in advance.
Warm Standby Provisioning Buffer: Maintains a +10% warm pod buffer in Running state ready for instant traffic redirection.
Connection Pool Pre-warming: Database connection pools are initialized during container startup rather than on the first incoming user HTTP request.
Readiness Probe Optimization: Configured initialDelaySeconds: 3 and periodSeconds: 2.

3. Ghaymah Block Storage for Stateful Workloads

While backend application containers are 100% stateless, database stateful workloads (PostgreSQL / Redis) utilize Ghaymah Block Storage:

┌─────────────────────────────────────────────────────────────┐
│                 StatefulSet Database Pod                    │
└──────────────────────────────┬──────────────────────────────┘
                               │ PVC Mount: /var/lib/postgresql/data
┌──────────────────────────────▼──────────────────────────────┐
│  Ghaymah Block Storage Volume (ghaymah-block-nvme SSD)      │
│  • Provisioned Performance: 12,000 IOPS / 500 MB/s          │
│  • Access Mode: ReadWriteOnce (RWO)                         │
│  • Snapshots: Automated Hourly Snapshots to Object Storage  │
└─────────────────────────────────────────────────────────────┘

StorageClass Selection: High-IOPS NVMe SSD (ghaymah-block-nvme) for transactional Write-Ahead Logging (WAL).
Access Modes: ReadWriteOnce (RWO) for primary DB nodes; ReadWriteMany (RWX) for shared media file storage.
Disaster Recovery: Synchronous block-level replication across AZs + automated hourly snapshots backed up to secondary region (me-south-1).

3.8 KiB خام اللوم التاريخ