# Scalability & Load Distribution — Architecture Report **Project:** Ghayma REST API **Platform:** ghaymah.systems (Ghayma Cloud) **Target Load:** 15,000 requests/second **Date:** July 26, 2026 --- ## Table of Contents 1. [Architecture Diagram — 15,000 req/s](#1-architecture-diagram--15000-reqs) 2. [Container Capacity Calculation](#2-container-capacity-calculation) 3. [Cold Start Strategy for New Containers](#3-cold-start-strategy-for-new-containers) 4. [Ghayma Block Storage for Stateful Workloads](#4-ghayma-block-storage-for-stateful-workloads) --- ## 1. Architecture Diagram — 15,000 req/s ### High-Level System Architecture ``` 15,000 req/s │ ▼ ┌───────────────────────┐ │ DNS / CDN │ │ (dns.ghaymah.systems)│ └───────────┬───────────┘ │ ▼ ┌───────────────────────┐ │ LOAD BALANCER │ │ (Layer 7 - HTTPS) │ │ │ │ • SSL Termination │ │ • Health Checks │ │ • Round Robin / │ │ Least Connections │ │ • Rate Limiting │ └───────────┬───────────┘ │ ┌────────────────────┼────────────────────┐ │ │ │ ▼ ▼ ▼ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ Zone A │ │ Zone B │ │ Zone C │ │ (Rack/AZ 1) │ │ (Rack/AZ 2) │ │ (Rack/AZ 3) │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │ │ ┌──────┴──────┐ ┌──────┴──────┐ ┌──────┴──────┐ │ ┌─────────┐ │ │ ┌─────────┐ │ │ ┌─────────┐ │ │ │Container│ │ │ │Container│ │ │ │Container│ │ │ │ 1-15 │ │ │ │ 16-30 │ │ │ │ 31-44 │ │ │ │ (15x) │ │ │ │ (15x) │ │ │ │ (14x) │ │ │ └─────────┘ │ │ └─────────┘ │ │ └─────────┘ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │ │ └───────────────────┼───────────────────┘ │ ┌─────────────────┼─────────────────┐ │ │ │ ▼ ▼ ▼ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ Database │ │ Redis │ │ Block Storage│ │ (Primary + │ │ (Session & │ │ (Persistent │ │ Replicas) │ │ Cache) │ │ Volumes) │ └──────────────┘ └──────────────┘ └──────────────┘ ``` ### Detailed Component Architecture ``` ┌──────────────────────────────────────────────────────────────────────────────────────┐ │ │ │ GHAYMA CLOUD PLATFORM │ │ │ │ ┌────────────────────────────────────────────────────────────────────────────────┐ │ │ │ NETWORKING LAYER │ │ │ │ │ │ │ │ Internet ──▶ [DNS] ──▶ [CDN/WAF] ──▶ [Load Balancer (L7)] │ │ │ │ │ │ │ │ │ ┌────────────────────┤ │ │ │ │ │ Health Checks │ │ │ │ │ │ GET /health (10s) │ │ │ │ │ │ Threshold: 3 fails │ │ │ │ │ └────────────────────┘ │ │ │ └────────────────────────────────┬───────────────────────────────────────────────┘ │ │ │ │ │ ┌────────────────────────────────┼───────────────────────────────────────────────┐ │ │ │ COMPUTE LAYER (44 Containers) │ │ │ │ │ │ │ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │ │ │ │ C-01 │ │ C-02 │ │ C-03 │ │ C-04 │ ... │ C-44 │ │ │ │ │ │ 500r/s │ │ 500r/s │ │ 500r/s │ │ 500r/s │ │ 500r/s │ │ │ │ │ │ 512MB │ │ 512MB │ │ 512MB │ │ 512MB │ │ 512MB │ │ │ │ │ │ 0.5 CPU │ │ 0.5 CPU │ │ 0.5 CPU │ │ 0.5 CPU │ │ 0.5 CPU │ │ │ │ │ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ │ │ │ │ └────────────┴────────────┴────────────┴──── ... ───────┘ │ │ │ │ │ │ │ │ │ ┌──────────────────────────────────────────────────────────────────┐ │ │ │ │ │ AUTO-SCALER (HPA) │ │ │ │ │ │ Min: 44 | Max: 66 | Scale at: CPU>75% or Mem>70% │ │ │ │ │ └──────────────────────────────────────────────────────────────────┘ │ │ │ └────────────────────────────────┬───────────────────────────────────────────────┘ │ │ │ │ │ ┌────────────────────────────────┼───────────────────────────────────────────────┐ │ │ │ DATA LAYER │ │ │ │ │ │ │ │ ┌──────────────────┐ ┌─────────────┐ ┌──────────────────────────────┐ │ │ │ │ │ PostgreSQL DB │ │ Redis │ │ Ghayma Block Storage │ │ │ │ │ │ Primary + 2 │ │ Cluster │ │ (Persistent Volumes) │ │ │ │ │ │ Read Replicas │ │ (Cache + │ │ • Database data (100GB) │ │ │ │ │ │ │ │ Sessions) │ │ • Logs & uploads (50GB) │ │ │ │ │ └──────────────────┘ └─────────────┘ └──────────────────────────────┘ │ │ │ └────────────────────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌────────────────────────────────────────────────────────────────────────────────┐ │ │ │ OBSERVABILITY LAYER │ │ │ │ │ │ │ │ [Prometheus] ──▶ [Grafana Dashboards] ──▶ [AlertManager] ──▶ [PagerDuty] │ │ │ │ [Container Logs] ──▶ [Centralized Logging] ──▶ [Log Analytics] │ │ │ └────────────────────────────────────────────────────────────────────────────────┘ │ │ │ └──────────────────────────────────────────────────────────────────────────────────────┘ ``` ### Load Balancer Distribution Strategy ``` 15,000 req/s incoming │ ┌────────┴────────┐ │ LOAD BALANCER │ │ │ │ Algorithm: │ │ Least │ │ Connections │ └────────┬────────┘ │ ┌────────────────┼────────────────┐ │ │ │ Zone A (33%) Zone B (33%) Zone C (34%) ~5,000 r/s ~5,000 r/s ~5,000 r/s 15 containers 15 containers 14 containers ~333 r/s each ~333 r/s each ~357 r/s each ``` > Each container handles **~341 req/s on average** — well within the 500 req/s capacity, leaving the 30% safety margin intact. --- ## 2. Container Capacity Calculation ### Given Parameters | Parameter | Value | | :--- | :--- | | **Total traffic** | 15,000 req/s | | **Capacity per container** | 500 req/s | | **Safety margin** | 30% | ### Step-by-Step Calculation #### Step 1: Calculate Effective Capacity per Container The 30% margin means each container should only be utilized at 70% of its max capacity: ``` Effective capacity = Max capacity × (1 - Safety margin) Effective capacity = 500 req/s × (1 - 0.30) Effective capacity = 500 × 0.70 Effective capacity = 350 req/s per container ``` #### Step 2: Calculate Minimum Containers Required ``` Minimum containers = Total traffic ÷ Effective capacity Minimum containers = 15,000 ÷ 350 Minimum containers = 42.86 Minimum containers = 43 containers (rounded up) ``` #### Step 3: Add Availability Buffer (+1) To handle single-container failures without breaching capacity: ``` Recommended containers = 43 + 1 (failure tolerance) Recommended containers = 44 containers ``` ### Summary Table | Metric | Value | Formula | | :--- | :--- | :--- | | **Total load** | 15,000 req/s | Given | | **Max capacity/container** | 500 req/s | Given | | **Safety margin** | 30% | Given | | **Effective capacity/container** | 350 req/s | 500 × 0.70 | | **Minimum containers** | 43 | ⌈15,000 ÷ 350⌉ | | **Recommended (with buffer)** | **44** | 43 + 1 | | **Actual utilization per container** | ~341 req/s | 15,000 ÷ 44 | | **Utilization percentage** | 68.2% | 341 ÷ 500 | | **Remaining headroom** | 31.8% | ✅ Exceeds 30% margin | ### Capacity Planning Visualization ``` Container Capacity: 500 req/s ├─────────────────────────────────────────────────┤ 0 500 Actual load per container: ~341 req/s (68.2%) ├──────────────────────────────────┤ 0 341 Safety margin zone (30%): ├──────────────────────────────────┤░░░░░░░░░░░░░░┤ 0 350 500 ▲ Threshold (don't exceed) ``` ### Auto-Scaling Tiers | Traffic Level | Containers Needed | Status | | :--- | :--- | :--- | | 5,000 req/s (low) | 15 | Minimum viable | | 10,000 req/s (moderate) | 30 | Normal scaling | | **15,000 req/s (target)** | **44** | **Design target** | | 20,000 req/s (spike) | 58 | Burst capacity | | 25,000 req/s (peak) | 72 | Maximum scaling | ### Resource Allocation per Container | Resource | Value | | :--- | :--- | | Memory Request | 256 Mi | | Memory Limit | 512 Mi | | CPU Request | 250m (0.25 vCPU) | | CPU Limit | 500m (0.5 vCPU) | ### Total Cluster Resources (44 containers) | Resource | Per Container | × 44 Containers | Total | | :--- | :--- | :--- | :--- | | **Memory** | 512 Mi | × 44 | **22 GB** | | **CPU** | 0.5 vCPU | × 44 | **22 vCPUs** | | **Throughput** | 350 req/s (effective) | × 44 | **15,400 req/s** | --- ## 3. Cold Start Strategy for New Containers ### What is a Cold Start? A **cold start** occurs when a new container instance is created and must initialize before serving traffic. During this period: - The container image is pulled from the registry - The Node.js runtime starts - Dependencies are loaded - The application connects to databases/caches - The first requests experience **higher latency** ``` Cold Start Timeline (without optimization) ────────────────────────────────────────────────────── │ Image Pull │ Runtime Init │ App Boot │ DB Connect │ Ready │ 0s 3s 4s 5s 7s 8s ▲ First request served ``` ### Cold Start Strategy — 5 Layers #### Layer 1: Pre-Warmed Container Pool Maintain a pool of **idle but ready** containers that can immediately accept traffic: ``` Active Containers (44) Pre-Warmed Pool (4-6) ┌────┬────┬────┬─ ... ─┬────┐ ┌────┬────┬────┬────┐ │ C1 │ C2 │ C3 │ │C44 │ │ W1 │ W2 │ W3 │ W4 │ │BUSY│BUSY│BUSY│ │BUSY│ │IDLE│IDLE│IDLE│IDLE│ └────┴────┴────┴─ ... ─┴────┘ └────┴────┴────┴────┘ │ Traffic spike detected │ ▼ W1, W2 immediately start serving (0s cold start) ``` **Implementation:** ```yaml # HPA with pre-warmed minimum apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler spec: minReplicas: 48 # 44 active + 4 pre-warmed maxReplicas: 72 ``` #### Layer 2: Optimized Docker Image (Fast Pulls) Reduce image pull time by using lightweight images: ```dockerfile # ✅ GOOD: Alpine-based image (~150MB, pulls in ~2s) FROM node:20-alpine # ❌ BAD: Full Debian image (~1GB, pulls in ~12s) FROM node:20 ``` | Image Base | Size | Pull Time | Cold Start Impact | | :--- | :--- | :--- | :--- | | `node:20` (Debian) | ~1 GB | ~12s | Very slow | | `node:20-slim` | ~250 MB | ~5s | Moderate | | `node:20-alpine` | ~150 MB | ~2s | ✅ **Fast** | #### Layer 3: Application-Level Readiness Implement a **readiness probe** that only marks the container as "ready" after it has fully initialized: ```javascript // In src/index.js let isReady = false; // Readiness check - only returns 200 when app is fully initialized app.get('/ready', (req, res) => { if (isReady) { res.status(200).json({ ready: true }); } else { res.status(503).json({ ready: false, message: 'Warming up...' }); } }); // App startup sequence async function initialize() { // 1. Connect to database await connectDatabase(); // 2. Connect to Redis cache await connectRedis(); // 3. Pre-load frequently accessed data into memory await preloadCache(); // 4. Mark as ready ONLY after everything is initialized isReady = true; console.log('✅ Application fully initialized and ready'); } app.listen(PORT, async () => { console.log(`Server starting on port ${PORT}...`); await initialize(); }); ``` ```yaml # Kubernetes readiness probe readinessProbe: httpGet: path: /ready # Not /health — separate from liveness port: 3000 initialDelaySeconds: 3 periodSeconds: 5 failureThreshold: 3 # Must pass 3 checks before receiving traffic ``` > The load balancer **will not route traffic** to the container until `/ready` returns HTTP 200. #### Layer 4: Proactive Scaling (Anticipate Demand) Scale **before** the spike hits, not after: ```yaml # CronJob-based pre-scaling for known traffic patterns apiVersion: batch/v1 kind: CronJob metadata: name: pre-scale-morning spec: # Scale up at 8:00 AM before morning traffic rush schedule: "0 8 * * 1-5" jobTemplate: spec: template: spec: containers: - name: scaler image: bitnami/kubectl:latest command: - kubectl - scale - deployment/ghayma-api - --replicas=50 ``` #### Layer 5: Connection Pooling & Warm-Up Requests After a container starts, send **internal warm-up requests** to pre-establish connections: ```javascript async function warmUp() { console.log('🔥 Running warm-up sequence...'); // Pre-establish database connection pool await db.query('SELECT 1'); // Prime V8 JIT compiler with typical request patterns const warmUpPaths = ['/health', '/api/v1/items']; for (const path of warmUpPaths) { await fetch(`http://localhost:${PORT}${path}`); } console.log('✅ Warm-up complete — ready for production traffic'); } ``` ### Cold Start Strategy Summary ``` ┌─────────────────────────────────────────────────────────────────┐ │ COLD START MITIGATION LAYERS │ │ │ │ Layer 1: Pre-Warmed Pool → 0s cold start (instant) │ │ Layer 2: Alpine Docker Image → 2s image pull (vs 12s) │ │ Layer 3: Readiness Probes → No traffic until fully ready │ │ Layer 4: Proactive Scaling → Scale before the spike │ │ Layer 5: Warm-Up Requests → Pre-establish connections │ │ │ │ Combined Effect: New containers ready in ~3-5s │ │ Without strategy: New containers ready in ~15-20s │ └─────────────────────────────────────────────────────────────────┘ ``` --- ## 4. Ghayma Block Storage for Stateful Workloads ### What is Ghayma Block Storage? **Ghayma Block Storage** is a persistent, network-attached block storage service provided by the Ghayma Cloud platform. It provides raw storage volumes that can be attached to containers, similar to a virtual hard drive. ``` ┌─────────────────────────────────────────────────────────────────┐ │ BLOCK STORAGE vs CONTAINER STORAGE │ │ │ │ Container (Ephemeral) Block Storage (Persistent) │ │ ┌──────────────────┐ ┌──────────────────────┐ │ │ │ Container FS │ │ Ghayma Block Volume │ │ │ │ │ │ │ │ │ │ ❌ Data lost │ │ ✅ Data survives │ │ │ │ when container│ │ container restarts│ │ │ │ stops/crashes │ │ and redeployments │ │ │ │ │ │ │ │ │ │ ❌ Not shared │ │ ✅ Can be reattached │ │ │ │ between │ │ to new containers │ │ │ │ containers │ │ │ │ │ └──────────────────┘ └──────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────┘ ``` ### Why is Block Storage Needed? Containers are **stateless and ephemeral** by design — when a container restarts, crashes, or is replaced during a deployment, all data written inside it is **lost**. This is a problem for: | Workload Type | Data at Risk | Block Storage Solution | | :--- | :--- | :--- | | **Databases** (PostgreSQL, MySQL) | Table data, indexes, WAL logs | Persistent volume for `/var/lib/postgresql/data` | | **Cache** (Redis with persistence) | RDB/AOF snapshots | Volume for `/data` | | **File Uploads** | User-uploaded images, documents | Volume for `/app/uploads` | | **Application Logs** | Log files for audit/compliance | Volume for `/var/log/app` | | **Session Storage** | User session files | Volume for `/tmp/sessions` | ### Architecture: Block Storage with Containers ``` ┌─────────────────────────────────────────────────────────────────────┐ │ STATEFUL WORKLOAD ARCHITECTURE │ │ │ │ ┌───────────────────┐ ┌───────────────────────────────┐ │ │ │ API Container │ │ Database Container │ │ │ │ (Stateless) │ │ (Stateful — PostgreSQL) │ │ │ │ │──────▶│ │ │ │ │ No volume │ │ ┌─────────────────────┐ │ │ │ │ needed │ │ │ /var/lib/postgresql │ │ │ │ │ │ │ │ (mount point) │ │ │ │ └───────────────────┘ │ └──────────┬──────────┘ │ │ │ │ │ │ │ │ └──────────────┼───────────────┘ │ │ │ │ │ ┌─────────▼──────────┐ │ │ │ GHAYMA BLOCK │ │ │ │ STORAGE VOLUME │ │ │ │ │ │ │ │ Name: db-data │ │ │ │ Size: 100 GB │ │ │ │ Type: SSD (gp3) │ │ │ │ IOPS: 3,000 │ │ │ │ Encrypted: Yes │ │ │ │ │ │ │ │ ✅ Persists across │ │ │ │ restarts │ │ │ │ ✅ Automatic │ │ │ │ snapshots │ │ │ │ ✅ Reattachable │ │ │ └────────────────────┘ │ │ │ └────────────────────────────────────────────────────────────────────┘ ``` ### How to Use Ghayma Block Storage #### Step 1: Create a Block Storage Volume ```bash # Using Ghayma CLI ghaymah storage create db-data \ --size 100GB \ --type ssd \ --region me-central-1 \ --encrypted ``` #### Step 2: Attach to a Container / Deployment ```yaml # Kubernetes PersistentVolumeClaim using Ghayma storage class apiVersion: v1 kind: PersistentVolumeClaim metadata: name: db-data-pvc spec: accessModes: - ReadWriteOnce # Single container read/write storageClassName: ghaymah-block-ssd resources: requests: storage: 100Gi --- # Deployment using the volume apiVersion: apps/v1 kind: Deployment metadata: name: postgres-db spec: replicas: 1 # Stateful — typically 1 primary template: spec: containers: - name: postgres image: postgres:16-alpine ports: - containerPort: 5432 volumeMounts: - name: db-storage mountPath: /var/lib/postgresql/data env: - name: POSTGRES_PASSWORD valueFrom: secretKeyRef: name: db-secret key: password volumes: - name: db-storage persistentVolumeClaim: claimName: db-data-pvc # Ghayma Block Storage volume ``` #### Step 3: Verify the Volume ```bash # Check volume status ghaymah storage list # Output: # NAME SIZE TYPE STATUS ATTACHED TO REGION # db-data 100GB SSD attached postgres-db-pod-0 me-central-1 ``` ### Block Storage Features on Ghayma Cloud | Feature | Description | | :--- | :--- | | **Persistence** | Data survives container restarts, crashes, and redeployments | | **SSD Performance** | Low-latency SSD storage with up to 16,000 IOPS | | **Encryption** | At-rest encryption (AES-256) for security compliance | | **Snapshots** | Point-in-time snapshots for backup and disaster recovery | | **Resizing** | Expand volume size without downtime | | **Reattachment** | Detach from one container and attach to another (for migration) | | **Availability Zones** | Volume and container must be in the same zone | ### When to Use Block Storage vs Other Options | Storage Type | Use Case | Persistence | Shared Access | Speed | | :--- | :--- | :--- | :--- | :--- | | **Container FS** | Temp files, build artifacts | ❌ Ephemeral | ❌ No | ⚡ Fastest | | **Ghayma Block Storage** | Databases, stateful apps, logs | ✅ Persistent | ❌ Single container | ⚡ Fast (SSD) | | **Object Storage (S3-like)** | Images, videos, backups | ✅ Persistent | ✅ Multi-container | 🐌 Moderate | | **Shared File Storage (NFS)** | Shared config, media libraries | ✅ Persistent | ✅ Multi-container | 🔄 Moderate | ### Data Lifecycle with Block Storage ``` Container crashes or redeploys │ ▼ ┌───────────────────────────────────────────┐ │ Old container is terminated │ │ ❌ Container filesystem is destroyed │ │ ✅ Block Storage volume is PRESERVED │ └────────────────────┬──────────────────────┘ │ ▼ ┌───────────────────────────────────────────┐ │ New container starts │ │ Block Storage volume is reattached │ │ Data is immediately available at the │ │ same mount path (/var/lib/postgresql) │ │ │ │ ✅ Zero data loss │ │ ✅ Database resumes from exact state │ └───────────────────────────────────────────┘ ``` --- ## Summary | Requirement | Solution | | :--- | :--- | | **Architecture for 15K req/s** | 3-zone architecture with L7 load balancer, 44 containers, auto-scaler, and data layer | | **Container count** | **44 containers** (15,000 ÷ 350 effective req/s + 1 buffer) at 68.2% utilization | | **Cold start strategy** | 5-layer approach: pre-warmed pool, Alpine images, readiness probes, proactive scaling, warm-up requests | | **Block Storage** | Persistent SSD volumes for databases and stateful workloads that survive container restarts | --- > **Document prepared for:** Ghayma Assessment — Scalability & Load Distribution section > **Last updated:** July 26, 2026