ghaymah-exam-OmarHussein-SRE/q4-scalability/calculations.md

Q4 — Scalability & Load Balancing (Ghaymah Cloud)

1. Architecture

See architecture.png in this folder.

Flow: Clients (15,000 req/s) → Load Balancer (round-robin + health checks) → Container Auto-Scaling Pool (39 containers) → In-Memory Cache / Database + Ghaymah Block Storage.

• The front tier (load balancer + containers) is stateless and scales horizontally.
• The data tier (database) is stateful and relies on Ghaymah Block Storage for durability, decoupled from the container lifecycle.

---

2. Container Capacity Calculation

Given:

• Total incoming load: 15,000 req/s
• Capacity per container: 500 req/s
• Safety margin: 30% (to absorb traffic spikes and node/container failures)

Formula:

Required capacity = Total load × (1 + margin)
                   = 15,000 × 1.30
                   = 19,500 req/s

Containers needed = Required capacity / Capacity per container
                   = 19,500 / 500
                   = 39 containers

Result: 39 containers

Tier	Count	Purpose
Min replicas (baseline)	30	Covers the raw 15,000 req/s
Target / desired	39	Includes the 30% margin
Max replicas (HPA ceiling)	50–55	Extra headroom for burst traffic

Recommendation: configure the Horizontal Pod Autoscaler (HPA) on a composite metric — requests-per-second (via a Prometheus adapter) as the primary signal and CPU utilization as a secondary safeguard — rather than CPU alone, since CPU can lag behind real request pressure.

---

3. Cold Start Strategy for New Containers

1. Pre-warmed pool (warm standby) — keep 2–3 idle, already-running containers outside the active traffic path so bursts are absorbed instantly instead of waiting for a container to be built from scratch.
2. Slim, optimized images — use minimal base images (Alpine/distroless), reduce layers, and pre-pull images on nodes to avoid registry pull latency during scale-out events.
3. Strict readiness probes — a container only joins the load balancer pool after passing a readinessProbe (DB/cache connectivity check), preventing 503s from traffic sent to a not-yet-ready instance.
4. Predictive/scheduled scaling alongside reactive HPA — if peak hours are known (campaigns, daily traffic peaks), trigger scheduled scale-up 5–10 minutes ahead of the peak instead of relying solely on reactive HPA, which lags behind the metrics collection window.
5. Connection pooling & pre-warmed init — open DB/cache connections in an init container or internal warm-up endpoint before the first real request arrives, instead of lazy-initializing on first use.
6. Graceful scale-down — use a preStop hook and a proper terminationGracePeriodSeconds to drain in-flight requests before terminating containers during scale-in, avoiding dropped requests on the way down.

---

4. Ghaymah Block Storage for Stateful Workloads

The containers above are intentionally stateless — any of them can be killed and replaced without losing data. Anything that needs durable state (database, message queues, uploaded files) must be decoupled from the container's lifecycle — this is where Block Storage comes in:

• Decouples data from container lifecycle — a block volume is attached to a container/VM as an independent raw block device. If the container is rescheduled or fails, the same volume is re-attached to the new instance/node without data loss — unlike ephemeral container storage, which is wiped on restart.
• Best fit — relational databases (PostgreSQL/MySQL), messaging systems (Kafka/RabbitMQ), and any workload needing low-latency random I/O, since block storage offers near-local-disk performance versus object storage.
• High availability — one block volume per database replica (never share a single volume across concurrently-active instances); pair it with application-level replication (e.g., Postgres streaming replication) rather than relying on storage replication alone.
• Snapshots & backup — schedule periodic snapshots (daily minimum, plus before any structural change/upgrade) to keep RPO/RTO low.
• Vertical scalability — volumes can typically be resized as data grows without downtime, complementing the horizontal scaling of the stateless container tier.

Summary: the front tier (load balancer + container pool) scales horizontally and fast because it's stateless, while the data tier relies on Ghaymah Block Storage for durability and consistency, with an in-memory cache absorbing read pressure and reducing latency.