106 أسطر
4.6 KiB
Markdown
106 أسطر
4.6 KiB
Markdown
# Q4 — Scalability & Load Balancing (Ghaymah Cloud)
|
||
|
||
## 1. Architecture
|
||
|
||
See `architecture.png` in this folder.
|
||
|
||
Flow: **Clients (15,000 req/s) → Load Balancer (round-robin + health checks) →
|
||
Container Auto-Scaling Pool (39 containers) → In-Memory Cache / Database +
|
||
Ghaymah Block Storage**.
|
||
|
||
- The front tier (load balancer + containers) is **stateless** and scales
|
||
horizontally.
|
||
- The data tier (database) is **stateful** and relies on Ghaymah Block Storage
|
||
for durability, decoupled from the container lifecycle.
|
||
|
||
---
|
||
|
||
## 2. Container Capacity Calculation
|
||
|
||
**Given:**
|
||
- Total incoming load: `15,000 req/s`
|
||
- Capacity per container: `500 req/s`
|
||
- Safety margin: `30%` (to absorb traffic spikes and node/container failures)
|
||
|
||
**Formula:**
|
||
|
||
```
|
||
Required capacity = Total load × (1 + margin)
|
||
= 15,000 × 1.30
|
||
= 19,500 req/s
|
||
|
||
Containers needed = Required capacity / Capacity per container
|
||
= 19,500 / 500
|
||
= 39 containers
|
||
```
|
||
|
||
**Result: 39 containers**
|
||
|
||
| Tier | Count | Purpose |
|
||
|---|---|---|
|
||
| Min replicas (baseline) | 30 | Covers the raw 15,000 req/s |
|
||
| Target / desired | 39 | Includes the 30% margin |
|
||
| Max replicas (HPA ceiling) | 50–55 | Extra headroom for burst traffic |
|
||
|
||
**Recommendation:** configure the Horizontal Pod Autoscaler (HPA) on a
|
||
composite metric — `requests-per-second` (via a Prometheus adapter) as the
|
||
primary signal and `CPU utilization` as a secondary safeguard — rather than
|
||
CPU alone, since CPU can lag behind real request pressure.
|
||
|
||
---
|
||
|
||
## 3. Cold Start Strategy for New Containers
|
||
|
||
1. **Pre-warmed pool (warm standby)** — keep 2–3 idle, already-running
|
||
containers outside the active traffic path so bursts are absorbed
|
||
instantly instead of waiting for a container to be built from scratch.
|
||
2. **Slim, optimized images** — use minimal base images (Alpine/distroless),
|
||
reduce layers, and pre-pull images on nodes to avoid registry pull latency
|
||
during scale-out events.
|
||
3. **Strict readiness probes** — a container only joins the load balancer
|
||
pool after passing a `readinessProbe` (DB/cache connectivity check),
|
||
preventing 503s from traffic sent to a not-yet-ready instance.
|
||
4. **Predictive/scheduled scaling alongside reactive HPA** — if peak hours
|
||
are known (campaigns, daily traffic peaks), trigger scheduled scale-up
|
||
5–10 minutes ahead of the peak instead of relying solely on reactive HPA,
|
||
which lags behind the metrics collection window.
|
||
5. **Connection pooling & pre-warmed init** — open DB/cache connections in an
|
||
init container or internal warm-up endpoint before the first real request
|
||
arrives, instead of lazy-initializing on first use.
|
||
6. **Graceful scale-down** — use a `preStop` hook and a proper
|
||
`terminationGracePeriodSeconds` to drain in-flight requests before
|
||
terminating containers during scale-in, avoiding dropped requests on the
|
||
way down.
|
||
|
||
---
|
||
|
||
## 4. Ghaymah Block Storage for Stateful Workloads
|
||
|
||
The containers above are intentionally **stateless** — any of them can be
|
||
killed and replaced without losing data. Anything that needs durable state
|
||
(database, message queues, uploaded files) must be decoupled from the
|
||
container's lifecycle — this is where **Block Storage** comes in:
|
||
|
||
- **Decouples data from container lifecycle** — a block volume is attached
|
||
to a container/VM as an independent raw block device. If the container is
|
||
rescheduled or fails, the same volume is re-attached to the new
|
||
instance/node without data loss — unlike ephemeral container storage,
|
||
which is wiped on restart.
|
||
- **Best fit** — relational databases (PostgreSQL/MySQL), messaging systems
|
||
(Kafka/RabbitMQ), and any workload needing low-latency random I/O, since
|
||
block storage offers near-local-disk performance versus object storage.
|
||
- **High availability** — one block volume per database replica (never share
|
||
a single volume across concurrently-active instances); pair it with
|
||
application-level replication (e.g., Postgres streaming replication)
|
||
rather than relying on storage replication alone.
|
||
- **Snapshots & backup** — schedule periodic snapshots (daily minimum, plus
|
||
before any structural change/upgrade) to keep RPO/RTO low.
|
||
- **Vertical scalability** — volumes can typically be resized as data grows
|
||
without downtime, complementing the horizontal scaling of the stateless
|
||
container tier.
|
||
|
||
**Summary:** the front tier (load balancer + container pool) scales
|
||
horizontally and fast because it's stateless, while the data tier relies on
|
||
Ghaymah Block Storage for durability and consistency, with an in-memory
|
||
cache absorbing read pressure and reducing latency.
|