Gamal0909
439a31ba93
- Created a GitHub Actions workflow for CI/CD to deploy to Ghyamah, including testing, building, and pushing Docker images. - Added architecture design document for handling 15,000 requests per second, detailing system components, capacity planning, and cold start strategies. - Introduced a Python-based uptime/latency/SSL monitor with a static dashboard, utilizing standard libraries only. - Included Dockerfile and entrypoint script for the monitoring application, ensuring it runs as a non-root user and handles process management. - Added a .dockerignore file to exclude unnecessary files from the Docker build context. - Created an HTML dashboard for visualizing monitoring metrics, including uptime, latency, and SSL certificate status.
7.1 KiB
High-Traffic Architecture Design (15,000 Requests/Second)
This document describes the proposed architecture for handling 15,000 requests per second (RPS) on the Ghaymah platform. It covers the system architecture, container capacity planning, cold start mitigation strategies, and the use of Ghaymah Block Storage for stateful workloads.
Note
The architecture diagram below is a placeholder. Replace it with your architecture image.
Table of Contents
- Architecture Overview
- Architecture Diagram
- Request Flow
- Container Capacity Planning
- Cold Start Strategy
- Ghaymah Block Storage for Stateful Workloads
- Key Design Decisions
Architecture Overview
The system is designed to process 15,000 requests per second while maintaining high availability, scalability, and fault tolerance.
The architecture consists of:
- WAF / CDN
- Ghaymah Load Balancer
- Auto-scaling application containers
- Redis cache cluster
- Primary database
- Read replicas
- Ghaymah Block Storage
The application containers remain stateless, while all persistent data is stored on external block storage.
Architecture Diagram
The architecture diagram below shows the proposed high-traffic deployment.
Request Flow
The following sequence illustrates how requests are processed.
- Clients send requests to the application.
- The WAF/CDN filters malicious traffic and caches static assets.
- Requests are forwarded to the Ghaymah Load Balancer.
- The load balancer distributes traffic across healthy application containers.
- Containers first attempt to retrieve data from the Redis cache.
- Cache misses are forwarded to the database.
- Read operations are served by database replicas whenever possible.
- Write operations are handled by the primary database.
- All database data is stored on Ghaymah Block Storage, ensuring persistence.
System Components
| Component | Responsibility |
|---|---|
| Clients | Generate incoming traffic (15,000 RPS) |
| WAF / CDN | Security filtering, DDoS protection, static content caching |
| Ghaymah Load Balancer | Evenly distributes requests across healthy containers |
| Application Containers | Stateless application processing |
| Redis Cluster | High-speed caching layer |
| Primary Database | Handles write operations |
| Read Replicas | Offload read traffic from the primary database |
| Ghaymah Block Storage | Persistent storage for stateful workloads |
Container Capacity Planning
The infrastructure must support 15,000 requests per second.
Assumptions
| Metric | Value |
|---|---|
| Expected Traffic | 15,000 req/s |
| Capacity per Container | 500 req/s |
Base Capacity
15,000 ÷ 500 = 30 Containers
Safety Buffer
To absorb unexpected traffic spikes:
30 × 30% = 9 Containers
Total Required Containers
30 + 9 = 39 Containers
| Calculation | Result |
|---|---|
| Base Containers | 30 |
| Safety Margin | 9 |
| Recommended Total | 39 Containers |
Recommendation
During peak traffic periods:
- Maintain approximately 39 running containers.
- Configure the auto-scaling policy to keep the minimum replica count close to this value.
- Scale beyond this threshold during sustained traffic increases.
Cold Start Strategy
Problem
When a new container starts, it requires time to:
- Pull the container image
- Initialize the runtime
- Establish database connections
- Load application dependencies
If traffic is routed before initialization completes, users may experience increased latency or request failures.
Recommended Mitigations
Readiness Probes
Configure readiness probes so that traffic is routed only after the application is fully initialized.
Example endpoint:
/health/ready
Only containers returning HTTP 200 OK should receive production traffic.
Pre-Warming
Initialize expensive resources during startup:
- Database connections
- Redis connections
- Configuration loading
- Dependency injection
- Frequently used libraries
Avoid performing heavy initialization during the first user request.
Image Optimization
Reduce startup time by:
- Using lightweight base images
- Removing unnecessary packages
- Minimizing image layers
- Keeping image sizes as small as possible
Examples:
- Alpine Linux
- Distroless Images
Capacity Buffer
Maintain spare capacity (30% safety margin) so existing containers can absorb traffic while new containers complete startup.
Benefits:
- Reduced request latency
- Smoother auto-scaling
- Improved user experience
Ghaymah Block Storage for Stateful Workloads
Containers are ephemeral by design.
If a container is deleted or restarted, its local filesystem is also removed.
Persistent workloads such as:
- PostgreSQL
- MySQL
- MariaDB
- MongoDB
- Message Brokers
must store data externally.
How It Works
- Provision a Ghaymah Block Storage volume.
- Attach the volume to the database container.
- Store all database files on the mounted volume.
- If the container fails, the storage remains intact.
- A replacement container automatically reattaches the existing volume.
Benefits
Persistent Data
Application data survives:
- Container restarts
- Platform upgrades
- Node failures
- Container replacements
Compute and Storage Separation
Containers remain disposable while storage persists independently.
This allows infrastructure updates without risking data loss.
Faster Recovery
If the database container crashes:
- A replacement container starts.
- The existing block storage volume is attached.
- The database resumes operation with its original data.
High Performance
Ghaymah Block Storage provides dedicated storage performance for demanding workloads.
Benefits include:
- High IOPS
- Low latency
- Reliable throughput
- Consistent database performance
Key Design Decisions
| Area | Decision |
|---|---|
| Compute | Stateless application containers |
| Scaling | Horizontal auto-scaling |
| Load Distribution | Ghaymah Load Balancer |
| Caching | Redis Cluster |
| Database | Primary + Read Replica architecture |
| Storage | Ghaymah Block Storage |
| Availability | Multi-container deployment |
| Cold Start Mitigation | Readiness probes, pre-warming, optimized images |
| Capacity Planning | 39 containers (30 base + 30% buffer) |
Summary
This architecture is designed to provide:
- High availability
- Horizontal scalability
- Fault tolerance
- Persistent storage
- Fast recovery from failures
- Efficient handling of 15,000 requests per second
By combining stateless application containers, intelligent load balancing, Redis caching, database replication, and Ghaymah Block Storage, the platform can maintain stable performance during normal operations as well as sudden traffic spikes.