Add CI/CD pipeline, architecture documentation, and monitoring application
- Created a GitHub Actions workflow for CI/CD to deploy to Ghyamah, including testing, building, and pushing Docker images. - Added architecture design document for handling 15,000 requests per second, detailing system components, capacity planning, and cold start strategies. - Introduced a Python-based uptime/latency/SSL monitor with a static dashboard, utilizing standard libraries only. - Included Dockerfile and entrypoint script for the monitoring application, ensuring it runs as a non-root user and handles process management. - Added a .dockerignore file to exclude unnecessary files from the Docker build context. - Created an HTML dashboard for visualizing monitoring metrics, including uptime, latency, and SSL certificate status.
هذا الالتزام موجود في:
307
Q4/README.md
Normal file
307
Q4/README.md
Normal file
@@ -0,0 +1,307 @@
|
||||
# High-Traffic Architecture Design (15,000 Requests/Second)
|
||||
|
||||
This document describes the proposed architecture for handling **15,000 requests per second (RPS)** on the Ghaymah platform. It covers the system architecture, container capacity planning, cold start mitigation strategies, and the use of Ghaymah Block Storage for stateful workloads.
|
||||
|
||||
> **Note**
|
||||
>
|
||||
> The architecture diagram below is a placeholder. Replace it with your architecture image.
|
||||
|
||||
---
|
||||
|
||||
# Table of Contents
|
||||
|
||||
- [Architecture Overview](#architecture-overview)
|
||||
- [Architecture Diagram](#architecture-diagram)
|
||||
- [Request Flow](#request-flow)
|
||||
- [Container Capacity Planning](#container-capacity-planning)
|
||||
- [Cold Start Strategy](#cold-start-strategy)
|
||||
- [Ghaymah Block Storage for Stateful Workloads](#ghaymah-block-storage-for-stateful-workloads)
|
||||
- [Key Design Decisions](#key-design-decisions)
|
||||
|
||||
---
|
||||
|
||||
# Architecture Overview
|
||||
|
||||
The system is designed to process **15,000 requests per second** while maintaining high availability, scalability, and fault tolerance.
|
||||
|
||||
The architecture consists of:
|
||||
|
||||
- WAF / CDN
|
||||
- Ghaymah Load Balancer
|
||||
- Auto-scaling application containers
|
||||
- Redis cache cluster
|
||||
- Primary database
|
||||
- Read replicas
|
||||
- Ghaymah Block Storage
|
||||
|
||||
The application containers remain **stateless**, while all persistent data is stored on external block storage.
|
||||
|
||||
---
|
||||
|
||||
# Architecture Diagram
|
||||
|
||||
The architecture diagram below shows the proposed high-traffic deployment.
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
# Request Flow
|
||||
|
||||
The following sequence illustrates how requests are processed.
|
||||
|
||||
1. Clients send requests to the application.
|
||||
2. The **WAF/CDN** filters malicious traffic and caches static assets.
|
||||
3. Requests are forwarded to the **Ghaymah Load Balancer**.
|
||||
4. The load balancer distributes traffic across healthy application containers.
|
||||
5. Containers first attempt to retrieve data from the **Redis cache**.
|
||||
6. Cache misses are forwarded to the database.
|
||||
7. Read operations are served by database replicas whenever possible.
|
||||
8. Write operations are handled by the primary database.
|
||||
9. All database data is stored on **Ghaymah Block Storage**, ensuring persistence.
|
||||
|
||||
---
|
||||
|
||||
# System Components
|
||||
|
||||
| Component | Responsibility |
|
||||
|-----------|----------------|
|
||||
| **Clients** | Generate incoming traffic (15,000 RPS) |
|
||||
| **WAF / CDN** | Security filtering, DDoS protection, static content caching |
|
||||
| **Ghaymah Load Balancer** | Evenly distributes requests across healthy containers |
|
||||
| **Application Containers** | Stateless application processing |
|
||||
| **Redis Cluster** | High-speed caching layer |
|
||||
| **Primary Database** | Handles write operations |
|
||||
| **Read Replicas** | Offload read traffic from the primary database |
|
||||
| **Ghaymah Block Storage** | Persistent storage for stateful workloads |
|
||||
|
||||
---
|
||||
|
||||
# Container Capacity Planning
|
||||
|
||||
The infrastructure must support **15,000 requests per second**.
|
||||
|
||||
## Assumptions
|
||||
|
||||
| Metric | Value |
|
||||
|---------|------:|
|
||||
| Expected Traffic | 15,000 req/s |
|
||||
| Capacity per Container | 500 req/s |
|
||||
|
||||
---
|
||||
|
||||
## Base Capacity
|
||||
|
||||
```
|
||||
15,000 ÷ 500 = 30 Containers
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Safety Buffer
|
||||
|
||||
To absorb unexpected traffic spikes:
|
||||
|
||||
```
|
||||
30 × 30% = 9 Containers
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Total Required Containers
|
||||
|
||||
```
|
||||
30 + 9 = 39 Containers
|
||||
```
|
||||
|
||||
| Calculation | Result |
|
||||
|-------------|-------:|
|
||||
| Base Containers | 30 |
|
||||
| Safety Margin | 9 |
|
||||
| **Recommended Total** | **39 Containers** |
|
||||
|
||||
---
|
||||
|
||||
## Recommendation
|
||||
|
||||
During peak traffic periods:
|
||||
|
||||
- Maintain approximately **39 running containers**.
|
||||
- Configure the auto-scaling policy to keep the minimum replica count close to this value.
|
||||
- Scale beyond this threshold during sustained traffic increases.
|
||||
|
||||
---
|
||||
|
||||
# Cold Start Strategy
|
||||
|
||||
## Problem
|
||||
|
||||
When a new container starts, it requires time to:
|
||||
|
||||
- Pull the container image
|
||||
- Initialize the runtime
|
||||
- Establish database connections
|
||||
- Load application dependencies
|
||||
|
||||
If traffic is routed before initialization completes, users may experience increased latency or request failures.
|
||||
|
||||
---
|
||||
|
||||
## Recommended Mitigations
|
||||
|
||||
### Readiness Probes
|
||||
|
||||
Configure readiness probes so that traffic is routed only after the application is fully initialized.
|
||||
|
||||
Example endpoint:
|
||||
|
||||
```
|
||||
/health/ready
|
||||
```
|
||||
|
||||
Only containers returning **HTTP 200 OK** should receive production traffic.
|
||||
|
||||
---
|
||||
|
||||
### Pre-Warming
|
||||
|
||||
Initialize expensive resources during startup:
|
||||
|
||||
- Database connections
|
||||
- Redis connections
|
||||
- Configuration loading
|
||||
- Dependency injection
|
||||
- Frequently used libraries
|
||||
|
||||
Avoid performing heavy initialization during the first user request.
|
||||
|
||||
---
|
||||
|
||||
### Image Optimization
|
||||
|
||||
Reduce startup time by:
|
||||
|
||||
- Using lightweight base images
|
||||
- Removing unnecessary packages
|
||||
- Minimizing image layers
|
||||
- Keeping image sizes as small as possible
|
||||
|
||||
Examples:
|
||||
|
||||
- Alpine Linux
|
||||
- Distroless Images
|
||||
|
||||
---
|
||||
|
||||
### Capacity Buffer
|
||||
|
||||
Maintain spare capacity (30% safety margin) so existing containers can absorb traffic while new containers complete startup.
|
||||
|
||||
Benefits:
|
||||
|
||||
- Reduced request latency
|
||||
- Smoother auto-scaling
|
||||
- Improved user experience
|
||||
|
||||
---
|
||||
|
||||
# Ghaymah Block Storage for Stateful Workloads
|
||||
|
||||
Containers are **ephemeral** by design.
|
||||
|
||||
If a container is deleted or restarted, its local filesystem is also removed.
|
||||
|
||||
Persistent workloads such as:
|
||||
|
||||
- PostgreSQL
|
||||
- MySQL
|
||||
- MariaDB
|
||||
- MongoDB
|
||||
- Message Brokers
|
||||
|
||||
must store data externally.
|
||||
|
||||
---
|
||||
|
||||
## How It Works
|
||||
|
||||
1. Provision a **Ghaymah Block Storage** volume.
|
||||
2. Attach the volume to the database container.
|
||||
3. Store all database files on the mounted volume.
|
||||
4. If the container fails, the storage remains intact.
|
||||
5. A replacement container automatically reattaches the existing volume.
|
||||
|
||||
---
|
||||
|
||||
## Benefits
|
||||
|
||||
### Persistent Data
|
||||
|
||||
Application data survives:
|
||||
|
||||
- Container restarts
|
||||
- Platform upgrades
|
||||
- Node failures
|
||||
- Container replacements
|
||||
|
||||
---
|
||||
|
||||
### Compute and Storage Separation
|
||||
|
||||
Containers remain disposable while storage persists independently.
|
||||
|
||||
This allows infrastructure updates without risking data loss.
|
||||
|
||||
---
|
||||
|
||||
### Faster Recovery
|
||||
|
||||
If the database container crashes:
|
||||
|
||||
1. A replacement container starts.
|
||||
2. The existing block storage volume is attached.
|
||||
3. The database resumes operation with its original data.
|
||||
|
||||
---
|
||||
|
||||
### High Performance
|
||||
|
||||
Ghaymah Block Storage provides dedicated storage performance for demanding workloads.
|
||||
|
||||
Benefits include:
|
||||
|
||||
- High IOPS
|
||||
- Low latency
|
||||
- Reliable throughput
|
||||
- Consistent database performance
|
||||
|
||||
---
|
||||
|
||||
# Key Design Decisions
|
||||
|
||||
| Area | Decision |
|
||||
|------|----------|
|
||||
| Compute | Stateless application containers |
|
||||
| Scaling | Horizontal auto-scaling |
|
||||
| Load Distribution | Ghaymah Load Balancer |
|
||||
| Caching | Redis Cluster |
|
||||
| Database | Primary + Read Replica architecture |
|
||||
| Storage | Ghaymah Block Storage |
|
||||
| Availability | Multi-container deployment |
|
||||
| Cold Start Mitigation | Readiness probes, pre-warming, optimized images |
|
||||
| Capacity Planning | 39 containers (30 base + 30% buffer) |
|
||||
|
||||
---
|
||||
|
||||
# Summary
|
||||
|
||||
This architecture is designed to provide:
|
||||
|
||||
- High availability
|
||||
- Horizontal scalability
|
||||
- Fault tolerance
|
||||
- Persistent storage
|
||||
- Fast recovery from failures
|
||||
- Efficient handling of **15,000 requests per second**
|
||||
|
||||
By combining stateless application containers, intelligent load balancing, Redis caching, database replication, and Ghaymah Block Storage, the platform can maintain stable performance during normal operations as well as sudden traffic spikes.
|
||||
المرجع في مشكلة جديدة
حظر مستخدم