Gamal0909/Ghaymah-Exam-GamalMohamed-SRE
1
0
نسخ 0
الشفرة المشكلات طلبات السحب Actions Packages المشاريع الإصدارات الويكي النشاط

Add CI/CD pipeline, architecture documentation, and monitoring application

تصفح المصدر 
- Created a GitHub Actions workflow for CI/CD to deploy to Ghyamah, including testing, building, and pushing Docker images.
- Added architecture design document for handling 15,000 requests per second, detailing system components, capacity planning, and cold start strategies.
- Introduced a Python-based uptime/latency/SSL monitor with a static dashboard, utilizing standard libraries only.
- Included Dockerfile and entrypoint script for the monitoring application, ensuring it runs as a non-root user and handles process management.
- Added a .dockerignore file to exclude unnecessary files from the Docker build context.
- Created an HTML dashboard for visualizing monitoring metrics, including uptime, latency, and SSL certificate status.
هذا الالتزام موجود في:
Gamal0909
2026-07-27 23:08:39 +03:00
الأصل c3368b9124
التزام 439a31ba93
16 ملفات معدلة مع 1753 إضافات و0 حذوفات
تنزيل ملف التصحيح تنزيل ملف الاختلاف توسيع جميع الملفات طي جميع الملفات

6
Q1/.dockerignore Normal file
عرض الملف

@@ -0,0 +1,6 @@
data/
*.pyc
__pycache__/
.git
.gitignore
README.md

13
Q1/Dockerfile Normal file
عرض الملف

@@ -0,0 +1,13 @@
FROM node:20-alpine3.16
WORKDIR /app
COPY --chown=node:node server.js .
RUN npm init -y && npm install express cors
USER node
EXPOSE 3000
CMD ["node" , "server.js"]

60
Q1/index.html Normal file
عرض الملف

@@ -0,0 +1,60 @@
<!DOCTYPE html>
<html lang="ar" dir="rtl">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>لوحة مراقبة الـ API</title>
<style>
body { font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; background: #f4f7f6; display: flex; justify-content: center; align-items: center; height: 100vh; margin: 0; }
.dashboard { background: white; padding: 30px; border-radius: 10px; box-shadow: 0 4px 15px rgba(0,0,0,0.1); text-align: center; width: 400px; }
h2 { color: #333; margin-bottom: 20px; }
.metric { background: #eef2f5; margin: 10px 0; padding: 15px; border-radius: 8px; font-size: 1.2em; display: flex; justify-content: space-between; }
.metric span { font-weight: bold; color: #0056b3; }
.status-dot { display: inline-block; width: 12px; height: 12px; border-radius: 50%; background: gray; margin-left: 8px; }
.online { background: #28a745; }
.offline { background: #dc3545; }
</style>
</head>
<body>
<div class="dashboard">
<h2>مراقبة النظام <span id="statusDot" class="status-dot"></span></h2>
<div class="metric">
الحالة: <span id="statusText">جاري التحميل...</span>
</div>
<div class="metric">
زمن الاستجابة: <span id="responseTime">0 ms</span>
</div>
<div class="metric">
عدد الطلبات الإجمالي: <span id="requestCount">0</span>
</div>
</div>
<script>
const API_URL = "";
async function fetchMetrics() {
try {
const res = await fetch(API_URL);
const data = await res.json();
document.getElementById('statusText').innerText = data.status;
document.getElementById('statusText').style.color = '#28a745';
document.getElementById('statusDot').className = 'status-dot online';
document.getElementById('responseTime').innerText = data.response_time_ms + ' ms';
document.getElementById('requestCount').innerText = data.request_count;
} catch (error) {
document.getElementById('statusText').innerText = 'انقطاع الاتصال (Offline)';
document.getElementById('statusText').style.color = '#dc3545';
document.getElementById('statusDot').className = 'status-dot offline';
document.getElementById('responseTime').innerText = '-';
}
}
setInterval(fetchMetrics, 5000);
fetchMetrics();
</script>
</body>
</html>

28
Q1/monitor.sh Executable file
عرض الملف

@@ -0,0 +1,28 @@
#!/bin/bash
API_URL=$1
echo "Starting monitor script..."
echo "API URL: $API_URL"
echo "------------------------------------------------"
while true; do
HTTP_STATUS=$(curl -o /dev/null -s -w "%{http_code}" -m 5 "$API_URL")
if [ "$HTTP_STATUS" -eq 200 ]; then
response=$(curl -s -m 5 "$API_URL")
status=$(echo "$response" | jq -r '.status' 2>/dev/null)
if [ -n "$status" ] && [ "$status" != "null" ]; then
echo "Status: $status | Response Time: ${response_time}ms"
else
echo "HTTP 200 OK, but invalid JSON format received."
fi
elif [ "$HTTP_STATUS" -eq 000 ]; then
echo "Application is Offline or unreachable (Timeout)."
else
echo "Application is unhealthy. HTTP Status: $HTTP_STATUS"
fi
sleep 30
done

35
Q1/server.js Normal file
عرض الملف

@@ -0,0 +1,35 @@
const express = require('express');
const cors = require('cors');
const app = express();
app.use(cors());
let requestCount = 0;
let lastResponseTime = 0;
app.use((req, res, next) => {
requestCount++;
const start = Date.now();
res.on('finish', () => {
lastResponseTime = Date.now() - start;
});
next();
});
app.get('/health', (req, res) => {
res.json({
status: 'Online',
uptime_seconds: process.uptime(),
request_count: requestCount,
response_time_ms: lastResponseTime
});
});
app.get('/', (req, res) => {
res.send('Welcome to the Backend API!');
});
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
console.log(`API is running on port ${PORT}`);
});

271
Q2/readme,md Normal file
عرض الملف

@@ -0,0 +1,271 @@
# Incident Postmortem: Core Backend API Service Interruption
> **Incident Date:** July 27, 2026
> **Affected Service:** Core .NET Backend API (Ghaymah Platform)
> **Severity:** SEV-1 (Critical)
> **Status:** Resolved
---
# Table of Contents
- [Incident Overview](#incident-overview)
- [Incident Metadata](#incident-metadata)
- [Executive Summary](#executive-summary)
- [Incident Timeline](#incident-timeline)
- [Root Cause Analysis](#root-cause-analysis)
- [Proposed Auto-Scaling Architecture](#proposed-auto-scaling-architecture)
- [Horizontal Pod Autoscaler Configuration](#horizontal-pod-autoscaler-configuration)
- [Observability & Early Detection](#observability--early-detection)
- [Action Items](#action-items)
- [Lessons Learned](#lessons-learned)
---
# Incident Overview
On **July 27, 2026**, the **Core .NET Backend API** experienced a complete service outage lasting **45 minutes**.
The outage was caused by repeated **OOMKilled (Exit Code 137)** events after the application exhausted its available memory during a sudden traffic spike.
The platform entered a continuous **CrashLoop** state, resulting in **100% API failure** until memory resources were increased and the containers were restarted.
---
# Incident Metadata
| Category | Details |
|-----------|---------|
| **Affected Service** | Core .NET Backend API |
| **Platform** | Ghaymah |
| **Date** | 2026-07-27 |
| **Downtime** | 45 Minutes |
| **Time** | 14:00 – 14:45 EEST |
| **Severity** | SEV-1 (Critical) |
| **Customer Impact** | 100% API transaction failures |
| **Observed Errors** | 502 Bad Gateway |
---
# Executive Summary
A sudden **400% increase in traffic** caused the backend API to consume memory rapidly.
The application maintained an **unbounded in-memory cache**, storing increasingly large datasets without expiration or eviction.
Once the container reached its configured memory limit, the Linux kernel terminated the process (**OOMKilled - Exit Code 137**) to protect node stability.
The orchestration platform continuously restarted the containers, causing a **CrashLoopBackOff** cycle and preventing the service from recovering automatically.
Service was restored after:
- Increasing the container memory limit
- Restarting the backend pods
- Verifying successful application startup
Future mitigation requires:
- Memory-based auto-scaling
- Cache eviction policies
- API pagination
- Improved monitoring and alerting
---
# Incident Timeline
| Time | Event |
|------|-------|
| **13:50** | Traffic increased by approximately **400%** due to an unexpected external campaign. |
| **13:58** | Container memory utilization exceeded **95%** of allocated memory. |
| **14:00** | First container terminated with **OOMKilled (Exit Code 137)**. Service degradation begins. |
| **14:05 – 14:30** | Containers repeatedly restarted by the orchestrator, resulting in a CrashLoop and complete outage. |
| **14:30** | On-call infrastructure engineer identified repeated OOMKilled events from platform logs. |
| **14:35** | Memory limit increased from **512Mi** to **2Gi** and backend pods restarted manually. |
| **14:45** | Service stabilized and API responses returned **HTTP 200 OK**. Incident closed. |
---
# Root Cause Analysis
## Direct Cause
The Linux kernel terminated the backend process because the application attempted to allocate more memory than the container's configured memory limit.
```
Exit Code: 137
Reason: OOMKilled
```
---
## Underlying Causes
### Unbounded In-Memory Cache
The application stored data inside a local in-memory dictionary that had:
- No Time-To-Live (TTL)
- No maximum cache size
- No eviction policy
Large requests continuously expanded the cache until the process exhausted available memory.
---
### Missing API Pagination
Several endpoints returned very large datasets.
Without pagination:
- Large payloads were cached
- Memory usage increased rapidly
- Garbage collection became inefficient
---
### Lack of Horizontal Auto-Scaling
The deployment relied on static memory limits and a fixed number of replicas.
As traffic increased:
- No new replicas were created
- Existing containers absorbed all incoming traffic
- Memory utilization reached critical levels
---
# Proposed Auto-Scaling Architecture
To prevent similar incidents, backend workloads should scale automatically based on memory utilization.
| Parameter | Recommended Value |
|-----------|-------------------|
| **Scaling Metric** | Average Memory Utilization |
| **Target Utilization** | 75% |
| **Minimum Replicas** | 3 |
| **Maximum Replicas** | 12 |
| **Scale-Up Policy** | Add up to 4 replicas immediately |
| **Scale-Down Policy** | Wait 5 minutes after memory falls below 40% |
---
# Horizontal Pod Autoscaler Configuration
```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: backend-api-scaler
spec:
minReplicas: 3
maxReplicas: 12
metrics:
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 75
```
---
# Observability & Early Detection
To move from reactive troubleshooting to proactive monitoring, the following observability improvements should be implemented.
## Alerting Rules
### Memory Utilization Alert
Trigger notification when:
- Memory utilization exceeds **70%**
- Sustained for **2 consecutive minutes**
Notification targets:
- Email
- Slack
- Webhook
---
### CrashLoop Detection
Create a high-priority incident whenever:
- Container restart count exceeds **3**
- Within a **10-minute** window
---
## Dashboard Metrics
Create a dedicated monitoring dashboard showing:
- Container memory usage
- Container memory limits
- Application heap size
- Garbage Collection duration
- API request rate
- Current replica count
- Container restart count
- CPU utilization
- Response latency
- Error rate (4xx / 5xx)
---
# Action Items
| Task | Owner | Priority | Status |
|------|-------|----------|--------|
| Implement cache eviction (LRU + TTL) | Development Team | Critical | In Progress |
| Enforce API pagination | Development Team | Critical | In Progress |
| Deploy memory-based Horizontal Pod Autoscaler | DevOps Team | High | Not Started |
| Create synthetic load tests (5× traffic) | QA Team | Medium | Not Started |
| Update on-call operational runbooks | Operations Team | Low | Completed |
---
# Lessons Learned
The incident highlighted several architectural improvements required to increase platform resilience.
## Infrastructure
- Configure Horizontal Pod Autoscaler (HPA)
- Define resource requests and limits carefully
- Monitor memory consumption continuously
## Application
- Implement cache eviction (LRU)
- Apply cache expiration (TTL)
- Enforce pagination on all large API endpoints
- Optimize memory allocation patterns
## Operations
- Improve proactive alerting
- Expand observability dashboards
- Regularly execute load and stress testing
- Update incident response runbooks
---
# Resolution Summary
| Item | Result |
|------|--------|
| **Root Cause** | Unbounded in-memory cache caused container OOM |
| **Immediate Fix** | Increased memory limit (512Mi → 2Gi) and restarted pods |
| **Long-Term Fixes** | HPA, cache eviction, pagination, monitoring improvements |
| **Incident Status** | ✅ Resolved |
```

149
Q3-CICD/README.md Normal file
عرض الملف

@@ -0,0 +1,149 @@
# Ghaymah CLI Integration & Environment Strategy
This document outlines the deployment environment strategy and provides comprehensive documentation for installing, configuring, and authenticating with the **Ghaymah Command Line Interface (CLI)**.
---
# Table of Contents
- [Environment Strategy](#environment-strategy)
- [Staging Environment](#staging-environment)
- [Production Environment](#production-environment)
- [Ghaymah CLI](#ghaymah-cli)
- [Prerequisites](#prerequisites)
- [Installation](#installation)
- [Authentication](#authentication)
- [Core Commands](#core-commands)
---
# Environment Strategy
To ensure reliable deployments and stable software releases, Ghaymah uses separate environments for testing and production workloads.
| Feature | Staging | Production |
|----------|----------|------------|
| **Purpose** | Final testing, QA, and integration validation. Mirrors production configuration as closely as possible. | Live customer-facing environment with maximum stability and availability. |
| **Users** | Developers, QA engineers, and internal stakeholders. | End users and customers. |
| **Data** | Mock, seeded, or anonymized datasets. | Real production data. |
| **Deployment** | Automatic deployment after merging into the staging branch (Continuous Deployment). | Manual approval with version tracking (release tags or commit SHA). |
| **Resources** | Lower CPU and memory allocation to reduce infrastructure costs. | High availability, load balancing, monitoring, and autoscaling. |
---
# Ghaymah CLI
The **Ghaymah CLI** allows developers and CI/CD pipelines to interact with the platform directly from the terminal.
It can be used to:
- Authenticate with the platform
- Manage applications
- Deploy services
- View running applications
- Integrate deployments into CI/CD workflows
---
## Prerequisites
Before using the CLI, you must generate a **Personal Access Token**.
1. Log in to the **Ghaymah Web Console**.
2. Navigate to:
```
Account Settings
└── Developer Settings
```
3. Generate a new **Personal Access Token**.
4. Copy the token immediately.
> **Note**
>
> The token is displayed only once. Store it securely.
---
## Installation
### Linux (Debian/Ubuntu/Linux Mint)
Install the CLI using:
```bash
curl -sL https://cli.ghaymah.systems/install.sh | sudo bash
```
Verify the installation:
```bash
ghaymah --version
```
---
## Authentication
For local development or automated CI/CD pipelines, authenticate using your Personal Access Token.
### Step 1 — Export the Token
```bash
export GHAYMAH_TOKEN="your_personal_access_token_here"
```
### Step 2 — Login
```bash
ghaymah login --token "$GHAYMAH_TOKEN"
```
If authentication succeeds, the CLI is ready to use.
---
# Core Commands
## List Applications
Display all deployed applications and their current health status.
```bash
ghaymah apps list
```
Example output:
```text
NAME STATUS REGION
frontend Running eu-central
backend Running eu-central
database Running eu-central
```
---
## Check CLI Version
```bash
ghaymah --version
```
---
## Display Help
```bash
ghaymah --help
```
---
# Environment Summary
| Environment | Deployment | Data | Approval |
|-------------|------------|------|----------|
| **Staging** | Automatic | Test/Mock | Not Required |
| **Production** | Manual | Live | Required |

112
Q3-CICD/workflow.yml Normal file
عرض الملف

@@ -0,0 +1,112 @@
name: CI/CD Pipeline to Ghyamah
on:
push:
branches:
- main
pull_request:
branches:
- main
env:
GHYAMAH_REGISTRY: app.gitpasha.com
GHYAMAH_USERNAME: ${{ github.actor }}
ImageName: ${{ env.GHYAMAH_USERNAME }}/my-app
jobs:
###############################
# Test App Code Job
#################################
test:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: '16'
- name: Install dependencies
run: npm install
- name: Run tests
run: npm test
- name: Build project
run: npm run build
########################################
# build and push docker image to ghyamah registry
########################################
BuildAndPush:
runs-on: ubuntu-latest
needs: test
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Log in to Ghyamah Docker registry
uses: docker/login-action@v2
with:
registry: ${{ env.GHYAMAH_REGISTRY }}
username: ${{ env.GHYAMAH_USERNAME }}
password: ${{ secrets.GHYAMAH_PASSWORD }}
- name: Build and Push Docker Image
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: |
${{ env.GHYAMAH_REGISTRY }}/${{ env.ImageName }}:${{ github.sha }}
${{ env.GHYAMAH_REGISTRY }}/${{ env.ImageName }}:latest
#################################
# Deploy to Staging and Production Jobs
#################################
deploy-staging:
runs-on: ubuntu-latest
needs: BuildAndPush
steps:
- name: Install Ghyamah CLI
run: |
curl -sSL https://get.ghaymah.systems/cli/install.sh | sh
echo "$HOME/.ghaymah/bin" >> $GITHUB_PATH
- name: Log in to Ghyamah
run: |
ghyamah login --username ${{ env.GHYAMAH_USERNAME }} --password ${{ secrets.GHYAMAH_PASSWORD }}
- name: Deploy to Staging
env:
GHYAMAH_TOKEN: ${{ secrets.GHYAMAH_PASSWORD }}
run: |
ghayamah app update my-staging-app --image ${{ env.GHYAMAH_REGISTRY }}/${{ env.ImageName }}:latest
#######################################
# Deploy to Production Job with Manual Approval
#######################################
deploy-production:
runs-on: ubuntu-latest
needs: BuildAndPush
steps:
- name: Wait for Manual Approval
uses: trstringer/manual-approval@v1
with:
secret: ${{ secrets.GHYAMAH_PASSWORD }}
issue-title: "Approval Required for Deployment to Production"
issue-body: "Please review the changes and approve the deployment to production."
- name: Install Ghyamah CLI
run: |
curl -sSL https://get.ghaymah.systems/cli/install.sh | sh
echo "$HOME/.ghaymah/bin" >> $GITHUB_PATH
- name: Log in to Ghyamah
run: |
ghyamah login --username ${{ env.GHYAMAH_USERNAME }} --password ${{ secrets.GHYAMAH_PASSWORD }}
- name: Deploy to Production
env:
GHYAMAH_TOKEN: ${{ secrets.GHYAMAH_PASSWORD }}
run: |
ghayamah app update my-production-app --image ${{ env.GHYAMAH_REGISTRY }}/${{ env.ImageName }}:latest

ثنائية
Q4/GhaymahAPI.png Normal file
عرض الملف

ملف ثنائي غير معروض.
جنبًا إلى جنب

بعد

العرض:  |  الارتفاع:  |  الحجم: 1.3 MiB

307
Q4/README.md Normal file
عرض الملف

@@ -0,0 +1,307 @@
# High-Traffic Architecture Design (15,000 Requests/Second)
This document describes the proposed architecture for handling **15,000 requests per second (RPS)** on the Ghaymah platform. It covers the system architecture, container capacity planning, cold start mitigation strategies, and the use of Ghaymah Block Storage for stateful workloads.
> **Note**
>
> The architecture diagram below is a placeholder. Replace it with your architecture image.
---
# Table of Contents
- [Architecture Overview](#architecture-overview)
- [Architecture Diagram](#architecture-diagram)
- [Request Flow](#request-flow)
- [Container Capacity Planning](#container-capacity-planning)
- [Cold Start Strategy](#cold-start-strategy)
- [Ghaymah Block Storage for Stateful Workloads](#ghaymah-block-storage-for-stateful-workloads)
- [Key Design Decisions](#key-design-decisions)
---
# Architecture Overview
The system is designed to process **15,000 requests per second** while maintaining high availability, scalability, and fault tolerance.
The architecture consists of:
- WAF / CDN
- Ghaymah Load Balancer
- Auto-scaling application containers
- Redis cache cluster
- Primary database
- Read replicas
- Ghaymah Block Storage
The application containers remain **stateless**, while all persistent data is stored on external block storage.
---
# Architecture Diagram
The architecture diagram below shows the proposed high-traffic deployment.
![Architecture Diagram](./GhaymahAPI.png)
---
# Request Flow
The following sequence illustrates how requests are processed.
1. Clients send requests to the application.
2. The **WAF/CDN** filters malicious traffic and caches static assets.
3. Requests are forwarded to the **Ghaymah Load Balancer**.
4. The load balancer distributes traffic across healthy application containers.
5. Containers first attempt to retrieve data from the **Redis cache**.
6. Cache misses are forwarded to the database.
7. Read operations are served by database replicas whenever possible.
8. Write operations are handled by the primary database.
9. All database data is stored on **Ghaymah Block Storage**, ensuring persistence.
---
# System Components
| Component | Responsibility |
|-----------|----------------|
| **Clients** | Generate incoming traffic (15,000 RPS) |
| **WAF / CDN** | Security filtering, DDoS protection, static content caching |
| **Ghaymah Load Balancer** | Evenly distributes requests across healthy containers |
| **Application Containers** | Stateless application processing |
| **Redis Cluster** | High-speed caching layer |
| **Primary Database** | Handles write operations |
| **Read Replicas** | Offload read traffic from the primary database |
| **Ghaymah Block Storage** | Persistent storage for stateful workloads |
---
# Container Capacity Planning
The infrastructure must support **15,000 requests per second**.
## Assumptions
| Metric | Value |
|---------|------:|
| Expected Traffic | 15,000 req/s |
| Capacity per Container | 500 req/s |
---
## Base Capacity
```
15,000 ÷ 500 = 30 Containers
```
---
## Safety Buffer
To absorb unexpected traffic spikes:
```
30 × 30% = 9 Containers
```
---
## Total Required Containers
```
30 + 9 = 39 Containers
```
| Calculation | Result |
|-------------|-------:|
| Base Containers | 30 |
| Safety Margin | 9 |
| **Recommended Total** | **39 Containers** |
---
## Recommendation
During peak traffic periods:
- Maintain approximately **39 running containers**.
- Configure the auto-scaling policy to keep the minimum replica count close to this value.
- Scale beyond this threshold during sustained traffic increases.
---
# Cold Start Strategy
## Problem
When a new container starts, it requires time to:
- Pull the container image
- Initialize the runtime
- Establish database connections
- Load application dependencies
If traffic is routed before initialization completes, users may experience increased latency or request failures.
---
## Recommended Mitigations
### Readiness Probes
Configure readiness probes so that traffic is routed only after the application is fully initialized.
Example endpoint:
```
/health/ready
```
Only containers returning **HTTP 200 OK** should receive production traffic.
---
### Pre-Warming
Initialize expensive resources during startup:
- Database connections
- Redis connections
- Configuration loading
- Dependency injection
- Frequently used libraries
Avoid performing heavy initialization during the first user request.
---
### Image Optimization
Reduce startup time by:
- Using lightweight base images
- Removing unnecessary packages
- Minimizing image layers
- Keeping image sizes as small as possible
Examples:
- Alpine Linux
- Distroless Images
---
### Capacity Buffer
Maintain spare capacity (30% safety margin) so existing containers can absorb traffic while new containers complete startup.
Benefits:
- Reduced request latency
- Smoother auto-scaling
- Improved user experience
---
# Ghaymah Block Storage for Stateful Workloads
Containers are **ephemeral** by design.
If a container is deleted or restarted, its local filesystem is also removed.
Persistent workloads such as:
- PostgreSQL
- MySQL
- MariaDB
- MongoDB
- Message Brokers
must store data externally.
---
## How It Works
1. Provision a **Ghaymah Block Storage** volume.
2. Attach the volume to the database container.
3. Store all database files on the mounted volume.
4. If the container fails, the storage remains intact.
5. A replacement container automatically reattaches the existing volume.
---
## Benefits
### Persistent Data
Application data survives:
- Container restarts
- Platform upgrades
- Node failures
- Container replacements
---
### Compute and Storage Separation
Containers remain disposable while storage persists independently.
This allows infrastructure updates without risking data loss.
---
### Faster Recovery
If the database container crashes:
1. A replacement container starts.
2. The existing block storage volume is attached.
3. The database resumes operation with its original data.
---
### High Performance
Ghaymah Block Storage provides dedicated storage performance for demanding workloads.
Benefits include:
- High IOPS
- Low latency
- Reliable throughput
- Consistent database performance
---
# Key Design Decisions
| Area | Decision |
|------|----------|
| Compute | Stateless application containers |
| Scaling | Horizontal auto-scaling |
| Load Distribution | Ghaymah Load Balancer |
| Caching | Redis Cluster |
| Database | Primary + Read Replica architecture |
| Storage | Ghaymah Block Storage |
| Availability | Multi-container deployment |
| Cold Start Mitigation | Readiness probes, pre-warming, optimized images |
| Capacity Planning | 39 containers (30 base + 30% buffer) |
---
# Summary
This architecture is designed to provide:
- High availability
- Horizontal scalability
- Fault tolerance
- Persistent storage
- Fast recovery from failures
- Efficient handling of **15,000 requests per second**
By combining stateless application containers, intelligent load balancing, Redis caching, database replication, and Ghaymah Block Storage, the platform can maintain stable performance during normal operations as well as sudden traffic spikes.

6
Q5/.dockerignore Normal file
عرض الملف

@@ -0,0 +1,6 @@
data/
*.pyc
__pycache__/
.git
.gitignore
README.md

40
Q5/Dockerfile Normal file
عرض الملف

@@ -0,0 +1,40 @@
# syntax=docker/dockerfile:1
FROM python:3.12-alpine
# Standard-library only — no requirements.txt needed. ca-certificates is
# required so ssl.create_default_context() can validate the target's
# certificate chain when checking SSL expiry.
RUN apk add --no-cache ca-certificates && update-ca-certificates
# Run as a non-root user
RUN addgroup -S monitor && adduser -S monitor -G monitor
WORKDIR /app
COPY monitor.py /app/monitor.py
COPY Entrypoint.sh /app/entrypoint.sh
COPY index.html /app/web/index.html
RUN chmod +x /app/entrypoint.sh \
&& mkdir -p /app/web/data \
&& chown -R monitor:monitor /app
USER monitor
# Defaults — override any of these at `docker run` / platform deploy time.
ENV TARGET_URL="https://mithal.space" \
SEARCH_PATH="/search?q=test" \
CHECK_INTERVAL=60 \
RETENTION_HOURS=24 \
REQUEST_TIMEOUT=10 \
DATA_FILE="/app/web/data/metrics.json" \
WEB_DIR="/app/web" \
PORT=8080
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
CMD python3 -c "import urllib.request,sys; sys.exit(0) if urllib.request.urlopen('http://127.0.0.1:${PORT}/', timeout=3).status==200 else sys.exit(1)"
ENTRYPOINT ["/app/entrypoint.sh"]

43
Q5/Entrypoint.sh Normal file
عرض الملف

@@ -0,0 +1,43 @@
#!/bin/sh
# entrypoint.sh — runs the Python monitoring loop and a static HTTP server
# side by side in a single container. POSIX sh so it works on Alpine's
# default shell.
set -eu
PORT="${PORT:-8080}"
WEB_DIR="${WEB_DIR:-/app/web}"
echo "[entrypoint] starting monitor loop"
python3 /app/monitor.py &
MONITOR_PID=$!
echo "[entrypoint] serving dashboard from ${WEB_DIR} on 0.0.0.0:${PORT}"
cd "${WEB_DIR}"
python3 -m http.server "${PORT}" --bind 0.0.0.0 &
SERVER_PID=$!
# Forward termination signals to both children and wait for them so the
# container shuts down cleanly (e.g. on `docker stop` / platform redeploys).
term_handler() {
echo "[entrypoint] shutting down..."
kill -TERM "$MONITOR_PID" "$SERVER_PID" 2>/dev/null || true
wait "$MONITOR_PID" "$SERVER_PID" 2>/dev/null || true
exit 0
}
trap term_handler TERM INT
# busybox ash (Alpine's /bin/sh) has no `wait -n`, so poll instead: if
# either child dies unexpectedly, bring the whole container down so the
# orchestrator (Docker/Kubernetes/Cloud Run/etc.) can restart it.
while true; do
if ! kill -0 "$MONITOR_PID" 2>/dev/null; then
echo "[entrypoint] monitor loop exited unexpectedly — stopping container"
term_handler
fi
if ! kill -0 "$SERVER_PID" 2>/dev/null; then
echo "[entrypoint] http server exited unexpectedly — stopping container"
term_handler
fi
sleep 2
done

80
Q5/README.md Normal file
عرض الملف

@@ -0,0 +1,80 @@
# mithal-space-monitor
A lightweight, stdlib-only uptime/latency/SSL monitor with a single-page
Chart.js dashboard, packaged into one container.
## What's inside
| File | Purpose |
|---|---|
| `monitor.py` | Standard-library-only Python script. Every `CHECK_INTERVAL` seconds it checks DNS resolution time, HTTP status + latency, search-endpoint latency, and SSL cert expiry, then writes a rolling `RETENTION_HOURS` window to a JSON file. |
| `index.html` | Single-page dashboard (HTML/CSS/JS + Chart.js via CDN). Polls the JSON file every 30s and renders 24h uptime %, a 60-minute latency line chart, SSL expiry, and a table of the last 10 checks. |
| `entrypoint.sh` | Starts `monitor.py` and `python -m http.server` side by side, forwards signals, and exits the container if either process dies (so the orchestrator restarts it). |
| `Dockerfile` | `python:3.12-alpine` base, non-root user, healthcheck, no external Python deps. |
## Configuration (environment variables)
| Variable | Default | Description |
|---|---|---|
| `TARGET_URL` | `https://mithal.space` | URL to monitor |
| `SEARCH_PATH` | `/search?q=test` | Path appended to the target's origin for the search-latency check |
| `CHECK_INTERVAL` | `60` | Seconds between checks |
| `RETENTION_HOURS` | `24` | Rolling window kept in the JSON log |
| `REQUEST_TIMEOUT` | `10` | Per-request timeout (seconds) |
| `PORT` | `8080` | Dashboard HTTP server port |
## Build & run locally
```bash
docker build -t mithal-space-monitor .
docker run -d \
--name mithal-monitor \
-p 8080:8080 \
-e TARGET_URL="https://mithal.space" \
-e SEARCH_PATH="/search?q=test" \
-v mithal_monitor_data:/app/data \
mithal-space-monitor
# open http://localhost:8080
```
The `-v mithal_monitor_data:/app/data` volume is optional but recommended so
your 24h history survives a container restart/redeploy.
## Push to a registry
```bash
docker tag mithal-space-monitor registry.example.com/yourorg/mithal-space-monitor:latest
docker push registry.example.com/yourorg/mithal-space-monitor:latest
```
## Deploy
This image is a single process group exposing one HTTP port, so it runs
as-is on most container platforms:
- **Cloud Run / Container Apps / Fly.io**: deploy the image, set `PORT`
to match the platform's expected port (Cloud Run injects `PORT`
automatically - the entrypoint already respects it), mount a persistent
volume if the platform supports one (otherwise history resets on redeploy,
which is fine - it just rebuilds over the next `RETENTION_HOURS`).
- **Kubernetes**: run as a `Deployment` with 1 replica, a `Service` of type
`ClusterIP`/`LoadBalancer`, and optionally a `PersistentVolumeClaim`
mounted at `/app/data`. The built-in `HEALTHCHECK` maps naturally to a
liveness probe on `GET /`.
- **Plain VM / docker-compose**: use the `docker run` command above behind
your existing reverse proxy / TLS terminator.
## Notes & extension points
- Everything in `monitor.py` uses only the Python standard library
(`urllib`, `socket`, `ssl`, `json`) - no `pip install` step, no
dependency surface in the image.
- Data is written atomically (`write → temp file → os.replace`) so the
dashboard never reads a half-written JSON file.
- To monitor multiple targets, run one container per target (each with its
own `TARGET_URL`/port), or extend `monitor.py` to loop over a list of
targets and extend `index.html` with a target selector.
- Add basic auth / IP allowlisting at your reverse proxy if the dashboard
shouldn't be public.

395
Q5/index.html Normal file
عرض الملف

@@ -0,0 +1,395 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Site Watch — Status</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link href="https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;500;600;700&family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
<script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/4.4.4/chart.umd.min.js"></script>
<style>
:root{
--bg:#0b1220;
--surface:#121b2e;
--surface-alt:#182541;
--border:#23324f;
--text:#e7ecf5;
--muted:#8a97b3;
--accent:#5eead4;
--ok:#34d399;
--warn:#fbbf24;
--down:#f87171;
--mono:'IBM Plex Mono', ui-monospace, monospace;
--sans:'Inter', system-ui, sans-serif;
}
*{box-sizing:border-box;}
html,body{margin:0;padding:0;background:var(--bg);color:var(--text);font-family:var(--sans);}
body{min-height:100vh;padding:32px 24px 64px;}
a{color:var(--accent);}
.wrap{max-width:1080px;margin:0 auto;}
/* ---- Header / pulse signature ---- */
header{
display:flex;
flex-direction:column;
gap:18px;
border:1px solid var(--border);
background:linear-gradient(180deg,var(--surface) 0%, var(--surface-alt) 100%);
border-radius:14px;
padding:22px 26px;
position:relative;
overflow:hidden;
}
.header-top{display:flex;justify-content:space-between;align-items:flex-start;flex-wrap:wrap;gap:14px;}
.brand{display:flex;flex-direction:column;gap:4px;}
.eyebrow{font-family:var(--mono);font-size:11px;letter-spacing:.14em;text-transform:uppercase;color:var(--muted);}
.target{font-family:var(--mono);font-size:22px;font-weight:600;color:var(--text);word-break:break-all;}
.status-pill{
display:inline-flex;align-items:center;gap:8px;
font-family:var(--mono);font-size:12px;font-weight:600;letter-spacing:.04em;
padding:7px 14px;border-radius:999px;border:1px solid var(--border);
background:rgba(255,255,255,0.02);white-space:nowrap;height:fit-content;
}
.dot{width:8px;height:8px;border-radius:50%;background:var(--muted);box-shadow:0 0 0 0 rgba(0,0,0,0);}
.dot.up{background:var(--ok);animation:pulse-dot 2s infinite;}
.dot.down{background:var(--down);animation:pulse-dot 1s infinite;}
@keyframes pulse-dot{
0%{box-shadow:0 0 0 0 rgba(52,211,153,.55);}
70%{box-shadow:0 0 0 8px rgba(52,211,153,0);}
100%{box-shadow:0 0 0 0 rgba(52,211,153,0);}
}
.pulse-line{width:100%;height:44px;opacity:.85;}
.pulse-line path{
fill:none;stroke:var(--accent);stroke-width:1.6;
stroke-dasharray:1200;stroke-dashoffset:1200;
animation:draw-pulse 3.2s linear infinite;
}
@keyframes draw-pulse{
0%{stroke-dashoffset:1200;}
100%{stroke-dashoffset:0;}
}
@media (prefers-reduced-motion: reduce){
.dot.up,.dot.down,.pulse-line path{animation:none;}
}
.meta-row{display:flex;gap:18px;flex-wrap:wrap;font-family:var(--mono);font-size:12px;color:var(--muted);}
.meta-row span b{color:var(--text);font-weight:600;}
/* ---- Stat cards ---- */
.stats{
display:grid;grid-template-columns:repeat(4,1fr);gap:14px;margin-top:22px;
}
@media (max-width:820px){.stats{grid-template-columns:repeat(2,1fr);}}
.stat-card{
background:var(--surface);border:1px solid var(--border);border-radius:12px;
padding:16px 18px;display:flex;flex-direction:column;gap:6px;
}
.stat-label{font-family:var(--mono);font-size:11px;text-transform:uppercase;letter-spacing:.1em;color:var(--muted);}
.stat-value{font-family:var(--mono);font-size:28px;font-weight:700;line-height:1.1;}
.stat-sub{font-size:12px;color:var(--muted);}
.stat-value.ok{color:var(--ok);}
.stat-value.warn{color:var(--warn);}
.stat-value.down{color:var(--down);}
/* ---- Chart panel ---- */
.panel{
margin-top:22px;background:var(--surface);border:1px solid var(--border);
border-radius:12px;padding:20px 22px;
}
.panel-title{display:flex;justify-content:space-between;align-items:baseline;margin-bottom:14px;flex-wrap:wrap;gap:8px;}
.panel-title h2{font-size:14px;margin:0;font-family:var(--mono);text-transform:uppercase;letter-spacing:.08em;color:var(--text);}
.panel-title .hint{font-size:12px;color:var(--muted);}
.chart-holder{height:280px;position:relative;}
/* ---- Table ---- */
table{width:100%;border-collapse:collapse;font-family:var(--mono);font-size:12.5px;}
thead th{
text-align:left;color:var(--muted);font-weight:600;text-transform:uppercase;
font-size:10.5px;letter-spacing:.08em;padding:8px 10px;border-bottom:1px solid var(--border);
}
tbody td{padding:9px 10px;border-bottom:1px solid rgba(255,255,255,0.04);color:var(--text);}
tbody tr:hover{background:rgba(255,255,255,0.02);}
.badge{
display:inline-block;padding:2px 8px;border-radius:6px;font-size:11px;font-weight:600;
}
.badge.ok{background:rgba(52,211,153,.12);color:var(--ok);}
.badge.down{background:rgba(248,113,113,.12);color:var(--down);}
footer{margin-top:26px;text-align:center;font-family:var(--mono);font-size:11px;color:var(--muted);}
.empty{color:var(--muted);font-size:13px;padding:20px 0;text-align:center;}
</style>
</head>
<body>
<div class="wrap">
<header>
<div class="header-top">
<div class="brand">
<span class="eyebrow">Uptime &amp; Performance</span>
<span class="target" id="targetUrl">—</span>
</div>
<div class="status-pill">
<span class="dot" id="statusDot"></span>
<span id="statusText">CHECKING…</span>
</div>
</div>
<svg class="pulse-line" viewBox="0 0 600 44" preserveAspectRatio="none">
<path d="M0,22 L120,22 L140,6 L160,38 L180,22 L260,22 L280,10 L300,34 L320,22 L600,22"/>
</svg>
<div class="meta-row">
<span>Checks every <b id="metaInterval">—</b></span>
<span>Window <b id="metaRetention">—</b></span>
<span>Last check <b id="metaUpdated">—</b></span>
</div>
</header>
<div class="stats">
<div class="stat-card">
<span class="stat-label">Uptime · 24h</span>
<span class="stat-value" id="statUptime">—</span>
<span class="stat-sub" id="statUptimeSub">— checks recorded</span>
</div>
<div class="stat-card">
<span class="stat-label">Latency · latest</span>
<span class="stat-value" id="statLatency">—</span>
<span class="stat-sub" id="statLatencySub">HTTP response time</span>
</div>
<div class="stat-card">
<span class="stat-label">DNS resolve</span>
<span class="stat-value" id="statDns">—</span>
<span class="stat-sub">Latest lookup time</span>
</div>
<div class="stat-card">
<span class="stat-label">SSL expires in</span>
<span class="stat-value" id="statSsl">—</span>
<span class="stat-sub" id="statSslSub">Certificate validity</span>
</div>
</div>
<div class="panel">
<div class="panel-title">
<h2>Latency — last 60 minutes</h2>
<span class="hint" id="chartHint">site vs. search endpoint, ms</span>
</div>
<div class="chart-holder"><canvas id="latencyChart"></canvas></div>
</div>
<div class="panel">
<div class="panel-title">
<h2>Recent checks</h2>
<span class="hint">last 10</span>
</div>
<table>
<thead>
<tr>
<th>Time (UTC)</th>
<th>Status</th>
<th>Latency</th>
<th>DNS</th>
<th>Search</th>
<th>SSL days</th>
</tr>
</thead>
<tbody id="checksBody">
<tr><td colspan="6" class="empty">Waiting for first data…</td></tr>
</tbody>
</table>
</div>
<footer>Reads <code>data/metrics.json</code> · refreshes every 30s</footer>
</div>
<script>
const DATA_URL = 'data/metrics.json';
const REFRESH_MS = 30000;
let chart;
function fmtMs(v){ return (v === null || v === undefined) ? '—' : Math.round(v) + ' ms'; }
function fmtDays(v){ return (v === null || v === undefined) ? '—' : v + 'd'; }
function fmtTime(iso){
try{
const d = new Date(iso);
return d.toISOString().substr(11,8);
}catch(e){ return iso; }
}
function classifySsl(days){
if(days === null || days === undefined) return '';
if(days < 7) return 'down';
if(days < 21) return 'warn';
return 'ok';
}
function renderStats(payload){
const checks = payload.checks || [];
document.getElementById('targetUrl').textContent = payload.target || '—';
document.getElementById('metaInterval').textContent = (payload.check_interval_seconds || '—') + 's';
document.getElementById('metaRetention').textContent = (payload.retention_hours || '—') + 'h';
document.getElementById('metaUpdated').textContent = payload.updated_at ? fmtTime(payload.updated_at) + ' UTC' : '—';
if(checks.length === 0){
document.getElementById('statusText').textContent = 'NO DATA';
return;
}
const latest = checks[checks.length - 1];
const successCount = checks.filter(c => c.success).length;
const uptimePct = (successCount / checks.length * 100);
// Status pill
const dot = document.getElementById('statusDot');
const statusText = document.getElementById('statusText');
if(latest.success){
dot.className = 'dot up';
statusText.textContent = `UP · HTTP ${latest.http_status}`;
} else {
dot.className = 'dot down';
statusText.textContent = latest.http_status ? `DEGRADED · HTTP ${latest.http_status}` : 'DOWN';
}
// Uptime stat
const uptimeEl = document.getElementById('statUptime');
uptimeEl.textContent = uptimePct.toFixed(2) + '%';
uptimeEl.className = 'stat-value ' + (uptimePct >= 99.5 ? 'ok' : uptimePct >= 97 ? 'warn' : 'down');
document.getElementById('statUptimeSub').textContent = `${successCount}/${checks.length} checks succeeded`;
// Latency stat
const latEl = document.getElementById('statLatency');
latEl.textContent = fmtMs(latest.latency_ms);
latEl.className = 'stat-value ' + (latest.latency_ms == null ? 'down' : latest.latency_ms < 500 ? 'ok' : latest.latency_ms < 1500 ? 'warn' : 'down');
// DNS stat
document.getElementById('statDns').textContent = fmtMs(latest.dns_ms);
// SSL stat
const sslEl = document.getElementById('statSsl');
sslEl.textContent = fmtDays(latest.ssl_days_remaining);
sslEl.className = 'stat-value ' + classifySsl(latest.ssl_days_remaining);
document.getElementById('statSslSub').textContent = latest.ssl_days_remaining != null
? 'Certificate validity' : 'Not monitored over HTTPS';
}
function renderChart(payload){
const checks = payload.checks || [];
const cutoff = Date.now() - 60 * 60 * 1000;
const recent = checks.filter(c => new Date(c.timestamp).getTime() >= cutoff);
const source = recent.length > 0 ? recent : checks.slice(-60);
const labels = source.map(c => fmtTime(c.timestamp));
const siteData = source.map(c => c.latency_ms);
const searchData = source.map(c => c.search_latency_ms);
const hasSearch = searchData.some(v => v !== null && v !== undefined);
document.getElementById('chartHint').textContent = hasSearch
? 'site vs. search endpoint, ms' : 'site response time, ms';
const datasets = [{
label: 'Site',
data: siteData,
borderColor: '#5eead4',
backgroundColor: 'rgba(94,234,212,0.12)',
borderWidth: 2,
pointRadius: 0,
tension: 0.25,
fill: true,
}];
if(hasSearch){
datasets.push({
label: 'Search endpoint',
data: searchData,
borderColor: '#fbbf24',
backgroundColor: 'transparent',
borderWidth: 2,
borderDash: [4,3],
pointRadius: 0,
tension: 0.25,
});
}
const cfg = {
type: 'line',
data: { labels, datasets },
options: {
responsive: true,
maintainAspectRatio: false,
interaction: { mode: 'index', intersect: false },
scales: {
x: {
ticks: { color: '#8a97b3', maxTicksLimit: 8, font: { family: 'IBM Plex Mono', size: 10 } },
grid: { color: 'rgba(255,255,255,0.04)' },
},
y: {
beginAtZero: true,
ticks: { color: '#8a97b3', font: { family: 'IBM Plex Mono', size: 10 } },
grid: { color: 'rgba(255,255,255,0.06)' },
title: { display: true, text: 'ms', color: '#8a97b3', font: { size: 11 } },
},
},
plugins: {
legend: {
display: hasSearch,
labels: { color: '#e7ecf5', font: { family: 'IBM Plex Mono', size: 11 }, boxWidth: 12 },
},
tooltip: {
backgroundColor: '#182541',
borderColor: '#23324f',
borderWidth: 1,
titleFont: { family: 'IBM Plex Mono' },
bodyFont: { family: 'IBM Plex Mono' },
},
},
},
};
if(chart){
chart.data = cfg.data;
chart.update('none');
} else {
chart = new Chart(document.getElementById('latencyChart'), cfg);
}
}
function renderTable(payload){
const checks = (payload.checks || []).slice(-10).reverse();
const body = document.getElementById('checksBody');
if(checks.length === 0){
body.innerHTML = '<tr><td colspan="6" class="empty">Waiting for first data…</td></tr>';
return;
}
body.innerHTML = checks.map(c => `
<tr>
<td>${fmtTime(c.timestamp)}</td>
<td><span class="badge ${c.success ? 'ok' : 'down'}">${c.http_status ?? 'ERR'}</span></td>
<td>${fmtMs(c.latency_ms)}</td>
<td>${fmtMs(c.dns_ms)}</td>
<td>${fmtMs(c.search_latency_ms)}</td>
<td>${fmtDays(c.ssl_days_remaining)}</td>
</tr>
`).join('');
}
async function refresh(){
try{
const res = await fetch(DATA_URL + '?t=' + Date.now(), { cache: 'no-store' });
if(!res.ok) throw new Error('HTTP ' + res.status);
const payload = await res.json();
renderStats(payload);
renderChart(payload);
renderTable(payload);
}catch(err){
document.getElementById('statusText').textContent = 'DASHBOARD ERROR';
document.getElementById('statusDot').className = 'dot down';
console.error('Failed to load metrics:', err);
}
}
refresh();
setInterval(refresh, REFRESH_MS);
</script>
</body>
</html>

208
Q5/monitor.py Normal file
عرض الملف

@@ -0,0 +1,208 @@
#!/usr/bin/env python3
"""
monitor.py — Lightweight uptime / latency / SSL monitor.
Standard-library only (urllib, ssl, socket, json, time). Runs an infinite
loop, checks a target URL (and optionally a search endpoint) every
CHECK_INTERVAL seconds, and persists a rolling RETENTION_HOURS window of
results to a JSON file that the static dashboard reads.
Configuration is via environment variables so the same image can monitor
any site without a rebuild:
TARGET_URL Full URL to monitor (default: https://mithal.space)
SEARCH_PATH Path appended to origin for a (default: /search?q=test)
secondary "search" check. Set to "" to disable.
CHECK_INTERVAL Seconds between checks (default: 60)
RETENTION_HOURS How much history to keep (default: 24)
REQUEST_TIMEOUT Per-request timeout, seconds (default: 10)
DATA_FILE Where to write the JSON log (default: /app/web/data/metrics.json)
"""
import json
import os
import socket
import ssl
import sys
import time
import urllib.error
import urllib.request
from datetime import datetime, timezone
from urllib.parse import urlparse
# --------------------------------------------------------------------------
# Configuration
# --------------------------------------------------------------------------
TARGET_URL = os.environ.get("TARGET_URL", "https://mithal.space")
SEARCH_PATH = os.environ.get("SEARCH_PATH", "/search?q=test")
CHECK_INTERVAL = int(os.environ.get("CHECK_INTERVAL", "60"))
RETENTION_HOURS = float(os.environ.get("RETENTION_HOURS", "24"))
REQUEST_TIMEOUT = float(os.environ.get("REQUEST_TIMEOUT", "10"))
DATA_FILE = os.environ.get("DATA_FILE", "/app/web/data/metrics.json")
USER_AGENT = "uptime-monitor/1.0 (+standard-library)"
_parsed = urlparse(TARGET_URL)
HOSTNAME = _parsed.hostname
PORT = _parsed.port or (443 if _parsed.scheme == "https" else 80)
SEARCH_URL = f"{_parsed.scheme}://{_parsed.netloc}{SEARCH_PATH}" if SEARCH_PATH else None
def now_iso() -> str:
return datetime.now(timezone.utc).isoformat(timespec="seconds")
def measure_dns(hostname: str):
"""Return DNS resolution time in milliseconds, or None on failure."""
start = time.perf_counter()
try:
socket.getaddrinfo(hostname, None)
except socket.gaierror as exc:
return None, str(exc)
elapsed_ms = (time.perf_counter() - start) * 1000
return round(elapsed_ms, 2), None
def timed_get(url: str, timeout: float):
"""
Perform an HTTP GET and return (status_code, latency_ms, error_str).
latency_ms measures time-to-first-byte-of-full-response (connect + TLS
+ request + response), matching what a real visitor experiences.
"""
req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT})
start = time.perf_counter()
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
resp.read(1) # confirm the body actually starts streaming
status = resp.status
latency_ms = (time.perf_counter() - start) * 1000
return status, round(latency_ms, 2), None
except urllib.error.HTTPError as exc:
# Still a "successful" connection from a monitoring standpoint —
# the server responded, just with an error status.
latency_ms = (time.perf_counter() - start) * 1000
return exc.code, round(latency_ms, 2), None
except (urllib.error.URLError, socket.timeout, TimeoutError) as exc:
latency_ms = (time.perf_counter() - start) * 1000
return None, round(latency_ms, 2), str(getattr(exc, "reason", exc))
def measure_ssl_expiry(hostname: str, port: int, timeout: float):
"""Return (days_remaining, error_str) for the TLS certificate."""
if not hostname:
return None, "no hostname"
try:
ctx = ssl.create_default_context()
with socket.create_connection((hostname, port), timeout=timeout) as sock:
with ctx.wrap_socket(sock, server_hostname=hostname) as ssock:
cert = ssock.getpeercert()
not_after = cert.get("notAfter")
if not not_after:
return None, "no notAfter field in certificate"
expiry_dt = datetime.strptime(not_after, "%b %d %H:%M:%S %Y %Z").replace(
tzinfo=timezone.utc
)
days_remaining = (expiry_dt - datetime.now(timezone.utc)).total_seconds() / 86400
return round(days_remaining, 1), None
except Exception as exc: # noqa: BLE001 — monitoring must never crash the loop
return None, str(exc)
def run_check() -> dict:
dns_ms, dns_err = measure_dns(HOSTNAME)
status, latency_ms, http_err = timed_get(TARGET_URL, REQUEST_TIMEOUT)
search_status, search_latency_ms, search_err = None, None, None
if SEARCH_URL:
search_status, search_latency_ms, search_err = timed_get(SEARCH_URL, REQUEST_TIMEOUT)
ssl_days, ssl_err = (None, None)
if _parsed.scheme == "https":
ssl_days, ssl_err = measure_ssl_expiry(HOSTNAME, PORT, REQUEST_TIMEOUT)
success = status is not None and 200 <= status < 400
return {
"timestamp": now_iso(),
"target": TARGET_URL,
"http_status": status,
"success": success,
"latency_ms": latency_ms,
"dns_ms": dns_ms,
"search_status": search_status,
"search_latency_ms": search_latency_ms,
"ssl_days_remaining": ssl_days,
"error": http_err or dns_err or ssl_err or search_err,
}
def load_history(path: str) -> list:
try:
with open(path, "r", encoding="utf-8") as f:
data = json.load(f)
return data.get("checks", []) if isinstance(data, dict) else data
except (FileNotFoundError, json.JSONDecodeError):
return []
def prune(history: list, retention_hours: float) -> list:
cutoff = time.time() - retention_hours * 3600
pruned = []
for entry in history:
try:
ts = datetime.fromisoformat(entry["timestamp"]).timestamp()
except (KeyError, ValueError):
continue
if ts >= cutoff:
pruned.append(entry)
return pruned
def save_history(path: str, history: list) -> None:
os.makedirs(os.path.dirname(path), exist_ok=True)
payload = {
"target": TARGET_URL,
"search_url": SEARCH_URL,
"updated_at": now_iso(),
"check_interval_seconds": CHECK_INTERVAL,
"retention_hours": RETENTION_HOURS,
"checks": history,
}
tmp_path = f"{path}.tmp"
with open(tmp_path, "w", encoding="utf-8") as f:
json.dump(payload, f, indent=2)
os.replace(tmp_path, path) # atomic write so the dashboard never reads a half-written file
def main() -> None:
print(f"[monitor] target={TARGET_URL} interval={CHECK_INTERVAL}s "
f"retention={RETENTION_HOURS}h data_file={DATA_FILE}", flush=True)
history = load_history(DATA_FILE)
while True:
cycle_start = time.time()
result = run_check()
history.append(result)
history = prune(history, RETENTION_HOURS)
save_history(DATA_FILE, history)
status_str = result["http_status"] if result["http_status"] is not None else "ERR"
print(
f"[monitor] {result['timestamp']} status={status_str} "
f"latency={result['latency_ms']}ms dns={result['dns_ms']}ms "
f"ssl_days={result['ssl_days_remaining']} "
f"search_latency={result['search_latency_ms']}ms "
f"{'error=' + result['error'] if result['error'] else 'ok'}",
flush=True,
)
elapsed = time.time() - cycle_start
time.sleep(max(0, CHECK_INTERVAL - elapsed))
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
sys.exit(0)