Ghaymah Cloud Internship — SRE & SecOps Track

Deployment, Monitoring, CI/CD, Scalability, and Observability on ghaymah.systems

Author: Mohamed Moustafa Track: SRE & SecOps Platform: ghaymah.systems

---

Project Overview

This repository contains all six tasks from the Ghaymah Cloud internship exam, covering the full lifecycle of cloud-native applications — from containerized deployment to post-incident analysis, CI/CD automation, scalability design, and real-time monitoring.

Task	Topic	Score	Status
Q1	Deploy & Monitor on Ghaymah	20/20	Done
Q2	Postmortem — OOMKilled Incident	20/20	Done
Q3	CI/CD Pipeline with GitHub Actions	20/20	Done
Q4	Scalability & Load Balancing	20/20	Done
Q5	Mithal.space Monitoring Dashboard	20/20	Done
Q6	Mortakaz Integration Proposals	15/15	Done

---

Repository Structure

ghaymah-exam-mohamed-sre/
├── README.md                          ← You are here
│
├── q1-deploy-monitor/
│   ├── Dockerfile                     ← Multi-stage Node.js build
│   ├── server.js                      ← Express API with /health endpoint
│   ├── package.json
│   ├── health-check.sh                ← Bash monitor (30s interval)
│   ├── ghaymah.json                   ← Ghaymah CLI config
│   ├── architecture.svg               ← Architecture diagram
│   └── public/
│       ├── dashboard.html             ← Monitoring dashboard
│       ├── dashboard.css
│       └── dashboard.js
│
├── q2-postmortem/
│   ├── postmortem-report.md           ← Full incident report
│   └── timeline.svg                   ← Visual incident timeline
│
├── q3-cicd/
│   ├── workflow.yml                   ← GitHub Actions workflow
│   ├── pipeline.svg                   ← CI/CD pipeline diagram
│   └── Readme.md                      ← Staging vs Production + CLI guide
│
├── q4-scalability/
│   ├── calculations.md                ← Container math & strategy
│   ├── architecture.svg               ← 15K req/s architecture
│   └── architecture.png               ← PNG version
│
├── q5-mithal-monitor/
│   ├── monitor.py                     ← Python monitoring script
│   ├── dashboard.html                 ← Real-time dashboard
│   ├── style.css                      ← Dashboard styles
│   ├── script.js                      ← Dashboard logic
│   ├── Dockerfile                     ← Docker image (Python + Nginx)
│   ├── nginx.conf                     ← Nginx config
│   ├── entrypoint.sh                  ← Container entrypoint
│   ├── requirements.txt               ← Python dependencies
│   ├── README.md                      ← Detailed documentation
│   └── screenshots/
│       ├── dashboard-preview.svg      ← Dashboard preview
│       └── monitor-preview.svg        ← Terminal preview
│
├── common-mortakaz/
│   ├── integration-1.md               ← Circle Panel proposal
│   └── integration-2.md               ← Bilya AI Assistant proposal
│
└── common-qabilah/
    └── qabilah-profile.txt            ← Qabilah profile link

---

Q1 — Deploy & Monitor on Ghaymah

Goal: Containerize a Node.js API, deploy to ghaymah.systems, and monitor it.

Architecture

What Was Built

• Node.js API (Express) with /health endpoint returning status and request count
• Multi-stage Dockerfile — builder stage for npm ci, production stage on Alpine
• Health check script (health-check.sh) — polls /health every 30 seconds, logs status
• Monitoring dashboard — live status, response time, total requests

Key Files

File	Purpose
Dockerfile	Multi-stage build: node:22 → node:22-alpine
server.js	Express server with /health endpoint
health-check.sh	Bash monitoring loop (30s interval)
ghaymah.json	Ghaymah CLI configuration

Deploy Commands

# Install Ghaymah CLI
curl -sSL https://cli.ghaymah.systems/install.sh | bash

# Authenticate
$HOME/ghaymah/bin/gy auth login --email "EMAIL" --password "PW"

# Deploy
$HOME/ghaymah/bin/gy resource app launch

# Run monitor
chmod +x health-check.sh
./health-check.sh

---

Q2 — Postmortem: OOMKilled Incident

Goal: Document a 45-minute outage caused by repeated OOMKilled events.

Incident Timeline

Summary

Field	Detail
Duration	45 minutes
Severity	High
Root Cause	Memory leak exceeded container limit
Impact	100% request failure, 12+ restarts

Key Sections

• Root Cause: Application consumed more memory than the container's configured limit
• Timeline: Deploy → Memory spike → OOMKilled → Crash loop → Fix
• Auto-Scaling Policy: Scale-out at 80% memory, scale-in at 40%, min 2 / max 10 instances
• Early Detection: Prometheus + Grafana for memory metrics, alerts at 80% threshold

---

Q3 — CI/CD Pipeline

Goal: Automate build, push, and deploy with manual approval gate.

Pipeline

Workflow Steps

1. Trigger — git push to main branch
2. Build — Docker image from Dockerfile
3. Test — npm test verification
4. Push — Image to Ghaymah Container Registry
5. Approval — Manual gate before production
6. Deploy — gy resource app launch

Staging vs Production

Feature	Staging	Production
Users	Developers / QA	End Users
Data	Test Data	Production Data
Approval	Optional	Required
Stability	Medium	High

Ghaymah CLI Integration

# Install
curl -sSL https://cli.ghaymah.systems/install.sh | bash

# Login
gy auth login --email "EMAIL" --password "PW"

# Deploy
gy resource app launch

# Monitor
gy app logs

---

Q4 — Scalability & Load Balancing

Goal: Design architecture for 15,000 req/s on Ghaymah Cloud.

Architecture

Calculations

Traffic:       15,000 req/s
Per Container:  500 req/s
Base Need:      15,000 / 500 = 30 containers
Safety Margin:  30% → 30 × 1.3 = 39 containers

Cold Start Strategy

• Keep 2–3 warm containers ready
• Use lightweight Docker images (Alpine-based)
• Configure health checks before routing traffic
• Trigger auto-scaling at 70% CPU or 80% memory

Block Storage for Stateful Workloads

Ghaymah Block Storage persists data across container restarts:

• Databases: PostgreSQL, MySQL, MongoDB
• File Storage: User uploads, recordings
• Logs: Persistent application logs
• Backups: Automated backup storage

---

Q5 — Mithal.space Monitoring Dashboard

Goal: Build a production-quality monitoring solution for mithal.space.

Dashboard Preview

Monitor Output

Features

Feature	Implementation
HTTP Latency	requests library, measures GET response time
Uptime	Status code check (200–399 = UP)
SSL Certificate	ssl module, checks expiry date
DNS Lookup	socket.getaddrinfo() timing
Search Response	Real /search?q= request measurement
Data Storage	metrics.json (1,440 records ≈ 24h)
Dashboard	HTML/CSS/JS with Chart.js
Dark Mode	Toggle with localStorage persistence
CSV Export	One-click metrics export

Quick Start

cd q5-mithal-monitor
pip install -r requirements.txt
python monitor.py --interval 60

# Dashboard
python3 -m http.server 8080
# Open http://localhost:8080/dashboard.html

Docker

docker build -t mithal-monitor .
docker run -d -p 8080:80 mithal-monitor

---

Q6 — Mortakaz Integration Proposals

Goal: Propose integrations between mortakaz.com products and Ghaymah Cloud.

Product 1: Circle Panel

User research platform for Arabic-speaking product teams.

• Ghaymah Integration: Containerized microservices (Frontend, Backend, AI Processing)
• Block Storage: Interview recordings and research data
• mithal.space: Searchable documentation and resources

Product 2: Bilya AI Assistant

AI-powered virtual assistant for automotive service centers.

• Ghaymah Integration: Microservices (Frontend, API, AI, Booking)
• Block Storage: Customer conversations and booking records
• mithal.space: FAQ and documentation discoverability

Recommendation

Bilya AI Assistant is the strongest candidate — its architecture naturally benefits from containers, auto-scaling, persistent storage, and CI/CD pipelines.

---

Qabilah Profile

qabilah.com/profile/mohamed-moustafa20

---

Technology Stack

Layer	Technology
Runtime	Node.js 22, Python 3.12
Framework	Express.js
Container	Docker (multi-stage)
Web Server	Nginx
Monitoring	Python + Bash
Dashboard	HTML/CSS/JS + Chart.js
CI/CD	GitHub Actions
Cloud	ghaymah.systems
Registry	Ghaymah Container Registry

---

License

This project was created for the Ghaymah Cloud Internship Program.