Ghaymah Test — Step-by-Step Checklist

Work through this top to bottom. Each step is tagged with who does it: [YOU] = manual work · [OPUS] = prompt Opus 5 · [SOL] = prompt Sol/Codex Steps inside the same phase that have different tags can run at the same time.

PHASE 0 — Setup & Research (Day 1 morning, ~2–3h)

0.1 [YOU] Create a Git repo ghaymah-test with this folder structure:

q1-deploy-monitor/{app,monitor,dashboard}
q2-postmortem/
q3-cicd/
q4-scalability/
q5-mithal-dashboard/{collector,data,dashboard}
README.md

0.2 [YOU] Create/log in to your ghaymah.systems account.
0.3 [YOU] Explore and take notes + screenshots on:
- Container platform (how to create a service, set port, get public URL)
- Container Registry (registry URL, how to log in/push)
- ghaymah CLI (install command, auth command, deploy command)
- Monitoring tools (metrics, graphs, alert rules)
- Block Storage (how volumes attach, limits)
These notes go into every agent prompt — Q2/Q3/Q4 must cite real Ghaymah features.
0.4 [YOU] Open mithal.space in a browser, run a search, and copy the search URL pattern (e.g. /search?q=test). Save it for step 4.1.
0.5 [YOU] Decide stack: Python + FastAPI (assumed below).

PHASE 1 — Dispatch both agents (Day 1 afternoon)

Send these two prompts at the same time — the tracks don't share files.

1.1 [OPUS] Prompt Opus with: full Q1 text + your platform notes + repo layout. Ask for:
- q1-deploy-monitor/app/main.py — FastAPI with GET /, GET /health (returns {"status":"ok","uptime_s":...,"timestamp":...}), GET /metrics (in-memory request counter via middleware)
- app/requirements.txt (fastapi, uvicorn)
- app/Dockerfile — python:3.12-slim, non-root user, layer-cached COPY order, EXPOSE 8080, HEALTHCHECK hitting /health, plus .dockerignore
- monitor/monitor.py — loop every 30s: GET $APP_URL/health (5s timeout), append {"ts","status","code","latency_ms","requests"} to monitor/data/checks.json; "down" on exception; alert print after 3 consecutive fails
- dashboard/index.html — single file, no build step: green/red status badge, latency line chart (Chart.js CDN), request-count tile, fetches checks.json, auto-refresh 30s
1.2 [SOL] Prompt Sol with: full Q2 text + your monitoring-tool notes. Ask for q2-postmortem/POSTMORTEM.md containing:
- Summary (45-min outage, OOMKilled exit 137, impact, severity)
- Minute-by-minute timeline table (memory creep → OOMKill → crash-loop → alert → limit raise + rollback → recovery)
- Root cause via 5-Whys (memory leak in release X + undersized limit + no memory alerting/CI check)
- Recommendations table (owner + priority)
- Auto-scaling policy: scale out at mem>70% or CPU>65% for 2 min, min 2 replicas, scale-in cooldown 5–10 min — and an explicit note that HPA alone doesn't fix a leak
- Early-detection section citing real Ghaymah monitoring features; every platform claim marked

PHASE 2 — Q1 goes live (Day 2 morning)

2.1 [YOU] Review Opus's Q1 output. Build & test locally:
```
docker build -t ghaymah-api ./q1-deploy-monitor/app
docker run -p 8080:8080 ghaymah-api
```
- http://localhost:8080/health returns 200 JSON
- /metrics counter increments
2.2 [YOU] Push image to ghaymah Container Registry (docker login → docker tag → docker push).
2.3 [YOU] Create the container service on ghaymah.systems (port 8080), note the public URL, screenshot the running service + /health response.
2.4 [YOU] Start monitor.py with APP_URL=<public URL> and leave it running so real data accumulates before submission.
2.5 [YOU] Open the Q1 dashboard against the growing checks.json — verify badge, chart, counter all render.
2.6 [OPUS] Send any deployment fixes back to Opus, then immediately give it Q5 (step 4.1) — don't wait.

PHASE 3 — CI/CD (Day 2 morning, parallel with Phase 2)

3.1 [SOL] Prompt Sol with: full Q3 text + your CLI/registry notes. Ask for:
- .github/workflows/deploy.yml — trigger on push to main + workflow_dispatch; jobs:
  - build-push: checkout → docker/login-action (secrets GHAYMAH_REGISTRY_USER/TOKEN) → docker/build-push-action tagging :${{ github.sha }} and :latest
  - deploy-staging (needs build-push): deploy via ghaymah CLI
  - deploy-production (needs staging): bound to GitHub Environment production (this is the manual-approval gate)
- q3-cicd/CICD.md — staging vs production section (data, scale, secrets, access, alerting, cadence + promotion flow diagram) and ghaymah CLI integration section (install, auth login with GHAYMAH_API_TOKEN secret, deploy command); placeholders marked 
3.2 [YOU] Push repo to GitHub. In repo settings:
- Add secrets: GHAYMAH_REGISTRY_USER, GHAYMAH_REGISTRY_TOKEN, GHAYMAH_API_TOKEN
- Settings → Environments → create production → add yourself as required reviewer
3.3 [YOU] Trigger the workflow, let it pause at the production gate, screenshot the approval prompt, then approve.

PHASE 4 — Q5 build & deploy (Day 2 afternoon)

4.1 [OPUS] Prompt Opus with: full Q5 text + the mithal.space search URL from 0.4. Ask for:
- collector/collect.py — every 60s (plus --once flag) measure:
  - Latency: timed GET https://mithal.space
  - Uptime: up if status 200–399
  - SSL: ssl/socket cert notAfter → days remaining
  - DNS: timed socket.getaddrinfo
  - Search: timed request to the search URL (configurable)
  - Append {"ts","up","code","latency_ms","dns_ms","ssl_days_left","search_ms"} to data/metrics.json, prune >48h
- dashboard/index.html — uptime % tile (24h), latency line chart (last hour, Chart.js), SSL card color-coded (green>30d / yellow 8–30 / red≤7), last-10-checks table, auto-refresh 60s; same visual style as Q1
- Dockerfile — one container serving the dashboard statically and running the collector in the background (shell entrypoint launching both), collector writes into the served data/ dir
4.2 [YOU] Test locally (--once first, then the loop; open the dashboard). Fix issues via Opus.
4.3 [YOU] Push image + deploy to ghaymah exactly like 2.2–2.3. Screenshot the live dashboard URL.
4.4 [YOU] Leave it running so the 24h uptime % and hourly chart fill with real data.

PHASE 5 — Q4 scalability doc (Day 2 afternoon, parallel with Phase 4)

5.1 [SOL] Prompt Sol with: full Q4 text + your Block Storage notes. Ask for q4-scalability/SCALABILITY.md:
- Mermaid architecture diagram: DNS → CDN → L7 load balancer → stateless API fleet (auto-scaled) → Redis cache → DB primary+replicas on Block Storage; monitoring on the side; annotate the 15k req/s path
- Container math shown explicitly: 15,000 ÷ 500 = 30 → ×1.30 = 39 containers, plus notes on rounding up and N+1 for rolling deploys
- Cold-start strategy: warm pool / min replicas, slim pre-pulled images, readiness-probe gating, predictive scaling before peaks, step scale-out policies
- Block Storage section: ephemeral containers vs persistent volumes, one-writer semantics, snapshots/backups, IOPS; API tier stays diskless —  markers on platform specifics

PHASE 6 — Verification & polish (Day 3 morning)

6.1 [YOU] Search the whole repo for  and resolve every one against real Ghaymah docs (fix CLI commands, registry URL, monitoring feature names, Block Storage details).
6.2 [YOU] Insert real values everywhere: live app URL (Q1), registry image name (Q3), live dashboard URL (Q5).
6.3 [YOU] Add all screenshots to the repo: Q1 service + /health, Q3 approval gate, Q5 live dashboard.
6.4 [SOL] Send Sol any corrections needed in Q2/Q3/Q4 docs from your verification pass.
6.5 [OPUS] Ask Opus for final dashboard polish now that real data exists (empty-state handling, chart scaling).

PHASE 7 — Final review & submit (Day 3 afternoon)

7.1 [YOU] Rubric pass — check every numbered sub-requirement one by one:
- Q1: Dockerfile · deployed on ghaymah · /health · 30s monitor script · dashboard (status, latency, request count)
- Q2: postmortem (summary, timeline, root cause, recommendations) · auto-scaling policy · early-detection with Ghaymah tools
- Q3: build+push workflow · manual approval before production · staging vs production explained · CLI integration documented
- Q4: architecture diagram · container count (39) with math · cold-start strategy · Block Storage explanation
- Q5: collector (latency, uptime, SSL, DNS, search) every minute · CSV/JSON storage · dashboard (uptime %, latency chart, SSL, last 10 checks) · deployed on ghaymah
7.2 [YOU] Write README.md as an index: one section per question, links to every file, live URLs, screenshots.
7.3 [YOU] Confirm both dashboards show real accumulated data (not empty charts).
7.4 [YOU] Final commit + push. Submit.

9.3 KiB خام اللوم التاريخ إلغاء الهروب الهروب