# Copy-Paste Agent Prompts + Ghaymah Info to Collect

> **PART C (added after research): the actual notes to paste — see bottom of this file. Docs turned out sparse, so Part C replaces Part A.**

## PART A — What to grab from ghaymah.systems FIRST (~20 min)

Log in and write down these exact values. Paste them into the `[GHAYMAH NOTES]` block of each prompt below.

1. **Registry URL** — the container registry hostname (e.g. `registry.ghaymah.systems` or similar) and how login works (username + token? access key?).
2. **How to deploy a container** — dashboard steps or CLI command to create a service from an image, set the port, and get the public URL. Note what the public URLs look like.
3. **CLI** — exact install command, exact auth/login command, exact deploy/update command (copy them verbatim from the docs page).
4. **Monitoring** — what metrics/graphs exist per container (CPU? memory? restarts?), whether alert rules exist and what they can trigger on, notification channels.
5. **Auto-scaling** — does the platform have built-in autoscaling settings? What knobs (min/max replicas, CPU/memory thresholds)?
6. **Block Storage** — product name as they call it, how a volume attaches to a container, size limits, snapshot support.
7. **Environments** — can you run two services (staging + production)? Any built-in env concept, or just two separate services?
8. Also: open **mithal.space**, do a search, copy the result URL pattern (e.g. `https://mithal.space/search?q=test`).

Screenshot everything as you go — you need screenshots for the submission anyway.

---

## PART B — The 5 prompts

Fill `[GHAYMAH NOTES: ...]` with the relevant items from Part A before sending. Send Prompt 1 (Opus) and Prompt 2 (Sol) simultaneously; then 3 & 4; then 5.

---

### PROMPT 1 → OPUS (Q1: app + Dockerfile + monitor + dashboard)

```
Build a complete deliverable for this task, in the folder q1-deploy-monitor/:

TASK: Deploy and monitor a simple API app on the Ghaymah cloud container platform.

Create exactly these files:

1. app/main.py — Python FastAPI app with:
   - GET /  → basic info JSON
   - GET /health → {"status":"ok","uptime_s":<float>,"timestamp":<iso8601>} with HTTP 200, no external dependencies
   - GET /metrics → {"requests_total":<int>, "started_at":<iso8601>} using an in-memory counter incremented by a middleware on every request
   - Runs with uvicorn on 0.0.0.0:8080

2. app/requirements.txt — fastapi + uvicorn, pinned versions.

3. app/Dockerfile — python:3.12-slim, non-root user, COPY requirements.txt and pip install BEFORE copying code (layer caching), EXPOSE 8080, HEALTHCHECK curl -f http://localhost:8080/health, CMD uvicorn. Also app/.dockerignore.

4. monitor/monitor.py — Python monitoring script:
   - Reads APP_URL from env var
   - Every 30 seconds: GET $APP_URL/health with 5s timeout, and GET /metrics
   - Appends one JSON object per check to monitor/data/checks.json (a JSON array):
     {"ts":"<iso8601>","status":"up"|"down","code":<int|null>,"latency_ms":<float|null>,"requests":<int|null>}
   - Any exception/timeout → status "down"
   - Prints an ALERT line after 3 consecutive failures
   - Also supports --once flag for a single check

5. dashboard/index.html — ONE self-contained file (inline CSS/JS, Chart.js from CDN allowed) that fetches ../monitor/data/checks.json (path configurable in a const at top) and shows:
   - Big status badge, green UP / red DOWN, from the latest check
   - Line chart of latency_ms over time
   - Request-count tile from the latest check
   - Last-updated timestamp, auto-refresh every 30s
   - Handle empty/missing data gracefully (show "no data yet", don't crash)
   Make it look clean and professional (dark theme is fine).

Also write q1-deploy-monitor/README.md: how to build/run the Docker image locally, how to push it to the Ghaymah registry and deploy it, how to run the monitor, how to open the dashboard.

[GHAYMAH NOTES: <paste registry URL, deploy steps, public URL format here>]
```

---

### PROMPT 2 → SOL (Q2: postmortem)

```
Write q2-postmortem/POSTMORTEM.md — a professional, blameless postmortem for this scenario:

SCENARIO: An application on the Ghaymah cloud platform was down for 45 minutes due to repeated OOMKilled container restarts.

Required sections:

1. Summary — one paragraph: duration 45 min, cause OOMKilled (exit code 137) crash loop, user impact (est. error rate / full downtime), severity SEV-2.

2. Timeline — a table with minute-by-minute realistic entries (invent plausible timestamps): gradual memory growth after a release → first OOMKill → crash loop worsened by retry traffic → alert fires → engineer investigates → mitigation (memory limit raised + rollback of the leaking release) → full recovery at minute 45. Columns: time, event, actor/action.

3. Root cause — 5-Whys analysis. Root: a memory leak introduced in the latest release (unbounded in-process cache), combined with a memory limit sized for the old baseline, no memory-usage alerting, and no memory regression testing in CI.

4. Recommendations — table with columns: action, owner (role), priority (P0/P1/P2). Include: fix the leak, right-size memory limits, memory alert at 80% for 5 min, restart-count alert, add auto-scaling, soak/load tests in CI, write a runbook.

5. Auto-scaling policy for Ghaymah — a concrete policy spec: horizontal scaling, scale OUT when avg memory > 70% OR CPU > 65% sustained 2 min; min 2 replicas, max N; scale-in cooldown 5–10 min to prevent flapping. Then an explicit honest paragraph: why autoscaling alone does NOT fix a memory leak (every replica leaks; scaling buys time and absorbs the retry storm while the leak is fixed) — plus vertical headroom and restart policies as complements.

6. Early detection with Ghaymah monitoring — how to catch this before an outage using the platform's monitoring: per-container memory graphs (watch the sawtooth/creep pattern), alert rules (memory > 80% for 5 min, restarts > 3 in 10 min, OOMKilled events), dashboards and notification channels. Wrap any claim about a specific Ghaymah feature name in <!-- VERIFY --> comments so I can check it against the real docs.

Tone: professional SRE postmortem, markdown tables, no fluff.

[GHAYMAH NOTES: <paste what monitoring/alerting/autoscaling features you saw here>]
```

---

### PROMPT 3 → SOL (Q3: CI/CD)

```
Create two files for a CI/CD deliverable targeting the Ghaymah cloud platform:

1. .github/workflows/deploy.yml — GitHub Actions workflow:
   - Triggers: push to main, and workflow_dispatch
   - Job build-push: checkout → docker/login-action to the Ghaymah container registry using secrets GHAYMAH_REGISTRY_USER and GHAYMAH_REGISTRY_TOKEN → docker/build-push-action building q1-deploy-monitor/app, tagging both <REGISTRY>/<user>/ghaymah-api:${{ github.sha }} and :latest
   - Job deploy-staging (needs: build-push): installs the ghaymah CLI, authenticates with secret GHAYMAH_API_TOKEN, deploys/updates the STAGING service to the new image tag
   - Job deploy-production (needs: deploy-staging): environment: production — this is the manual approval gate — then same deploy against the PRODUCTION service
   - Mark every Ghaymah-specific command with a # VERIFY comment

2. q3-cicd/CICD.md with:
   - Pipeline overview + a mermaid diagram: commit → build+push → staging (auto) → manual approval → production
   - "Manual approval" section: explain it's implemented via a GitHub Environment named production with required reviewers, configured in Settings → Environments (not in YAML), and the run pauses until approved
   - "Staging vs Production" section: purpose of each, differences table (data: synthetic vs real; scale/replicas; secrets; access control; alerting thresholds; deploy cadence; who can approve)
   - "Ghaymah CLI integration" section: install, auth login with API token, the deploy/update command, and how the workflow steps call it. Wrap uncertain command names in <!-- VERIFY -->.

[GHAYMAH NOTES: <paste exact registry URL + exact CLI install/auth/deploy commands here>]
```

---

### PROMPT 4 → SOL (Q4: scalability)

```
Write q4-scalability/SCALABILITY.md answering this task about the Ghaymah cloud platform:

1. Architecture diagram (mermaid, renders on GitHub) for an app receiving 15,000 req/s:
   DNS → CDN/edge cache → L7 load balancer → auto-scaled fleet of stateless API containers → Redis cache → PostgreSQL primary + read replicas (volumes on Ghaymah Block Storage) → and a monitoring/alerting component observing everything. Annotate the request flow and where bursts get absorbed (CDN + cache).

2. Container count calculation — show the arithmetic explicitly:
   15,000 req/s ÷ 500 req/s per container = 30 containers
   +30% headroom: 30 × 1.30 = 39 containers
   Answer: 39. Add notes: always round up; keep N+1 extra for rolling deploys; headroom also covers traffic spikes and a zone failure.

3. Cold-start strategy for new containers, as concrete bullets: minimum warm replica floor (never scale to zero for this tier), slim pre-pulled images (small base, few layers) for fast pulls, fast app boot (lazy-load non-critical work), readiness probe so the load balancer only routes to warmed containers, predictive/scheduled scale-up before known peaks, step-based scale-out (add several at once under sharp load).

4. Ghaymah Block Storage for stateful workloads: containers are ephemeral so state must live on attached volumes; use Block Storage for the database, queues, uploads; one-writer-per-volume semantics mean the stateful tier scales via replication (primary/replica) not by cloning volumes; snapshots for backup; IOPS/size considerations; the API tier stays completely diskless so it can scale freely. Wrap platform-specific claims in <!-- VERIFY -->.

[GHAYMAH NOTES: <paste Block Storage + autoscaling details here>]
```

---

### PROMPT 5 → OPUS (Q5: mithal.space monitoring)

```
Build a monitoring system for the website https://mithal.space (a search engine), in q5-mithal-dashboard/:

1. collector/collect.py — Python (stdlib + requests only):
   Every 60 seconds (loop mode; also support --once) measure:
   - latency_ms: timed GET https://mithal.space (10s timeout)
   - up: true if status code 200–399
   - code: the status code (null on connection failure)
   - ssl_days_left: open an ssl socket to mithal.space:443, read the cert notAfter, compute days remaining
   - dns_ms: timed socket.getaddrinfo("mithal.space", 443)
   - search_ms: timed GET to the search endpoint — make the URL a config constant, default: <PASTE THE SEARCH URL PATTERN YOU FOUND, e.g. https://mithal.space/search?q=test>
   Append one object per run to data/metrics.json (JSON array):
   {"ts":"<iso8601 utc>","up":bool,"code":int|null,"latency_ms":float|null,"dns_ms":float|null,"ssl_days_left":int|null,"search_ms":float|null}
   Prune entries older than 48h on each write. Never crash on a failed check — record nulls.

2. dashboard/index.html — ONE self-contained file (Chart.js CDN allowed) fetching data/metrics.json (path const at top):
   - Uptime % tile over the last 24h: up_checks / total_checks × 100, one decimal
   - Line chart of latency_ms for the LAST HOUR (optionally search_ms as second series)
   - SSL card: "X days remaining", green if >30, yellow 8–30, red ≤7
   - Table of the last 10 checks: time, ✅/❌, code, latency, dns, search
   - Auto-refresh every 60s, graceful empty state, clean professional look consistent with a dark ops-dashboard style

3. Dockerfile (in q5-mithal-dashboard/) — a single container that BOTH serves the dashboard as static files on port 8080 AND runs the collector in the background writing into the served directory. Simplest approach: python:3.12-slim, a start.sh entrypoint that launches collect.py in the background then runs python -m http.server 8080 (or uvicorn static serving) from the dashboard directory, with data/ inside the served path so the dashboard can fetch metrics.json.

4. README.md: run locally, build the image, deploy to Ghaymah.

[GHAYMAH NOTES: <paste deploy steps here>]
```

---

## PART C — Ready [GHAYMAH NOTES] blocks (paste these as-is)

Research result: Ghaymah docs only document **deploy-from-image-URL** (image URL + app name + port + public access toggle + env vars → Deploy). No registry, CLI, monitoring, autoscaling, or block-storage details are documented. Strategy: use **Docker Hub** as the registry, and have agents label undocumented platform specifics as **"Proposed design"** instead of inventing features. mithal.space search URL confirmed: `https://mithal.space/search?q=<query>`.

### Paste into PROMPT 1 (Opus, Q1):
```
[GHAYMAH NOTES]
- Ghaymah deploys containers from a public image URL. Deployment flow (dashboard): enter Container Image URL (e.g. docker.io/<user>/ghaymah-api:latest) → set Application Name → set Port Number (must match the EXPOSEd port) → enable Public Access → add Environment Variables → click Deploy.
- So in the README, document: build locally → docker push to Docker Hub (docker.io/<MY_DOCKERHUB_USER>/ghaymah-api) → deploy on Ghaymah by pasting that image URL with port 8080 and Public Access enabled.
- Ghaymah's docs don't document the public URL format; write "<the public URL Ghaymah assigns>" as a placeholder.
```

### Paste into PROMPT 2 (Sol, Q2):
```
[GHAYMAH NOTES]
- Ghaymah's public docs do not document monitoring/alerting or autoscaling features. IMPORTANT: do NOT invent Ghaymah feature names. Instead:
  - Section 5 (auto-scaling policy): the task literally asks to DESIGN a policy for the platform — present it as a proposed policy design ("Proposed auto-scaling policy for Ghaymah"), which is exactly what's asked.
  - Section 6 (early detection): frame as "monitoring approach on Ghaymah" using platform-agnostic container signals (memory %, restart count, OOMKilled events, exit code 137) and note these can be collected via an external monitor hitting /health plus container runtime stats, since that's verifiable. Keep <!-- VERIFY --> markers only where you reference a platform capability.
```

### Paste into PROMPT 3 (Sol, Q3):
```
[GHAYMAH NOTES]
- Ghaymah deploys from a container image URL entered in its dashboard (image URL + app name + port + public access + env vars → Deploy). No Ghaymah-hosted registry or CLI is documented publicly.
- Therefore: build the workflow to push to Docker Hub (docker/login-action with secrets DOCKERHUB_USERNAME / DOCKERHUB_TOKEN, image docker.io/<user>/ghaymah-api:${{ github.sha }} and :latest). Add a note: "if a Ghaymah-hosted registry is available, only the login server and image prefix change."
- For the deploy jobs: since no CLI is documented, implement deploy-staging and deploy-production as jobs that (a) print the exact image tag to deploy and (b) call a placeholder script scripts/ghaymah_deploy.sh marked # VERIFY, and document in CICD.md that per current docs the deploy step is updating the image URL/tag in the Ghaymah dashboard for the app — with the CLI section written as "integration guide (to be confirmed against Ghaymah CLI docs)".
- Staging vs production on Ghaymah: two separate deployed apps, e.g. myapp-staging and myapp-production, each with its own env vars; the workflow deploys staging automatically and production only after the GitHub Environment approval.
```

### Paste into PROMPT 4 (Sol, Q4):
```
[GHAYMAH NOTES]
- Ghaymah's public docs don't detail Block Storage or autoscaling specifics. Write section 4 as a correct general explanation of block storage for stateful container workloads applied to Ghaymah ("Ghaymah Block Storage"), and keep <!-- VERIFY --> only on hard specifics (size limits, snapshot support). The architecture/math/cold-start sections are platform-agnostic — no changes needed.
```

### Paste into PROMPT 5 (Opus, Q5):
```
[GHAYMAH NOTES]
- Search endpoint CONFIRMED working: https://mithal.space/search?q=test (returns a results page). Use that as the default search URL constant.
- Deployment: Ghaymah deploys from a public image URL (dashboard: image URL + app name + port 8080 + Public Access enabled + env vars → Deploy). README should say: push image to Docker Hub, then deploy by pasting docker.io/<MY_DOCKERHUB_USER>/mithal-monitor:latest into Ghaymah with port 8080.
```

### Your remaining manual to-dos (updated)
1. Make/confirm a **Docker Hub** account; create repos `ghaymah-api` and `mithal-monitor` (public is simplest for Ghaymah to pull).
2. In GitHub: secrets `DOCKERHUB_USERNAME`, `DOCKERHUB_TOKEN` (access token from Docker Hub → Account Settings → Security); environment `production` with you as required reviewer.
3. In the Ghaymah dashboard, while deploying, **look for tabs/menus the docs didn't cover** (Registry, CLI, Metrics/Logs, Scaling, Volumes). If any exist, screenshot them and resolve the matching <!-- VERIFY --> markers — instant answer upgrades for Q2/Q3/Q4.
4. Screenshots needed: Q1 app running + /health JSON, Q3 workflow paused at approval, Q5 live dashboard.