From 033c43a2211ff22f0408800b670d45896384d5cd Mon Sep 17 00:00:00 2001 From: yassinelagamy <284392805+yassinelagamy@users.noreply.github.com> Date: Sun, 26 Jul 2026 23:54:10 +0300 Subject: [PATCH] docs: record post-wake verification and history [skip ci] --- README.md | 25 +++++-- docs/evidence/README.md | 7 +- q1-deploy-monitor/README.md | 53 ++++++++++++--- q1-deploy-monitor/monitor/data/checks.json | 77 ++++++++++++++++++++++ q3-cicd/CICD.md | 6 +- q5-mithal-dashboard/README.md | 20 +++++- 6 files changed, 165 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index a112ec8..1537669 100644 --- a/README.md +++ b/README.md @@ -8,13 +8,16 @@ monitoring dashboard. | Question | Deliverable | Status | |---|---|---| -| Q1 — Deploy and monitor an API | [Application, monitor, dashboard, and deployment guide](q1-deploy-monitor/README.md) | Live | +| Q1 — Deploy and monitor an API | [Application, monitor, dashboard, and deployment guide](q1-deploy-monitor/README.md) | HTTP 200 verified 2026-07-26 20:46:07 UTC | | Q2 — OOMKilled postmortem | [Professional blameless postmortem](q2-postmortem/POSTMORTEM.md) | Complete | | Q3 — CI/CD pipeline | [Workflow](.github/workflows/deploy.yml) and [documentation](q3-cicd/CICD.md) | Complete | | Q4 — Scalability | [Architecture, capacity calculation, cold starts, and storage](q4-scalability/SCALABILITY.md) | Complete | -| Q5 — `mithal.space` monitoring | [Collector and dashboard](q5-mithal-dashboard/) | Live | +| Q5 — `mithal.space` monitoring | [Collector and dashboard](q5-mithal-dashboard/) | HTTP 200 verified 2026-07-26 20:46:07 UTC | -## Live services +## Service endpoints + +Both application endpoints returned HTTP 200 in a fresh check at +**2026-07-26 20:46:07 UTC**: - Q1 API: - Q1 health: @@ -35,7 +38,7 @@ The curated evidence set is documented in |---|---| | Ghaymah project and applications | [Project](docs/evidence/screenshots/01-ghaymah-project.png) | | Q1 API service | [Q1 service](docs/evidence/screenshots/02-q1-service.png) | -| Q1 live health response | [Q1 health](docs/evidence/screenshots/03-q1-health.png) | +| Q1 health response recorded 2026-07-26 18:48:19 UTC | [Q1 health](docs/evidence/screenshots/03-q1-health.png) | | Q1 monitoring dashboard | [Q1 dashboard](docs/evidence/screenshots/04-q1-dashboard.png) | | Q5 Ghaymah service | [Q5 service](docs/evidence/screenshots/05-q5-service.png) | | Q5 monitoring dashboard | [Q5 dashboard](docs/evidence/screenshots/06-q5-dashboard.png) | @@ -52,7 +55,7 @@ The workflow: 4. Starts the immutable image in an ephemeral staging container. 5. Requires `status=ok` and the expected SHA from `/health`. 6. Pauses for the required reviewer on the GitHub `production` Environment. -7. Verifies that the live Ghaymah application serves the approved SHA. +7. Verifies that the deployed Ghaymah application serves the approved SHA. Ghaymah CLI `0.0.24` documents interactive email/password login but does not publish API-token authentication or the external-image field for @@ -62,7 +65,17 @@ workflow does not report a stale or unverified deployment as successful. The account's five-resource free-plan limit prevents a third persistent Ghaymah application, so staging runs as an isolated ephemeral container on the Actions -runner. Production and the Q5 dashboard remain live on Ghaymah. +runner. Production and the Q5 dashboard both returned HTTP 200 at +2026-07-26 20:46:07 UTC. + +## Free-tier hibernation policy + +Both applications run on the Ghaymah free tier. They may be put into +**Hibernate** in the Ghaymah dashboard when the assessment is not being reviewed +to conserve container credits; hibernated public URLs return HTTP 503. Before +submission or review, open each application and select **Wake Up**, then verify +both public endpoints and record the UTC check time. Hibernation preserves the +deployment configuration but stops the Q5 in-container collector. ## Verified platform information diff --git a/docs/evidence/README.md b/docs/evidence/README.md index 199e4d5..1607142 100644 --- a/docs/evidence/README.md +++ b/docs/evidence/README.md @@ -8,7 +8,7 @@ duplicate captures are not included. |---:|---|---| | 01 | `01-ghaymah-project.png` | Authenticated Ghaymah project and deployed applications. | | 02 | `02-q1-service.png` | Q1 API application in Ghaymah. | -| 03 | `03-q1-health.png` | Live HTTP 200 health response with release identifier. | +| 03 | `03-q1-health.png` | HTTP 200 health response with release identifier, recorded 2026-07-26 18:48:19 UTC. | | 04 | `04-q1-dashboard.png` | Q1 status, latency, request count, uptime, and history. | | 05 | `05-q5-service.png` | Q5 monitoring application in Ghaymah. | | 06 | `06-q5-dashboard.png` | Q5 uptime, latency chart, SSL, DNS/search metrics, and recent checks. | @@ -18,5 +18,10 @@ duplicate captures are not included. `health-response.html` records the URL, UTC check time, HTTP status, and JSON used for screenshot 03. +Operational status was rechecked after waking both free-tier applications: +Q1 `/health` and Q5 `/index.html` returned HTTP 200 at +**2026-07-26 20:46:07 UTC**. Q5's ephemeral in-container metric history restarted +at **2026-07-26 20:45:34.818046 UTC** and resumed its one-minute growth cadence. + All service URLs and Docker image names are also listed in the repository root [README](../../README.md). diff --git a/q1-deploy-monitor/README.md b/q1-deploy-monitor/README.md index a6ca54f..b22a721 100644 --- a/q1-deploy-monitor/README.md +++ b/q1-deploy-monitor/README.md @@ -25,11 +25,25 @@ q1-deploy-monitor/ | Method | Path | Response | |---|---|---| -| `GET` | `/` | service name, version, endpoint list, start time | -| `GET` | `/health` | `{"status":"ok","uptime_s":12.34,"timestamp":"2026-07-26T15:36:50.283265+00:00"}` — HTTP 200 | +| `GET` | `/` | service name, version, `release`, endpoint list, start time | +| `GET` | `/health` | `{"status":"ok","release":"","uptime_s":12.34,"timestamp":"2026-07-26T20:46:06.847603+00:00"}` — HTTP 200 | | `GET` | `/metrics` | `{"requests_total":42,"started_at":""}` | | `GET` | `/docs` | interactive OpenAPI docs (FastAPI built-in) | +Example `/` response: + +```json +{ + "service": "ghaymah-api", + "version": "1.0.0", + "release": "60b9e2c38e3f1a509383c6d61a7f76b9d043214a", + "message": "Ghaymah deployment demo API", + "endpoints": ["/", "/health", "/metrics", "/docs"], + "started_at": "2026-07-26T20:46:03.315313+00:00", + "timestamp": "2026-07-26T20:52:18.907569+00:00" +} +``` + `/health` performs **no** downstream checks (no DB, no network) — it reports process liveness only, so a failing check always means "restart me", which is exactly the signal an orchestrator's health probe should act on. @@ -135,7 +149,7 @@ anonymously from the URL you paste in. Then click **Deploy**. Once the deployment reports as running, Ghaymah assigns the service a public URL. -### 3.3 Verify the live deployment +### 3.3 Verify the deployment ```bash curl -f https://ghaymah-api-615e99f13665.hosted.ghaymah.systems/health @@ -173,6 +187,21 @@ Every 30 seconds it issues `GET $APP_URL/health` with a 5 s timeout, then `GET $APP_URL/metrics`, and appends one record to `monitor/data/checks.json` (a JSON array, created on first run): +The current `/health` response shape is: + +```json +{ + "status": "ok", + "release": "60b9e2c38e3f1a509383c6d61a7f76b9d043214a", + "uptime_s": 3.532, + "timestamp": "2026-07-26T20:46:06.847603+00:00" +} +``` + +The monitor converts that response into the following derived history record. +It intentionally stores status, HTTP code, latency, and request count rather +than duplicating every response field: + ```json { "ts": "2026-07-26T15:37:29.886772+00:00", @@ -257,7 +286,7 @@ const DATA_URL = '../monitor/data/checks.json'; | Check | Result | |---|---| | `pip install -r requirements.txt` (pinned versions) | fastapi 0.115.6, uvicorn 0.34.0 installed cleanly | -| `GET /health` | HTTP 200 · `{"status":"ok","uptime_s":7.797,"timestamp":"..."}` | +| `GET /health` | HTTP 200 · `{"status":"ok","release":"local","uptime_s":7.797,"timestamp":"..."}` | | `GET /` and `GET /metrics` | valid JSON; `requests_total` increments per request | | `monitor.py --once` against the running app | `UP code=200 latency=45.8ms requests=5`, exit 0 | | `monitor.py` loop across an app shutdown | up records → down records → `ALERT` printed on the 3rd consecutive failure | @@ -266,9 +295,9 @@ const DATA_URL = '../monitor/data/checks.json'; | `docker build` | image builds clean (249 MB) | | `docker run` | container reports `(healthy)` via the `HEALTHCHECK`; all three endpoints respond | -### Against the live Ghaymah deployment +### Against the Ghaymah deployment -Final release checked on 2026-07-26 at 18:48 UTC: +Final release checked on 2026-07-26 at 20:46:07 UTC: | Endpoint | Result | |---|---| @@ -276,7 +305,11 @@ Final release checked on 2026-07-26 at 18:48 UTC: | `/` | Service metadata and the same immutable release identifier | | `/metrics` | Valid request counter and process start time; the counter reflects real monitor traffic | -The monitor polls the live URL every 30 seconds. The committed -`monitor/data/checks.json` is a submission snapshot; ongoing checks continue in -the `ghaymah-q1-monitor-history` Docker volume so normal monitoring does not -dirty the repository. +After the endpoint returned HTTP 200 at 2026-07-26 20:46:07 UTC, the monitor was +started with `DATA_FILE=/data/checks.json`, where `/data` is a read-write bind to +the repository's `monitor/data` directory. Its `/monitor` code bind remains +read-only. This lets atomic JSON updates append directly to the submitted +history without allowing the container to alter its monitor source code. The +older `ghaymah-q1-monitor-history` volume is deliberately not copied into the +repository because it contains checks made while the free-tier service was +hibernated. diff --git a/q1-deploy-monitor/monitor/data/checks.json b/q1-deploy-monitor/monitor/data/checks.json index 4ea3ece..f9bb929 100644 --- a/q1-deploy-monitor/monitor/data/checks.json +++ b/q1-deploy-monitor/monitor/data/checks.json @@ -1433,5 +1433,82 @@ "code": 200, "latency_ms": 881.37, "requests": 77 + }, + { + "ts": "2026-07-26T20:47:19.374072+00:00", + "status": "up", + "code": 200, + "latency_ms": 879.36, + "requests": 21 + }, + { + "ts": "2026-07-26T20:47:50.990356+00:00", + "status": "up", + "code": 200, + "latency_ms": 815.81, + "requests": 23 + }, + { + "ts": "2026-07-26T20:48:22.486604+00:00", + "status": "up", + "code": 200, + "latency_ms": 837.14, + "requests": 25 + }, + { + "ts": "2026-07-26T20:48:55.259585+00:00", + "status": "up", + "code": 200, + "latency_ms": 776.02, + "requests": 27 + }, + { + "ts": "2026-07-26T20:49:26.925213+00:00", + "status": "up", + "code": 200, + "latency_ms": 801.65, + "requests": 29 + }, + { + "ts": "2026-07-26T20:49:58.707677+00:00", + "status": "up", + "code": 200, + "latency_ms": 913.24, + "requests": 31 + }, + { + "ts": "2026-07-26T20:50:29.887394+00:00", + "status": "up", + "code": 200, + "latency_ms": 692.46, + "requests": 33 + }, + { + "ts": "2026-07-26T20:51:02.112889+00:00", + "status": "up", + "code": 200, + "latency_ms": 1722.28, + "requests": 35 + }, + { + "ts": "2026-07-26T20:51:33.111656+00:00", + "status": "up", + "code": 200, + "latency_ms": 481.18, + "requests": 37 + }, + { + "ts": "2026-07-26T20:52:04.126187+00:00", + "status": "up", + "code": 200, + "latency_ms": 511.67, + "requests": 39 + }, + { + "ts": "2026-07-26T20:52:35.120780+00:00", + "status": "up", + "code": 200, + "latency_ms": 472.67, + "requests": 43 } ] diff --git a/q3-cicd/CICD.md b/q3-cicd/CICD.md index 3f41f9c..47d1d11 100644 --- a/q3-cicd/CICD.md +++ b/q3-cicd/CICD.md @@ -21,7 +21,7 @@ flowchart LR ``` The free Ghaymah account used for this assessment permits five total resources. -The two required live applications (`ghaymah-api` and `mithal-monitor`) already +The two required deployed applications (`ghaymah-api` and `mithal-monitor`) already consume the available allocation together with their supporting resources. A third Ghaymah application was tested and rejected by the platform with the documented account message `Resource limit exceeded ... maximum 5 resources`. @@ -36,7 +36,7 @@ The workflow runs on a push to `main` and through `workflow_dispatch`. It uses: | Secret `DOCKERHUB_USERNAME` | Docker Hub account and image namespace. | | Secret `DOCKERHUB_TOKEN` | Read/write Docker Hub access token. | | Variable `GHAYMAH_STAGING_APP` | Display name for the ephemeral staging service. | -| Variable `GHAYMAH_PRODUCTION_APP` | Name of the live Ghaymah application. | +| Variable `GHAYMAH_PRODUCTION_APP` | Name of the deployed Ghaymah application. | | Variable `GHAYMAH_PRODUCTION_URL` | Public base URL used for the production verification. | Images are published as: @@ -136,7 +136,7 @@ bash scripts/verify_deployment.sh \ --expected-release "$GITHUB_SHA" ``` -It retries `/health`, requires `status=ok`, compares the live `release` to the +It retries `/health`, requires `status=ok`, compares the deployed `release` to the approved SHA, writes evidence to the GitHub job summary, and exits nonzero for an unavailable or stale deployment. diff --git a/q5-mithal-dashboard/README.md b/q5-mithal-dashboard/README.md index 870b1b8..35155f1 100644 --- a/q5-mithal-dashboard/README.md +++ b/q5-mithal-dashboard/README.md @@ -1,7 +1,7 @@ # Q5 — Monitoring dashboard for mithal.space A single container that continuously measures the availability and performance of -**https://mithal.space** and serves a live dashboard of the results on port 8080. +**https://mithal.space** and serves a monitoring dashboard of the results on port 8080. ``` q5-mithal-dashboard/ @@ -188,7 +188,7 @@ Then, in the Ghaymah dashboard: | Environment Variables | *(optional)* `INTERVAL_S=60`, `TARGET_URL=https://mithal.space`, `SEARCH_URL=https://mithal.space/search?q=test` | Click **Deploy**, then open `https://mithal-monitor-292f00f076b1.hosted.ghaymah.systems/index.html` and -screenshot the live dashboard. Leave it running so the 24 h uptime tile and the +screenshot the deployed dashboard. Leave it running so the 24 h uptime tile and the hourly chart fill with real history before submission. --- @@ -207,7 +207,7 @@ Measured against the live site on 2026-07-26: | `docker build` | image builds clean (200 MB) | | `docker run` | `start.sh` launches the collector and the static server; container reports `(healthy)`; the collector wrote records into the served directory and the dashboard rendered them | -### Against the live Ghaymah deployment +### Against the Ghaymah deployment Checked on 2026-07-26 at 17:23 UTC, ~30 minutes after deployment: @@ -223,3 +223,17 @@ storage or second service involved. The `dashboard/data/metrics.json` in this repo holds those first real samples; the collector prunes anything older than 48 h automatically. + +### Post-hibernation collection + +The application was woken for final submission verification. `/index.html` +returned HTTP 200 at **2026-07-26 20:46:07 UTC**. The collector's new +in-container history began at **2026-07-26 20:45:34.818046 UTC** and grew from +3 to 4 records by **2026-07-26 20:48:44 UTC**, confirming the one-record-per-minute +loop resumed. + +This history restarted at wake time because `/app/dashboard/data/metrics.json` +is on the container's ephemeral filesystem, not an attached volume. Hibernation +and restart therefore demonstrate the persistence boundary described in Q4: +production history that must survive container replacement belongs on Ghaymah +Block Storage or another external durable store.