docs: record post-wake verification and history [skip ci]

This commit is contained in:
yassinelagamy
2026-07-26 23:54:10 +03:00
parent deb96349b8
commit 033c43a221
6 changed files with 165 additions and 23 deletions
+19 -6
View File
@@ -8,13 +8,16 @@ monitoring dashboard.
| Question | Deliverable | Status |
|---|---|---|
| Q1 — Deploy and monitor an API | [Application, monitor, dashboard, and deployment guide](q1-deploy-monitor/README.md) | Live |
| Q1 — Deploy and monitor an API | [Application, monitor, dashboard, and deployment guide](q1-deploy-monitor/README.md) | HTTP 200 verified 2026-07-26 20:46:07 UTC |
| Q2 — OOMKilled postmortem | [Professional blameless postmortem](q2-postmortem/POSTMORTEM.md) | Complete |
| Q3 — CI/CD pipeline | [Workflow](.github/workflows/deploy.yml) and [documentation](q3-cicd/CICD.md) | Complete |
| Q4 — Scalability | [Architecture, capacity calculation, cold starts, and storage](q4-scalability/SCALABILITY.md) | Complete |
| Q5 — `mithal.space` monitoring | [Collector and dashboard](q5-mithal-dashboard/) | Live |
| Q5 — `mithal.space` monitoring | [Collector and dashboard](q5-mithal-dashboard/) | HTTP 200 verified 2026-07-26 20:46:07 UTC |
## Live services
## Service endpoints
Both application endpoints returned HTTP 200 in a fresh check at
**2026-07-26 20:46:07 UTC**:
- Q1 API: <https://ghaymah-api-615e99f13665.hosted.ghaymah.systems>
- Q1 health: <https://ghaymah-api-615e99f13665.hosted.ghaymah.systems/health>
@@ -35,7 +38,7 @@ The curated evidence set is documented in
|---|---|
| Ghaymah project and applications | [Project](docs/evidence/screenshots/01-ghaymah-project.png) |
| Q1 API service | [Q1 service](docs/evidence/screenshots/02-q1-service.png) |
| Q1 live health response | [Q1 health](docs/evidence/screenshots/03-q1-health.png) |
| Q1 health response recorded 2026-07-26 18:48:19 UTC | [Q1 health](docs/evidence/screenshots/03-q1-health.png) |
| Q1 monitoring dashboard | [Q1 dashboard](docs/evidence/screenshots/04-q1-dashboard.png) |
| Q5 Ghaymah service | [Q5 service](docs/evidence/screenshots/05-q5-service.png) |
| Q5 monitoring dashboard | [Q5 dashboard](docs/evidence/screenshots/06-q5-dashboard.png) |
@@ -52,7 +55,7 @@ The workflow:
4. Starts the immutable image in an ephemeral staging container.
5. Requires `status=ok` and the expected SHA from `/health`.
6. Pauses for the required reviewer on the GitHub `production` Environment.
7. Verifies that the live Ghaymah application serves the approved SHA.
7. Verifies that the deployed Ghaymah application serves the approved SHA.
Ghaymah CLI `0.0.24` documents interactive email/password login but does not
publish API-token authentication or the external-image field for
@@ -62,7 +65,17 @@ workflow does not report a stale or unverified deployment as successful.
The account's five-resource free-plan limit prevents a third persistent Ghaymah
application, so staging runs as an isolated ephemeral container on the Actions
runner. Production and the Q5 dashboard remain live on Ghaymah.
runner. Production and the Q5 dashboard both returned HTTP 200 at
2026-07-26 20:46:07 UTC.
## Free-tier hibernation policy
Both applications run on the Ghaymah free tier. They may be put into
**Hibernate** in the Ghaymah dashboard when the assessment is not being reviewed
to conserve container credits; hibernated public URLs return HTTP 503. Before
submission or review, open each application and select **Wake Up**, then verify
both public endpoints and record the UTC check time. Hibernation preserves the
deployment configuration but stops the Q5 in-container collector.
## Verified platform information
+6 -1
View File
@@ -8,7 +8,7 @@ duplicate captures are not included.
|---:|---|---|
| 01 | `01-ghaymah-project.png` | Authenticated Ghaymah project and deployed applications. |
| 02 | `02-q1-service.png` | Q1 API application in Ghaymah. |
| 03 | `03-q1-health.png` | Live HTTP 200 health response with release identifier. |
| 03 | `03-q1-health.png` | HTTP 200 health response with release identifier, recorded 2026-07-26 18:48:19 UTC. |
| 04 | `04-q1-dashboard.png` | Q1 status, latency, request count, uptime, and history. |
| 05 | `05-q5-service.png` | Q5 monitoring application in Ghaymah. |
| 06 | `06-q5-dashboard.png` | Q5 uptime, latency chart, SSL, DNS/search metrics, and recent checks. |
@@ -18,5 +18,10 @@ duplicate captures are not included.
`health-response.html` records the URL, UTC check time, HTTP status, and JSON
used for screenshot 03.
Operational status was rechecked after waking both free-tier applications:
Q1 `/health` and Q5 `/index.html` returned HTTP 200 at
**2026-07-26 20:46:07 UTC**. Q5's ephemeral in-container metric history restarted
at **2026-07-26 20:45:34.818046 UTC** and resumed its one-minute growth cadence.
All service URLs and Docker image names are also listed in the repository root
[README](../../README.md).
+43 -10
View File
@@ -25,11 +25,25 @@ q1-deploy-monitor/
| Method | Path | Response |
|---|---|---|
| `GET` | `/` | service name, version, endpoint list, start time |
| `GET` | `/health` | `{"status":"ok","uptime_s":12.34,"timestamp":"2026-07-26T15:36:50.283265+00:00"}` — HTTP 200 |
| `GET` | `/` | service name, version, `release`, endpoint list, start time |
| `GET` | `/health` | `{"status":"ok","release":"<git-commit-sha>","uptime_s":12.34,"timestamp":"2026-07-26T20:46:06.847603+00:00"}` — HTTP 200 |
| `GET` | `/metrics` | `{"requests_total":42,"started_at":"<iso8601>"}` |
| `GET` | `/docs` | interactive OpenAPI docs (FastAPI built-in) |
Example `/` response:
```json
{
"service": "ghaymah-api",
"version": "1.0.0",
"release": "60b9e2c38e3f1a509383c6d61a7f76b9d043214a",
"message": "Ghaymah deployment demo API",
"endpoints": ["/", "/health", "/metrics", "/docs"],
"started_at": "2026-07-26T20:46:03.315313+00:00",
"timestamp": "2026-07-26T20:52:18.907569+00:00"
}
```
`/health` performs **no** downstream checks (no DB, no network) — it reports
process liveness only, so a failing check always means "restart me", which is
exactly the signal an orchestrator's health probe should act on.
@@ -135,7 +149,7 @@ anonymously from the URL you paste in.
Then click **Deploy**. Once the deployment reports as running, Ghaymah assigns the
service a public URL.
### 3.3 Verify the live deployment
### 3.3 Verify the deployment
```bash
curl -f https://ghaymah-api-615e99f13665.hosted.ghaymah.systems/health
@@ -173,6 +187,21 @@ Every 30 seconds it issues `GET $APP_URL/health` with a 5 s timeout, then
`GET $APP_URL/metrics`, and appends one record to `monitor/data/checks.json`
(a JSON array, created on first run):
The current `/health` response shape is:
```json
{
"status": "ok",
"release": "60b9e2c38e3f1a509383c6d61a7f76b9d043214a",
"uptime_s": 3.532,
"timestamp": "2026-07-26T20:46:06.847603+00:00"
}
```
The monitor converts that response into the following derived history record.
It intentionally stores status, HTTP code, latency, and request count rather
than duplicating every response field:
```json
{
"ts": "2026-07-26T15:37:29.886772+00:00",
@@ -257,7 +286,7 @@ const DATA_URL = '../monitor/data/checks.json';
| Check | Result |
|---|---|
| `pip install -r requirements.txt` (pinned versions) | fastapi 0.115.6, uvicorn 0.34.0 installed cleanly |
| `GET /health` | HTTP 200 · `{"status":"ok","uptime_s":7.797,"timestamp":"..."}` |
| `GET /health` | HTTP 200 · `{"status":"ok","release":"local","uptime_s":7.797,"timestamp":"..."}` |
| `GET /` and `GET /metrics` | valid JSON; `requests_total` increments per request |
| `monitor.py --once` against the running app | `UP code=200 latency=45.8ms requests=5`, exit 0 |
| `monitor.py` loop across an app shutdown | up records → down records → `ALERT` printed on the 3rd consecutive failure |
@@ -266,9 +295,9 @@ const DATA_URL = '../monitor/data/checks.json';
| `docker build` | image builds clean (249 MB) |
| `docker run` | container reports `(healthy)` via the `HEALTHCHECK`; all three endpoints respond |
### Against the live Ghaymah deployment
### Against the Ghaymah deployment
Final release checked on 2026-07-26 at 18:48 UTC:
Final release checked on 2026-07-26 at 20:46:07 UTC:
| Endpoint | Result |
|---|---|
@@ -276,7 +305,11 @@ Final release checked on 2026-07-26 at 18:48 UTC:
| `/` | Service metadata and the same immutable release identifier |
| `/metrics` | Valid request counter and process start time; the counter reflects real monitor traffic |
The monitor polls the live URL every 30 seconds. The committed
`monitor/data/checks.json` is a submission snapshot; ongoing checks continue in
the `ghaymah-q1-monitor-history` Docker volume so normal monitoring does not
dirty the repository.
After the endpoint returned HTTP 200 at 2026-07-26 20:46:07 UTC, the monitor was
started with `DATA_FILE=/data/checks.json`, where `/data` is a read-write bind to
the repository's `monitor/data` directory. Its `/monitor` code bind remains
read-only. This lets atomic JSON updates append directly to the submitted
history without allowing the container to alter its monitor source code. The
older `ghaymah-q1-monitor-history` volume is deliberately not copied into the
repository because it contains checks made while the free-tier service was
hibernated.
@@ -1433,5 +1433,82 @@
"code": 200,
"latency_ms": 881.37,
"requests": 77
},
{
"ts": "2026-07-26T20:47:19.374072+00:00",
"status": "up",
"code": 200,
"latency_ms": 879.36,
"requests": 21
},
{
"ts": "2026-07-26T20:47:50.990356+00:00",
"status": "up",
"code": 200,
"latency_ms": 815.81,
"requests": 23
},
{
"ts": "2026-07-26T20:48:22.486604+00:00",
"status": "up",
"code": 200,
"latency_ms": 837.14,
"requests": 25
},
{
"ts": "2026-07-26T20:48:55.259585+00:00",
"status": "up",
"code": 200,
"latency_ms": 776.02,
"requests": 27
},
{
"ts": "2026-07-26T20:49:26.925213+00:00",
"status": "up",
"code": 200,
"latency_ms": 801.65,
"requests": 29
},
{
"ts": "2026-07-26T20:49:58.707677+00:00",
"status": "up",
"code": 200,
"latency_ms": 913.24,
"requests": 31
},
{
"ts": "2026-07-26T20:50:29.887394+00:00",
"status": "up",
"code": 200,
"latency_ms": 692.46,
"requests": 33
},
{
"ts": "2026-07-26T20:51:02.112889+00:00",
"status": "up",
"code": 200,
"latency_ms": 1722.28,
"requests": 35
},
{
"ts": "2026-07-26T20:51:33.111656+00:00",
"status": "up",
"code": 200,
"latency_ms": 481.18,
"requests": 37
},
{
"ts": "2026-07-26T20:52:04.126187+00:00",
"status": "up",
"code": 200,
"latency_ms": 511.67,
"requests": 39
},
{
"ts": "2026-07-26T20:52:35.120780+00:00",
"status": "up",
"code": 200,
"latency_ms": 472.67,
"requests": 43
}
]
+3 -3
View File
@@ -21,7 +21,7 @@ flowchart LR
```
The free Ghaymah account used for this assessment permits five total resources.
The two required live applications (`ghaymah-api` and `mithal-monitor`) already
The two required deployed applications (`ghaymah-api` and `mithal-monitor`) already
consume the available allocation together with their supporting resources. A
third Ghaymah application was tested and rejected by the platform with the
documented account message `Resource limit exceeded ... maximum 5 resources`.
@@ -36,7 +36,7 @@ The workflow runs on a push to `main` and through `workflow_dispatch`. It uses:
| Secret `DOCKERHUB_USERNAME` | Docker Hub account and image namespace. |
| Secret `DOCKERHUB_TOKEN` | Read/write Docker Hub access token. |
| Variable `GHAYMAH_STAGING_APP` | Display name for the ephemeral staging service. |
| Variable `GHAYMAH_PRODUCTION_APP` | Name of the live Ghaymah application. |
| Variable `GHAYMAH_PRODUCTION_APP` | Name of the deployed Ghaymah application. |
| Variable `GHAYMAH_PRODUCTION_URL` | Public base URL used for the production verification. |
Images are published as:
@@ -136,7 +136,7 @@ bash scripts/verify_deployment.sh \
--expected-release "$GITHUB_SHA"
```
It retries `/health`, requires `status=ok`, compares the live `release` to the
It retries `/health`, requires `status=ok`, compares the deployed `release` to the
approved SHA, writes evidence to the GitHub job summary, and exits nonzero for
an unavailable or stale deployment.
+17 -3
View File
@@ -1,7 +1,7 @@
# Q5 — Monitoring dashboard for mithal.space
A single container that continuously measures the availability and performance of
**https://mithal.space** and serves a live dashboard of the results on port 8080.
**https://mithal.space** and serves a monitoring dashboard of the results on port 8080.
```
q5-mithal-dashboard/
@@ -188,7 +188,7 @@ Then, in the Ghaymah dashboard:
| Environment Variables | *(optional)* `INTERVAL_S=60`, `TARGET_URL=https://mithal.space`, `SEARCH_URL=https://mithal.space/search?q=test` |
Click **Deploy**, then open `https://mithal-monitor-292f00f076b1.hosted.ghaymah.systems/index.html` and
screenshot the live dashboard. Leave it running so the 24 h uptime tile and the
screenshot the deployed dashboard. Leave it running so the 24 h uptime tile and the
hourly chart fill with real history before submission.
---
@@ -207,7 +207,7 @@ Measured against the live site on 2026-07-26:
| `docker build` | image builds clean (200 MB) |
| `docker run` | `start.sh` launches the collector and the static server; container reports `(healthy)`; the collector wrote records into the served directory and the dashboard rendered them |
### Against the live Ghaymah deployment
### Against the Ghaymah deployment
Checked on 2026-07-26 at 17:23 UTC, ~30 minutes after deployment:
@@ -223,3 +223,17 @@ storage or second service involved.
The `dashboard/data/metrics.json` in this repo holds those first real samples; the
collector prunes anything older than 48 h automatically.
### Post-hibernation collection
The application was woken for final submission verification. `/index.html`
returned HTTP 200 at **2026-07-26 20:46:07 UTC**. The collector's new
in-container history began at **2026-07-26 20:45:34.818046 UTC** and grew from
3 to 4 records by **2026-07-26 20:48:44 UTC**, confirming the one-record-per-minute
loop resumed.
This history restarted at wake time because `/app/dashboard/data/metrics.json`
is on the container's ephemeral filesystem, not an attached volume. Hibernation
and restart therefore demonstrate the persistence boundary described in Q4:
production history that must survive container replacement belongs on Ghaymah
Block Storage or another external durable store.