docs: record post-wake verification and history [skip ci]
This commit is contained in:
@@ -8,13 +8,16 @@ monitoring dashboard.
|
||||
|
||||
| Question | Deliverable | Status |
|
||||
|---|---|---|
|
||||
| Q1 — Deploy and monitor an API | [Application, monitor, dashboard, and deployment guide](q1-deploy-monitor/README.md) | Live |
|
||||
| Q1 — Deploy and monitor an API | [Application, monitor, dashboard, and deployment guide](q1-deploy-monitor/README.md) | HTTP 200 verified 2026-07-26 20:46:07 UTC |
|
||||
| Q2 — OOMKilled postmortem | [Professional blameless postmortem](q2-postmortem/POSTMORTEM.md) | Complete |
|
||||
| Q3 — CI/CD pipeline | [Workflow](.github/workflows/deploy.yml) and [documentation](q3-cicd/CICD.md) | Complete |
|
||||
| Q4 — Scalability | [Architecture, capacity calculation, cold starts, and storage](q4-scalability/SCALABILITY.md) | Complete |
|
||||
| Q5 — `mithal.space` monitoring | [Collector and dashboard](q5-mithal-dashboard/) | Live |
|
||||
| Q5 — `mithal.space` monitoring | [Collector and dashboard](q5-mithal-dashboard/) | HTTP 200 verified 2026-07-26 20:46:07 UTC |
|
||||
|
||||
## Live services
|
||||
## Service endpoints
|
||||
|
||||
Both application endpoints returned HTTP 200 in a fresh check at
|
||||
**2026-07-26 20:46:07 UTC**:
|
||||
|
||||
- Q1 API: <https://ghaymah-api-615e99f13665.hosted.ghaymah.systems>
|
||||
- Q1 health: <https://ghaymah-api-615e99f13665.hosted.ghaymah.systems/health>
|
||||
@@ -35,7 +38,7 @@ The curated evidence set is documented in
|
||||
|---|---|
|
||||
| Ghaymah project and applications | [Project](docs/evidence/screenshots/01-ghaymah-project.png) |
|
||||
| Q1 API service | [Q1 service](docs/evidence/screenshots/02-q1-service.png) |
|
||||
| Q1 live health response | [Q1 health](docs/evidence/screenshots/03-q1-health.png) |
|
||||
| Q1 health response recorded 2026-07-26 18:48:19 UTC | [Q1 health](docs/evidence/screenshots/03-q1-health.png) |
|
||||
| Q1 monitoring dashboard | [Q1 dashboard](docs/evidence/screenshots/04-q1-dashboard.png) |
|
||||
| Q5 Ghaymah service | [Q5 service](docs/evidence/screenshots/05-q5-service.png) |
|
||||
| Q5 monitoring dashboard | [Q5 dashboard](docs/evidence/screenshots/06-q5-dashboard.png) |
|
||||
@@ -52,7 +55,7 @@ The workflow:
|
||||
4. Starts the immutable image in an ephemeral staging container.
|
||||
5. Requires `status=ok` and the expected SHA from `/health`.
|
||||
6. Pauses for the required reviewer on the GitHub `production` Environment.
|
||||
7. Verifies that the live Ghaymah application serves the approved SHA.
|
||||
7. Verifies that the deployed Ghaymah application serves the approved SHA.
|
||||
|
||||
Ghaymah CLI `0.0.24` documents interactive email/password login but does not
|
||||
publish API-token authentication or the external-image field for
|
||||
@@ -62,7 +65,17 @@ workflow does not report a stale or unverified deployment as successful.
|
||||
|
||||
The account's five-resource free-plan limit prevents a third persistent Ghaymah
|
||||
application, so staging runs as an isolated ephemeral container on the Actions
|
||||
runner. Production and the Q5 dashboard remain live on Ghaymah.
|
||||
runner. Production and the Q5 dashboard both returned HTTP 200 at
|
||||
2026-07-26 20:46:07 UTC.
|
||||
|
||||
## Free-tier hibernation policy
|
||||
|
||||
Both applications run on the Ghaymah free tier. They may be put into
|
||||
**Hibernate** in the Ghaymah dashboard when the assessment is not being reviewed
|
||||
to conserve container credits; hibernated public URLs return HTTP 503. Before
|
||||
submission or review, open each application and select **Wake Up**, then verify
|
||||
both public endpoints and record the UTC check time. Hibernation preserves the
|
||||
deployment configuration but stops the Q5 in-container collector.
|
||||
|
||||
## Verified platform information
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ duplicate captures are not included.
|
||||
|---:|---|---|
|
||||
| 01 | `01-ghaymah-project.png` | Authenticated Ghaymah project and deployed applications. |
|
||||
| 02 | `02-q1-service.png` | Q1 API application in Ghaymah. |
|
||||
| 03 | `03-q1-health.png` | Live HTTP 200 health response with release identifier. |
|
||||
| 03 | `03-q1-health.png` | HTTP 200 health response with release identifier, recorded 2026-07-26 18:48:19 UTC. |
|
||||
| 04 | `04-q1-dashboard.png` | Q1 status, latency, request count, uptime, and history. |
|
||||
| 05 | `05-q5-service.png` | Q5 monitoring application in Ghaymah. |
|
||||
| 06 | `06-q5-dashboard.png` | Q5 uptime, latency chart, SSL, DNS/search metrics, and recent checks. |
|
||||
@@ -18,5 +18,10 @@ duplicate captures are not included.
|
||||
`health-response.html` records the URL, UTC check time, HTTP status, and JSON
|
||||
used for screenshot 03.
|
||||
|
||||
Operational status was rechecked after waking both free-tier applications:
|
||||
Q1 `/health` and Q5 `/index.html` returned HTTP 200 at
|
||||
**2026-07-26 20:46:07 UTC**. Q5's ephemeral in-container metric history restarted
|
||||
at **2026-07-26 20:45:34.818046 UTC** and resumed its one-minute growth cadence.
|
||||
|
||||
All service URLs and Docker image names are also listed in the repository root
|
||||
[README](../../README.md).
|
||||
|
||||
+43
-10
@@ -25,11 +25,25 @@ q1-deploy-monitor/
|
||||
|
||||
| Method | Path | Response |
|
||||
|---|---|---|
|
||||
| `GET` | `/` | service name, version, endpoint list, start time |
|
||||
| `GET` | `/health` | `{"status":"ok","uptime_s":12.34,"timestamp":"2026-07-26T15:36:50.283265+00:00"}` — HTTP 200 |
|
||||
| `GET` | `/` | service name, version, `release`, endpoint list, start time |
|
||||
| `GET` | `/health` | `{"status":"ok","release":"<git-commit-sha>","uptime_s":12.34,"timestamp":"2026-07-26T20:46:06.847603+00:00"}` — HTTP 200 |
|
||||
| `GET` | `/metrics` | `{"requests_total":42,"started_at":"<iso8601>"}` |
|
||||
| `GET` | `/docs` | interactive OpenAPI docs (FastAPI built-in) |
|
||||
|
||||
Example `/` response:
|
||||
|
||||
```json
|
||||
{
|
||||
"service": "ghaymah-api",
|
||||
"version": "1.0.0",
|
||||
"release": "60b9e2c38e3f1a509383c6d61a7f76b9d043214a",
|
||||
"message": "Ghaymah deployment demo API",
|
||||
"endpoints": ["/", "/health", "/metrics", "/docs"],
|
||||
"started_at": "2026-07-26T20:46:03.315313+00:00",
|
||||
"timestamp": "2026-07-26T20:52:18.907569+00:00"
|
||||
}
|
||||
```
|
||||
|
||||
`/health` performs **no** downstream checks (no DB, no network) — it reports
|
||||
process liveness only, so a failing check always means "restart me", which is
|
||||
exactly the signal an orchestrator's health probe should act on.
|
||||
@@ -135,7 +149,7 @@ anonymously from the URL you paste in.
|
||||
Then click **Deploy**. Once the deployment reports as running, Ghaymah assigns the
|
||||
service a public URL.
|
||||
|
||||
### 3.3 Verify the live deployment
|
||||
### 3.3 Verify the deployment
|
||||
|
||||
```bash
|
||||
curl -f https://ghaymah-api-615e99f13665.hosted.ghaymah.systems/health
|
||||
@@ -173,6 +187,21 @@ Every 30 seconds it issues `GET $APP_URL/health` with a 5 s timeout, then
|
||||
`GET $APP_URL/metrics`, and appends one record to `monitor/data/checks.json`
|
||||
(a JSON array, created on first run):
|
||||
|
||||
The current `/health` response shape is:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"release": "60b9e2c38e3f1a509383c6d61a7f76b9d043214a",
|
||||
"uptime_s": 3.532,
|
||||
"timestamp": "2026-07-26T20:46:06.847603+00:00"
|
||||
}
|
||||
```
|
||||
|
||||
The monitor converts that response into the following derived history record.
|
||||
It intentionally stores status, HTTP code, latency, and request count rather
|
||||
than duplicating every response field:
|
||||
|
||||
```json
|
||||
{
|
||||
"ts": "2026-07-26T15:37:29.886772+00:00",
|
||||
@@ -257,7 +286,7 @@ const DATA_URL = '../monitor/data/checks.json';
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `pip install -r requirements.txt` (pinned versions) | fastapi 0.115.6, uvicorn 0.34.0 installed cleanly |
|
||||
| `GET /health` | HTTP 200 · `{"status":"ok","uptime_s":7.797,"timestamp":"..."}` |
|
||||
| `GET /health` | HTTP 200 · `{"status":"ok","release":"local","uptime_s":7.797,"timestamp":"..."}` |
|
||||
| `GET /` and `GET /metrics` | valid JSON; `requests_total` increments per request |
|
||||
| `monitor.py --once` against the running app | `UP code=200 latency=45.8ms requests=5`, exit 0 |
|
||||
| `monitor.py` loop across an app shutdown | up records → down records → `ALERT` printed on the 3rd consecutive failure |
|
||||
@@ -266,9 +295,9 @@ const DATA_URL = '../monitor/data/checks.json';
|
||||
| `docker build` | image builds clean (249 MB) |
|
||||
| `docker run` | container reports `(healthy)` via the `HEALTHCHECK`; all three endpoints respond |
|
||||
|
||||
### Against the live Ghaymah deployment
|
||||
### Against the Ghaymah deployment
|
||||
|
||||
Final release checked on 2026-07-26 at 18:48 UTC:
|
||||
Final release checked on 2026-07-26 at 20:46:07 UTC:
|
||||
|
||||
| Endpoint | Result |
|
||||
|---|---|
|
||||
@@ -276,7 +305,11 @@ Final release checked on 2026-07-26 at 18:48 UTC:
|
||||
| `/` | Service metadata and the same immutable release identifier |
|
||||
| `/metrics` | Valid request counter and process start time; the counter reflects real monitor traffic |
|
||||
|
||||
The monitor polls the live URL every 30 seconds. The committed
|
||||
`monitor/data/checks.json` is a submission snapshot; ongoing checks continue in
|
||||
the `ghaymah-q1-monitor-history` Docker volume so normal monitoring does not
|
||||
dirty the repository.
|
||||
After the endpoint returned HTTP 200 at 2026-07-26 20:46:07 UTC, the monitor was
|
||||
started with `DATA_FILE=/data/checks.json`, where `/data` is a read-write bind to
|
||||
the repository's `monitor/data` directory. Its `/monitor` code bind remains
|
||||
read-only. This lets atomic JSON updates append directly to the submitted
|
||||
history without allowing the container to alter its monitor source code. The
|
||||
older `ghaymah-q1-monitor-history` volume is deliberately not copied into the
|
||||
repository because it contains checks made while the free-tier service was
|
||||
hibernated.
|
||||
|
||||
@@ -1433,5 +1433,82 @@
|
||||
"code": 200,
|
||||
"latency_ms": 881.37,
|
||||
"requests": 77
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-26T20:47:19.374072+00:00",
|
||||
"status": "up",
|
||||
"code": 200,
|
||||
"latency_ms": 879.36,
|
||||
"requests": 21
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-26T20:47:50.990356+00:00",
|
||||
"status": "up",
|
||||
"code": 200,
|
||||
"latency_ms": 815.81,
|
||||
"requests": 23
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-26T20:48:22.486604+00:00",
|
||||
"status": "up",
|
||||
"code": 200,
|
||||
"latency_ms": 837.14,
|
||||
"requests": 25
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-26T20:48:55.259585+00:00",
|
||||
"status": "up",
|
||||
"code": 200,
|
||||
"latency_ms": 776.02,
|
||||
"requests": 27
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-26T20:49:26.925213+00:00",
|
||||
"status": "up",
|
||||
"code": 200,
|
||||
"latency_ms": 801.65,
|
||||
"requests": 29
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-26T20:49:58.707677+00:00",
|
||||
"status": "up",
|
||||
"code": 200,
|
||||
"latency_ms": 913.24,
|
||||
"requests": 31
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-26T20:50:29.887394+00:00",
|
||||
"status": "up",
|
||||
"code": 200,
|
||||
"latency_ms": 692.46,
|
||||
"requests": 33
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-26T20:51:02.112889+00:00",
|
||||
"status": "up",
|
||||
"code": 200,
|
||||
"latency_ms": 1722.28,
|
||||
"requests": 35
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-26T20:51:33.111656+00:00",
|
||||
"status": "up",
|
||||
"code": 200,
|
||||
"latency_ms": 481.18,
|
||||
"requests": 37
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-26T20:52:04.126187+00:00",
|
||||
"status": "up",
|
||||
"code": 200,
|
||||
"latency_ms": 511.67,
|
||||
"requests": 39
|
||||
},
|
||||
{
|
||||
"ts": "2026-07-26T20:52:35.120780+00:00",
|
||||
"status": "up",
|
||||
"code": 200,
|
||||
"latency_ms": 472.67,
|
||||
"requests": 43
|
||||
}
|
||||
]
|
||||
|
||||
+3
-3
@@ -21,7 +21,7 @@ flowchart LR
|
||||
```
|
||||
|
||||
The free Ghaymah account used for this assessment permits five total resources.
|
||||
The two required live applications (`ghaymah-api` and `mithal-monitor`) already
|
||||
The two required deployed applications (`ghaymah-api` and `mithal-monitor`) already
|
||||
consume the available allocation together with their supporting resources. A
|
||||
third Ghaymah application was tested and rejected by the platform with the
|
||||
documented account message `Resource limit exceeded ... maximum 5 resources`.
|
||||
@@ -36,7 +36,7 @@ The workflow runs on a push to `main` and through `workflow_dispatch`. It uses:
|
||||
| Secret `DOCKERHUB_USERNAME` | Docker Hub account and image namespace. |
|
||||
| Secret `DOCKERHUB_TOKEN` | Read/write Docker Hub access token. |
|
||||
| Variable `GHAYMAH_STAGING_APP` | Display name for the ephemeral staging service. |
|
||||
| Variable `GHAYMAH_PRODUCTION_APP` | Name of the live Ghaymah application. |
|
||||
| Variable `GHAYMAH_PRODUCTION_APP` | Name of the deployed Ghaymah application. |
|
||||
| Variable `GHAYMAH_PRODUCTION_URL` | Public base URL used for the production verification. |
|
||||
|
||||
Images are published as:
|
||||
@@ -136,7 +136,7 @@ bash scripts/verify_deployment.sh \
|
||||
--expected-release "$GITHUB_SHA"
|
||||
```
|
||||
|
||||
It retries `/health`, requires `status=ok`, compares the live `release` to the
|
||||
It retries `/health`, requires `status=ok`, compares the deployed `release` to the
|
||||
approved SHA, writes evidence to the GitHub job summary, and exits nonzero for
|
||||
an unavailable or stale deployment.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Q5 — Monitoring dashboard for mithal.space
|
||||
|
||||
A single container that continuously measures the availability and performance of
|
||||
**https://mithal.space** and serves a live dashboard of the results on port 8080.
|
||||
**https://mithal.space** and serves a monitoring dashboard of the results on port 8080.
|
||||
|
||||
```
|
||||
q5-mithal-dashboard/
|
||||
@@ -188,7 +188,7 @@ Then, in the Ghaymah dashboard:
|
||||
| Environment Variables | *(optional)* `INTERVAL_S=60`, `TARGET_URL=https://mithal.space`, `SEARCH_URL=https://mithal.space/search?q=test` |
|
||||
|
||||
Click **Deploy**, then open `https://mithal-monitor-292f00f076b1.hosted.ghaymah.systems/index.html` and
|
||||
screenshot the live dashboard. Leave it running so the 24 h uptime tile and the
|
||||
screenshot the deployed dashboard. Leave it running so the 24 h uptime tile and the
|
||||
hourly chart fill with real history before submission.
|
||||
|
||||
---
|
||||
@@ -207,7 +207,7 @@ Measured against the live site on 2026-07-26:
|
||||
| `docker build` | image builds clean (200 MB) |
|
||||
| `docker run` | `start.sh` launches the collector and the static server; container reports `(healthy)`; the collector wrote records into the served directory and the dashboard rendered them |
|
||||
|
||||
### Against the live Ghaymah deployment
|
||||
### Against the Ghaymah deployment
|
||||
|
||||
Checked on 2026-07-26 at 17:23 UTC, ~30 minutes after deployment:
|
||||
|
||||
@@ -223,3 +223,17 @@ storage or second service involved.
|
||||
|
||||
The `dashboard/data/metrics.json` in this repo holds those first real samples; the
|
||||
collector prunes anything older than 48 h automatically.
|
||||
|
||||
### Post-hibernation collection
|
||||
|
||||
The application was woken for final submission verification. `/index.html`
|
||||
returned HTTP 200 at **2026-07-26 20:46:07 UTC**. The collector's new
|
||||
in-container history began at **2026-07-26 20:45:34.818046 UTC** and grew from
|
||||
3 to 4 records by **2026-07-26 20:48:44 UTC**, confirming the one-record-per-minute
|
||||
loop resumed.
|
||||
|
||||
This history restarted at wake time because `/app/dashboard/data/metrics.json`
|
||||
is on the container's ephemeral filesystem, not an attached volume. Hibernation
|
||||
and restart therefore demonstrate the persistence boundary described in Q4:
|
||||
production history that must survive container replacement belongs on Ghaymah
|
||||
Block Storage or another external durable store.
|
||||
|
||||
Reference in New Issue
Block a user