Add CI/CD pipeline, architecture documentation, and monitoring application

- Created a GitHub Actions workflow for CI/CD to deploy to Ghyamah, including testing, building, and pushing Docker images.
- Added architecture design document for handling 15,000 requests per second, detailing system components, capacity planning, and cold start strategies.
- Introduced a Python-based uptime/latency/SSL monitor with a static dashboard, utilizing standard libraries only.
- Included Dockerfile and entrypoint script for the monitoring application, ensuring it runs as a non-root user and handles process management.
- Added a .dockerignore file to exclude unnecessary files from the Docker build context.
- Created an HTML dashboard for visualizing monitoring metrics, including uptime, latency, and SSL certificate status.
هذا الالتزام موجود في:
2026-07-27 23:08:39 +03:00
الأصل c3368b9124
التزام 439a31ba93
16 ملفات معدلة مع 1753 إضافات و0 حذوفات

6
Q5/.dockerignore Normal file
عرض الملف

@@ -0,0 +1,6 @@
data/
*.pyc
__pycache__/
.git
.gitignore
README.md

40
Q5/Dockerfile Normal file
عرض الملف

@@ -0,0 +1,40 @@
# syntax=docker/dockerfile:1
FROM python:3.12-alpine
# Standard-library only — no requirements.txt needed. ca-certificates is
# required so ssl.create_default_context() can validate the target's
# certificate chain when checking SSL expiry.
RUN apk add --no-cache ca-certificates && update-ca-certificates
# Run as a non-root user
RUN addgroup -S monitor && adduser -S monitor -G monitor
WORKDIR /app
COPY monitor.py /app/monitor.py
COPY Entrypoint.sh /app/entrypoint.sh
COPY index.html /app/web/index.html
RUN chmod +x /app/entrypoint.sh \
&& mkdir -p /app/web/data \
&& chown -R monitor:monitor /app
USER monitor
# Defaults — override any of these at `docker run` / platform deploy time.
ENV TARGET_URL="https://mithal.space" \
SEARCH_PATH="/search?q=test" \
CHECK_INTERVAL=60 \
RETENTION_HOURS=24 \
REQUEST_TIMEOUT=10 \
DATA_FILE="/app/web/data/metrics.json" \
WEB_DIR="/app/web" \
PORT=8080
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
CMD python3 -c "import urllib.request,sys; sys.exit(0) if urllib.request.urlopen('http://127.0.0.1:${PORT}/', timeout=3).status==200 else sys.exit(1)"
ENTRYPOINT ["/app/entrypoint.sh"]

43
Q5/Entrypoint.sh Normal file
عرض الملف

@@ -0,0 +1,43 @@
#!/bin/sh
# entrypoint.sh — runs the Python monitoring loop and a static HTTP server
# side by side in a single container. POSIX sh so it works on Alpine's
# default shell.
set -eu
PORT="${PORT:-8080}"
WEB_DIR="${WEB_DIR:-/app/web}"
echo "[entrypoint] starting monitor loop"
python3 /app/monitor.py &
MONITOR_PID=$!
echo "[entrypoint] serving dashboard from ${WEB_DIR} on 0.0.0.0:${PORT}"
cd "${WEB_DIR}"
python3 -m http.server "${PORT}" --bind 0.0.0.0 &
SERVER_PID=$!
# Forward termination signals to both children and wait for them so the
# container shuts down cleanly (e.g. on `docker stop` / platform redeploys).
term_handler() {
echo "[entrypoint] shutting down..."
kill -TERM "$MONITOR_PID" "$SERVER_PID" 2>/dev/null || true
wait "$MONITOR_PID" "$SERVER_PID" 2>/dev/null || true
exit 0
}
trap term_handler TERM INT
# busybox ash (Alpine's /bin/sh) has no `wait -n`, so poll instead: if
# either child dies unexpectedly, bring the whole container down so the
# orchestrator (Docker/Kubernetes/Cloud Run/etc.) can restart it.
while true; do
if ! kill -0 "$MONITOR_PID" 2>/dev/null; then
echo "[entrypoint] monitor loop exited unexpectedly — stopping container"
term_handler
fi
if ! kill -0 "$SERVER_PID" 2>/dev/null; then
echo "[entrypoint] http server exited unexpectedly — stopping container"
term_handler
fi
sleep 2
done

80
Q5/README.md Normal file
عرض الملف

@@ -0,0 +1,80 @@
# mithal-space-monitor
A lightweight, stdlib-only uptime/latency/SSL monitor with a single-page
Chart.js dashboard, packaged into one container.
## What's inside
| File | Purpose |
|---|---|
| `monitor.py` | Standard-library-only Python script. Every `CHECK_INTERVAL` seconds it checks DNS resolution time, HTTP status + latency, search-endpoint latency, and SSL cert expiry, then writes a rolling `RETENTION_HOURS` window to a JSON file. |
| `index.html` | Single-page dashboard (HTML/CSS/JS + Chart.js via CDN). Polls the JSON file every 30s and renders 24h uptime %, a 60-minute latency line chart, SSL expiry, and a table of the last 10 checks. |
| `entrypoint.sh` | Starts `monitor.py` and `python -m http.server` side by side, forwards signals, and exits the container if either process dies (so the orchestrator restarts it). |
| `Dockerfile` | `python:3.12-alpine` base, non-root user, healthcheck, no external Python deps. |
## Configuration (environment variables)
| Variable | Default | Description |
|---|---|---|
| `TARGET_URL` | `https://mithal.space` | URL to monitor |
| `SEARCH_PATH` | `/search?q=test` | Path appended to the target's origin for the search-latency check |
| `CHECK_INTERVAL` | `60` | Seconds between checks |
| `RETENTION_HOURS` | `24` | Rolling window kept in the JSON log |
| `REQUEST_TIMEOUT` | `10` | Per-request timeout (seconds) |
| `PORT` | `8080` | Dashboard HTTP server port |
## Build & run locally
```bash
docker build -t mithal-space-monitor .
docker run -d \
--name mithal-monitor \
-p 8080:8080 \
-e TARGET_URL="https://mithal.space" \
-e SEARCH_PATH="/search?q=test" \
-v mithal_monitor_data:/app/data \
mithal-space-monitor
# open http://localhost:8080
```
The `-v mithal_monitor_data:/app/data` volume is optional but recommended so
your 24h history survives a container restart/redeploy.
## Push to a registry
```bash
docker tag mithal-space-monitor registry.example.com/yourorg/mithal-space-monitor:latest
docker push registry.example.com/yourorg/mithal-space-monitor:latest
```
## Deploy
This image is a single process group exposing one HTTP port, so it runs
as-is on most container platforms:
- **Cloud Run / Container Apps / Fly.io**: deploy the image, set `PORT`
to match the platform's expected port (Cloud Run injects `PORT`
automatically - the entrypoint already respects it), mount a persistent
volume if the platform supports one (otherwise history resets on redeploy,
which is fine - it just rebuilds over the next `RETENTION_HOURS`).
- **Kubernetes**: run as a `Deployment` with 1 replica, a `Service` of type
`ClusterIP`/`LoadBalancer`, and optionally a `PersistentVolumeClaim`
mounted at `/app/data`. The built-in `HEALTHCHECK` maps naturally to a
liveness probe on `GET /`.
- **Plain VM / docker-compose**: use the `docker run` command above behind
your existing reverse proxy / TLS terminator.
## Notes & extension points
- Everything in `monitor.py` uses only the Python standard library
(`urllib`, `socket`, `ssl`, `json`) - no `pip install` step, no
dependency surface in the image.
- Data is written atomically (`write → temp file → os.replace`) so the
dashboard never reads a half-written JSON file.
- To monitor multiple targets, run one container per target (each with its
own `TARGET_URL`/port), or extend `monitor.py` to loop over a list of
targets and extend `index.html` with a target selector.
- Add basic auth / IP allowlisting at your reverse proxy if the dashboard
shouldn't be public.

395
Q5/index.html Normal file
عرض الملف

@@ -0,0 +1,395 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Site Watch — Status</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link href="https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;500;600;700&family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
<script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/4.4.4/chart.umd.min.js"></script>
<style>
:root{
--bg:#0b1220;
--surface:#121b2e;
--surface-alt:#182541;
--border:#23324f;
--text:#e7ecf5;
--muted:#8a97b3;
--accent:#5eead4;
--ok:#34d399;
--warn:#fbbf24;
--down:#f87171;
--mono:'IBM Plex Mono', ui-monospace, monospace;
--sans:'Inter', system-ui, sans-serif;
}
*{box-sizing:border-box;}
html,body{margin:0;padding:0;background:var(--bg);color:var(--text);font-family:var(--sans);}
body{min-height:100vh;padding:32px 24px 64px;}
a{color:var(--accent);}
.wrap{max-width:1080px;margin:0 auto;}
/* ---- Header / pulse signature ---- */
header{
display:flex;
flex-direction:column;
gap:18px;
border:1px solid var(--border);
background:linear-gradient(180deg,var(--surface) 0%, var(--surface-alt) 100%);
border-radius:14px;
padding:22px 26px;
position:relative;
overflow:hidden;
}
.header-top{display:flex;justify-content:space-between;align-items:flex-start;flex-wrap:wrap;gap:14px;}
.brand{display:flex;flex-direction:column;gap:4px;}
.eyebrow{font-family:var(--mono);font-size:11px;letter-spacing:.14em;text-transform:uppercase;color:var(--muted);}
.target{font-family:var(--mono);font-size:22px;font-weight:600;color:var(--text);word-break:break-all;}
.status-pill{
display:inline-flex;align-items:center;gap:8px;
font-family:var(--mono);font-size:12px;font-weight:600;letter-spacing:.04em;
padding:7px 14px;border-radius:999px;border:1px solid var(--border);
background:rgba(255,255,255,0.02);white-space:nowrap;height:fit-content;
}
.dot{width:8px;height:8px;border-radius:50%;background:var(--muted);box-shadow:0 0 0 0 rgba(0,0,0,0);}
.dot.up{background:var(--ok);animation:pulse-dot 2s infinite;}
.dot.down{background:var(--down);animation:pulse-dot 1s infinite;}
@keyframes pulse-dot{
0%{box-shadow:0 0 0 0 rgba(52,211,153,.55);}
70%{box-shadow:0 0 0 8px rgba(52,211,153,0);}
100%{box-shadow:0 0 0 0 rgba(52,211,153,0);}
}
.pulse-line{width:100%;height:44px;opacity:.85;}
.pulse-line path{
fill:none;stroke:var(--accent);stroke-width:1.6;
stroke-dasharray:1200;stroke-dashoffset:1200;
animation:draw-pulse 3.2s linear infinite;
}
@keyframes draw-pulse{
0%{stroke-dashoffset:1200;}
100%{stroke-dashoffset:0;}
}
@media (prefers-reduced-motion: reduce){
.dot.up,.dot.down,.pulse-line path{animation:none;}
}
.meta-row{display:flex;gap:18px;flex-wrap:wrap;font-family:var(--mono);font-size:12px;color:var(--muted);}
.meta-row span b{color:var(--text);font-weight:600;}
/* ---- Stat cards ---- */
.stats{
display:grid;grid-template-columns:repeat(4,1fr);gap:14px;margin-top:22px;
}
@media (max-width:820px){.stats{grid-template-columns:repeat(2,1fr);}}
.stat-card{
background:var(--surface);border:1px solid var(--border);border-radius:12px;
padding:16px 18px;display:flex;flex-direction:column;gap:6px;
}
.stat-label{font-family:var(--mono);font-size:11px;text-transform:uppercase;letter-spacing:.1em;color:var(--muted);}
.stat-value{font-family:var(--mono);font-size:28px;font-weight:700;line-height:1.1;}
.stat-sub{font-size:12px;color:var(--muted);}
.stat-value.ok{color:var(--ok);}
.stat-value.warn{color:var(--warn);}
.stat-value.down{color:var(--down);}
/* ---- Chart panel ---- */
.panel{
margin-top:22px;background:var(--surface);border:1px solid var(--border);
border-radius:12px;padding:20px 22px;
}
.panel-title{display:flex;justify-content:space-between;align-items:baseline;margin-bottom:14px;flex-wrap:wrap;gap:8px;}
.panel-title h2{font-size:14px;margin:0;font-family:var(--mono);text-transform:uppercase;letter-spacing:.08em;color:var(--text);}
.panel-title .hint{font-size:12px;color:var(--muted);}
.chart-holder{height:280px;position:relative;}
/* ---- Table ---- */
table{width:100%;border-collapse:collapse;font-family:var(--mono);font-size:12.5px;}
thead th{
text-align:left;color:var(--muted);font-weight:600;text-transform:uppercase;
font-size:10.5px;letter-spacing:.08em;padding:8px 10px;border-bottom:1px solid var(--border);
}
tbody td{padding:9px 10px;border-bottom:1px solid rgba(255,255,255,0.04);color:var(--text);}
tbody tr:hover{background:rgba(255,255,255,0.02);}
.badge{
display:inline-block;padding:2px 8px;border-radius:6px;font-size:11px;font-weight:600;
}
.badge.ok{background:rgba(52,211,153,.12);color:var(--ok);}
.badge.down{background:rgba(248,113,113,.12);color:var(--down);}
footer{margin-top:26px;text-align:center;font-family:var(--mono);font-size:11px;color:var(--muted);}
.empty{color:var(--muted);font-size:13px;padding:20px 0;text-align:center;}
</style>
</head>
<body>
<div class="wrap">
<header>
<div class="header-top">
<div class="brand">
<span class="eyebrow">Uptime &amp; Performance</span>
<span class="target" id="targetUrl"></span>
</div>
<div class="status-pill">
<span class="dot" id="statusDot"></span>
<span id="statusText">CHECKING…</span>
</div>
</div>
<svg class="pulse-line" viewBox="0 0 600 44" preserveAspectRatio="none">
<path d="M0,22 L120,22 L140,6 L160,38 L180,22 L260,22 L280,10 L300,34 L320,22 L600,22"/>
</svg>
<div class="meta-row">
<span>Checks every <b id="metaInterval"></b></span>
<span>Window <b id="metaRetention"></b></span>
<span>Last check <b id="metaUpdated"></b></span>
</div>
</header>
<div class="stats">
<div class="stat-card">
<span class="stat-label">Uptime · 24h</span>
<span class="stat-value" id="statUptime"></span>
<span class="stat-sub" id="statUptimeSub">— checks recorded</span>
</div>
<div class="stat-card">
<span class="stat-label">Latency · latest</span>
<span class="stat-value" id="statLatency"></span>
<span class="stat-sub" id="statLatencySub">HTTP response time</span>
</div>
<div class="stat-card">
<span class="stat-label">DNS resolve</span>
<span class="stat-value" id="statDns"></span>
<span class="stat-sub">Latest lookup time</span>
</div>
<div class="stat-card">
<span class="stat-label">SSL expires in</span>
<span class="stat-value" id="statSsl"></span>
<span class="stat-sub" id="statSslSub">Certificate validity</span>
</div>
</div>
<div class="panel">
<div class="panel-title">
<h2>Latency — last 60 minutes</h2>
<span class="hint" id="chartHint">site vs. search endpoint, ms</span>
</div>
<div class="chart-holder"><canvas id="latencyChart"></canvas></div>
</div>
<div class="panel">
<div class="panel-title">
<h2>Recent checks</h2>
<span class="hint">last 10</span>
</div>
<table>
<thead>
<tr>
<th>Time (UTC)</th>
<th>Status</th>
<th>Latency</th>
<th>DNS</th>
<th>Search</th>
<th>SSL days</th>
</tr>
</thead>
<tbody id="checksBody">
<tr><td colspan="6" class="empty">Waiting for first data…</td></tr>
</tbody>
</table>
</div>
<footer>Reads <code>data/metrics.json</code> · refreshes every 30s</footer>
</div>
<script>
const DATA_URL = 'data/metrics.json';
const REFRESH_MS = 30000;
let chart;
function fmtMs(v){ return (v === null || v === undefined) ? '—' : Math.round(v) + ' ms'; }
function fmtDays(v){ return (v === null || v === undefined) ? '—' : v + 'd'; }
function fmtTime(iso){
try{
const d = new Date(iso);
return d.toISOString().substr(11,8);
}catch(e){ return iso; }
}
function classifySsl(days){
if(days === null || days === undefined) return '';
if(days < 7) return 'down';
if(days < 21) return 'warn';
return 'ok';
}
function renderStats(payload){
const checks = payload.checks || [];
document.getElementById('targetUrl').textContent = payload.target || '—';
document.getElementById('metaInterval').textContent = (payload.check_interval_seconds || '—') + 's';
document.getElementById('metaRetention').textContent = (payload.retention_hours || '—') + 'h';
document.getElementById('metaUpdated').textContent = payload.updated_at ? fmtTime(payload.updated_at) + ' UTC' : '—';
if(checks.length === 0){
document.getElementById('statusText').textContent = 'NO DATA';
return;
}
const latest = checks[checks.length - 1];
const successCount = checks.filter(c => c.success).length;
const uptimePct = (successCount / checks.length * 100);
// Status pill
const dot = document.getElementById('statusDot');
const statusText = document.getElementById('statusText');
if(latest.success){
dot.className = 'dot up';
statusText.textContent = `UP · HTTP ${latest.http_status}`;
} else {
dot.className = 'dot down';
statusText.textContent = latest.http_status ? `DEGRADED · HTTP ${latest.http_status}` : 'DOWN';
}
// Uptime stat
const uptimeEl = document.getElementById('statUptime');
uptimeEl.textContent = uptimePct.toFixed(2) + '%';
uptimeEl.className = 'stat-value ' + (uptimePct >= 99.5 ? 'ok' : uptimePct >= 97 ? 'warn' : 'down');
document.getElementById('statUptimeSub').textContent = `${successCount}/${checks.length} checks succeeded`;
// Latency stat
const latEl = document.getElementById('statLatency');
latEl.textContent = fmtMs(latest.latency_ms);
latEl.className = 'stat-value ' + (latest.latency_ms == null ? 'down' : latest.latency_ms < 500 ? 'ok' : latest.latency_ms < 1500 ? 'warn' : 'down');
// DNS stat
document.getElementById('statDns').textContent = fmtMs(latest.dns_ms);
// SSL stat
const sslEl = document.getElementById('statSsl');
sslEl.textContent = fmtDays(latest.ssl_days_remaining);
sslEl.className = 'stat-value ' + classifySsl(latest.ssl_days_remaining);
document.getElementById('statSslSub').textContent = latest.ssl_days_remaining != null
? 'Certificate validity' : 'Not monitored over HTTPS';
}
function renderChart(payload){
const checks = payload.checks || [];
const cutoff = Date.now() - 60 * 60 * 1000;
const recent = checks.filter(c => new Date(c.timestamp).getTime() >= cutoff);
const source = recent.length > 0 ? recent : checks.slice(-60);
const labels = source.map(c => fmtTime(c.timestamp));
const siteData = source.map(c => c.latency_ms);
const searchData = source.map(c => c.search_latency_ms);
const hasSearch = searchData.some(v => v !== null && v !== undefined);
document.getElementById('chartHint').textContent = hasSearch
? 'site vs. search endpoint, ms' : 'site response time, ms';
const datasets = [{
label: 'Site',
data: siteData,
borderColor: '#5eead4',
backgroundColor: 'rgba(94,234,212,0.12)',
borderWidth: 2,
pointRadius: 0,
tension: 0.25,
fill: true,
}];
if(hasSearch){
datasets.push({
label: 'Search endpoint',
data: searchData,
borderColor: '#fbbf24',
backgroundColor: 'transparent',
borderWidth: 2,
borderDash: [4,3],
pointRadius: 0,
tension: 0.25,
});
}
const cfg = {
type: 'line',
data: { labels, datasets },
options: {
responsive: true,
maintainAspectRatio: false,
interaction: { mode: 'index', intersect: false },
scales: {
x: {
ticks: { color: '#8a97b3', maxTicksLimit: 8, font: { family: 'IBM Plex Mono', size: 10 } },
grid: { color: 'rgba(255,255,255,0.04)' },
},
y: {
beginAtZero: true,
ticks: { color: '#8a97b3', font: { family: 'IBM Plex Mono', size: 10 } },
grid: { color: 'rgba(255,255,255,0.06)' },
title: { display: true, text: 'ms', color: '#8a97b3', font: { size: 11 } },
},
},
plugins: {
legend: {
display: hasSearch,
labels: { color: '#e7ecf5', font: { family: 'IBM Plex Mono', size: 11 }, boxWidth: 12 },
},
tooltip: {
backgroundColor: '#182541',
borderColor: '#23324f',
borderWidth: 1,
titleFont: { family: 'IBM Plex Mono' },
bodyFont: { family: 'IBM Plex Mono' },
},
},
},
};
if(chart){
chart.data = cfg.data;
chart.update('none');
} else {
chart = new Chart(document.getElementById('latencyChart'), cfg);
}
}
function renderTable(payload){
const checks = (payload.checks || []).slice(-10).reverse();
const body = document.getElementById('checksBody');
if(checks.length === 0){
body.innerHTML = '<tr><td colspan="6" class="empty">Waiting for first data…</td></tr>';
return;
}
body.innerHTML = checks.map(c => `
<tr>
<td>${fmtTime(c.timestamp)}</td>
<td><span class="badge ${c.success ? 'ok' : 'down'}">${c.http_status ?? 'ERR'}</span></td>
<td>${fmtMs(c.latency_ms)}</td>
<td>${fmtMs(c.dns_ms)}</td>
<td>${fmtMs(c.search_latency_ms)}</td>
<td>${fmtDays(c.ssl_days_remaining)}</td>
</tr>
`).join('');
}
async function refresh(){
try{
const res = await fetch(DATA_URL + '?t=' + Date.now(), { cache: 'no-store' });
if(!res.ok) throw new Error('HTTP ' + res.status);
const payload = await res.json();
renderStats(payload);
renderChart(payload);
renderTable(payload);
}catch(err){
document.getElementById('statusText').textContent = 'DASHBOARD ERROR';
document.getElementById('statusDot').className = 'dot down';
console.error('Failed to load metrics:', err);
}
}
refresh();
setInterval(refresh, REFRESH_MS);
</script>
</body>
</html>

208
Q5/monitor.py Normal file
عرض الملف

@@ -0,0 +1,208 @@
#!/usr/bin/env python3
"""
monitor.py — Lightweight uptime / latency / SSL monitor.
Standard-library only (urllib, ssl, socket, json, time). Runs an infinite
loop, checks a target URL (and optionally a search endpoint) every
CHECK_INTERVAL seconds, and persists a rolling RETENTION_HOURS window of
results to a JSON file that the static dashboard reads.
Configuration is via environment variables so the same image can monitor
any site without a rebuild:
TARGET_URL Full URL to monitor (default: https://mithal.space)
SEARCH_PATH Path appended to origin for a (default: /search?q=test)
secondary "search" check. Set to "" to disable.
CHECK_INTERVAL Seconds between checks (default: 60)
RETENTION_HOURS How much history to keep (default: 24)
REQUEST_TIMEOUT Per-request timeout, seconds (default: 10)
DATA_FILE Where to write the JSON log (default: /app/web/data/metrics.json)
"""
import json
import os
import socket
import ssl
import sys
import time
import urllib.error
import urllib.request
from datetime import datetime, timezone
from urllib.parse import urlparse
# --------------------------------------------------------------------------
# Configuration
# --------------------------------------------------------------------------
TARGET_URL = os.environ.get("TARGET_URL", "https://mithal.space")
SEARCH_PATH = os.environ.get("SEARCH_PATH", "/search?q=test")
CHECK_INTERVAL = int(os.environ.get("CHECK_INTERVAL", "60"))
RETENTION_HOURS = float(os.environ.get("RETENTION_HOURS", "24"))
REQUEST_TIMEOUT = float(os.environ.get("REQUEST_TIMEOUT", "10"))
DATA_FILE = os.environ.get("DATA_FILE", "/app/web/data/metrics.json")
USER_AGENT = "uptime-monitor/1.0 (+standard-library)"
_parsed = urlparse(TARGET_URL)
HOSTNAME = _parsed.hostname
PORT = _parsed.port or (443 if _parsed.scheme == "https" else 80)
SEARCH_URL = f"{_parsed.scheme}://{_parsed.netloc}{SEARCH_PATH}" if SEARCH_PATH else None
def now_iso() -> str:
return datetime.now(timezone.utc).isoformat(timespec="seconds")
def measure_dns(hostname: str):
"""Return DNS resolution time in milliseconds, or None on failure."""
start = time.perf_counter()
try:
socket.getaddrinfo(hostname, None)
except socket.gaierror as exc:
return None, str(exc)
elapsed_ms = (time.perf_counter() - start) * 1000
return round(elapsed_ms, 2), None
def timed_get(url: str, timeout: float):
"""
Perform an HTTP GET and return (status_code, latency_ms, error_str).
latency_ms measures time-to-first-byte-of-full-response (connect + TLS
+ request + response), matching what a real visitor experiences.
"""
req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT})
start = time.perf_counter()
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
resp.read(1) # confirm the body actually starts streaming
status = resp.status
latency_ms = (time.perf_counter() - start) * 1000
return status, round(latency_ms, 2), None
except urllib.error.HTTPError as exc:
# Still a "successful" connection from a monitoring standpoint —
# the server responded, just with an error status.
latency_ms = (time.perf_counter() - start) * 1000
return exc.code, round(latency_ms, 2), None
except (urllib.error.URLError, socket.timeout, TimeoutError) as exc:
latency_ms = (time.perf_counter() - start) * 1000
return None, round(latency_ms, 2), str(getattr(exc, "reason", exc))
def measure_ssl_expiry(hostname: str, port: int, timeout: float):
"""Return (days_remaining, error_str) for the TLS certificate."""
if not hostname:
return None, "no hostname"
try:
ctx = ssl.create_default_context()
with socket.create_connection((hostname, port), timeout=timeout) as sock:
with ctx.wrap_socket(sock, server_hostname=hostname) as ssock:
cert = ssock.getpeercert()
not_after = cert.get("notAfter")
if not not_after:
return None, "no notAfter field in certificate"
expiry_dt = datetime.strptime(not_after, "%b %d %H:%M:%S %Y %Z").replace(
tzinfo=timezone.utc
)
days_remaining = (expiry_dt - datetime.now(timezone.utc)).total_seconds() / 86400
return round(days_remaining, 1), None
except Exception as exc: # noqa: BLE001 — monitoring must never crash the loop
return None, str(exc)
def run_check() -> dict:
dns_ms, dns_err = measure_dns(HOSTNAME)
status, latency_ms, http_err = timed_get(TARGET_URL, REQUEST_TIMEOUT)
search_status, search_latency_ms, search_err = None, None, None
if SEARCH_URL:
search_status, search_latency_ms, search_err = timed_get(SEARCH_URL, REQUEST_TIMEOUT)
ssl_days, ssl_err = (None, None)
if _parsed.scheme == "https":
ssl_days, ssl_err = measure_ssl_expiry(HOSTNAME, PORT, REQUEST_TIMEOUT)
success = status is not None and 200 <= status < 400
return {
"timestamp": now_iso(),
"target": TARGET_URL,
"http_status": status,
"success": success,
"latency_ms": latency_ms,
"dns_ms": dns_ms,
"search_status": search_status,
"search_latency_ms": search_latency_ms,
"ssl_days_remaining": ssl_days,
"error": http_err or dns_err or ssl_err or search_err,
}
def load_history(path: str) -> list:
try:
with open(path, "r", encoding="utf-8") as f:
data = json.load(f)
return data.get("checks", []) if isinstance(data, dict) else data
except (FileNotFoundError, json.JSONDecodeError):
return []
def prune(history: list, retention_hours: float) -> list:
cutoff = time.time() - retention_hours * 3600
pruned = []
for entry in history:
try:
ts = datetime.fromisoformat(entry["timestamp"]).timestamp()
except (KeyError, ValueError):
continue
if ts >= cutoff:
pruned.append(entry)
return pruned
def save_history(path: str, history: list) -> None:
os.makedirs(os.path.dirname(path), exist_ok=True)
payload = {
"target": TARGET_URL,
"search_url": SEARCH_URL,
"updated_at": now_iso(),
"check_interval_seconds": CHECK_INTERVAL,
"retention_hours": RETENTION_HOURS,
"checks": history,
}
tmp_path = f"{path}.tmp"
with open(tmp_path, "w", encoding="utf-8") as f:
json.dump(payload, f, indent=2)
os.replace(tmp_path, path) # atomic write so the dashboard never reads a half-written file
def main() -> None:
print(f"[monitor] target={TARGET_URL} interval={CHECK_INTERVAL}s "
f"retention={RETENTION_HOURS}h data_file={DATA_FILE}", flush=True)
history = load_history(DATA_FILE)
while True:
cycle_start = time.time()
result = run_check()
history.append(result)
history = prune(history, RETENTION_HOURS)
save_history(DATA_FILE, history)
status_str = result["http_status"] if result["http_status"] is not None else "ERR"
print(
f"[monitor] {result['timestamp']} status={status_str} "
f"latency={result['latency_ms']}ms dns={result['dns_ms']}ms "
f"ssl_days={result['ssl_days_remaining']} "
f"search_latency={result['search_latency_ms']}ms "
f"{'error=' + result['error'] if result['error'] else 'ok'}",
flush=True,
)
elapsed = time.time() - cycle_start
time.sleep(max(0, CHECK_INTERVAL - elapsed))
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
sys.exit(0)