# Lightweight SIEM Deployment Guide on Ghaymah Cloud Infrastructure

## Overview

This guide explains how to deploy the `siem.py` log analyzer and `dashboard.html` on Ghaymah cloud infrastructure, using Block Storage to persist logs and alert data.

---

## Architecture

```mermaid
graph TD
    BS["🗄️ Block Storage Volume\n/mnt/logs"]
    BS --> AL["/mnt/logs/auth.log"]
    BS --> NL["/mnt/logs/nginx/access.log"]
    BS --> AP["/mnt/logs/app/application.log"]

    AL & NL & AP --> SIEM["⚙️ siem.py\nsystemd service"]

    SIEM -->|"writes every 30s"| AJ["📄 alerts.json\n/var/www/siem/"]

    AJ -->|"served by Nginx"| DASH["🖥️ dashboard.html\nNear Real-Time UI"]
```

---

## Step 1: Create and Attach Block Storage

In the Ghaymah Cloud Console:

1. Go to **Storage → Block Storage → Create Volume**
2. Set name: `siem-logs-volume`
3. Set size: `50 GB` (adjust based on log retention needs)
4. Attach to your cloud instance

Then format and mount the volume:

```bash
# Find the new volume (usually /dev/sdb or /dev/vdb)
lsblk

# Format the volume
sudo mkfs.ext4 /dev/sdb

# Create mount point
sudo mkdir -p /mnt/logs

# Mount the volume
sudo mount /dev/sdb /mnt/logs

# Persist mount across reboots — use UUID to avoid /dev/sdX changes after reboot
# First find the UUID:
sudo blkid /dev/sdb
# Then replace UUID=xxxx with the actual value shown:
echo 'UUID=xxxx-xxxx /mnt/logs ext4 defaults,nofail 0 2' | sudo tee -a /etc/fstab

# Create log directories
sudo mkdir -p /mnt/logs/nginx /mnt/logs/app
sudo chown -R $USER:$USER /mnt/logs
```

> **Encryption:** Enable volume encryption during Block Storage creation in the Ghaymah console before writing any data to the volume.

---

## Step 2: Configure Services to Write Logs to Block Storage

### Nginx
```bash
# Edit nginx.conf to write access log to block storage
sudo nano /etc/nginx/nginx.conf
# Change:
# access_log /var/log/nginx/access.log;
# To:
# access_log /mnt/logs/nginx/access.log;

sudo systemctl reload nginx
```

### Application (Docker)
```yaml
# docker-compose.yml
services:
  api:
    volumes:
      - /mnt/logs/app:/var/log/app
    environment:
      LOG_PATH: /var/log/app/application.log
```

> **Secrets management note:** For production deployments, avoid plain `.env` files on disk.
> Prefer **Kubernetes Secrets** (mounted as env vars or files) or a cloud secret manager
> (e.g., HashiCorp Vault, AWS Secrets Manager, or your cloud provider's equivalent).
> `.env` files are acceptable for local development but should never be committed to Git.

### SSH Auth Log Forwarding
```bash
# Create the auth log file with correct permissions for rsyslog
sudo touch /mnt/logs/auth.log
sudo chown syslog:adm /mnt/logs/auth.log
sudo chmod 640 /mnt/logs/auth.log

# Direct rsyslog to write auth logs to block storage
echo 'auth,authpriv.* /mnt/logs/auth.log' | sudo tee /etc/rsyslog.d/99-siem.conf
sudo systemctl restart rsyslog
```

---

## Step 3: Deploy siem.py as a Systemd Service

```bash
# Install Python 3
sudo apt-get update && sudo apt-get install -y python3

# Create a shared group so both siem.py (ubuntu) and nginx (www-data) can access alerts.json
sudo groupadd siem
sudo usermod -aG siem ubuntu
sudo usermod -aG siem www-data

# Copy SIEM files
sudo mkdir -p /opt/siem /var/www/siem
sudo cp q4-siem/siem.py /opt/siem/siem.py
sudo cp q4-siem/dashboard.html /var/www/siem/dashboard.html

# Set permissions: ubuntu writes, www-data reads via shared siem group
sudo touch /var/www/siem/alerts.json
sudo chown ubuntu:siem /var/www/siem/alerts.json
sudo chmod 640 /var/www/siem/alerts.json

# Edit log paths in siem.py to match block storage
sudo nano /opt/siem/siem.py
# Set:
# LOG_SOURCES = {
#     "auth":  "/mnt/logs/auth.log",
#     "nginx": "/mnt/logs/nginx/access.log",
#     "app":   "/mnt/logs/app/application.log",
# }
# OUTPUT_FILE = "/var/www/siem/alerts.json"
```

Create the systemd service:

```ini
# /etc/systemd/system/siem.service
[Unit]
Description=Ghaymah SIEM Log Analyzer
After=network.target

[Service]
Type=simple
User=ubuntu
WorkingDirectory=/opt/siem
ExecStart=/usr/bin/python3 /opt/siem/siem.py
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
```

```bash
sudo systemctl daemon-reload
sudo systemctl enable siem
sudo systemctl start siem
sudo systemctl status siem
```

---

## Step 4: Serve the Dashboard via Nginx

```nginx
# /etc/nginx/sites-available/siem
server {
    listen 8443 ssl;
    server_name siem.ghaymah.systems;

    ssl_certificate     /etc/letsencrypt/live/siem.ghaymah.systems/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/siem.ghaymah.systems/privkey.pem;

    root /var/www/siem;
    index dashboard.html;

    # Restrict access to internal network only
    allow 10.0.0.0/8;
    allow 192.168.0.0/16;
    deny all;

    # alerts.json is served to the dashboard browser client.
    # The parent allow/deny block already restricts access to internal IPs only.
    location = /alerts.json {
        allow 10.0.0.0/8;
        allow 192.168.0.0/16;
        deny all;
        add_header Content-Type application/json;
    }

    location / {
        try_files $uri $uri/ =404;
    }
}
```

```bash
sudo ln -s /etc/nginx/sites-available/siem /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
```

---

## Step 5: Log Rotation

```bash
# /etc/logrotate.d/siem-logs
/mnt/logs/*.log
/mnt/logs/app/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 640 syslog adm
    postrotate
        systemctl restart rsyslog
    endscript
}

/mnt/logs/nginx/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 640 www-data adm
    postrotate
        systemctl reload nginx
    endscript
}
```

---

## Verification

```bash
# Check SIEM is running
sudo systemctl status siem

# Check alerts output
cat /var/www/siem/alerts.json | python3 -m json.tool | head -30

# View SIEM logs
sudo journalctl -u siem -f

# Test dashboard
curl -k https://siem.ghaymah.systems:8443/dashboard.html
```

---

## Security Notes

| Control | Implementation |
|---|---|
| Dashboard access | Restricted to internal IP range (10.0.0.0/8) |
| HTTPS | TLS required, no HTTP access |
| `alerts.json` access | Internal IPs only via `allow 10.0.0.0/8; deny all;` inside `location = /alerts.json` |
| File permissions | `alerts.json` owned `ubuntu:siem`, mode `640` — owner writes, group (nginx) reads, others none |
| Block storage | Encrypted at rest (enable during volume creation in Ghaymah console) |
| Audit logs | All SIEM activity written to systemd journal |

---

## Future Integrations

The SIEM is designed to grow. The following integrations can be added as next steps:

| Integration | How |
|---|---|
| **AlertManager webhook** | Extend `siem.py` to POST critical alerts to a webhook endpoint. Pairs directly with the Prometheus alert rules in `q2-attack-simulation/alert-rule.yml`. |
| **Slack notifications** | Send alerts to a `#security-alerts` channel via Slack Incoming Webhooks when severity is CRITICAL or HIGH. |
| **PagerDuty / on-call** | Route CRITICAL alerts to PagerDuty for 24/7 on-call escalation using the Events API. |
| **Email alerts** | Use `smtplib` in `siem.py` to send summary emails to the security team at the end of each scan cycle. |
| **Cloud monitoring export** | Ship `alerts.json` to a cloud-native monitoring service (e.g., Ghaymah monitoring, Datadog, or CloudWatch) for long-term trend analysis and dashboards. |
| **Log shipping to SIEM platform** | Forward structured logs to a dedicated SIEM platform (e.g., Elastic/OpenSearch) for retention and correlation across multiple services. |

---

## Backup Strategy

Block Storage volumes are the persistence layer for all logs and alert data. Schedule regular snapshots to protect against data loss.

**Recommended snapshot schedule:**

| Frequency | Retention | Purpose |
|---|---|---|
| Daily | 7 days | Short-term recovery from accidental deletion or corruption |
| Weekly | 4 weeks | Recovery from persistent misconfiguration issues |
| Monthly | 3 months | Compliance and long-term trend analysis |

**How to enable in Ghaymah console:**

1. Go to **Storage → Block Storage → `siem-logs-volume`**
2. Navigate to **Snapshots → Create Snapshot Policy**
3. Set a daily schedule with 7-day retention
4. Enable email notification on snapshot failure

**Restore procedure (monthly test recommended):**

```bash
# Restore a snapshot to a new volume in the Ghaymah console,
# then mount it temporarily to verify data integrity:
sudo mkdir -p /mnt/logs-restore
sudo mount /dev/sdc /mnt/logs-restore
ls -lh /mnt/logs-restore/
sudo umount /mnt/logs-restore
```

> Restore tests should be performed monthly to confirm backups are valid and recovery time is within acceptable limits.