325 أسطر
8.7 KiB
Markdown
325 أسطر
8.7 KiB
Markdown
# Lightweight SIEM Deployment Guide on Ghaymah Cloud Infrastructure
|
|
|
|
## Overview
|
|
|
|
This guide explains how to deploy the `siem.py` log analyzer and `dashboard.html` on Ghaymah cloud infrastructure, using Block Storage to persist logs and alert data.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```mermaid
|
|
graph TD
|
|
BS["🗄️ Block Storage Volume\n/mnt/logs"]
|
|
BS --> AL["/mnt/logs/auth.log"]
|
|
BS --> NL["/mnt/logs/nginx/access.log"]
|
|
BS --> AP["/mnt/logs/app/application.log"]
|
|
|
|
AL & NL & AP --> SIEM["⚙️ siem.py\nsystemd service"]
|
|
|
|
SIEM -->|"writes every 30s"| AJ["📄 alerts.json\n/var/www/siem/"]
|
|
|
|
AJ -->|"served by Nginx"| DASH["🖥️ dashboard.html\nNear Real-Time UI"]
|
|
```
|
|
|
|
---
|
|
|
|
## Step 1: Create and Attach Block Storage
|
|
|
|
In the Ghaymah Cloud Console:
|
|
|
|
1. Go to **Storage → Block Storage → Create Volume**
|
|
2. Set name: `siem-logs-volume`
|
|
3. Set size: `50 GB` (adjust based on log retention needs)
|
|
4. Attach to your cloud instance
|
|
|
|
Then format and mount the volume:
|
|
|
|
```bash
|
|
# Find the new volume (usually /dev/sdb or /dev/vdb)
|
|
lsblk
|
|
|
|
# Format the volume
|
|
sudo mkfs.ext4 /dev/sdb
|
|
|
|
# Create mount point
|
|
sudo mkdir -p /mnt/logs
|
|
|
|
# Mount the volume
|
|
sudo mount /dev/sdb /mnt/logs
|
|
|
|
# Persist mount across reboots — use UUID to avoid /dev/sdX changes after reboot
|
|
# First find the UUID:
|
|
sudo blkid /dev/sdb
|
|
# Then replace UUID=xxxx with the actual value shown:
|
|
echo 'UUID=xxxx-xxxx /mnt/logs ext4 defaults,nofail 0 2' | sudo tee -a /etc/fstab
|
|
|
|
# Create log directories
|
|
sudo mkdir -p /mnt/logs/nginx /mnt/logs/app
|
|
sudo chown -R $USER:$USER /mnt/logs
|
|
```
|
|
|
|
> **Encryption:** Enable volume encryption during Block Storage creation in the Ghaymah console before writing any data to the volume.
|
|
|
|
---
|
|
|
|
## Step 2: Configure Services to Write Logs to Block Storage
|
|
|
|
### Nginx
|
|
```bash
|
|
# Edit nginx.conf to write access log to block storage
|
|
sudo nano /etc/nginx/nginx.conf
|
|
# Change:
|
|
# access_log /var/log/nginx/access.log;
|
|
# To:
|
|
# access_log /mnt/logs/nginx/access.log;
|
|
|
|
sudo systemctl reload nginx
|
|
```
|
|
|
|
### Application (Docker)
|
|
```yaml
|
|
# docker-compose.yml
|
|
services:
|
|
api:
|
|
volumes:
|
|
- /mnt/logs/app:/var/log/app
|
|
environment:
|
|
LOG_PATH: /var/log/app/application.log
|
|
```
|
|
|
|
> **Secrets management note:** For production deployments, avoid plain `.env` files on disk.
|
|
> Prefer **Kubernetes Secrets** (mounted as env vars or files) or a cloud secret manager
|
|
> (e.g., HashiCorp Vault, AWS Secrets Manager, or your cloud provider's equivalent).
|
|
> `.env` files are acceptable for local development but should never be committed to Git.
|
|
|
|
### SSH Auth Log Forwarding
|
|
```bash
|
|
# Create the auth log file with correct permissions for rsyslog
|
|
sudo touch /mnt/logs/auth.log
|
|
sudo chown syslog:adm /mnt/logs/auth.log
|
|
sudo chmod 640 /mnt/logs/auth.log
|
|
|
|
# Direct rsyslog to write auth logs to block storage
|
|
echo 'auth,authpriv.* /mnt/logs/auth.log' | sudo tee /etc/rsyslog.d/99-siem.conf
|
|
sudo systemctl restart rsyslog
|
|
```
|
|
|
|
---
|
|
|
|
## Step 3: Deploy siem.py as a Systemd Service
|
|
|
|
```bash
|
|
# Install Python 3
|
|
sudo apt-get update && sudo apt-get install -y python3
|
|
|
|
# Create a shared group so both siem.py (ubuntu) and nginx (www-data) can access alerts.json
|
|
sudo groupadd siem
|
|
sudo usermod -aG siem ubuntu
|
|
sudo usermod -aG siem www-data
|
|
|
|
# Copy SIEM files
|
|
sudo mkdir -p /opt/siem /var/www/siem
|
|
sudo cp q4-siem/siem.py /opt/siem/siem.py
|
|
sudo cp q4-siem/dashboard.html /var/www/siem/dashboard.html
|
|
|
|
# Set permissions: ubuntu writes, www-data reads via shared siem group
|
|
sudo touch /var/www/siem/alerts.json
|
|
sudo chown ubuntu:siem /var/www/siem/alerts.json
|
|
sudo chmod 640 /var/www/siem/alerts.json
|
|
|
|
# Edit log paths in siem.py to match block storage
|
|
sudo nano /opt/siem/siem.py
|
|
# Set:
|
|
# LOG_SOURCES = {
|
|
# "auth": "/mnt/logs/auth.log",
|
|
# "nginx": "/mnt/logs/nginx/access.log",
|
|
# "app": "/mnt/logs/app/application.log",
|
|
# }
|
|
# OUTPUT_FILE = "/var/www/siem/alerts.json"
|
|
```
|
|
|
|
Create the systemd service:
|
|
|
|
```ini
|
|
# /etc/systemd/system/siem.service
|
|
[Unit]
|
|
Description=Ghaymah SIEM Log Analyzer
|
|
After=network.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=ubuntu
|
|
WorkingDirectory=/opt/siem
|
|
ExecStart=/usr/bin/python3 /opt/siem/siem.py
|
|
Restart=always
|
|
RestartSec=10
|
|
StandardOutput=journal
|
|
StandardError=journal
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
```
|
|
|
|
```bash
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl enable siem
|
|
sudo systemctl start siem
|
|
sudo systemctl status siem
|
|
```
|
|
|
|
---
|
|
|
|
## Step 4: Serve the Dashboard via Nginx
|
|
|
|
```nginx
|
|
# /etc/nginx/sites-available/siem
|
|
server {
|
|
listen 8443 ssl;
|
|
server_name siem.ghaymah.systems;
|
|
|
|
ssl_certificate /etc/letsencrypt/live/siem.ghaymah.systems/fullchain.pem;
|
|
ssl_certificate_key /etc/letsencrypt/live/siem.ghaymah.systems/privkey.pem;
|
|
|
|
root /var/www/siem;
|
|
index dashboard.html;
|
|
|
|
# Restrict access to internal network only
|
|
allow 10.0.0.0/8;
|
|
allow 192.168.0.0/16;
|
|
deny all;
|
|
|
|
# alerts.json is served to the dashboard browser client.
|
|
# The parent allow/deny block already restricts access to internal IPs only.
|
|
location = /alerts.json {
|
|
allow 10.0.0.0/8;
|
|
allow 192.168.0.0/16;
|
|
deny all;
|
|
add_header Content-Type application/json;
|
|
}
|
|
|
|
location / {
|
|
try_files $uri $uri/ =404;
|
|
}
|
|
}
|
|
```
|
|
|
|
```bash
|
|
sudo ln -s /etc/nginx/sites-available/siem /etc/nginx/sites-enabled/
|
|
sudo nginx -t && sudo systemctl reload nginx
|
|
```
|
|
|
|
---
|
|
|
|
## Step 5: Log Rotation
|
|
|
|
```bash
|
|
# /etc/logrotate.d/siem-logs
|
|
/mnt/logs/*.log
|
|
/mnt/logs/app/*.log {
|
|
daily
|
|
rotate 30
|
|
compress
|
|
delaycompress
|
|
missingok
|
|
notifempty
|
|
create 640 syslog adm
|
|
postrotate
|
|
systemctl restart rsyslog
|
|
endscript
|
|
}
|
|
|
|
/mnt/logs/nginx/*.log {
|
|
daily
|
|
rotate 30
|
|
compress
|
|
delaycompress
|
|
missingok
|
|
notifempty
|
|
create 640 www-data adm
|
|
postrotate
|
|
systemctl reload nginx
|
|
endscript
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Verification
|
|
|
|
```bash
|
|
# Check SIEM is running
|
|
sudo systemctl status siem
|
|
|
|
# Check alerts output
|
|
cat /var/www/siem/alerts.json | python3 -m json.tool | head -30
|
|
|
|
# View SIEM logs
|
|
sudo journalctl -u siem -f
|
|
|
|
# Test dashboard
|
|
curl -k https://siem.ghaymah.systems:8443/dashboard.html
|
|
```
|
|
|
|
---
|
|
|
|
## Security Notes
|
|
|
|
| Control | Implementation |
|
|
|---|---|
|
|
| Dashboard access | Restricted to internal IP range (10.0.0.0/8) |
|
|
| HTTPS | TLS required, no HTTP access |
|
|
| `alerts.json` access | Internal IPs only via `allow 10.0.0.0/8; deny all;` inside `location = /alerts.json` |
|
|
| File permissions | `alerts.json` owned `ubuntu:siem`, mode `640` — owner writes, group (nginx) reads, others none |
|
|
| Block storage | Encrypted at rest (enable during volume creation in Ghaymah console) |
|
|
| Audit logs | All SIEM activity written to systemd journal |
|
|
|
|
---
|
|
|
|
## Future Integrations
|
|
|
|
The SIEM is designed to grow. The following integrations can be added as next steps:
|
|
|
|
| Integration | How |
|
|
|---|---|
|
|
| **AlertManager webhook** | Extend `siem.py` to POST critical alerts to a webhook endpoint. Pairs directly with the Prometheus alert rules in `q2-attack-simulation/alert-rule.yml`. |
|
|
| **Slack notifications** | Send alerts to a `#security-alerts` channel via Slack Incoming Webhooks when severity is CRITICAL or HIGH. |
|
|
| **PagerDuty / on-call** | Route CRITICAL alerts to PagerDuty for 24/7 on-call escalation using the Events API. |
|
|
| **Email alerts** | Use `smtplib` in `siem.py` to send summary emails to the security team at the end of each scan cycle. |
|
|
| **Cloud monitoring export** | Ship `alerts.json` to a cloud-native monitoring service (e.g., Ghaymah monitoring, Datadog, or CloudWatch) for long-term trend analysis and dashboards. |
|
|
| **Log shipping to SIEM platform** | Forward structured logs to a dedicated SIEM platform (e.g., Elastic/OpenSearch) for retention and correlation across multiple services. |
|
|
|
|
---
|
|
|
|
## Backup Strategy
|
|
|
|
Block Storage volumes are the persistence layer for all logs and alert data. Schedule regular snapshots to protect against data loss.
|
|
|
|
**Recommended snapshot schedule:**
|
|
|
|
| Frequency | Retention | Purpose |
|
|
|---|---|---|
|
|
| Daily | 7 days | Short-term recovery from accidental deletion or corruption |
|
|
| Weekly | 4 weeks | Recovery from persistent misconfiguration issues |
|
|
| Monthly | 3 months | Compliance and long-term trend analysis |
|
|
|
|
**How to enable in Ghaymah console:**
|
|
|
|
1. Go to **Storage → Block Storage → `siem-logs-volume`**
|
|
2. Navigate to **Snapshots → Create Snapshot Policy**
|
|
3. Set a daily schedule with 7-day retention
|
|
4. Enable email notification on snapshot failure
|
|
|
|
**Restore procedure (monthly test recommended):**
|
|
|
|
```bash
|
|
# Restore a snapshot to a new volume in the Ghaymah console,
|
|
# then mount it temporarily to verify data integrity:
|
|
sudo mkdir -p /mnt/logs-restore
|
|
sudo mount /dev/sdc /mnt/logs-restore
|
|
ls -lh /mnt/logs-restore/
|
|
sudo umount /mnt/logs-restore
|
|
```
|
|
|
|
> Restore tests should be performed monthly to confirm backups are valid and recovery time is within acceptable limits.
|