added
هذا الالتزام موجود في:
324
q4-siem/deploy-guide.md
Normal file
324
q4-siem/deploy-guide.md
Normal file
@@ -0,0 +1,324 @@
|
||||
# Lightweight SIEM Deployment Guide on Ghaymah Cloud Infrastructure
|
||||
|
||||
## Overview
|
||||
|
||||
This guide explains how to deploy the `siem.py` log analyzer and `dashboard.html` on Ghaymah cloud infrastructure, using Block Storage to persist logs and alert data.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
BS["🗄️ Block Storage Volume\n/mnt/logs"]
|
||||
BS --> AL["/mnt/logs/auth.log"]
|
||||
BS --> NL["/mnt/logs/nginx/access.log"]
|
||||
BS --> AP["/mnt/logs/app/application.log"]
|
||||
|
||||
AL & NL & AP --> SIEM["⚙️ siem.py\nsystemd service"]
|
||||
|
||||
SIEM -->|"writes every 30s"| AJ["📄 alerts.json\n/var/www/siem/"]
|
||||
|
||||
AJ -->|"served by Nginx"| DASH["🖥️ dashboard.html\nNear Real-Time UI"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Create and Attach Block Storage
|
||||
|
||||
In the Ghaymah Cloud Console:
|
||||
|
||||
1. Go to **Storage → Block Storage → Create Volume**
|
||||
2. Set name: `siem-logs-volume`
|
||||
3. Set size: `50 GB` (adjust based on log retention needs)
|
||||
4. Attach to your cloud instance
|
||||
|
||||
Then format and mount the volume:
|
||||
|
||||
```bash
|
||||
# Find the new volume (usually /dev/sdb or /dev/vdb)
|
||||
lsblk
|
||||
|
||||
# Format the volume
|
||||
sudo mkfs.ext4 /dev/sdb
|
||||
|
||||
# Create mount point
|
||||
sudo mkdir -p /mnt/logs
|
||||
|
||||
# Mount the volume
|
||||
sudo mount /dev/sdb /mnt/logs
|
||||
|
||||
# Persist mount across reboots — use UUID to avoid /dev/sdX changes after reboot
|
||||
# First find the UUID:
|
||||
sudo blkid /dev/sdb
|
||||
# Then replace UUID=xxxx with the actual value shown:
|
||||
echo 'UUID=xxxx-xxxx /mnt/logs ext4 defaults,nofail 0 2' | sudo tee -a /etc/fstab
|
||||
|
||||
# Create log directories
|
||||
sudo mkdir -p /mnt/logs/nginx /mnt/logs/app
|
||||
sudo chown -R $USER:$USER /mnt/logs
|
||||
```
|
||||
|
||||
> **Encryption:** Enable volume encryption during Block Storage creation in the Ghaymah console before writing any data to the volume.
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Configure Services to Write Logs to Block Storage
|
||||
|
||||
### Nginx
|
||||
```bash
|
||||
# Edit nginx.conf to write access log to block storage
|
||||
sudo nano /etc/nginx/nginx.conf
|
||||
# Change:
|
||||
# access_log /var/log/nginx/access.log;
|
||||
# To:
|
||||
# access_log /mnt/logs/nginx/access.log;
|
||||
|
||||
sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
### Application (Docker)
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
services:
|
||||
api:
|
||||
volumes:
|
||||
- /mnt/logs/app:/var/log/app
|
||||
environment:
|
||||
LOG_PATH: /var/log/app/application.log
|
||||
```
|
||||
|
||||
> **Secrets management note:** For production deployments, avoid plain `.env` files on disk.
|
||||
> Prefer **Kubernetes Secrets** (mounted as env vars or files) or a cloud secret manager
|
||||
> (e.g., HashiCorp Vault, AWS Secrets Manager, or your cloud provider's equivalent).
|
||||
> `.env` files are acceptable for local development but should never be committed to Git.
|
||||
|
||||
### SSH Auth Log Forwarding
|
||||
```bash
|
||||
# Create the auth log file with correct permissions for rsyslog
|
||||
sudo touch /mnt/logs/auth.log
|
||||
sudo chown syslog:adm /mnt/logs/auth.log
|
||||
sudo chmod 640 /mnt/logs/auth.log
|
||||
|
||||
# Direct rsyslog to write auth logs to block storage
|
||||
echo 'auth,authpriv.* /mnt/logs/auth.log' | sudo tee /etc/rsyslog.d/99-siem.conf
|
||||
sudo systemctl restart rsyslog
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Deploy siem.py as a Systemd Service
|
||||
|
||||
```bash
|
||||
# Install Python 3
|
||||
sudo apt-get update && sudo apt-get install -y python3
|
||||
|
||||
# Create a shared group so both siem.py (ubuntu) and nginx (www-data) can access alerts.json
|
||||
sudo groupadd siem
|
||||
sudo usermod -aG siem ubuntu
|
||||
sudo usermod -aG siem www-data
|
||||
|
||||
# Copy SIEM files
|
||||
sudo mkdir -p /opt/siem /var/www/siem
|
||||
sudo cp q4-siem/siem.py /opt/siem/siem.py
|
||||
sudo cp q4-siem/dashboard.html /var/www/siem/dashboard.html
|
||||
|
||||
# Set permissions: ubuntu writes, www-data reads via shared siem group
|
||||
sudo touch /var/www/siem/alerts.json
|
||||
sudo chown ubuntu:siem /var/www/siem/alerts.json
|
||||
sudo chmod 640 /var/www/siem/alerts.json
|
||||
|
||||
# Edit log paths in siem.py to match block storage
|
||||
sudo nano /opt/siem/siem.py
|
||||
# Set:
|
||||
# LOG_SOURCES = {
|
||||
# "auth": "/mnt/logs/auth.log",
|
||||
# "nginx": "/mnt/logs/nginx/access.log",
|
||||
# "app": "/mnt/logs/app/application.log",
|
||||
# }
|
||||
# OUTPUT_FILE = "/var/www/siem/alerts.json"
|
||||
```
|
||||
|
||||
Create the systemd service:
|
||||
|
||||
```ini
|
||||
# /etc/systemd/system/siem.service
|
||||
[Unit]
|
||||
Description=Ghaymah SIEM Log Analyzer
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=ubuntu
|
||||
WorkingDirectory=/opt/siem
|
||||
ExecStart=/usr/bin/python3 /opt/siem/siem.py
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable siem
|
||||
sudo systemctl start siem
|
||||
sudo systemctl status siem
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Serve the Dashboard via Nginx
|
||||
|
||||
```nginx
|
||||
# /etc/nginx/sites-available/siem
|
||||
server {
|
||||
listen 8443 ssl;
|
||||
server_name siem.ghaymah.systems;
|
||||
|
||||
ssl_certificate /etc/letsencrypt/live/siem.ghaymah.systems/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/siem.ghaymah.systems/privkey.pem;
|
||||
|
||||
root /var/www/siem;
|
||||
index dashboard.html;
|
||||
|
||||
# Restrict access to internal network only
|
||||
allow 10.0.0.0/8;
|
||||
allow 192.168.0.0/16;
|
||||
deny all;
|
||||
|
||||
# alerts.json is served to the dashboard browser client.
|
||||
# The parent allow/deny block already restricts access to internal IPs only.
|
||||
location = /alerts.json {
|
||||
allow 10.0.0.0/8;
|
||||
allow 192.168.0.0/16;
|
||||
deny all;
|
||||
add_header Content-Type application/json;
|
||||
}
|
||||
|
||||
location / {
|
||||
try_files $uri $uri/ =404;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo ln -s /etc/nginx/sites-available/siem /etc/nginx/sites-enabled/
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Log Rotation
|
||||
|
||||
```bash
|
||||
# /etc/logrotate.d/siem-logs
|
||||
/mnt/logs/*.log
|
||||
/mnt/logs/app/*.log {
|
||||
daily
|
||||
rotate 30
|
||||
compress
|
||||
delaycompress
|
||||
missingok
|
||||
notifempty
|
||||
create 640 syslog adm
|
||||
postrotate
|
||||
systemctl restart rsyslog
|
||||
endscript
|
||||
}
|
||||
|
||||
/mnt/logs/nginx/*.log {
|
||||
daily
|
||||
rotate 30
|
||||
compress
|
||||
delaycompress
|
||||
missingok
|
||||
notifempty
|
||||
create 640 www-data adm
|
||||
postrotate
|
||||
systemctl reload nginx
|
||||
endscript
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# Check SIEM is running
|
||||
sudo systemctl status siem
|
||||
|
||||
# Check alerts output
|
||||
cat /var/www/siem/alerts.json | python3 -m json.tool | head -30
|
||||
|
||||
# View SIEM logs
|
||||
sudo journalctl -u siem -f
|
||||
|
||||
# Test dashboard
|
||||
curl -k https://siem.ghaymah.systems:8443/dashboard.html
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Notes
|
||||
|
||||
| Control | Implementation |
|
||||
|---|---|
|
||||
| Dashboard access | Restricted to internal IP range (10.0.0.0/8) |
|
||||
| HTTPS | TLS required, no HTTP access |
|
||||
| `alerts.json` access | Internal IPs only via `allow 10.0.0.0/8; deny all;` inside `location = /alerts.json` |
|
||||
| File permissions | `alerts.json` owned `ubuntu:siem`, mode `640` — owner writes, group (nginx) reads, others none |
|
||||
| Block storage | Encrypted at rest (enable during volume creation in Ghaymah console) |
|
||||
| Audit logs | All SIEM activity written to systemd journal |
|
||||
|
||||
---
|
||||
|
||||
## Future Integrations
|
||||
|
||||
The SIEM is designed to grow. The following integrations can be added as next steps:
|
||||
|
||||
| Integration | How |
|
||||
|---|---|
|
||||
| **AlertManager webhook** | Extend `siem.py` to POST critical alerts to a webhook endpoint. Pairs directly with the Prometheus alert rules in `q2-attack-simulation/alert-rule.yml`. |
|
||||
| **Slack notifications** | Send alerts to a `#security-alerts` channel via Slack Incoming Webhooks when severity is CRITICAL or HIGH. |
|
||||
| **PagerDuty / on-call** | Route CRITICAL alerts to PagerDuty for 24/7 on-call escalation using the Events API. |
|
||||
| **Email alerts** | Use `smtplib` in `siem.py` to send summary emails to the security team at the end of each scan cycle. |
|
||||
| **Cloud monitoring export** | Ship `alerts.json` to a cloud-native monitoring service (e.g., Ghaymah monitoring, Datadog, or CloudWatch) for long-term trend analysis and dashboards. |
|
||||
| **Log shipping to SIEM platform** | Forward structured logs to a dedicated SIEM platform (e.g., Elastic/OpenSearch) for retention and correlation across multiple services. |
|
||||
|
||||
---
|
||||
|
||||
## Backup Strategy
|
||||
|
||||
Block Storage volumes are the persistence layer for all logs and alert data. Schedule regular snapshots to protect against data loss.
|
||||
|
||||
**Recommended snapshot schedule:**
|
||||
|
||||
| Frequency | Retention | Purpose |
|
||||
|---|---|---|
|
||||
| Daily | 7 days | Short-term recovery from accidental deletion or corruption |
|
||||
| Weekly | 4 weeks | Recovery from persistent misconfiguration issues |
|
||||
| Monthly | 3 months | Compliance and long-term trend analysis |
|
||||
|
||||
**How to enable in Ghaymah console:**
|
||||
|
||||
1. Go to **Storage → Block Storage → `siem-logs-volume`**
|
||||
2. Navigate to **Snapshots → Create Snapshot Policy**
|
||||
3. Set a daily schedule with 7-day retention
|
||||
4. Enable email notification on snapshot failure
|
||||
|
||||
**Restore procedure (monthly test recommended):**
|
||||
|
||||
```bash
|
||||
# Restore a snapshot to a new volume in the Ghaymah console,
|
||||
# then mount it temporarily to verify data integrity:
|
||||
sudo mkdir -p /mnt/logs-restore
|
||||
sudo mount /dev/sdc /mnt/logs-restore
|
||||
ls -lh /mnt/logs-restore/
|
||||
sudo umount /mnt/logs-restore
|
||||
```
|
||||
|
||||
> Restore tests should be performed monthly to confirm backups are valid and recovery time is within acceptable limits.
|
||||
المرجع في مشكلة جديدة
حظر مستخدم