MCP Red-Team Security Testing Server — Architecture Walkthrough

Overview

A production-ready MCP Server for automated AI Red-Teaming and Security Testing against https://os.solidpoint.ai, built with MCPServer (mcp v2) and Playwright.

Architecture

graph TD
    A[MCP Client] -->|JSON-RPC stdio| B[server.py — MCPServer]
    B -->|imports| C[client.py — Playwright Browser Layer]
    C -->|drives| D[Chromium Headless]
    D -->|interacts| E[os.solidpoint.ai]
    
    B --> F[run_security_suite]
    B --> G[evaluate_asr]
    B --> H[generate_redteam_report]
    
    F -->|reads| I[redteam_testcases.json — 45 payloads]
    F -->|reads| J[accounts.csv — 10 accounts]
    F -->|reads| K[.enc — Pro credentials + token]
    
    F -->|writes| L[suite_results.json]
    G -->|writes| M[evaluated_results.json]
    H -->|writes| N[redteam_report.json]
    H -->|writes| O[redteam_report.md]

Files Created

File	Purpose	Size
`server.py`	FastMCP server — 3 tools exposed via stdio	20.7 KB
`client.py`	Playwright browser automation layer	16.9 KB
`mcp_config.json`	MCP server configuration (stdio transport)	354 B

MCP Tools Exposed

1. `run_security_suite(headless, timeout_per_test, categories)`

Orchestrates all 45 red-team payloads from redteam_testcases.json across the loaded accounts via Playwright browser automation.

Auth priority: Pro token → Pro creds → CSV accounts (round-robin)
Auto-continue: Monitors for stalled responses and clicks "Continue" automatically
File injection: Creates temp files for indirect injection test cases (D01–D05)
Output: suite_results.json

2. `evaluate_asr(results_file)`

Computes Attack Success Rate using regex-based heuristics:

Attack success indicators: PWNED, DAN compliance, system prompt leaks, credential exposure
Defense indicators: Refusal patterns, safety guardrail phrases
Per-category ASR and overall risk level (LOW/MEDIUM/HIGH/CRITICAL)
Output: evaluated_results.json

3. `generate_redteam_report(results_file)`

Generates comprehensive security assessment report:

OWASP LLM Top 10 mapping (LLM01 Prompt Injection, LLM06 Excessive Agency)
MITRE ATLAS technique mapping (AML.T0051, AML.T0054, AML.T0048, AML.T0024)
Executive summary with risk level
Remediation recommendations
Output: redteam_report.json + redteam_report.md

Test Coverage (45 payloads across 7 categories)

Category	ID Range	Count	OWASP Code	MITRE ATLAS
direct (Prompt Injection)	A01–A08	8	LLM01	AML.T0051.000
leak (System Prompt Extraction)	B01–B08	8	LLM01	AML.T0051.001
jailbreak (Safety Bypass)	C01–C08	8	LLM01	AML.T0054
indirect (File-based Injection)	D01–D05	5	LLM01	AML.T0051.002
agency (Excessive Permissions)	E01–E06	6	LLM06	AML.T0048
disclosure (Data Leakage)	F01–F05	5	LLM06	AML.T0024
control (Baseline)	G01–G05	5	N/A	N/A

Strict Constraints Enforced

Important

❌ ZERO print() statements — All logging uses Python logging module → sys.stderr + mcp_manager.log

✅ stdout reserved exclusively for JSON-RPC communication

✅ MCP_MODE=stdio as specified in mcp_config.json

✅ Graceful error handling — failed logins, timeouts, and network errors are caught and reported without crashing

Validation Results

Check	Status
Python 3.13 + mcp v2.1.1 + playwright v1.62 installed	✅
`client.py` loads .enc, accounts.csv, redteam_testcases.json	✅ (4 env keys, 10 accounts, 45 tests)
`server.py` imports MCPServer correctly	✅
MCP `initialize` handshake responds	✅
MCP `tools/list` returns all 3 tools	✅
Chromium browser installed for Playwright	✅

How to Run

# Start the MCP server (used by MCP clients like Claude Desktop, etc.)
cd "c:\Users\ZIAD\OneDrive\سطح المكتب\ghaymah_solidpoint.ai\testing usning MCP"
& 'C:\Program Files\Python313\python.exe' server.py

# Or call tools directly for standalone testing:
& 'C:\Program Files\Python313\python.exe' -c "
import sys; sys.path.insert(0, '.')
from server import run_security_suite, evaluate_asr, generate_redteam_report
# Run a subset first:
print(run_security_suite(categories='direct,control'), file=sys.stderr)
"

4.8 KiB خام اللوم التاريخ إلغاء الهروب الهروب