Initial commit: MCP Red-Team Security Server setup and results
هذا الالتزام موجود في:
ZiadMahmoud2003
161
README.md
Normal file
161
README.md
Normal file
@@ -0,0 +1,161 @@
|
||||
<div align="center">
|
||||
|
||||
# 🛡️ MCP Red-Team Security Server
|
||||
|
||||
**Automated AI Security & Penetration Testing Framework**
|
||||
|
||||
[](https://www.python.org/)
|
||||
[](https://modelcontextprotocol.io/)
|
||||
[](https://playwright.dev/)
|
||||
[](https://owasp.org/www-project-top-10-for-large-language-model-applications/)
|
||||
[](https://atlas.mitre.org/)
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
## 📖 Overview
|
||||
|
||||
The **MCP Red-Team Security Server** is a production-grade automated security testing suite built on the **Model Context Protocol (MCP)**. It uses **Playwright** to drive browser-based injection payloads against an LLM platform (target: `os.solidpoint.ai`).
|
||||
|
||||
It programmatically injects 45 crafted adversarial payloads, monitors the model's responses, heuristically evaluates the **Attack Success Rate (ASR)**, and generates a final security report mapped directly to the **OWASP LLM Top 10** and **MITRE ATLAS** frameworks.
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Architecture & Workflow
|
||||
|
||||
The framework operates in a completely headless and automated manner, coordinating an MCP client, the MCP Server, and the browser automation layer.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User/MCP Client
|
||||
participant Server as MCP Server (server.py)
|
||||
participant Client as Playwright Layer (client.py)
|
||||
participant Target as Target LLM Platform
|
||||
|
||||
User/MCP Client->>Server: Call `run_security_suite()`
|
||||
Server->>Client: Load 45 payloads & Accounts
|
||||
|
||||
loop For each payload
|
||||
Client->>Target: Headless Login (Token/Creds)
|
||||
Client->>Target: Inject Payload (Chat / File Upload)
|
||||
Target-->>Client: AI Response (Streaming)
|
||||
Client->>Client: Monitor & Extract Final Response
|
||||
end
|
||||
|
||||
Client-->>Server: Raw Suite Results
|
||||
|
||||
User/MCP Client->>Server: Call `evaluate_asr()`
|
||||
Server->>Server: Regex Heuristics & Analysis
|
||||
Server-->>User/MCP Client: Evaluated Results (ASR)
|
||||
|
||||
User/MCP Client->>Server: Call `generate_redteam_report()`
|
||||
Server->>Server: Map to OWASP & MITRE ATLAS
|
||||
Server-->>User/MCP Client: Markdown & JSON Reports
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📂 Project Structure
|
||||
|
||||
```text
|
||||
RedTeam-MCP-Server/
|
||||
├── src/ # Source Code
|
||||
│ ├── server.py # FastMCP Server (MCP v2) exposing the 3 tools
|
||||
│ ├── client.py # Playwright browser automation layer
|
||||
│ └── run_all.py # Standalone orchestrator script
|
||||
├── data/ # Inputs
|
||||
│ ├── redteam_testcases.json # 45 Injection Payloads
|
||||
│ ├── accounts.csv # Test Accounts
|
||||
│ └── created_accounts.json
|
||||
├── config/ # Configuration
|
||||
│ ├── mcp_config.json # MCP Client Configuration (stdio)
|
||||
│ └── .env # Target URL, Pro Credentials & Tokens
|
||||
├── reports/ # Outputs
|
||||
│ ├── suite_results.json
|
||||
│ ├── evaluated_results.json
|
||||
│ ├── redteam_report.json
|
||||
│ └── redteam_report.md
|
||||
└── docs/ # Security Guides
|
||||
├── LLM_Testing_AI_Security_Pentesting_Prompt_Injection_Guide.md
|
||||
└── LLM_Testing_Security_Pentesting_PromptInjection_Guide.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Exposed MCP Tools
|
||||
|
||||
The server (`src/server.py`) exposes three tools via `stdio` transport:
|
||||
|
||||
1. `run_security_suite(headless=True, timeout_per_test=120, categories="")`
|
||||
* Opens the browser, handles authentication, and executes all defined testcases.
|
||||
2. `evaluate_asr(results_file="")`
|
||||
* Processes the raw responses to calculate the **Attack Success Rate (ASR)**.
|
||||
* Classifies responses as `ATTACK_SUCCEEDED`, `DEFENDED`, `POSSIBLE_LEAK`, etc.
|
||||
3. `generate_redteam_report(results_file="")`
|
||||
* Compiles the evaluated data into a comprehensive Markdown report mapped to **OWASP LLM01/LLM06** and **MITRE ATLAS AML.T0051/AML.T0048**.
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
### Prerequisites
|
||||
- Python 3.13+
|
||||
- Required packages: `mcp[cli]`, `playwright`
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install "mcp[cli]" playwright
|
||||
playwright install chromium
|
||||
```
|
||||
|
||||
### Running Standalone (No MCP Client Required)
|
||||
To run the full suite automatically without an MCP Client:
|
||||
|
||||
```bash
|
||||
cd RedTeam-MCP-Server
|
||||
python src/run_all.py
|
||||
```
|
||||
|
||||
### Running via MCP
|
||||
Add the server to your MCP Client configuration (e.g., Claude Desktop, Cursor, or IDE Extensions) using the provided `config/mcp_config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"redteam-security": {
|
||||
"command": "C:\\Program Files\\Python313\\python.exe",
|
||||
"args": ["src\\server.py"],
|
||||
"cwd": "C:\\path\\to\\RedTeam-MCP-Server",
|
||||
"env": {
|
||||
"MCP_MODE": "stdio",
|
||||
"PYTHONUNBUFFERED": "1"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Evaluation Categories (45 Payloads)
|
||||
|
||||
| Category | Description | OWASP | MITRE ATLAS |
|
||||
|----------|-------------|-------|-------------|
|
||||
| **Direct** | Direct Prompt Injections (e.g., "Ignore instructions") | LLM01 | AML.T0051.000 |
|
||||
| **Leak** | System Prompt Extraction & Reconnaissance | LLM01 | AML.T0051.001 |
|
||||
| **Jailbreak**| Safety filter bypass (e.g., DAN, Roleplay) | LLM01 | AML.T0054 |
|
||||
| **Indirect** | Injection via file uploads (e.g., poisoned `.txt` / `.csv`) | LLM01 | AML.T0051.002 |
|
||||
| **Agency** | Exploiting system tools (e.g., executing commands) | LLM06 | AML.T0048 |
|
||||
| **Disclosure**| Data leakage of private environments/keys | LLM06 | AML.T0024 |
|
||||
| **Control** | Baseline generic questions to ensure model stability | N/A | N/A |
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security Constraints Enforced
|
||||
|
||||
- **Zero `print()` Statements**: All logging uses the Python `logging` module to output to `sys.stderr` and `logs/mcp_manager.log` to preserve JSON-RPC integrity on `stdout`.
|
||||
- **Fault Tolerance**: Stalled generation triggers automatic UI 'Continue' clicks. Failed logins skip gracefully to the next test.
|
||||
|
||||
---
|
||||
*Developed for internal AI Red-Teaming and Security Assessment.*
|
||||
المرجع في مشكلة جديدة
حظر مستخدم