Files
ghaymah-v2-analysis/reports/REDTEAM_SECURITY_ASSESSMENT.md

17 KiB
Raw Permalink Blame History

Ghaymah Cloud V2 — Red Team Penetration Testing & Security Assessment Report

Classification: CONFIDENTIAL / RED TEAM REPORT
Target: https://deploy.ghaymah.systems/dashboard/apps & Associated Microservices
Assessment Date: October 2026
Scope: Single Page Application (SPA), Hasura GraphQL Engine, Hasura Auth Service, Gateway, GenAI/LiteLLM Gateway, Centralized Logging API, Integration Services
Assessor: Advanced Red Team Operator


1. Executive Summary

During this engagement, a black-box and authenticated red team assessment was conducted against the Ghaymah Cloud V2 platform, focusing on the newly deployed dashboard at https://deploy.ghaymah.systems/dashboard/apps and its supporting microservices ecosystem.

The assessment revealed multiple critical and high-severity security vulnerabilities spanning broken authentication/authorization patterns, unauthorized configuration and console exposure, sensitive API key leakage, unsafe development configurations in production, and architectural fragility under load.

Risk Overview Dashboard

Severity ID Finding Title CVSS v3.1 Status
🔴 CRITICAL GHY-SEC-01 Hasura Admin Console Publicly Accessible with Environment Disclosure 9.1 Confirmed Exploit
🔴 CRITICAL GHY-SEC-02 Unauthenticated GraphQL Introspection Exposing Complete Schema 8.2 Confirmed Exploit
🟠 HIGH GHY-SEC-03 Plaintext GenAI / LiteLLM Virtual API Key Leakage & Model Abuse 7.7 Confirmed Exploit
🟠 HIGH GHY-SEC-04 Auth Microservice Running in Development Mode ("version":"dev") 7.5 Confirmed
🟠 HIGH GHY-SEC-05 Internal Cascading DoS & Pod Log Probe via Centralized Logging API 7.2 Confirmed Exploit
🟡 MEDIUM GHY-SEC-06 Live Stripe Merchant ID & Billing Portal Session Disclosure 5.3 Confirmed
🟡 MEDIUM GHY-SEC-07 Permissive CSP with 'unsafe-inline' Enabling Stored & DOM XSS 6.1 Confirmed
🟡 MEDIUM GHY-SEC-08 Business Logic Quota Flaw (Negative Allocated Resources -14/5) 5.5 Confirmed
🔵 LOW GHY-SEC-09 Missing Security & Anti-Framing Headers on GraphQL Engine 4.3 Confirmed

2. Infrastructure & Attack Surface Mapping

                                      [ATTACKER / CLIENT]
                                               │
              ┌────────────────────────────────┼────────────────────────────────┐
              │                                │                                │
              ▼                                ▼                                ▼
    deploy.ghaymah.systems          graphql.ghaymah.systems           auth.ghaymah.systems
    [React 19 SPA / Vite]           [Hasura v2.35.1 Pro-Lite]         [Hasura Auth - DEV MODE]
    - Permissive CSP                 - Public /console (200 OK)       - HS256 Signed JWTs
    - Exposes client bundles         - Introspection Open             - 17 User Refresh Tokens
              │                                │                                │
              ▼                                ▼                                ▼
     gateway.ghaymah.systems         genai.ghaymah.systems            logs.ghaymah.systems
    [Billing & Plans API]           [LiteLLM AI Proxy]               [K8s Pod Log Collector]
    - Live Stripe Account ID         - Plaintext Key in GraphQL       - Sync HTTP Auth Validation
    - Card setup endpoints           - 7 High-end AI Models           - Leaks HTTP 429 Status
              │                                │                                │
              └────────────────────────────────┼────────────────────────────────┘
                                               │
                                               ▼
                              KUBERNETES CLUSTER & INTERNAL MESH
                      ┌──────────────────────────────────────────────┐
                      │  - API Server: 10.43.0.1:443                 │
                      │  - CoreDNS: 10.43.0.10:53                    │
                      │  - Redis Cache: soc-redis:6379               │
                      │  - Customer Pods: *.hosted.cumin.dev         │
                      └──────────────────────────────────────────────┘

3. Detailed Technical Findings & Proof-of-Concepts (PoC)

GHY-SEC-01: Hasura Admin Console Publicly Accessible (Critical)

  • CVSS Score: 9.1 (CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N)
  • Target: https://graphql.ghaymah.systems/console
  • Vulnerability Type: CWE-200 (Exposure of Sensitive Information) / CWE-16 (Configuration Vulnerability)

Description

The Hasura GraphQL Engine web management console is publicly accessible over the internet without IP restrictions or perimeter protection, returning HTTP 200 OK.

Evidence & Leaked Configuration

An HTTP GET request to https://graphql.ghaymah.systems/console renders the full Hasura administrative interface and leaks critical internal configuration parameters embedded in window.__env:

window.__env = {
  consoleMode: "server",
  urlPrefix: "/console",
  consolePath: "/console",
  isAccessKeySet: true,
  isAdminSecretSet: true,
  enableTelemetry: false,
  assetsPath: "https://graphql-engine-cdn.hasura.io/console/assets",
  assetsVersion: "channel/stable/v2.35",
  cdnAssets: true,
  serverVersion: "v2.35.1",
  consoleType: "pro-lite",
  consoleSentryDsn: ""
}

Exploitability Analysis & CVE Correlation

  1. Engine Version Identified: Hasura GraphQL Engine v2.35.1 (Community / Pro-Lite).
  2. Known CVE Exposure: Hasura versions < 2.45.5 and < 2.49.2 are vulnerable to critical GraphQL injection and unauthenticated resource consumption bugs (e.g., CVE-2024-43368 / CVE-2024-38515).
  3. Admin Secret Brute-Force Vector: The endpoint exposes the admin login interface, allowing automated brute-force attacks against x-hasura-admin-secret.

GHY-SEC-02: Unauthenticated GraphQL Schema Introspection (Critical)

  • CVSS Score: 8.2 (CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N)
  • Target: POST https://graphql.ghaymah.systems/v1/graphql
  • Vulnerability Type: CWE-200 (Information Disclosure)

Description

The GraphQL endpoint does not restrict schema introspection queries for unauthenticated users. An attacker with zero credentials can dump the entire database schema, internal data models, relationships, custom action handlers, and mutation signatures.

Proof of Concept

curl -s -X POST https://graphql.ghaymah.systems/v1/graphql \
  -H "Content-Type: application/json" \
  -d '{"query":"query Introspect { __schema { types { name fields { name } } } }"}'

Exposed Attack Surface Discovered

  • 15 Queries: ghaymah_cloud_resources, ghaymah_cloud_managed_resources, ghaymah_cloud_store, ghaymah_cloud_user_plans, ghaymah_cloud_genai_apikeys, ghaymah_cloud_row_event, authRefreshTokens
  • 41 Mutations: createApp, updateApp, deleteApp, listApps, createPostgresDb, createS3Bucket, createSecret, createVolume, deleteAuthRefreshTokens
  • 22 Streaming Subscriptions: Real-time event streams on all underlying tables.

GHY-SEC-03: Plaintext GenAI / LiteLLM API Key Leakage & Model Abuse (High)

  • CVSS Score: 7.7 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:L/A:L)
  • Target: ghaymah_cloud_genai_apikeys & https://genai.ghaymah.systems
  • Vulnerability Type: CWE-312 (Cleartext Storage of Sensitive Information) / CWE-284 (Improper Access Control)

Description

LiteLLM Virtual API Keys assigned to users are stored in plaintext and accessible via GraphQL queries (ghaymah_cloud_genai_apikeys). Once obtained, this key can be used directly against the LiteLLM gateway (https://genai.ghaymah.systems) to consume AI resources, list internal user identifiers, and query key metadata.

Proof of Concept

  1. Retrieve Plaintext Key via GraphQL:

    query {
      ghaymah_cloud_genai_apikeys {
        key_id
        key_name
        key_value
        user_id
      }
    }
    

    Output: Returns sk-ANg_uw-fjr4E...

  2. Query LiteLLM API Endpoints:

    # List Available Models
    curl -s -H "Authorization: Bearer sk-ANg_uw-..." https://genai.ghaymah.systems/models
    

    Output: Confirmed access to 7 models:

    • Kimi-K3
    • DeepSeek-Coder
    • QwQ-32B
    • DeepSeek-V4-Flash
    • GLM-5.3-Flash
    • Qwen3.6-35B-A3B
    • gemma-4-31B-it
  3. Leaked User and Key Identifiers:

    • Key Hash: 54c0ee4ef62bb869f800c5cc59bdaa9e48b7bfbf2ed3eeb3007bab5b421ec406
    • User ID: 6ec6de28-36c8-4ecb-b879-b34757616c21
    • Spend: $0 with unconstrained soft budget cooldown

GHY-SEC-04: Auth Microservice Running in Development Mode (High)

  • CVSS Score: 7.5 (CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N)
  • Target: https://auth.ghaymah.systems/version
  • Vulnerability Type: CWE-1188 (Insecure Default Initialization)

Description

Querying the authentication microservice metadata endpoint confirms that the service is running in development mode:

HTTP/2 200 OK
content-type: application/json; charset=utf-8

{"version":"dev"}

Running authentication engines in development mode frequently disables critical security mitigations, such as strict cryptographic checks, secure cookie attributes, PKCE validation on OAuth flows, and debug logging suppression.

Additionally, JWT tokens are signed using the symmetric HS256 algorithm rather than asymmetric RS256/EdDSA. If the shared secret is cracked offline via hashcat or leaked, an attacker can forge administrative JWTs with arbitrary claims (x-hasura-default-role: admin).


GHY-SEC-05: Cascading Denial of Service & Information Leakage in Logging API (High)

  • CVSS Score: 7.2 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:M/I:N/A:H)
  • Target: POST https://logs.ghaymah.systems/logs
  • Vulnerability Type: CWE-209 (Generation of Error Message Containing Sensitive Information) / CWE-400 (Uncontrolled Resource Consumption)

Description

The centralized logging microservice performs a synchronous network request to https://auth.ghaymah.systems on every log query to validate the Bearer token.

When rapid requests are sent, auth.ghaymah.systems rate-limits the internal validation call with HTTP 429. The logging service does not gracefully handle this failure, but instead crashes and leaks internal error details to the client:

HTTP/2 500 Internal Server Error
Failed to validate and extract user from token: token validation failed with status: 429
Unauthorized
No pods found

Impact

  1. Architectural Fragility: Attackers can induce cascading denial-of-service across the entire platform's logging and pod inspection infrastructure simply by generating burst traffic.
  2. Pod Namespace Oracle: When the token succeeds, the response directly queries the underlying Kubernetes cluster:
    • Valid pod: Returns streaming pod stdout/stderr.
    • Non-existent pod: Returns No pods found. This allows attackers to brute-force or enumerate active Kubernetes pods and namespaces across the cluster.

GHY-SEC-06: Live Stripe Merchant Account & Billing Portal Session Disclosure (Medium)

  • CVSS Score: 5.3 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:M/I:N/A:N)
  • Target: POST https://gateway.ghaymah.systems/v1/billing/portal & /setup-card
  • Vulnerability Type: CWE-200 (Information Disclosure)

Description

The Gateway billing service directly generates Stripe checkout and customer portal sessions for authenticated accounts.

  • Leaked Stripe Merchant Account: acct_1UDgQ25HrO9nbYqv (decoded from the live session token live_YWNjdF8xVURnUTI1SHJPOW5iWXF2LF9...).
  • Live Checkout Session Tokens: Unauthenticated or authenticated users can trigger card setup sessions that expose live Stripe API configuration parameters.

GHY-SEC-07: Permissive CSP Allowing 'unsafe-inline' Script Execution (Medium)

  • CVSS Score: 6.1 (CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:L/I:L/A:N)
  • Target: https://deploy.ghaymah.systems
  • Vulnerability Type: CWE-79 (Cross-Site Scripting) / CWE-1021 (Improper Restriction of Rendered UI)

Description

The Content Security Policy (CSP) header returned by deploy.ghaymah.systems explicitly includes 'unsafe-inline' in its script-src directive:

content-security-policy: default-src 'self'; script-src 'self' 'unsafe-inline' https://js.stripe.com https://*.posthog.com https://*.i.posthog.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' https://fonts.gstatic.com data:; img-src 'self' data: https: blob:; connect-src 'self' https://graphql.ghaymah.systems https://auth.ghaymah.systems https://*.posthog.com https://*.i.posthog.com https://gateway.ghaymah.systems https://integrations.ghaymah.systems https://logs.ghaymah.systems https://status.ghaymah.systems https://genai.ghaymah.systems;

Because 'unsafe-inline' is permitted, any stored XSS vulnerability in application names, environment variable keys/values, or markdown documentation renders will execute arbitrary JavaScript, allowing complete account takeover and session hijacking.


GHY-SEC-08: Business Logic Quota Flaw (Negative Resource Allocation) (Medium)

  • CVSS Score: 5.5 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:L/A:L)
  • Target: ghaymah_cloud_user_plans
  • Vulnerability Type: CWE-840 (Business Logic Errors)

Description

Querying the ghaymah_cloud_user_plans table reveals an inconsistent quota calculation state:

  • Plan Type: free
  • Max Allowed Resources: 5
  • Currently Calculated Resources: -14 / 5

A negative allocation integer demonstrates a severe flaw in the resource tracking lifecycle (e.g., race conditions when apps are deleted, failure to increment on create, or unhandled subtraction underflows). This flaw can allow attackers to bypass maximum container quota restrictions and deploy unlimited instances without payment.


4. End-to-End Attack Chains & Pivot Analysis

Attack Chain 1: Unauthenticated Recon to Model Hijacking

  1. Introspection Query: Attacker queries https://graphql.ghaymah.systems/v1/graphql without credentials and discovers the ghaymah_cloud_genai_apikeys table schema.
  2. Account Creation & Key Extraction: Attacker registers a free account, queries the table, and retrieves their LiteLLM virtual key (sk-ANg_uw-...).
  3. Model Resource Exhaustion: Attacker connects directly to https://genai.ghaymah.systems, bypassing dashboard rate limits, and uses heavy compute models (QwQ-32B, DeepSeek-Coder, gemma-4-31B-it) to incur server costs on the platform operator.

Attack Chain 2: Hasura Console Exposure to Cluster Pivot

  1. Console Identification: Attacker accesses https://graphql.ghaymah.systems/console (200 OK) and identifies Hasura version v2.35.1.
  2. Admin Secret Brute-Force / CVE Exploit: Using known vulnerabilities in Hasura v2.35.1 or password spraying against x-hasura-admin-secret, attacker gains administrative control over Hasura.
  3. Database & Cluster Pivot: With admin access to Hasura, attacker executes raw SQL via the Metadata API (/v1/metadata), exports Postgres connection strings, and pivots to internal Kubernetes service accounts (10.43.0.1:443).

5. Remediation & Hardening Roadmap

Immediate Priorities (Within 24–48 Hours)

  1. Disable Public Hasura Console:
    • Configure Hasura environment variable: HASURA_GRAPHQL_ENABLE_CONSOLE=false.
    • Restrict /console access at the ingress controller level via VPN / IP whitelisting.
  2. Disable Schema Introspection in Production:
    • Set HASURA_GRAPHQL_ENABLE_INTROSPECTION=false in the production environment.
  3. Fix Permissive CSP:
    • Remove 'unsafe-inline' from script-src. Implement cryptographic nonces ('nonce-...') or SHA-256 hashes for legitimate inline scripts.
  4. Transition Auth Service to Production:
    • Ensure the authentication microservice runs with NODE_ENV=production and suppress internal version flags.
    • Migrate JWT signing from symmetric HS256 to asymmetric RS256 or EdDSA.

Architectural Enhancements (Within 1–2 Weeks)

  1. Asynchronous / Local Token Validation:
    • Avoid synchronous HTTP round-trips from logs.ghaymah.systems to auth.ghaymah.systems. Verify JWT signatures locally in the logging service using the public verification key to eliminate the cascading HTTP 429 bottleneck.
  2. Enforce Strict Quota Invariants:
    • Add database check constraints: CONSTRAINT positive_resource_count CHECK (allocated_resources >= 0).
  3. Upgrade Hasura GraphQL Engine:
    • Upgrade Hasura from v2.35.1 to the latest stable release (v2.49+) to patch known GraphQL engine denial-of-service and injection vulnerabilities.