updated
هذا الالتزام موجود في:
552
Readme.md
552
Readme.md
@@ -0,0 +1,552 @@
|
||||
# Ghaymah CLI — Manual Test Report
|
||||
|
||||
> **CLI Version:** `gy v0.0.24`
|
||||
> **Test Date:** 2026-08-21
|
||||
> **Tester:** Mahmoud
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Overview](#overview)
|
||||
2. [Test Summary](#test-summary)
|
||||
3. [Core Commands](#core-commands)
|
||||
- [version](#1-version)
|
||||
- [help](#2-help)
|
||||
- [signup](#3-signup)
|
||||
- [login](#4-login)
|
||||
- [logout](#5-logout)
|
||||
- [whoami](#6-whoami)
|
||||
- [list](#7-list)
|
||||
4. [Deployment Tests](#deployment-tests)
|
||||
- [Node.js](#nodejs-deployment)
|
||||
- [Flask](#flask-deployment)
|
||||
- [Go (Non-Web)](#go-non-web-deployment)
|
||||
- [Go (Web)](#go-web-deployment)
|
||||
- [FastAPI](#fastapi-deployment)
|
||||
5. [Tunneling](#tunneling)
|
||||
6. [Delete Operations](#delete-operations)
|
||||
7. [Security Tests](#security-tests)
|
||||
- [Secret Exposure in CLI Output](#secret-exposure-in-cli-output)
|
||||
- [Sensitive File Inclusion](#sensitive-file-inclusion)
|
||||
- [Git Metadata Exposure](#git-metadata-exposure)
|
||||
- [Dockerfile Hardcoded Secret Detection](#dockerfile-hardcoded-secret-detection)
|
||||
- [AI-Assisted Secret Detection](#ai-assisted-secret-detection)
|
||||
8. [Known Issues & Bugs](#known-issues--bugs)
|
||||
9. [Verdict Summary Table](#verdict-summary-table)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This document reports the results of manual functional and security testing performed on the **Ghaymah CLI (`gy`)**. Tests cover core authentication commands, multi-language deployment scenarios, tunneling, resource management, and several security-focused scenarios including secret exposure and file inclusion.
|
||||
|
||||
---
|
||||
|
||||
## Test Summary
|
||||
|
||||
| Category | Total Tests | ✅ Pass | ⚠️ Partial | ❌ Fail |
|
||||
|---|---|---|---|---|
|
||||
| Core Commands | 7 | 6 | 0 | 1 |
|
||||
| Deployments | 5 | 3 | 1 | 1 |
|
||||
| Tunneling | 1 | 0 | 0 | 1 |
|
||||
| Delete Operations | 2 | 1 | 1 | 0 |
|
||||
| Security Tests | 5 | 2 | 0 | 3 |
|
||||
| **Total** | **20** | **12** | **2** | **5** |
|
||||
|
||||
---
|
||||
|
||||
## Core Commands
|
||||
|
||||
### 1. Version
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy version` |
|
||||
| **Expected** | Display current CLI version and update status |
|
||||
| **Result** | ✅ Pass |
|
||||
|
||||
**Output:**
|
||||
```
|
||||
Ghaymah CLI v0.0.24
|
||||
You're up to date!
|
||||
```
|
||||
|
||||
The CLI correctly reports its current version and confirms it is up to date.
|
||||
|
||||
---
|
||||
|
||||
### 2. Help
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy --help` |
|
||||
| **Expected** | Display CLI description, available commands, usage examples, and global flags |
|
||||
| **Result** | ✅ Pass |
|
||||
|
||||
The help output is clear and provides all necessary information about the available commands and CLI options.
|
||||
|
||||
---
|
||||
|
||||
### 3. Signup
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy signup` |
|
||||
| **Expected** | Create a new account and auto-login |
|
||||
| **Result** | ✅ Pass |
|
||||
|
||||
**Output:**
|
||||
```
|
||||
✅ Account created! Logged in as username@gmail.com
|
||||
```
|
||||
|
||||
> **Note:** An informational log `config file not found, using defaults` appeared during signup. This is expected behavior and is not an issue.
|
||||
|
||||
---
|
||||
|
||||
### 4. Login
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy login` |
|
||||
| **Expected** | Authenticate with existing Ghaymah account credentials |
|
||||
| **Result** | ✅ Pass |
|
||||
|
||||
Login succeeds using the email associated with the Ghaymah account. If no account exists, `gy signup` should be used instead.
|
||||
|
||||
---
|
||||
|
||||
### 5. Logout
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy logout` |
|
||||
| **Expected** | Sign out and clear local session |
|
||||
| **Result** | ⚠️ Non-blocking issue |
|
||||
|
||||
**Output:**
|
||||
```
|
||||
Remote sign-out call failed with 400 error: "refreshToken" is required
|
||||
✅ Logged out
|
||||
```
|
||||
|
||||
Logout succeeds **locally** and clears the local config. However, the **remote sign-out request fails** because the `refreshToken` is not being included in the request body.
|
||||
|
||||
> **Bug:** The API call for remote sign-out is missing the required `refreshToken` field. The session may remain active server-side until token expiry.
|
||||
|
||||
---
|
||||
|
||||
### 6. Whoami
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy whoami` |
|
||||
| **Expected** | Display the currently authenticated user's email and User ID |
|
||||
| **Result** | ✅ Pass |
|
||||
|
||||
**Output:**
|
||||
```
|
||||
✅ Logged in as username@gmail.com
|
||||
User ID: <id>
|
||||
```
|
||||
|
||||
No authentication tokens or secrets are exposed. The command correctly identifies the logged-in user.
|
||||
|
||||
---
|
||||
|
||||
### 7. List
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy list` |
|
||||
| **Expected** | List all projects associated with the authenticated account |
|
||||
| **Result** | ✅ Pass |
|
||||
|
||||
**Output:**
|
||||
```
|
||||
[proj] synced ghaymah-exam-mahmoud-secops
|
||||
```
|
||||
|
||||
Projects are listed with their synchronization status as expected.
|
||||
|
||||
---
|
||||
|
||||
## Deployment Tests
|
||||
|
||||
### Node.js Deployment
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy deploy` |
|
||||
| **Requirements** | `package.json`, `package-lock.json`, HTTP server listening on port `8080` |
|
||||
| **Result** | ✅ Pass |
|
||||
|
||||
**Steps:**
|
||||
1. Created a minimal Node.js Hello World project.
|
||||
2. Initial attempt without a lockfile failed with an `npm ci` error → generated `package-lock.json`.
|
||||
3. First runtime test failed because `index.js` was missing and later returned `502` because the app wasn't listening on port `8080`.
|
||||
4. Updated `index.js` to start an HTTP server on port `8080`, redeployed, and verified using `gy config`, `gy logs`, and the deployed URL.
|
||||
|
||||
**Output:**
|
||||
```
|
||||
Hello from Node.js!
|
||||
```
|
||||
|
||||
> **Note:** The project **must** include a `package-lock.json` and the application **must** listen on port `8080` (or the configured port) for deployment to succeed.
|
||||
|
||||
---
|
||||
|
||||
### Flask Deployment
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy deploy --port=5000` |
|
||||
| **Requirements** | Flask application listening on the configured port |
|
||||
| **Result** | ✅ Pass |
|
||||
|
||||
**Steps:**
|
||||
1. Initial deployment using the default port `8080` resulted in `502 Bad Gateway`.
|
||||
2. Changed the application to listen on port `5000`.
|
||||
3. Redeployed with `gy deploy --port=5000` — deployment completed successfully.
|
||||
|
||||
> **Note:** The `--port` flag must match the port the application actually listens on.
|
||||
|
||||
---
|
||||
|
||||
### Go (Non-Web) Deployment
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy deploy` |
|
||||
| **Requirements** | Valid Go project |
|
||||
| **Result** | ❌ Fail |
|
||||
|
||||
**Steps:**
|
||||
1. Deployed a simple Go program that prints `Hello from Go!` — not a web server.
|
||||
2. The CLI generated a Dockerfile, built, and uploaded the artifact successfully.
|
||||
3. However, resource creation failed with a `resources_pkey` uniqueness violation.
|
||||
4. In an earlier attempt, the CLI waited for **15 minutes** before timing out, while `gy logs go-test` showed `Hello from Go!`.
|
||||
|
||||
**Output:**
|
||||
```
|
||||
resources_pkey uniqueness violation
|
||||
(earlier) Deployment timeout after 15 minutes
|
||||
Hello from Go! ← from logs
|
||||
```
|
||||
|
||||
> **Bug:** The CLI successfully detects and builds the Go application, but deployment does not complete and leaves the state inconsistent. See [Known Issues](#known-issues--bugs).
|
||||
|
||||
---
|
||||
|
||||
### Go (Web) Deployment
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy deploy --port=8000` |
|
||||
| **Requirements** | Go HTTP server listening on the configured port |
|
||||
| **Result** | ✅ Pass |
|
||||
|
||||
**Steps:**
|
||||
1. Created a minimal Go HTTP server using `net/http` listening on port `8000`.
|
||||
2. Deployed without providing a Dockerfile — the CLI auto-detected Go and generated one.
|
||||
3. Deployment completed and the app was reachable.
|
||||
|
||||
**Output:**
|
||||
```
|
||||
Hello from Go Web!
|
||||
```
|
||||
|
||||
The CLI correctly detects, builds, and deploys a Go web application when the app listens on the configured port.
|
||||
|
||||
---
|
||||
|
||||
### FastAPI Deployment
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy deploy --port=8000` |
|
||||
| **Requirements** | FastAPI application with Uvicorn listening on `0.0.0.0:8000` |
|
||||
| **Result** | ⚠️ Partial — CLI Issue |
|
||||
|
||||
**Steps:**
|
||||
1. Deployed a FastAPI application using the CLI-generated Dockerfile.
|
||||
2. Deployment completed successfully, but accessing the URL returned `502 Bad Gateway`.
|
||||
3. Investigation revealed the auto-generated Dockerfile used `CMD ["sh", "-c", "python main.py"]`.
|
||||
4. Since `main.py` only defines the FastAPI app object and does not start a server, the container never listens on the port.
|
||||
5. Verified locally using `uvicorn main:app --host 0.0.0.0 --port 8000` — worked correctly.
|
||||
|
||||
**Output:**
|
||||
```
|
||||
Deployment completed successfully.
|
||||
URL: 502 Bad Gateway
|
||||
```
|
||||
|
||||
> **Bug:** The auto-generated Dockerfile for FastAPI uses `python main.py` instead of a Uvicorn command. The correct CMD should be:
|
||||
> ```dockerfile
|
||||
> CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
> ```
|
||||
|
||||
---
|
||||
|
||||
## Tunneling
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy tunnel start <name>` |
|
||||
| **Requirements** | A successfully deployed application with a configured port |
|
||||
| **Result** | ❌ Fail |
|
||||
|
||||
**Steps:**
|
||||
1. Started the tunnel for a deployed application.
|
||||
2. The CLI reported the tunnel as live but the endpoint was unreachable.
|
||||
3. The tunnel incorrectly targeted `localhost:8080` instead of the configured application port.
|
||||
4. The tunnel URL returned was empty: `https://`.
|
||||
|
||||
**Output:**
|
||||
```
|
||||
Tunnel is live at: https://
|
||||
```
|
||||
|
||||
> **Bug:** Three separate issues were identified (see [Known Issues](#known-issues--bugs)):
|
||||
> - Hardcoded port `8080` instead of using the configured application port.
|
||||
> - Empty tunnel URL returned.
|
||||
> - Tunnel reported as live despite being unreachable.
|
||||
|
||||
---
|
||||
|
||||
## Delete Operations
|
||||
|
||||
### Delete Project
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy delete project ghaymah-exam-mahmoud-secops` |
|
||||
| **Result** | ✅ Pass |
|
||||
|
||||
**Output:**
|
||||
```
|
||||
✅ Deleted project 'ghaymah-exam-mahmoud-secops'
|
||||
```
|
||||
|
||||
A confirmation prompt appeared before deletion. The project and all its associated apps were deleted successfully.
|
||||
|
||||
---
|
||||
|
||||
### Delete App
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy delete app <name>` |
|
||||
| **Result** | ⚠️ Works for unique names; ambiguous for duplicates |
|
||||
|
||||
App deletion works correctly when the app name is unique. However, if **duplicate app names exist**, the CLI cannot uniquely identify the target and the deletion becomes unreliable.
|
||||
|
||||
> **Bug:** The CLI relies solely on the friendly name for app deletion. Duplicate names should either be prevented or the CLI should support deletion by unique resource ID or interactive selection.
|
||||
|
||||
---
|
||||
|
||||
## Security Tests
|
||||
|
||||
### Secret Exposure in CLI Output
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy deploy --debug` |
|
||||
| **Test** | Checked whether fake credentials in `.env` and `config.txt` appear in CLI output or app logs |
|
||||
| **Result** | ✅ Pass |
|
||||
|
||||
Deployment completed successfully. The CLI output and application logs did **not** contain the test credentials.
|
||||
|
||||
---
|
||||
|
||||
### Sensitive File Inclusion
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy deploy --debug` |
|
||||
| **Test** | Verified whether `.dockerignore` rules are honored and sensitive files are excluded |
|
||||
| **Result** | ✅ Pass (with `.dockerignore`) / ❌ Fail (without `.dockerignore`) |
|
||||
|
||||
**With `.dockerignore`:**
|
||||
Created `.dockerignore` rules excluding `.env` and `config.txt`. The deployed container contained only the allowed files.
|
||||
|
||||
```
|
||||
total 16
|
||||
.dockerignore
|
||||
Dockerfile
|
||||
```
|
||||
|
||||
**Without `.dockerignore`:**
|
||||
Deployed a project containing `.env` and `config.txt` without a `.dockerignore`. Both files were present in the deployed container.
|
||||
|
||||
```
|
||||
.env
|
||||
Dockerfile
|
||||
config.txt
|
||||
```
|
||||
|
||||
> **Security Issue:** Sensitive files are **not automatically excluded** from the deployment. Developers must manually configure `.dockerignore`. The CLI should warn users when sensitive-looking files (e.g., `.env`, `*.key`, `config.txt`) are included in the build context.
|
||||
|
||||
---
|
||||
|
||||
### Git Metadata Exposure
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy deploy --debug` |
|
||||
| **Test** | Verified whether `.git` directory and metadata are included in the deployed container |
|
||||
| **Result** | ✅ Pass |
|
||||
|
||||
Created a Git repository and committed project files. Added a Dockerfile that checks for `.git` and `.git/config`. The deployed container did **not** contain the `.git` directory.
|
||||
|
||||
**Output:**
|
||||
```
|
||||
=== GIT CHECK ===
|
||||
ls: .git: No such file or directory
|
||||
=== CONFIG CHECK ===
|
||||
cat: can't open '.git/config': No such file or directory
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Dockerfile Hardcoded Secret Detection
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy deploy --debug` |
|
||||
| **Test** | Verified whether the CLI warns about hardcoded secrets in `ENV` instructions |
|
||||
| **Result** | ❌ Fail |
|
||||
|
||||
Created a Dockerfile with `ENV TEST_SECRET="FAKE_SECRET_12345"`. Deployed the application and inspected the output and logs.
|
||||
|
||||
**Output:**
|
||||
```
|
||||
Application started
|
||||
TEST_SECRET=FAKE_SECRET_12345
|
||||
```
|
||||
|
||||
The CLI deployed the application **without any security warning**. The hardcoded secret was accessible inside the running container and appeared in the application logs.
|
||||
|
||||
> **Security Issue:** The CLI does not scan or warn about hardcoded secrets in `ENV` instructions within the Dockerfile.
|
||||
|
||||
---
|
||||
|
||||
### AI-Assisted Secret Detection
|
||||
|
||||
| Field | Details |
|
||||
|---|---|
|
||||
| **Command** | `gy deploy --debug --ai-key="$GY_AI_KEY"` |
|
||||
| **Test** | Verified whether the `--ai-key` flag enables detection of hardcoded secrets in source code |
|
||||
| **Result** | ❌ Fail |
|
||||
|
||||
Created a test project with a hardcoded API key in `app.sh`. Deployed with `--ai-key` flag enabled. No security warning or detection message was observed.
|
||||
|
||||
**Output:**
|
||||
```
|
||||
Application started
|
||||
```
|
||||
|
||||
> **Security Issue:** The AI-assisted secret detection feature (`--ai-key`) did not identify or warn about the hardcoded API key. This feature appears to be non-functional or not yet implemented.
|
||||
|
||||
---
|
||||
|
||||
## Known Issues & Bugs
|
||||
|
||||
### 🐛 Bug-01 — Duplicate App Names Cause Ambiguity
|
||||
|
||||
- **Affected Command:** `gy delete app <name>`
|
||||
- **Description:** The CLI identifies apps by friendly name only. When multiple apps share the same name, the CLI cannot uniquely target an app for deletion.
|
||||
- **Recommendation:** Prevent duplicate app names at creation time, or support deletion by unique resource ID / interactive disambiguation.
|
||||
|
||||
---
|
||||
|
||||
### 🐛 Bug-02 — Default Port Assumption Causes 502 Errors
|
||||
|
||||
- **Affected Commands:** `gy deploy`, `gy tunnel start`
|
||||
- **Description:** The CLI assumes port `8080` by default. Applications listening on a different port will result in `502 Bad Gateway`.
|
||||
- **Recommendation:** Detect the actual application port automatically, or require explicit port configuration with a clearer error message.
|
||||
|
||||
---
|
||||
|
||||
### 🐛 Bug-03 — Tunnel Fails Completely
|
||||
|
||||
- **Affected Command:** `gy tunnel start <name>`
|
||||
- **Description:** Three compounding issues:
|
||||
1. Port is hardcoded to `8080` instead of using the configured application port.
|
||||
2. The tunnel URL is returned empty (`https://`).
|
||||
3. The tunnel is reported as **live** despite being unreachable.
|
||||
- **Recommendation:** Fix port synchronization with deployment config, validate the tunnel URL before reporting success, and surface a proper error if the tunnel fails to establish.
|
||||
|
||||
---
|
||||
|
||||
### 🐛 Bug-04 — Go Deployment Timeout & Inconsistent State
|
||||
|
||||
- **Affected Command:** `gy deploy` (Go projects)
|
||||
- **Description:** The CLI timed out after 15 minutes for a non-web Go deployment. A `resources_pkey` uniqueness violation was also observed in a later attempt. The deployment state was left inconsistent — the app appeared to run (logs were visible) but was never confirmed as deployed.
|
||||
- **Recommendation:** Improve deployment state reporting, handle database constraint violations gracefully, and surface a clear error when a timeout occurs.
|
||||
|
||||
---
|
||||
|
||||
### 🐛 Bug-05 — FastAPI Auto-Generated Dockerfile Uses Wrong CMD
|
||||
|
||||
- **Affected Command:** `gy deploy` (FastAPI projects)
|
||||
- **Description:** The auto-generated Dockerfile for FastAPI uses `CMD ["sh", "-c", "python main.py"]`, which does not start a web server. The result is a `502 Bad Gateway` even though the deployment infrastructure succeeds.
|
||||
- **Recommendation:** Generate a correct Uvicorn command for FastAPI projects:
|
||||
```dockerfile
|
||||
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🔒 Security-01 — Sensitive File Not Auto-Excluded
|
||||
|
||||
- **Affected Command:** `gy deploy`
|
||||
- **Description:** Files such as `.env` and `config.txt` are included in the deployment by default if no `.dockerignore` is present.
|
||||
- **Impact:** Potential exposure of credentials or other sensitive configuration in the deployed container.
|
||||
- **Recommendation:** Warn users when sensitive-looking files are present in the build context, or automatically exclude common secret files (`.env`, `*.key`, `*.pem`, etc.) unless explicitly included.
|
||||
|
||||
---
|
||||
|
||||
### 🔒 Security-02 — Dockerfile `ENV` Secrets Not Detected
|
||||
|
||||
- **Affected Command:** `gy deploy`
|
||||
- **Description:** The CLI does not scan Dockerfiles for hardcoded secrets in `ENV` instructions. Secrets deployed this way are accessible inside the container and may appear in logs.
|
||||
- **Impact:** Potential exposure of credentials or sensitive values hardcoded in the Dockerfile.
|
||||
- **Recommendation:** Implement static analysis of the Dockerfile to detect and warn about `ENV` instructions containing secret-like values.
|
||||
|
||||
---
|
||||
|
||||
### 🔒 Security-03 — AI Secret Detection Not Functional
|
||||
|
||||
- **Affected Command:** `gy deploy --ai-key`
|
||||
- **Description:** The `--ai-key` flag is documented as enabling AI-assisted secret detection, but no warning or detection was observed when deploying a project with a hardcoded API key in source code.
|
||||
- **Impact:** Users may rely on this feature as a security safeguard, but it provides no protection in its current state.
|
||||
- **Recommendation:** Fix or fully implement the AI-assisted secret scanning feature, and clearly document its current limitations.
|
||||
|
||||
---
|
||||
|
||||
## Verdict Summary Table
|
||||
|
||||
| Test | Command | Status | Notes |
|
||||
|---|---|---|---|
|
||||
| Version | `gy version` | ✅ Pass | Correct version and update status |
|
||||
| Help | `gy --help` | ✅ Pass | Clear output |
|
||||
| Signup | `gy signup` | ✅ Pass | Creates account + auto-login |
|
||||
| Login | `gy login` | ✅ Pass | — |
|
||||
| Logout | `gy logout` | ⚠️ Non-blocking | Remote sign-out missing `refreshToken` |
|
||||
| Whoami | `gy whoami` | ✅ Pass | No token leakage |
|
||||
| List | `gy list` | ✅ Pass | Shows sync status |
|
||||
| Node.js Deploy | `gy deploy` | ✅ Pass | Requires lockfile + port 8080 |
|
||||
| Flask Deploy | `gy deploy --port=5000` | ✅ Pass | Port flag required |
|
||||
| Go Deploy (non-web) | `gy deploy` | ❌ Fail | DB violation / timeout / inconsistent state |
|
||||
| Go Web Deploy | `gy deploy --port=8000` | ✅ Pass | Auto-detected + deployed |
|
||||
| FastAPI Deploy | `gy deploy --port=8000` | ⚠️ Partial | Wrong CMD in generated Dockerfile |
|
||||
| Tunnel | `gy tunnel start` | ❌ Fail | Wrong port + empty URL + false live status |
|
||||
| Delete Project | `gy delete project` | ✅ Pass | Confirmation prompt works |
|
||||
| Delete App | `gy delete app` | ⚠️ Partial | Fails on duplicate names |
|
||||
| Secret in CLI Output | `gy deploy --debug` | ✅ Pass | No leakage in output |
|
||||
| Sensitive File Inclusion | `gy deploy --debug` | ❌ Fail | `.env` included without `.dockerignore` |
|
||||
| Git Metadata Exposure | `gy deploy --debug` | ✅ Pass | `.git` excluded |
|
||||
| Dockerfile Secret Detection | `gy deploy --debug` | ❌ Fail | No warning for `ENV` secrets |
|
||||
| AI Secret Detection | `gy deploy --debug --ai-key` | ❌ Fail | Feature non-functional |
|
||||
|
||||
129
notes.txt
129
notes.txt
@@ -133,6 +133,115 @@ Output: Deployment completed successfully, but the generated URL returned 502 Ba
|
||||
|
||||
Verdict: Partial / CLI issue — FastAPI deployment succeeds, but the auto-generated Dockerfile incorrectly starts python main.py instead of Uvicorn, causing the deployed application to be unreachable.
|
||||
|
||||
|
||||
Secret Exposure Test
|
||||
|
||||
Command: gy deploy --debug
|
||||
|
||||
Test: Created a test project containing fake credentials in .env and config.txt.
|
||||
|
||||
Steps: Deployed the project and reviewed the CLI output and application logs to check whether the test credentials were exposed.
|
||||
|
||||
Result: Deployment completed successfully. The application logs did not contain the test credentials.
|
||||
|
||||
|
||||
Sensitive File Inclusion Test
|
||||
|
||||
Command: gy deploy --debug
|
||||
|
||||
Test: Verified whether sensitive files excluded by .dockerignore are included in the deployed container.
|
||||
|
||||
Steps: Created .dockerignore rules excluding .env and config.txt. Added a Dockerfile that lists the contents of /app, then deployed the project and inspected the application logs.
|
||||
|
||||
Result: Deployment completed successfully. The deployed container contained only the Dockerfile and .dockerignore; .env and config.txt were not included.
|
||||
|
||||
Output:
|
||||
total 16
|
||||
.dockerignore
|
||||
Dockerfile
|
||||
|
||||
Verdict: Pass — the deployment respected the .dockerignore rules and excluded the sensitive test files from the deployed container.
|
||||
|
||||
Output:
|
||||
Ready! Add a CMD to your Dockerfile
|
||||
|
||||
Verdict: No secret exposure observed in the CLI output or application logs during this test.
|
||||
|
||||
|
||||
Security – Sensitive File Exposure
|
||||
|
||||
Command: gy deploy
|
||||
|
||||
Requirements: A project containing sensitive-looking files such as .env and config.txt.
|
||||
|
||||
Steps: Created a test project containing a fake access key in .env and a fake password in config.txt. Deployed the project using gy deploy --debug, then checked the deployed container using gy logs.
|
||||
|
||||
Result: The deployed container filesystem contained both .env and config.txt, meaning these files were included in the deployment by default.
|
||||
|
||||
Output:
|
||||
.env
|
||||
Dockerfile
|
||||
config.txt
|
||||
|
||||
Verdict: Potential security issue: Sensitive files are not automatically excluded from the deployment. In a real project, this could expose credentials or other sensitive configuration.
|
||||
|
||||
|
||||
Git Metadata Exposure Test
|
||||
|
||||
Command: gy deploy --debug
|
||||
|
||||
Test: Verified whether the .git directory and Git metadata are included in the deployed container.
|
||||
|
||||
Steps: Created a Git repository in the test project and committed project files. Added a Dockerfile that checks for the .git directory and .git/config, then deployed the project and inspected the application logs.
|
||||
|
||||
Result: Deployment completed successfully. The deployed container did not contain the .git directory or Git metadata.
|
||||
|
||||
Output:
|
||||
|
||||
=== GIT CHECK ===
|
||||
ls: .git: No such file or directory
|
||||
=== CONFIG CHECK ===
|
||||
cat: can't open '.git/config': No such file or directory
|
||||
|
||||
Verdict: Pass — the deployment excluded the .git directory and Git metadata from the deployed container.
|
||||
|
||||
Verdict: No Git metadata exposure observed in the deployed container.
|
||||
|
||||
Dockerfile Hardcoded Secret Detection Test
|
||||
|
||||
Command: gy deploy --debug
|
||||
|
||||
Test: Verified whether the CLI detects and warns about hardcoded secrets defined in the Dockerfile.
|
||||
|
||||
Steps: Created a Dockerfile containing a fake secret using ENV TEST_SECRET="FAKE_SECRET_12345", deployed the application, and inspected the deployment output and application logs.
|
||||
|
||||
Result: Deployment completed successfully without any security warning or secret-detection message. The hardcoded secret was accessible inside the running container and appeared in the application logs.
|
||||
|
||||
Output:
|
||||
|
||||
Application started
|
||||
TEST_SECRET=FAKE_SECRET_12345
|
||||
|
||||
Verdict: Fail — The CLI did not detect or warn about the hardcoded secret defined in the Dockerfile ENV instruction.
|
||||
|
||||
AI Secret Detection Test
|
||||
|
||||
Command: `gy deploy --debug --ai-key="$GY_AI_KEY"`
|
||||
|
||||
Test: Verified whether the CLI's AI-assisted detection identifies and warns about hardcoded secrets in the project source code.
|
||||
|
||||
Steps: Created a test project containing a fake hardcoded API key in `app.sh`, deployed the application with AI-assisted detection enabled using `--ai-key="$GY_AI_KEY"`, and inspected the deployment output and application logs.
|
||||
|
||||
Result: Deployment completed successfully without any security warning or secret-detection message. The CLI did not indicate that the hardcoded API key was detected.
|
||||
|
||||
Output:
|
||||
|
||||
Application started
|
||||
|
||||
|
||||
Verdict: Fail — The CLI's AI-assisted detection did not detect or warn about the hardcoded API key during deployment.
|
||||
|
||||
|
||||
Problems
|
||||
|
||||
Duplicate Names Problem
|
||||
@@ -159,3 +268,23 @@ FastAPI Auto-Generated Dockerfile Problem
|
||||
The CLI successfully detects FastAPI and generates a Dockerfile, but the generated Dockerfile uses CMD ["sh", "-c", "python main.py"]. Since the FastAPI application only defines the app object and does not start a web server, the container does not listen on the configured port, resulting in 502 Bad Gateway. The CLI should generate a FastAPI-compatible command such as Uvicorn with 0.0.0.0 and the configured port.
|
||||
|
||||
|
||||
Security – Sensitive File Exposure
|
||||
|
||||
The CLI deploys sensitive-looking files such as .env and config.txt to the container by default. No automatic detection or exclusion was observed. In a real project, this could expose credentials or other sensitive configuration.
|
||||
|
||||
Impact: Potential exposure of secrets if sensitive files are included in the project.
|
||||
|
||||
Security – Dockerfile Secret Exposure
|
||||
|
||||
The CLI allows hardcoded secrets defined in the Dockerfile using the `ENV` instruction to be deployed without any security warning or detection. The secret was accessible inside the running container and appeared in the application logs.
|
||||
|
||||
Impact: Potential exposure of credentials or other sensitive values when secrets are hardcoded directly in the Dockerfile.
|
||||
|
||||
Security – AI Secret Detection
|
||||
|
||||
The CLI provides an AI-assisted secret detection feature through the `--ai-key` option, but no warning or detection was observed when deploying a project containing a hardcoded API key.
|
||||
|
||||
Impact: Potential exposure of hardcoded secrets if the AI-assisted detection fails to identify sensitive credentials before deployment.
|
||||
|
||||
Verdict: Fail — With AI-assisted detection enabled using `--ai-key`, the CLI successfully deployed the project without detecting or warning about the hardcoded API key.
|
||||
|
||||
|
||||
13
test-projects/ai-secret-test/.gy.json
Normal file
13
test-projects/ai-secret-test/.gy.json
Normal file
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"app": "ai-secret-test",
|
||||
"project": "ai-secret-test",
|
||||
"_app_id": "435cdcb4-f00f-40a4-8c87-88986a869925",
|
||||
"_project_id": "2083f529-564d-4e7d-a62c-bce955a2f117",
|
||||
"port": 8080,
|
||||
"publicAccess": {
|
||||
"enabled": true,
|
||||
"domain": "ai-secret-test-da24302cf9e6.hosted.ghaymah.systems"
|
||||
},
|
||||
"resourceTier": "t1",
|
||||
"_detected": "dockerfile"
|
||||
}
|
||||
6
test-projects/ai-secret-test/Dockerfile
Normal file
6
test-projects/ai-secret-test/Dockerfile
Normal file
@@ -0,0 +1,6 @@
|
||||
FROM alpine:latest
|
||||
|
||||
COPY app.sh /app.sh
|
||||
RUN chmod +x /app.sh
|
||||
|
||||
CMD ["/app.sh"]
|
||||
4
test-projects/ai-secret-test/app.sh
Normal file
4
test-projects/ai-secret-test/app.sh
Normal file
@@ -0,0 +1,4 @@
|
||||
#!/bin/sh
|
||||
API_KEY="FAKE_SECRET_12345"
|
||||
echo "Application started"
|
||||
sleep 3600
|
||||
13
test-projects/dockerfile-secret-test/.gy.json
Normal file
13
test-projects/dockerfile-secret-test/.gy.json
Normal file
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"app": "dockerfile-secret-test",
|
||||
"project": "dockerfile-secret-test",
|
||||
"_app_id": "1a610b79-dba3-4bc3-a10b-0c03d0f6ad6c",
|
||||
"_project_id": "cd3c975e-498a-4f1b-a6d3-583aa237782a",
|
||||
"port": 8080,
|
||||
"publicAccess": {
|
||||
"enabled": true,
|
||||
"domain": "dockerfile-secret-test-44651a81fd96.hosted.ghaymah.systems"
|
||||
},
|
||||
"resourceTier": "t1",
|
||||
"_detected": "dockerfile"
|
||||
}
|
||||
7
test-projects/dockerfile-secret-test/Dockerfile
Normal file
7
test-projects/dockerfile-secret-test/Dockerfile
Normal file
@@ -0,0 +1,7 @@
|
||||
FROM alpine:latest
|
||||
|
||||
ENV TEST_SECRET="FAKE_SECRET_12345"
|
||||
|
||||
RUN echo "Building application..."
|
||||
|
||||
CMD ["sh", "-c", "echo 'Application started'; echo \"TEST_SECRET=$TEST_SECRET\"; sleep 3600"]
|
||||
1
test-projects/git-exposure-test
Submodule
1
test-projects/git-exposure-test
Submodule
تمت إضافة الوحدة الفرعية test-projects/git-exposure-test في 57b6ac3939
1
test-projects/security-secret-test/.env
Normal file
1
test-projects/security-secret-test/.env
Normal file
@@ -0,0 +1 @@
|
||||
ACCESS_KEY_ID=FAKE_TEST_KEY_12345
|
||||
13
test-projects/security-secret-test/.gy.json
Normal file
13
test-projects/security-secret-test/.gy.json
Normal file
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"app": "security-secret-test",
|
||||
"project": "security-secret-test",
|
||||
"_app_id": "602ac150-27f6-4c58-9231-be2790561266",
|
||||
"_project_id": "ee34a34c-5d79-45a4-955b-acf048fe75b2",
|
||||
"port": 8080,
|
||||
"publicAccess": {
|
||||
"enabled": true,
|
||||
"domain": "security-secret-test-6ec4ac826b8b.hosted.ghaymah.systems"
|
||||
},
|
||||
"resourceTier": "t1",
|
||||
"_detected": "unknown"
|
||||
}
|
||||
4
test-projects/security-secret-test/Dockerfile
Normal file
4
test-projects/security-secret-test/Dockerfile
Normal file
@@ -0,0 +1,4 @@
|
||||
FROM alpine:latest
|
||||
WORKDIR /app
|
||||
COPY . .
|
||||
CMD ["sh", "-c", "ls -la /app"]
|
||||
1
test-projects/security-secret-test/config.txt
Normal file
1
test-projects/security-secret-test/config.txt
Normal file
@@ -0,0 +1 @@
|
||||
password=FAKE_PASSWORD_123
|
||||
المرجع في مشكلة جديدة
حظر مستخدم