This document reports the results of manual functional and security testing performed on the Ghaymah CLI (gy). Tests cover core authentication commands, multi-language deployment scenarios, tunneling, resource management, and several security-focused scenarios including secret exposure and file inclusion.

Test Summary

Category	Total Tests	✅ Pass	⚠️ Partial	❌ Fail
Core Commands	7	6	1	0
Deployments	5	3	1	1
Tunneling	1	0	0	1
Delete Operations	2	1	1	0
Security Tests	5	2	0	3
Edge & Error Cases	8	5	1	2
Total	28	17	4	7

Core Commands

1. Version

Field	Details
Command	`gy version`
Expected	Display current CLI version and update status
Result	✅ Pass

Output:

Ghaymah CLI v0.0.24
You're up to date!

The CLI correctly reports its current version and confirms it is up to date.

2. Help

Field	Details
Command	`gy --help`
Expected	Display CLI description, available commands, usage examples, and global flags
Result	✅ Pass

The help output is clear and provides all necessary information about the available commands and CLI options.

Field	Details
Command	`gy signup`
Expected	Create a new account and auto-login
Result	✅ Pass

Output:

✅ Account created! Logged in as username@gmail.com

Note: An informational log config file not found, using defaults appeared during signup. This is expected behavior and is not an issue.

Field	Details
Command	`gy login`
Expected	Authenticate with existing Ghaymah account credentials
Result	✅ Pass

Login succeeds using the email associated with the Ghaymah account. If no account exists, gy signup should be used instead.

5. Logout

Field	Details
Command	`gy logout`
Expected	Sign out and clear local session
Result	⚠️ Non-blocking issue

Output:

Remote sign-out call failed with 400 error: "refreshToken" is required
✅ Logged out

Logout succeeds locally and clears the local config. However, the remote sign-out request fails because the refreshToken is not being included in the request body.

Bug: The API call for remote sign-out is missing the required refreshToken field. The session may remain active server-side until token expiry.

6. Whoami

Field	Details
Command	`gy whoami`
Expected	Display the currently authenticated user's email and User ID
Result	✅ Pass

Output:

✅ Logged in as username@gmail.com
User ID: <id>

No authentication tokens or secrets are exposed. The command correctly identifies the logged-in user.

7. List

Field	Details
Command	`gy list`
Expected	List all projects associated with the authenticated account
Result	✅ Pass

Output:

[proj] synced ghaymah-exam-mahmoud-secops

Projects are listed with their synchronization status as expected.

Deployment Tests

Node.js Deployment

Field	Details
Command	`gy deploy`
Requirements	`package.json`, `package-lock.json`, HTTP server listening on port `8080`
Result	✅ Pass

Steps:

Created a minimal Node.js Hello World project.
Initial attempt without a lockfile failed with an npm ci error → generated package-lock.json.
First runtime test failed because index.js was missing and later returned 502 because the app wasn't listening on port 8080.
Updated index.js to start an HTTP server on port 8080, redeployed, and verified using gy config, gy logs, and the deployed URL.

Output:

Hello from Node.js!

Note: The project must include a package-lock.json and the application must listen on port 8080 (or the configured port) for deployment to succeed.

Flask Deployment

Field	Details
Command	`gy deploy --port=5000`
Requirements	Flask application listening on the configured port
Result	✅ Pass

Steps:

Initial deployment using the default port 8080 resulted in 502 Bad Gateway.
Changed the application to listen on port 5000.
Redeployed with gy deploy --port=5000 — deployment completed successfully.

Note: The --port flag must match the port the application actually listens on.

Go (Non-Web) Deployment

Field	Details
Command	`gy deploy`
Requirements	Valid Go project
Result	❌ Fail

Steps:

Deployed a simple Go program that prints Hello from Go! — not a web server.
The CLI generated a Dockerfile, built, and uploaded the artifact successfully.
However, resource creation failed with a resources_pkey uniqueness violation.
In an earlier attempt, the CLI waited for 15 minutes before timing out, while gy logs go-test showed Hello from Go!.

Output:

resources_pkey uniqueness violation
(earlier) Deployment timeout after 15 minutes
Hello from Go!  ← from logs

Bug: The CLI successfully detects and builds the Go application, but deployment does not complete and leaves the state inconsistent. See Known Issues.

Go (Web) Deployment

Field	Details
Command	`gy deploy --port=8000`
Requirements	Go HTTP server listening on the configured port
Result	✅ Pass

Steps:

Created a minimal Go HTTP server using net/http listening on port 8000.
Deployed without providing a Dockerfile — the CLI auto-detected Go and generated one.
Deployment completed and the app was reachable.

Output:

Hello from Go Web!

The CLI correctly detects, builds, and deploys a Go web application when the app listens on the configured port.

FastAPI Deployment

Field	Details
Command	`gy deploy --port=8000`
Requirements	FastAPI application with Uvicorn listening on `0.0.0.0:8000`
Result	⚠️ Partial — CLI Issue

Steps:

Deployed a FastAPI application using the CLI-generated Dockerfile.
Deployment completed successfully, but accessing the URL returned 502 Bad Gateway.
Investigation revealed the auto-generated Dockerfile used CMD ["sh", "-c", "python main.py"].
Since main.py only defines the FastAPI app object and does not start a server, the container never listens on the port.
Verified locally using uvicorn main:app --host 0.0.0.0 --port 8000 — worked correctly.

Output:

Deployment completed successfully.
URL: 502 Bad Gateway

Bug: The auto-generated Dockerfile for FastAPI uses python main.py instead of a Uvicorn command. The correct CMD should be:
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Tunneling

Field	Details
Command	`gy tunnel start <name>`
Requirements	A successfully deployed application with a configured port
Result	❌ Fail

Steps:

Started the tunnel for a deployed application.
The CLI reported the tunnel as live but the endpoint was unreachable.
The tunnel incorrectly targeted localhost:8080 instead of the configured application port.
The tunnel URL returned was empty: https://.

Output:

Tunnel is live at: https://

Bug: Three separate issues were identified (see Known Issues):

Hardcoded port 8080 instead of using the configured application port.

Empty tunnel URL returned.

Tunnel reported as live despite being unreachable.

Delete Operations

Delete Project

Field	Details
Command	`gy delete project ghaymah-exam-mahmoud-secops`
Result	✅ Pass

Output:

✅ Deleted project 'ghaymah-exam-mahmoud-secops'

A confirmation prompt appeared before deletion. The project and all its associated apps were deleted successfully.

Delete App

Field	Details
Command	`gy delete app <name>`
Result	⚠️ Works for unique names; ambiguous for duplicates

App deletion works correctly when the app name is unique. However, if duplicate app names exist, the CLI cannot uniquely identify the target and the deletion becomes unreliable.

Bug: The CLI relies solely on the friendly name for app deletion. Duplicate names should either be prevented or the CLI should support deletion by unique resource ID or interactive selection.

Security Tests

Secret Exposure in CLI Output

Field	Details
Command	`gy deploy --debug`
Test	Checked whether fake credentials in `.env` and `config.txt` appear in CLI output or app logs
Result	✅ Pass

Deployment completed successfully. The CLI output and application logs did not contain the test credentials.

Sensitive File Inclusion

Field	Details
Command	`gy deploy --debug`
Test	Verified whether `.dockerignore` rules are honored and sensitive files are excluded
Result	✅ Pass (with `.dockerignore`) / ❌ Fail (without `.dockerignore`)

With .dockerignore: Created .dockerignore rules excluding .env and config.txt. The deployed container contained only the allowed files.

total 16
.dockerignore
Dockerfile

Without .dockerignore: Deployed a project containing .env and config.txt without a .dockerignore. Both files were present in the deployed container.

.env
Dockerfile
config.txt

Security Issue: Sensitive files are not automatically excluded from the deployment. Developers must manually configure .dockerignore. The CLI should warn users when sensitive-looking files (e.g., .env, *.key, config.txt) are included in the build context.

Git Metadata Exposure

Field	Details
Command	`gy deploy --debug`
Test	Verified whether `.git` directory and metadata are included in the deployed container
Result	✅ Pass

Created a Git repository and committed project files. Added a Dockerfile that checks for .git and .git/config. The deployed container did not contain the .git directory.

Output:

=== GIT CHECK ===
ls: .git: No such file or directory
=== CONFIG CHECK ===
cat: can't open '.git/config': No such file or directory

Dockerfile Hardcoded Secret Detection

Field	Details
Command	`gy deploy --debug`
Test	Verified whether the CLI warns about hardcoded secrets in `ENV` instructions
Result	❌ Fail

Created a Dockerfile with ENV TEST_SECRET="FAKE_SECRET_12345". Deployed the application and inspected the output and logs.

Output:

Application started
TEST_SECRET=FAKE_SECRET_12345

The CLI deployed the application without any security warning. The hardcoded secret was accessible inside the running container and appeared in the application logs.

Security Issue: The CLI does not scan or warn about hardcoded secrets in ENV instructions within the Dockerfile.

AI-Assisted Secret Detection

Field	Details
Command	`gy deploy --debug --ai-key="$GY_AI_KEY"`
Test	Verified whether the `--ai-key` flag enables detection of hardcoded secrets in source code
Result	❌ Fail

Created a test project with a hardcoded API key in app.sh. Deployed with --ai-key flag enabled. No security warning or detection message was observed.

Output:

Application started

Security Issue: The AI-assisted secret detection feature (--ai-key) did not identify or warn about the hardcoded API key. This feature appears to be non-functional or not yet implemented.

Known Issues & Bugs

🐛 Bug-01 — Duplicate App Names Cause Ambiguity

Affected Command: gy delete app <name>
Description: The CLI identifies apps by friendly name only. When multiple apps share the same name, the CLI cannot uniquely target an app for deletion.
Recommendation: Prevent duplicate app names at creation time, or support deletion by unique resource ID / interactive disambiguation.

🐛 Bug-02 — Default Port Assumption Causes 502 Errors

Affected Commands: gy deploy, gy tunnel start
Description: The CLI assumes port 8080 by default. Applications listening on a different port will result in 502 Bad Gateway.
Recommendation: Detect the actual application port automatically, or require explicit port configuration with a clearer error message.

🐛 Bug-03 — Tunnel Fails Completely

Affected Command: gy tunnel start <name>
Description: Three compounding issues:
1. Port is hardcoded to 8080 instead of using the configured application port.
2. The tunnel URL is returned empty (https://).
3. The tunnel is reported as live despite being unreachable.
Recommendation: Fix port synchronization with deployment config, validate the tunnel URL before reporting success, and surface a proper error if the tunnel fails to establish.

🐛 Bug-04 — Go Deployment Timeout & Inconsistent State

Affected Command: gy deploy (Go projects)
Description: The CLI timed out after 15 minutes for a non-web Go deployment. A resources_pkey uniqueness violation was also observed in a later attempt. The deployment state was left inconsistent — the app appeared to run (logs were visible) but was never confirmed as deployed.
Recommendation: Improve deployment state reporting, handle database constraint violations gracefully, and surface a clear error when a timeout occurs.

🐛 Bug-05 — FastAPI Auto-Generated Dockerfile Uses Wrong CMD

Affected Command: gy deploy (FastAPI projects)
Description: The auto-generated Dockerfile for FastAPI uses CMD ["sh", "-c", "python main.py"], which does not start a web server. The result is a 502 Bad Gateway even though the deployment infrastructure succeeds.

Recommendation: Generate a correct Uvicorn command for FastAPI projects:

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

🔒 Security-01 — Sensitive File Not Auto-Excluded

Affected Command: gy deploy
Description: Files such as .env and config.txt are included in the deployment by default if no .dockerignore is present.
Impact: Potential exposure of credentials or other sensitive configuration in the deployed container.
Recommendation: Warn users when sensitive-looking files are present in the build context, or automatically exclude common secret files (.env, *.key, *.pem, etc.) unless explicitly included.

🔒 Security-02 — Dockerfile `ENV` Secrets Not Detected

Affected Command: gy deploy
Description: The CLI does not scan Dockerfiles for hardcoded secrets in ENV instructions. Secrets deployed this way are accessible inside the container and may appear in logs.
Impact: Potential exposure of credentials or sensitive values hardcoded in the Dockerfile.
Recommendation: Implement static analysis of the Dockerfile to detect and warn about ENV instructions containing secret-like values.

🔒 Security-03 — AI Secret Detection Not Functional

Affected Command: gy deploy --ai-key
Description: The --ai-key flag is documented as enabling AI-assisted secret detection, but no warning or detection was observed when deploying a project with a hardcoded API key in source code.
Impact: Users may rely on this feature as a security safeguard, but it provides no protection in its current state.
Recommendation: Fix or fully implement the AI-assisted secret scanning feature, and clearly document its current limitations.

Additional Tests — Edge & Error Cases

Field	Details
Command	`gy login`
Input	Invalid email + short password (`wrongpassword` → length < 3 test)
Result	✅ Pass — proper error returned

Output:

Error: login failed: sign in request failed: request failed with status 400:
"minimum string length is 3" (at /password)

The CLI surfaces a clear, actionable error. Exit code is 1.

Note: The raw OpenAPI schema validation error is exposed directly to the user. A more user-friendly message such as "Invalid email or password" would improve UX.

2. Delete Non-Existent Project

Field	Details
Command	`gy delete project this-project-does-not-exist`
Result	✅ Pass — clear error, no crash

Output:

Error: project 'this-project-does-not-exist' not found — create it with 'gy deploy' or check the name

The CLI handles the not-found case gracefully with a helpful suggestion.

3. gy logs — Existing App

Field	Details
Command	`gy logs <app-name> --no-follow`
Result	✅ Pass

Output:

ℹ️  Streaming logs for app <id>...
─────────────────────────────────
2026-08-21T09:07:02Z Ready! Add a CMD to your Dockerfile

The command correctly streams logs and respects the --no-follow flag to exit after printing.

4. gy logs — Non-Existent App

Field	Details
Command	`gy logs this-app-does-not-exist`
Result	✅ Pass — clear error

Output:

Error: app 'this-app-does-not-exist' not found — deploy it first with 'gy deploy'

Not-found case is handled correctly with a clear, actionable error message.

5. gy logs — Duplicate App Name

Field	Details
Command	`gy logs pem-inclusion-test --no-follow`
Result	❌ Fail — ambiguous target

Output:

Error: multiple apps named 'pem-inclusion-test' — use 'gy list apps' to see them all

The duplicate names bug (previously documented for gy delete app) also affects gy logs. Any command that resolves apps by friendly name is impacted. Running gy logs from within the project directory (without specifying a name) correctly resolves the app and works as expected.

6. Deploy Empty Directory

Field	Details
Command	`gy deploy` (from empty directory)
Result	⚠️ Partial — deploys successfully but produces no useful app

Steps:

Created a completely empty directory with no files.
Ran gy deploy — the CLI auto-detected the project type as Unknown, auto-generated a Dockerfile, and deployed successfully.
Checked logs: the container started but had no CMD to execute.

Output (deploy):

ℹ️  Detected: Unknown
ℹ️  Auto-generated: Dockerfile
⚠️  WARNING: We auto-generated a Dockerfile for you...
✅ Deploy complete!
   URL: https://empty-deploy-test-72d66f287027.hosted.ghaymah.systems

Output (logs):

Ready! Add a CMD to your Dockerfile

Bug: The CLI should warn or refuse to deploy an empty directory, not silently deploy a broken container. An empty deployment wastes resources and may confuse users who expect a valid running application.

7. PEM & secrets.json File Inclusion

Field	Details
Command	`gy deploy --debug`
Test	Verified whether `.pem` private key and `secrets.json` are included in the deployed container
Result	❌ Fail — both files appear in container

Steps:

Created a project containing fake_key.pem (RSA private key format) and secrets.json (fake DB password).
Deployed without a .dockerignore.
Inspected container via gy logs --no-follow.

Output (logs):

=== FILE LIST ===
-rw-rw-r-- Dockerfile
-rw-rw-r-- fake_key.pem
-rw-rw-r-- secrets.json
=== PEM CHECK ===
-----BEGIN RSA PRIVATE KEY-----
FAKEFAKEFAKEFAKEFAKEFAKE
-----END RSA PRIVATE KEY-----
=== SECRETS CHECK ===
FAKE_DB_PASSWORD=supersecret123

Security Issue: This confirms that the sensitive file inclusion problem is not limited to .env files. Any file — including private keys (.pem, id_rsa) and JSON secrets — is included in the deployed container if no .dockerignore is configured. The CLI provides no warning about sensitive files.

8. Dockerfile ARG Secret Detection

Field	Details
Command	`gy deploy --debug`
Test	Verified whether CLI detects hardcoded secrets in `ARG` instructions
Result	✅ Pass (with note) — `ARG` value not exposed at runtime

Dockerfile used:

FROM alpine
ARG BUILD_SECRET="FAKE_ARG_SECRET_99999"
RUN echo "Build done"
CMD ["sh", "-c", "echo BUILD_SECRET=${BUILD_SECRET}; sleep 30"]

Output (logs):

BUILD_SECRET=

The ARG value is empty at runtime because Docker ARG variables are scoped to the build stage and are not inherited by the running container unless explicitly promoted to ENV.

Note: While ARG secrets are not exposed at container runtime, they can appear in the Docker build history (docker history --no-trunc). The CLI still produced no warning about the ARG instruction containing a secret-like value.

Verdict Summary Table

Test	Command	Status	Notes
Version	`gy version`	✅ Pass	Correct version and update status
Help	`gy --help`	✅ Pass	Clear output
Signup	`gy signup`	✅ Pass	Creates account + auto-login
Login	`gy login`	✅ Pass	—
Logout	`gy logout`	⚠️ Non-blocking	Remote sign-out missing `refreshToken`
Whoami	`gy whoami`	✅ Pass	No token leakage
List	`gy list`	✅ Pass	Shows sync status
Node.js Deploy	`gy deploy`	✅ Pass	Requires lockfile + port 8080
Flask Deploy	`gy deploy --port=5000`	✅ Pass	Port flag required
Go Deploy (non-web)	`gy deploy`	❌ Fail	DB violation / timeout / inconsistent state
Go Web Deploy	`gy deploy --port=8000`	✅ Pass	Auto-detected + deployed
FastAPI Deploy	`gy deploy --port=8000`	⚠️ Partial	Wrong CMD in generated Dockerfile
Tunnel	`gy tunnel start`	❌ Fail	Wrong port + empty URL + false live status
Delete Project (existing)	`gy delete project`	✅ Pass	Confirmation prompt works
Delete App	`gy delete app`	⚠️ Partial	Fails on duplicate names
Secret in CLI Output	`gy deploy --debug`	✅ Pass	No leakage in output
Sensitive File Inclusion (.env)	`gy deploy --debug`	❌ Fail	`.env` included without `.dockerignore`
Git Metadata Exposure	`gy deploy --debug`	✅ Pass	`.git` excluded
Dockerfile `ENV` Secret	`gy deploy --debug`	❌ Fail	No warning for `ENV` secrets
AI Secret Detection	`gy deploy --debug --ai-key`	❌ Fail	Feature non-functional
Login — Wrong Credentials	`gy login`	✅ Pass	Returns 400 error; raw schema message exposed
Delete Non-Existent Project	`gy delete project`	✅ Pass	Clear not-found error
`gy logs` — Existing App	`gy logs <name> --no-follow`	✅ Pass	Streams and exits correctly
`gy logs` — Non-Existent App	`gy logs <name>`	✅ Pass	Clear not-found error
`gy logs` — Duplicate Name	`gy logs <name>`	❌ Fail	Ambiguous — same bug as delete
Deploy Empty Directory	`gy deploy`	⚠️ Partial	Deploys but produces broken container
PEM & secrets.json Inclusion	`gy deploy --debug`	❌ Fail	Private key + secrets exposed in container
Dockerfile `ARG` Secret	`gy deploy --debug`	✅ Pass (note)	`ARG` not exposed at runtime; no CLI warning

اللغات

Dockerfile 74.1%

Go 9.6%

JavaScript 7.4%

Python 7.1%

Shell 1.8%

Readme.md إلغاء الهروب الهروب

Ghaymah CLI — Manual Test Report

Table of Contents

Overview

Test Summary

Core Commands

1. Version

2. Help

3. Signup

4. Login

5. Logout

6. Whoami

7. List

Deployment Tests

Node.js Deployment

Flask Deployment

Go (Non-Web) Deployment

Go (Web) Deployment

FastAPI Deployment

Tunneling

Delete Operations

Delete Project

Delete App

Security Tests

Secret Exposure in CLI Output

Sensitive File Inclusion

Git Metadata Exposure

Dockerfile Hardcoded Secret Detection

AI-Assisted Secret Detection

Known Issues & Bugs

🐛 Bug-01 — Duplicate App Names Cause Ambiguity

🐛 Bug-02 — Default Port Assumption Causes 502 Errors

🐛 Bug-03 — Tunnel Fails Completely

🐛 Bug-04 — Go Deployment Timeout & Inconsistent State

🐛 Bug-05 — FastAPI Auto-Generated Dockerfile Uses Wrong CMD

🔒 Security-01 — Sensitive File Not Auto-Excluded

🔒 Security-02 — Dockerfile ENV Secrets Not Detected

🔒 Security-03 — AI Secret Detection Not Functional

Additional Tests — Edge & Error Cases

1. Login with Wrong Credentials

2. Delete Non-Existent Project

3. gy logs — Existing App

4. gy logs — Non-Existent App

5. gy logs — Duplicate App Name

6. Deploy Empty Directory

7. PEM & secrets.json File Inclusion

8. Dockerfile ARG Secret Detection

Verdict Summary Table

Readme.md

🔒 Security-02 — Dockerfile `ENV` Secrets Not Detected