Ghyamah-task-1-cli2/notes.txt

CLI Test


Version

Command: gy version
Result: Successfully displayed the installed CLI version and confirmed that it is up to date.
Output:
Ghaymah CLI v0.0.24
You're up to date!
Verdict: Works as expected. The CLI correctly reports its current version and update status

Help

Command: gy --help
Result: Successfully displayed the CLI description, available commands, usage examples, and global flags.
Verdict: Works as expected. The help command provides clear information about the available commands and CLI options.
Login
Command: gy login
Result: Logged in successfully (or gy signup if no account yet — email is the one used for the Ghaymah account).

Logout

Command: gy logout
Result: Logout succeeded, but a warning appeared first:
Remote sign-out call failed with 400 error: "refreshToken" is required
CLI still cleared local config and confirmed: ✅ Logged out
Verdict: Non-blocking issue. Logout succeeds locally, but the remote sign-out request should be fixed to include the required refreshToken.

Signup

Command: gy signup
Result: Account created successfully and auto-logged in
Same info log appeared: "config file not found, using defaults" (normal, non-issue)
Output: ✅ Account created! Logged in as username@gmail.com
Verdict: Works as expected. Signup creates account + logs in in one step.

Whoami

Command: gy whoami
Result: Successfully displayed the currently authenticated user's email and User ID.
Output:
✅ Logged in as username@gmail.com
User ID: also shown
Verdict: Works as expected. The command correctly identifies the currently logged-in user and displays the associated User ID. No security issue since no authentication tokens or secrets are exposed.

List

Command: gy list
Result: Successfully retrieved the projects associated with the authenticated account.
Output:
[proj] synced ghaymah-exam-mahmoud-secops
Verdict: Works as expected. The command successfully lists the available projects and correctly shows their synchronization status.

Tunneling

Command: gy tunnel start <name>
Requirements: A successfully deployed application with a reachable configured port.
Steps: Started the tunnel for the deployed application. The tunnel incorrectly targeted localhost:8080 instead of the configured application port, then returned an empty tunnel URL.
Result: Tunnel was reported as live, but the endpoint was unreachable.
Output: Tunnel is live at: https://
Verdict: Does not work as expected due to incorrect port handling and an invalid tunnel URL

Delete

project
Command: gy delete project ghaymah-exam-mahmoud-secops
Result: Confirmation prompt appeared → confirmed with y
Output: ✅ Deleted project 'ghaymah-exam-mahmoud-secops'
Verdict: Works as expected. Project and all its apps deleted successfully with confirmation step.

app
Command: gy delete app <name>
Requirements: An existing app with a unique name.
Steps: Tested deleting an app using its name. The delete operation works correctly, but duplicate app names can cause ambiguity when identifying the target app.
Result: App deletion works as expected, but duplicate names make app selection unreliable.
Output: Successful deletion / ambiguous target when duplicate names exist.
Verdict: Works as expected for unique app names; duplicate names should be prevented or handled to avoid ambiguity.

Deployment

Node.js Deployment

Command: gy deploy
Requirements: A package.json, package-lock.json, and a web server listening on port 8080.
Steps: Created a minimal Node.js Hello World project, initially tested without a lockfile and got an npm ci error, then generated package-lock.json. The first runtime test also failed because index.js was missing and later returned 502 because the app was not listening on port 8080. Updated index.js to run an HTTP server on port 8080, then redeployed and verified with gy config, gy logs, and the deployed URL.
Result: Successfully detected, built, and deployed the Node.js application.
Output: Hello from Node.js!
Verdict: Works as expected after meeting the required Node.js project and runtime requirements.

Flask Deployment

Command: gy deploy --port=5000
Requirements: A Flask application with a web server listening on the configured port.
Steps: Initially tested the Flask application using the default port 8080, which resulted in a 502 Bad Gateway. Changed the application to listen on port 5000, then deployed using gy deploy --port=5000 and verified that the deployment completed successfully.
Result: Successfully detected, built, and deployed the Flask application after configuring the correct port.
Output: Deployed successfully with the configured port 5000.
Verdict: Works as expected when the application port matches the configured deployment port

Go Deployment

Command: gy deploy
Requirements: A valid Go project. The test application is a simple non-web Go program that prints Hello from Go!.
Steps: Deployed the Go application using the CLI-generated Dockerfile. The project was created successfully, and the deployment artifact was built and uploaded successfully. However, resource creation failed with a resources_pkey uniqueness violation. In an earlier deployment attempt, the CLI also waited for 15 minutes before timing out, while gy logs go-test showed Hello from Go!.
Result: The application produced logs, but the deployment could not be confirmed as successfully completed in the cloud.
Output: resources_pkey uniqueness violation / deployment timeout after 15 minutes / Hello from Go!
Verdict: Does not work as expected: the CLI successfully detects, builds, and uploads the Go application, but deployment does not complete successfully and the CLI leaves the deployment state inconsistent.

Go Web Deployment

Command: gy deploy --port=8000

Requirements: A Go web application with an HTTP server listening on the configured port.

Steps: Created a minimal Go HTTP server using net/http that listens on port 8000 and returns a Hello World response. Deployed the application without providing a Dockerfile to test the CLI's Go detection and automatic Dockerfile generation.

Result: The CLI successfully detected the Go web application, generated the Dockerfile, built the deployment artifact, and deployed the application successfully.

Output: Hello from Go Web!

Verdict: Works as expected. The CLI successfully detects, builds, and deploys a Go web application when the application listens on the configured port.

FastAPI Deployment

Command: gy deploy --port=8000
Requirements: A FastAPI application with Uvicorn listening on 0.0.0.0:8000.

Steps: Initially deployed the FastAPI application using the CLI-generated Dockerfile. Deployment completed successfully, but accessing the generated URL resulted in a 502 Bad Gateway. Investigation showed that the auto-generated Dockerfile used CMD ["sh", "-c", "python main.py"], while main.py only defines the FastAPI application and does not start a web server. The application was then verified locally using Uvicorn with uvicorn main:app --host 0.0.0.0 --port 8000, which successfully served the application.

Result: Deployment infrastructure completed successfully, but the generated Dockerfile did not start the FastAPI server correctly, resulting in a 502 Bad Gateway.

Output: Deployment completed successfully, but the generated URL returned 502 Bad Gateway.

Verdict: Partial / CLI issue — FastAPI deployment succeeds, but the auto-generated Dockerfile incorrectly starts python main.py instead of Uvicorn, causing the deployed application to be unreachable.


Secret Exposure Test

Command: gy deploy --debug

Test: Created a test project containing fake credentials in .env and config.txt.

Steps: Deployed the project and reviewed the CLI output and application logs to check whether the test credentials were exposed.

Result: Deployment completed successfully. The application logs did not contain the test credentials.


Sensitive File Inclusion Test

Command: gy deploy --debug

Test: Verified whether sensitive files excluded by .dockerignore are included in the deployed container.

Steps: Created .dockerignore rules excluding .env and config.txt. Added a Dockerfile that lists the contents of /app, then deployed the project and inspected the application logs.

Result: Deployment completed successfully. The deployed container contained only the Dockerfile and .dockerignore; .env and config.txt were not included.

Output:
total 16
.dockerignore
Dockerfile

Verdict: Pass — the deployment respected the .dockerignore rules and excluded the sensitive test files from the deployed container.

Output:
Ready! Add a CMD to your Dockerfile

Verdict: No secret exposure observed in the CLI output or application logs during this test.


Security – Sensitive File Exposure

Command: gy deploy

Requirements: A project containing sensitive-looking files such as .env and config.txt.

Steps: Created a test project containing a fake access key in .env and a fake password in config.txt. Deployed the project using gy deploy --debug, then checked the deployed container using gy logs.

Result: The deployed container filesystem contained both .env and config.txt, meaning these files were included in the deployment by default.

Output:
.env
Dockerfile
config.txt

Verdict: Potential security issue: Sensitive files are not automatically excluded from the deployment. In a real project, this could expose credentials or other sensitive configuration.


Git Metadata Exposure Test

Command: gy deploy --debug

Test: Verified whether the .git directory and Git metadata are included in the deployed container.

Steps: Created a Git repository in the test project and committed project files. Added a Dockerfile that checks for the .git directory and .git/config, then deployed the project and inspected the application logs.

Result: Deployment completed successfully. The deployed container did not contain the .git directory or Git metadata.

Output:

=== GIT CHECK ===
ls: .git: No such file or directory
=== CONFIG CHECK ===
cat: can't open '.git/config': No such file or directory

Verdict: Pass — the deployment excluded the .git directory and Git metadata from the deployed container.

Verdict: No Git metadata exposure observed in the deployed container.

Dockerfile Hardcoded Secret Detection Test

Command: gy deploy --debug

Test: Verified whether the CLI detects and warns about hardcoded secrets defined in the Dockerfile.

Steps: Created a Dockerfile containing a fake secret using ENV TEST_SECRET="FAKE_SECRET_12345", deployed the application, and inspected the deployment output and application logs.

Result: Deployment completed successfully without any security warning or secret-detection message. The hardcoded secret was accessible inside the running container and appeared in the application logs.

Output:

Application started
TEST_SECRET=FAKE_SECRET_12345

Verdict: Fail — The CLI did not detect or warn about the hardcoded secret defined in the Dockerfile ENV instruction.

AI Secret Detection Test

Command: `gy deploy --debug --ai-key="$GY_AI_KEY"`

Test: Verified whether the CLI's AI-assisted detection identifies and warns about hardcoded secrets in the project source code.

Steps: Created a test project containing a fake hardcoded API key in `app.sh`, deployed the application with AI-assisted detection enabled using `--ai-key="$GY_AI_KEY"`, and inspected the deployment output and application logs.

Result: Deployment completed successfully without any security warning or secret-detection message. The CLI did not indicate that the hardcoded API key was detected.

Output:

Application started


Verdict: Fail — The CLI's AI-assisted detection did not detect or warn about the hardcoded API key during deployment.


Problems

Duplicate Names Problem

Duplicate app names cause ambiguity during deletion: gy delete app <NAME> relies only on the friendly name, so multiple apps with the same name cannot be uniquely targeted. The CLI should either prevent duplicate app names or support deletion by a unique resource ID / interactive selection.

Auto listening port problem

Incorrect port detection: The CLI assumes port 8080 by default, but applications may listen on a different port, causing deployment failures such as 502 Bad Gateway. The CLI should detect the actual application port or provide a clearer configuration mechanism.

Tunneling problem

Hardcoded 8080 port instead of using the configured application port.
Empty tunnel URL returned (https://).
Tunnel is reported as live despite being unreachable.
Tunnel port is not synchronized with the deployment configuration.

Deployment Timeout / Inconsistent State

The Go deployment timed out after 15 minutes, even though the application appears to be running and producing logs. The CLI does not clearly confirm the app's deployment status in the cloud, leaving the deployment state inconsistent.

FastAPI Auto-Generated Dockerfile Problem

The CLI successfully detects FastAPI and generates a Dockerfile, but the generated Dockerfile uses CMD ["sh", "-c", "python main.py"]. Since the FastAPI application only defines the app object and does not start a web server, the container does not listen on the configured port, resulting in 502 Bad Gateway. The CLI should generate a FastAPI-compatible command such as Uvicorn with 0.0.0.0 and the configured port.


Security – Sensitive File Exposure

The CLI deploys sensitive-looking files such as .env and config.txt to the container by default. No automatic detection or exclusion was observed. In a real project, this could expose credentials or other sensitive configuration.

Impact: Potential exposure of secrets if sensitive files are included in the project.

Security – Dockerfile Secret Exposure

The CLI allows hardcoded secrets defined in the Dockerfile using the `ENV` instruction to be deployed without any security warning or detection. The secret was accessible inside the running container and appeared in the application logs.

Impact: Potential exposure of credentials or other sensitive values when secrets are hardcoded directly in the Dockerfile.

Security – AI Secret Detection

The CLI provides an AI-assisted secret detection feature through the `--ai-key` option, but no warning or detection was observed when deploying a project containing a hardcoded API key.

Impact: Potential exposure of hardcoded secrets if the AI-assisted detection fails to identify sensitive credentials before deployment.

Verdict: Fail — With AI-assisted detection enabled using `--ai-key`, the CLI successfully deployed the project without detecting or warning about the hardcoded API key.