Files
gracesalmoun/docs/production-deployment.md

305 lines
17 KiB
Markdown

# Production Docker deployment and image-pull diagnosis
## Findings and scope
Audit date: 2026-11-11. The reported Kubernetes message means the container
image has not been pulled; the application has not started. Its exact cause
cannot be identified from that message alone. A Dockerfile change does not
repair registry authentication, a missing image/tag, or node networking.
This checkout initially locked Next.js 14.2.30 (React 18.3.1, TypeScript 5.7.2)
and declared Node 20.x. Next.js requires Node >=18.17.0. Node 20 is now EOL;
the Dockerfile and package engine now use supported Node 24 LTS. A compatible
Next.js 14.2.35 patch and non-breaking lockfile security fixes are included.
Next.js 14 remains unsupported and still has security advisories. A major
framework migration is deliberately outside this Docker-focused change.
Do not interpret a successful Docker build as a clean security audit.
The original multi-stage Dockerfile already used `npm ci`, standalone output,
`node server.js`, port 3000, and host 0.0.0.0. `next.config.mjs` already enables
`output: "standalone"`; its existing options are preserved. Confirmed gaps:
- The runtime ran as root and used EOL Node 20.
- The Docker context allowed environment files, alternate `.next*` builds,
local artifacts, and ZIP source backups. They are now excluded, along with
credentials and live CMS state. No actual secret file was found or read.
- The latest commit added a filesystem CMS, authentication, and uploads.
Production reproduction: login 200, upload 200, file exists, immediate image
GET 404. Next.js indexes `public` at startup. A dynamic `/uploads/[filename]`
handler now serves new images without restarting; it accepts only flat image
filenames and leaves the existing upload URL/API unchanged.
- Container testing found `/en/resume` returning a cached 308 with no HTTP
`Location` header, carrying only a client-side Next.js redirect payload. An
explicit Next.js configuration redirect preserves the intended `/resume`
destination for HTTP clients as well as browsers.
- The runtime creates writable `data`, `public/uploads`, and `.next/cache`
directories for UID/GID 1001. Persistent storage remains an operator task.
There is no database client, database connection setting, ORM, migration,
Kubernetes manifest, Helm chart, Compose file, or CI workflow in this checkout.
The CMS stores `data/cms-content.json`; initial content is compiled from the
TypeScript data modules. All `public` assets, including the tracked portfolio
PDF and existing upload, are copied into the image. No source data directory
is needed to read the compiled defaults.
The configured Git remote is `https://app.gitpasha.com/boutmoun/gracesalmoun.git`,
not a GitHub remote. Verify which repository, branch, and commit Ghaymah actually
builds. No registry/image name or tag is recorded in this repository.
## Build and runtime configuration
| Setting | Value |
| --- | --- |
| Docker build context | Repository root (`.`) |
| Dockerfile | `./Dockerfile` |
| Build | `npm ci`, then `npm run build` in separate Docker stages |
| Output | `.next/standalone`, `.next/static`, `public` |
| Container working directory | `/app` |
| Container start command | `node server.js` (image default) |
| Runtime user/group | `nextjs:nodejs`, UID/GID `1001:1001` |
| Application address | `0.0.0.0:3000` |
| Runtime defaults | `NODE_ENV=production`, `PORT=3000`, `HOSTNAME=0.0.0.0` |
| Health check | Unauthenticated HTTP GET `/` on port 3000; expect 200 |
Use the image's default command. `npm start` remains available for a normal
local Next.js build, but the standalone container starts `node server.js`.
Do not set `NEXT_BUILD_DIR` for the image; its build uses `.next` explicitly.
No external API, database, or admin secret is required to build the application.
| Environment variable | When | Requirement |
| --- | --- | --- |
| `ADMIN_PASSWORD` | Runtime only | Strong private value; missing value disables admin sign-in |
| `ADMIN_COOKIE_SECURE` | Runtime only | `true` for public HTTPS, including proxy-terminated HTTPS; `false` for local HTTP testing |
| `NEXT_PUBLIC_SITE_URL` | Build and runtime | Optional public origin; omitted value preserves `https://gracesalamoun.com` |
`NEXT_PUBLIC_SITE_URL` is compiled into metadata and static robots/sitemap
output. A runtime-only override cannot update the already built files.
Rebuild when changing it. Existing normalization of known production aliases
to `https://gracesalamoun.com` is retained. Never pass `ADMIN_PASSWORD`, registry
tokens, or private credentials as build arguments or commit them to Git.
## Build, test, and prepare an image
Run from the repository root in PowerShell. `npm.cmd` bypasses this workstation's
PowerShell policy restriction on `npm.ps1`; other shells can use `npm`.
```powershell
npm.cmd ci
npm.cmd run build
node node_modules/typescript/bin/tsc --noEmit --incremental false
npm.cmd audit
docker build -t boutmoun:production .
```
For a different public origin, build with its non-secret value:
```powershell
docker build --build-arg NEXT_PUBLIC_SITE_URL=https://your-public-domain.example -t boutmoun:production .
```
Check each exit code before continuing. Lint is currently unconfigured:
`npm run lint` invokes Next.js's interactive ESLint setup; the repository has
neither ESLint configuration nor its dependencies. No dependencies were added
just to run that setup.
For a local HTTP smoke test, create an untracked `.env.docker-smoke` containing
a temporary `ADMIN_PASSWORD` and `ADMIN_COOKIE_SECURE=false`, then run:
```powershell
docker run --rm --name boutmoun-smoke -p 127.0.0.1:3000:3000 --env-file .env.docker-smoke boutmoun:production
```
In another terminal, inspect `http://localhost:3000/`, `/ar`, `/resume`,
`/ar/resume`, `/admin`, `/robots.txt`, `/sitemap.xml`, and
`/documents/grace-salmoun-portfolio.pdf`. Check that English redirects still
work and unauthenticated `/api/admin/content` returns 401. Sign in, save a
content change, upload an image, and verify the returned image URL immediately.
Use disposable volumes when testing save/upload persistence across container
recreation. This temporary HTTP test must not use production credentials.
## Preserve CMS state before any production replacement
The JSON CMS and image uploads are the application's existing datastore. Back
up any deployed `data/cms-content.json` and the entire `public/uploads`
directory before replacing a running container. Restore them to persistent
storage; do not put live data or secrets into a new image.
Attach two persistent volume mounts:
- `/app/data` for `cms-content.json`.
- `/app/public/uploads` for uploaded images.
Mount only these directories, not all of `/app` or `/app/public`, which would
hide bundled application files/assets. New empty volumes can also hide the
image's existing upload: seed that directory with the existing bundled uploads
as well as any recovered production uploads. Avoid overwriting newer files.
Verify that UID/GID 1001 can read/write both mounts. Image ownership does not
override volume ownership. With Kubernetes control, `fsGroup: 1001` can be used
where the storage driver supports it; otherwise provision the mount ownership
through the platform's supported storage procedure.
Use one instance with the current JSON-file datastore. It has no concurrent
writer coordination; multiple replicas or overlapping rolling replacements
can lose updates. Serialize replacement of the writer. Storage architecture
changes and a database migration are outside this task.
## Ghaymah deployment steps (operator-run; nothing pushed or deployed here)
The documented image-based flow uses an already pushed registry image. No
Ghaymah-specific Git build fields or undocumented registry names are assumed.
1. Verify the connected source repository/branch/commit. If using an existing
source-build integration, its Docker context must be the repository root
and its Dockerfile `Dockerfile`. Inspect its build and image-push logs.
2. Build and test the image above. Choose an actual registry you control and
a unique release tag, preferably tied to the final committed Git SHA. Login
with a credential manager or interactive prompt; never embed tokens in commands.
3. When ready to publish, use the same exact full image reference throughout:
```powershell
$RegistryServer = Read-Host 'Registry hostname'
$RegistryImage = Read-Host 'Full registry/repository:unique-tag'
docker login $RegistryServer
docker tag boutmoun:production $RegistryImage
docker push $RegistryImage
docker buildx imagetools inspect $RegistryImage
```
Stop on any failed command. Confirm the manifest supports the cluster node
architecture. For a multi-platform release, use an initialized Buildx builder
and `docker buildx build --platform linux/amd64,linux/arm64 -t $RegistryImage
--push .`, then test the image on each target architecture. That push command
is an operator publishing action, not a step executed during this audit.
4. In Ghaymah, open Apps and edit the existing app, or use Deploy App. Set the
image to that exact pushed reference (or digest). Keep the existing app's
domains, database connections, and other production settings.
5. Expose container port **3000**, retain the image command, and set the runtime
variables described above. For a private image, select/configure the
private-registry credential for that registry. A local `docker login` does
not authenticate Ghaymah's Kubernetes nodes.
6. Attach the backed-up and seeded persistent mounts with correct permissions;
use one instance and avoid overlapping writers. Configure the port's HTTP
health check to GET `/` (200), allowing enough startup time. The health check
proves HTTP readiness, not CMS write permissions; verify saving separately.
7. Only after review, update/deploy in the dashboard. Inspect pull/provisioning
events first, then runtime logs and health. Repeat the route, login, save,
upload, and persistence checks over HTTPS. Retain the previous image digest
and data backup for rollback.
Official Ghaymah references: [deploying](https://docs.ghaymah.cloud/guides/deploying/),
[app fields, registry credentials, ports and health checks](https://docs.ghaymah.cloud/platform/apps/),
[persistent volumes](https://docs.ghaymah.cloud/platform/volumes/).
Dashboard wording may differ; these are documented capabilities, not invented
settings for this particular account.
## Diagnose the reported Kubernetes image-pull failure
The local `kubectl` binary has no current context or configured contexts. No
cluster requests were made against an assumed cluster. With authorized Ghaymah
cluster access, use the actual namespace and current replacement pod if the
reported pod has expired:
```powershell
kubectl config current-context
kubectl get pods -A --field-selector metadata.name=gracesalmounsite-33c5dfa7fb2d-v2-5955767578-ldg5q
$Namespace = Read-Host 'Namespace from the previous output'
$PodName = 'gracesalmounsite-33c5dfa7fb2d-v2-5955767578-ldg5q'
$PodInfo = kubectl get pod $PodName -n $Namespace -o json | ConvertFrom-Json
$PodInfo.spec.containers | Select-Object name,image,imagePullPolicy
$PodInfo.status.containerStatuses | Select-Object name,state
$PodInfo.spec.imagePullSecrets | Select-Object name
$PodInfo.spec | Select-Object serviceAccountName,nodeName
kubectl get events -n $Namespace --field-selector "involvedObject.name=$PodName" --sort-by=.lastTimestamp
kubectl get nodes -L kubernetes.io/arch,kubernetes.io/os
$ServiceAccount = $PodInfo.spec.serviceAccountName
kubectl get serviceaccount $ServiceAccount -n $Namespace -o 'jsonpath={.imagePullSecrets[*].name}'
$PullSecret = Read-Host 'Actual pull-secret name, if used'
kubectl get secret $PullSecret -n $Namespace -o 'jsonpath={.type}'
```
Do not share raw secret data or unredacted Pod descriptions containing literal
environment secrets. Pull-secret metadata should exist in the same namespace
and have a valid registry credential type (normally
`kubernetes.io/dockerconfigjson`). Do not print `.dockerconfigjson`.
After collecting the exact image string from the Pod:
```powershell
$RegistryImage = Read-Host 'Exact image reference from the Pod'
docker buildx imagetools inspect $RegistryImage
docker pull $RegistryImage
```
A successful workstation pull proves only workstation access. Nodes may have
different credentials, network routes, DNS, CA trust, and CPU architecture.
| Event detail | Investigation/action |
| --- | --- |
| `manifest unknown`, `not found`, `name unknown` | Compare full hostname, owner/repository, case, tag/digest with push logs and registry manifest; verify the push actually finished |
| `unauthorized`, `denied`, `no basic auth credentials` | Verify read access, token expiry/scope, correct registry host, Ghaymah credential binding, Pod/service-account pull-secret references, and same-namespace secret existence |
| `FailedToRetrieveImagePullSecret` | Correct the missing/wrong-namespace secret reference using the platform's credential workflow |
| `no matching manifest` | Match manifest OS/architecture to the scheduled node; rebuild for that Linux architecture or publish/test a multi-platform manifest |
| DNS failure, timeout, connection refused | Check node-to-registry DNS, routing, firewall/egress and registry availability; local application settings cannot fix this |
| `x509`, certificate/TLS error | Fix registry certificate chain/hostname or node trust configuration; do not disable TLS verification |
| `429`, rate limit | Check registry quotas/rate limits and authenticated pull allowance |
Without cluster access, collect from Ghaymah: project/app identifier, current
revision, complete image reference, selected registry credential name/status
(no token), source repository/branch/commit, build and push result/digest,
provisioning/Pod event detail, node OS/architecture, and network/TLS errors. The
container name in the original error is not the registry image reference.
Ask platform support for the Pod Events if only the log error is visible.
Do not rotate credentials or retry deployments blindly before reading Events.
References: [Kubernetes images and private registries](https://kubernetes.io/docs/concepts/containers/images/),
[Next.js standalone output](https://nextjs.org/docs/14/app/api-reference/next-config-js/output),
[Node support status](https://nodejs.org/en/about/previous-releases),
[Next.js support policy](https://nextjs.org/support-policy).
## Recorded verification results
Final verification on 2026-11-11:
| Check | Result |
| --- | --- |
| `npm.cmd ci` after lockfile updates | PASS; 108 packages installed reproducibly |
| `npm.cmd run build` | PASS on Node 24.16.0 / Next.js 14.2.35 |
| TypeScript `tsc --noEmit --incremental false` | PASS |
| `docker build -t boutmoun:production .` | PASS using Node 24.22.2 Alpine; Linux/amd64 |
| Real container HTTP/admin/storage checks | PASS; 58 assertions |
| `npm.cmd run lint` with closed stdin | UNAVAILABLE / exit 1; interactive setup, ESLint unconfigured |
| `npm.cmd audit` | FAIL / exit 1; 2 remaining findings: Next.js critical, PostCSS high (including Next.js's bundled PostCSS) |
| Kubernetes access | UNAVAILABLE; no configured contexts/current context, also checked outside sandbox |
The Windows build emits a webpack cache-snapshot warning but completes
successfully. The Linux Docker build completes without that warning. The
security audit is not clean: compatible patches reduced the original seven
package findings to two, but Next.js 14 is unsupported. npm proposes a breaking
Next.js 16 upgrade for the remaining dependency chain. The user explicitly
kept that migration outside scope. Applicability varies by advisory and enabled
features; no claim is made that the remaining vulnerabilities are mitigated.
Container tests verified UID 1001, `node server.js`, `0.0.0.0:3000`, English and
Arabic home/resume pages, admin page, static CSS/JS, hero image, icon, portfolio
PDF, existing upload, both English redirects (308 with correct `Location`),
unauthorized API rejection, invalid password rejection, successful login,
HttpOnly/SameSite cookie flags, CMS saves reflected in both languages,
immediate upload retrieval with matching bytes/type, missing/invalid image
paths, invalid save/upload rejection, absence of secret `.env` files in the
image, logout, and CMS/image persistence across container recreation using
two disposable volumes. Test containers and volumes were removed afterward;
no production data was modified. This verifies API/HTTP behavior, not a new
visual or browser-interaction audit.
The locally retained image is `boutmoun:production`, image ID
`sha256:94c3479226d94172f410aa3f80bca98a50145cad357b31fafd3eb2e5e9348afe`.
It was not pushed to a registry or deployed. Only Linux/amd64 was tested;
cluster architecture and registry access remain externally unverified.
Ignored local evidence: `.artifacts/deployment-smoke-results.json`,
`.artifacts/deployment-audit-before.json`, `.artifacts/deployment-audit-final.json`,
and the smoke/reproduction scripts in `.artifacts/`.
Changed files prepared for review: `Dockerfile`, `.dockerignore`,
`.env.example`, `.gitignore`, `package.json`, `package-lock.json`,
`next.config.mjs`, `app/uploads/[filename]/route.ts`, and this document.