compose.prod.yml has always intended to take the app and frontend off the
host interface and let nginx front them — "remove port exposure (nginx will
handle it)". It did not work: Compose concatenates list-valued fields when
merging -f files, so `ports: []` contributed nothing and 8000:8000 and
3001:80 survived from compose.yml.
The effect was that an SSL deployment still answered on both ports over
plain HTTP, bypassing TLS, HSTS, the security headers and the rate limits —
confirmed against the live host from off-box, where the API returned its
health payload unencrypted on :8000 and the panel loaded on :3001.
Use the !reset tag so the inherited value is dropped rather than extended.
Verified in place: the merged config now publishes only 80 and 443, both
former ports refuse connections from outside, and the panel still serves
over HTTPS.
nginx.conf configures 2048 worker_connections, but Docker's default
RLIMIT_NOFILE of 1024 capped what the workers could actually open — and
since a proxied connection needs a descriptor both downstream and upstream,
the real ceiling was about a quarter of the configured one. nginx warned
about exactly this on every config test.
Raise nofile to 8192 for the nginx container. Verified in place: the limit
goes from 1024 to 8192 and `nginx -t` is now free of warnings.
The SSL deployment aborted even when Let's Encrypt had issued a perfectly
valid certificate. obtain_ssl_certificate() confirmed success with
[ -d "nginx/certbot/conf/live/${domain}" ]
but certbot runs as root inside its container and creates live/, archive/
and accounts/ with mode 0700. Whenever deploy.sh runs as a non-root user
that test fails with EACCES no matter what is on disk, so every SSL deploy
fell back to the HTTP-only config and printed "Failed to obtain SSL
certificate" right after certbot reported the certificate was fine.
Check the certificate from inside a container instead, and treat the files
on disk as the source of truth rather than certbot's exit code, so a
certificate that is present and not yet due for renewal counts as success.
Pass --keep-until-expiring to make that case explicit.
Also drop the recursive chmod on nginx/certbot: it could not touch the
root-owned files it was aimed at (producing the wall of "Operation not
permitted" noise) and would have exposed the private keys if it had.
Verify nginx actually came up after switching to the SSL config and roll
back to HTTP if it did not — a crash-looping nginx takes the whole panel
offline, which is worse than serving without HTTPS.
Reload nginx every 6h so renewed certificates are picked up: nginx reads
the certificate once at startup, so the renewal loop would otherwise keep
serving the old one until the container happened to restart.
Ignore nginx/certbot/ — it holds Let's Encrypt private keys and was
untracked but not ignored.
