3 Commits

Author SHA1 Message Date
74b7a9fe73 fix(nginx): raise the container file descriptor limit to match worker_connections
Some checks failed
Deploy to Production / Trigger production update (push) Failing after 10s
nginx.conf configures 2048 worker_connections, but Docker's default
RLIMIT_NOFILE of 1024 capped what the workers could actually open — and
since a proxied connection needs a descriptor both downstream and upstream,
the real ceiling was about a quarter of the configured one. nginx warned
about exactly this on every config test.

Raise nofile to 8192 for the nginx container. Verified in place: the limit
goes from 1024 to 8192 and `nginx -t` is now free of warnings.
2026-08-11 20:47:49 +05:00
5f20b87824 fix(nginx): drop two directives that only produced reload warnings
`listen 443 ssl http2` has been deprecated since nginx 1.25.1 in favour of
a separate `http2 on;` directive, and Let's Encrypt no longer publishes an
OCSP responder URL in its certificates, so `ssl_stapling` had nothing to
staple. Both warned on every reload — noise that now matters more, since
nginx reloads itself every 6h to pick up renewed certificates.

Reloads are clean afterwards and HTTP/2 is still negotiated.
2026-08-11 20:38:03 +05:00
c6a32255df fix(ssl): stop reporting a working certificate as a failed deployment
The SSL deployment aborted even when Let's Encrypt had issued a perfectly
valid certificate. obtain_ssl_certificate() confirmed success with

    [ -d "nginx/certbot/conf/live/${domain}" ]

but certbot runs as root inside its container and creates live/, archive/
and accounts/ with mode 0700. Whenever deploy.sh runs as a non-root user
that test fails with EACCES no matter what is on disk, so every SSL deploy
fell back to the HTTP-only config and printed "Failed to obtain SSL
certificate" right after certbot reported the certificate was fine.

Check the certificate from inside a container instead, and treat the files
on disk as the source of truth rather than certbot's exit code, so a
certificate that is present and not yet due for renewal counts as success.
Pass --keep-until-expiring to make that case explicit.

Also drop the recursive chmod on nginx/certbot: it could not touch the
root-owned files it was aimed at (producing the wall of "Operation not
permitted" noise) and would have exposed the private keys if it had.

Verify nginx actually came up after switching to the SSL config and roll
back to HTTP if it did not — a crash-looping nginx takes the whole panel
offline, which is worse than serving without HTTPS.

Reload nginx every 6h so renewed certificates are picked up: nginx reads
the certificate once at startup, so the renewal loop would otherwise keep
serving the old one until the container happened to restart.

Ignore nginx/certbot/ — it holds Let's Encrypt private keys and was
untracked but not ignored.
2026-08-11 20:28:58 +05:00
4 changed files with 107 additions and 26 deletions

3
.gitignore vendored
View File

@@ -27,6 +27,9 @@ node_modules/
# Generated systemd files (created by deploy.sh when not root)
systemd/
# Let's Encrypt state written by certbot (contains private keys — never commit)
nginx/certbot/
# Environment
.env
.env.local

View File

@@ -15,6 +15,12 @@ services:
- ./nginx/ssl:/etc/nginx/ssl:ro
- ./nginx/certbot/conf:/etc/letsencrypt:ro
- ./nginx/certbot/www:/var/www/certbot:rw
# nginx reads the certificate once at startup, so a certificate renewed by
# the certbot service below would keep being served in its old form until
# the container restarts — i.e. the site would start serving an expired
# certificate roughly 30 days after the first successful renewal.
# Reload every 6h so a renewed certificate is picked up in time.
command: /bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g "daemon off;"'
depends_on:
app:
condition: service_healthy
@@ -28,6 +34,14 @@ services:
interval: 30s
timeout: 3s
retries: 3
# nginx.conf asks for 2048 worker_connections, but Docker's default file
# descriptor limit is 1024 — so the real ceiling was a quarter of the
# configured one (each proxied connection needs an fd downstream and
# upstream), and nginx warned about it on every config test.
ulimits:
nofile:
soft: 8192
hard: 8192
deploy:
resources:
limits:

106
deploy.sh
View File

@@ -371,16 +371,35 @@ setup_nginx_config() {
print_success "NGINX configuration created for domain: ${domain}"
}
# Check whether a usable certificate for the domain is already on disk.
#
# The check has to run as root inside a container: certbot creates
# /etc/letsencrypt/{live,archive,accounts} owned by root with mode 0700, so a
# plain `[ -d nginx/certbot/conf/live/$domain ]` from the (unprivileged) deploy
# user always fails with EACCES — even when the certificate is perfectly valid.
certificate_exists() {
local domain=$1
docker run --rm \
-v "$(pwd)/nginx/certbot/conf:/etc/letsencrypt" \
--entrypoint sh \
certbot/certbot -c \
"[ -s /etc/letsencrypt/live/${domain}/fullchain.pem ] && [ -s /etc/letsencrypt/live/${domain}/privkey.pem ]" \
>/dev/null 2>&1
}
obtain_ssl_certificate() {
local domain=$1
local email=$2
print_info "Obtaining SSL certificate for ${domain}..."
# Create certbot directories with proper permissions
# Create certbot directories. Only the ACME webroot needs relaxing — the
# rest belongs to certbot (root, 0700 on purpose: it holds private keys),
# and chmod -R here would only spam errors it cannot act on anyway.
mkdir -p nginx/certbot/conf nginx/certbot/www
chmod -R 755 nginx/certbot
chmod 755 nginx/certbot nginx/certbot/www 2>/dev/null || true
# Wait for nginx to be fully ready
print_info "Waiting for nginx to be ready..."
local max_wait=30
@@ -398,7 +417,9 @@ obtain_ssl_certificate() {
print_warning "NGINX may not be fully ready, continuing anyway..."
fi
# Request certificate using standalone certbot (more reliable)
# Request certificate via the webroot nginx is already serving.
# --keep-until-expiring makes an existing, still-valid certificate a plain
# success instead of an ambiguous "no action taken" result.
print_info "Requesting SSL certificate from Let's Encrypt..."
docker run --rm \
-v "$(pwd)/nginx/certbot/conf:/etc/letsencrypt" \
@@ -410,18 +431,26 @@ obtain_ssl_certificate() {
--agree-tos \
--no-eff-email \
--non-interactive \
--keep-until-expiring \
-d "${domain}"
local result=$?
if [ $result -eq 0 ] && [ -d "nginx/certbot/conf/live/${domain}" ]; then
print_success "SSL certificate obtained successfully"
# The certificate on disk is the source of truth, not certbot's exit code:
# a certificate that is present and not yet due for renewal is a success
# even on the runs where certbot reports it had nothing to do.
if certificate_exists "$domain"; then
if [ $result -ne 0 ]; then
print_warning "certbot exited with code ${result}, but a valid certificate for ${domain} is present"
fi
print_success "SSL certificate is available for ${domain}"
return 0
else
print_error "Failed to obtain SSL certificate"
print_warning "Continuing with HTTP only..."
return 1
fi
print_error "Failed to obtain SSL certificate (certbot exit code: ${result})"
print_warning "Check that ${domain} resolves to this server and that port 80 is reachable"
print_warning "Continuing with HTTP only..."
return 1
}
# Resilient Docker build: try with cache + --pull first; on failure retry once
@@ -479,6 +508,26 @@ check_stale_db_volume() {
fi
}
# Wait for the nginx container to settle into a running state and accept its
# configuration. A bad ssl_certificate path makes nginx exit at startup, so
# "up -d" returning 0 is not on its own proof that the site is being served.
verify_nginx_running() {
local max_wait=20
local waited=0
while [ $waited -lt $max_wait ]; do
if docker compose -f compose.yml -f compose.prod.yml ps nginx 2>/dev/null | grep -qi "running\|healthy"; then
if docker compose -f compose.yml -f compose.prod.yml exec -T nginx nginx -t >/dev/null 2>&1; then
return 0
fi
fi
sleep 2
waited=$((waited + 2))
done
return 1
}
deploy_with_nginx() {
local domain=$1
local use_ssl=$2
@@ -534,20 +583,33 @@ deploy_with_nginx() {
# Try to obtain SSL certificate
if obtain_ssl_certificate "$domain" "$email"; then
ssl_success=true
# Stop nginx to reconfigure with SSL
docker compose -f compose.yml -f compose.prod.yml stop nginx
# Setup SSL config
setup_nginx_config "$domain" true
print_info "Starting NGINX with SSL..."
docker compose -f compose.yml -f compose.prod.yml up -d nginx
# Start certbot renewal service
print_info "Starting certbot renewal service..."
docker compose -f compose.yml -f compose.prod.yml --profile ssl up -d certbot
if verify_nginx_running; then
ssl_success=true
print_success "NGINX is serving HTTPS for ${domain}"
# Start certbot renewal service
print_info "Starting certbot renewal service..."
docker compose -f compose.yml -f compose.prod.yml --profile ssl up -d certbot
else
# A crash-looping nginx takes the whole panel offline, which is
# worse than no HTTPS — roll back to the HTTP-only config.
print_error "NGINX failed to start with the SSL configuration:"
docker compose -f compose.yml -f compose.prod.yml logs --tail 20 nginx || true
print_warning "Rolling back to the HTTP-only configuration..."
docker compose -f compose.yml -f compose.prod.yml stop nginx
setup_nginx_config "$domain" false
docker compose -f compose.yml -f compose.prod.yml up -d nginx
ssl_success=false
fi
else
# Already running with non-SSL config, just continue
print_warning "Continuing with HTTP only (no SSL)..."

View File

@@ -20,8 +20,9 @@ server {
# HTTPS
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
listen 443 ssl;
listen [::]:443 ssl;
http2 on;
server_name DOMAIN_NAME;
# SSL certificates
@@ -34,8 +35,9 @@ server {
ssl_prefer_server_ciphers off;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
ssl_stapling on;
ssl_stapling_verify on;
# No OCSP stapling: Let's Encrypt no longer publishes an OCSP responder URL
# in its certificates, so ssl_stapling had nothing to staple and only
# produced a warning on every reload.
# Security headers
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;