Publishing a new service on `*.savethemall.eu`¶

Machine translation of the French original.

Complete procedure for exposing a service on a public name, protected by Authentik. Takes approximately 30 minutes, mainly waiting for DNS propagation.

In all that follows, replace monservice with the target subdomain and monservice:8080 with the Compose service name and port.

Assumptions of the procedure

The container is running and responding on HTTP on infrastructure_network.
Verification: from vm-docker, docker exec caddy wget -qO- http://monservice:8080/ | head -5. If this command does not return anything, stop here: nothing further will compensate for a silent upstream.

The Order, and Why It's Non-Negotiable¶

flowchart TB
    A["1 · Record A chez OVH"]
    B["2 · Certificat acme.sh<br/>validation DNS-01"]
    C["3 · Provider + application<br/>Authentik"]
    D["4 · Bloc Caddy vm-docker"]
    E["5 · Bloc Caddy nas-caddy"]
    F["6 · Secrets, s'il y en a"]
    G["7 · decisions.md, commit"]

    A -->|"sans le record,<br/>la validation DNS-01 échoue"| B
    B -->|"sans le cert, `caddy validate`<br/>refuse et le reload COUPE<br/>les 76 autres sites"| D
    C -->|"l'app doit exister,<br/>sinon 404 au lieu de 302"| D
    D -->|"le cert doit être copié<br/>sur le NAS"| E
    E --> G
    F --> G

    B -.->|"rsync quotidien<br/>ou déclenché à la main"| E

    classDef risk stroke-width:3px
    class D risk

The only step that can break anything is step 4: Caddy on vm-docker has 76 vhosts, and reloading an invalid configuration affects them all. Therefore, caddy validate and restart must come first, and the certificate before the block.

1. DNS Registration with OVH¶

Always first. acme.sh validates DNS-01: without the registration, certificate issuance fails, and the failure looks like an authentication issue.

ssh -p <port-ssh> <user>@vm-docker \
  "sudo /docker-mounts/homelab-vault/00-infra/scripts/ovh-record.sh \
     add savethemall.eu A monservice <IP publique> --apply"

Verification

dig +short monservice.savethemall.eu @1.1.1.1
# attendu : <IP publique>

Specifically not the local resolver: the internal DNS already responds for this name, using a wildcard, and would give a false positive.

And not the OVH authoritative servers — ns.ovh.net and dns1.ovh.net do not respond to direct requests from this network (measured on 2026-08-29 from vm-claude and from the VPS: communications error … timed out). A public resolver has already done the work.

2. TLS Certificate¶

Add the domain to the issuance list – this is the list that will be read on the next full deployment – and then issue the certificate alone.

# 1. inscrire le domaine dans la liste
ssh -p <port-ssh> <user>@vm-docker \
  "grep -q 'monservice.savethemall.eu' /opt/docker/acme/issue-all.sh || \
   sed -i 's|^)|  monservice.savethemall.eu\n)|' /opt/docker/acme/issue-all.sh"

# 2. émettre ce seul certificat
ssh -p <port-ssh> <user>@vm-docker \
  "docker exec acme acme.sh --issue --dns dns_ovh -d monservice.savethemall.eu \
     --cert-file      /certs/monservice.savethemall.eu/cert.pem \
     --key-file       /certs/monservice.savethemall.eu/privkey.pem \
     --fullchain-file /certs/monservice.savethemall.eu/fullchain.pem \
     --reloadcmd      \"curl -sf -X POST http://127.0.0.1:2019/load \
                        -H 'Content-Type: text/caddyfile' \
                        --data-binary @/etc/caddy/Caddyfile\""

--dnssleep is mandatory here, and the reason is not with OVH

Without it, issuance consistently fails on:

Timed out waiting for DNS.
Error checking DNS.

acme.sh verifies its TXT record via DoH on cloudflare-dns.com, and AdGuard resolves this name to 0.0.0.0: the anti-circumvention DNS filtering of the house blocks its own issuing tool. However, the TXT record is correctly set up at OVH, and then properly removed — which leads to a failure that resembles a problem with identifiers or propagation.

The control that breaks down into a single command:

docker exec acme sh -c 'nslookup cloudflare-dns.com | tail -3'
# Address: 0.0.0.0  → c'est AdGuard, pas OVH

--dnssleep replaces the verification with a fixed wait. This is the circumvention; the fix is to allow cloudflare-dns.com in AdGuard.

Control

ssh -p <port-ssh> <user>@vm-docker \
  "ls -l /opt/docker/acme/certs/monservice.savethemall.eu/fullchain.pem"

The NAS receives its copy at rsync daily. To avoid waiting, trigger it manually before step 5.

3. Application and Provider in Authentik¶

In the administration interface:

Applications → Providers → Create → Proxy Provider
- Mode: Forward auth (single application) — not Proxy, not Domain level.
- External host: https://monservice.savethemall.eu
- Authorization flow: the park's (implicit consent).
Applications → Create
- Slug: monservice
- Provider: the one that has just been created
- Icon: find the official logo before creating the application. In order: the GitHub repository of the project, the /favicon.svg of the official website, https://cdn.simpleicons.org/<nom>, then the catalog https://cdn.jsdelivr.net/gh/selfhst/icons/svg/<nom>.svg.
Applications → Outposts → authentik Embedded Outpost → Edit → Applications
- Add the application manually.

The embedded outpost does not automatically retrieve the application

This is the most common mistake in this procedure. Without this explicit addition, everything is correctly configured on the Authentik side, Caddy redirects to the login flow, and the connection still fails — without any message indicating the cause.

4. Caddy Block on vm-docker¶

Edit 10-stacks/caddy/Caddyfile in the vault, then synchronize. The vault file is not a symbolic link: both copies are real and will diverge if forgotten.

monservice.savethemall.eu {
    import tls_cert monservice.savethemall.eu
    # Endpoints de l'outpost embarqué (sign_out, start, callback) → Authentik
    handle /outpost.goauthentik.io/* {
        reverse_proxy authentik-server:9000
    }
    handle {
        import authentik
        reverse_proxy monservice:8080
    }
}

The order of handle is not decorative: see forward_auth comes before handle.

Synchronization is required after any edit:

ssh -p <port-ssh> <user>@vm-docker \
  "cp /docker-mounts/homelab-vault/10-stacks/caddy/Caddyfile /opt/docker/caddy/Caddyfile && \
   docker compose -f /opt/docker/caddy/docker-compose.yml restart"

Control – on LAN, where the DNS already points to vm-docker

curl -s -o /dev/null -w '%{http_code}\n' https://monservice.savethemall.eu/
# 302 vers auth.savethemall.eu → le mur est en place

curl -s -o /dev/null -w '%{http_code}\n' \
  https://monservice.savethemall.eu/outpost.goauthentik.io/zzz
# 404 → la route de l'outpost existe (302 ici = le handle est mal placé)

5. Caddy Block on nas-caddy – this is the one that opens the WAN¶

As long as this step is not performed, the service responds on the LAN and tailnet, and remains invisible from the Internet. The NAS is the edge.

File: /volume3/docker/nas-caddy/Caddyfile on the NAS.

monservice.savethemall.eu {
    import tls_cert monservice.savethemall.eu
    import to_vmdocker
}

Nothing else: authentication and routing remain downstream, on vm-docker. The snippet (to_vmdocker) already has header_up Host {host} – without it, the upstream would receive Host: vm-docker.

Variant: keep the service out of the WAN

For a service that needs to remain internal, add import lan_only to the block instead of not declaring it. Here, remote_ip is the real client: nas-caddy is the edge, there is no upstream proxy to obscure the source.

Control – the real public path

curl -s -o /dev/null -w '%{http_code}\n' \
  --resolve monservice.savethemall.eu:443:<IP publique> \
  https://monservice.savethemall.eu/

Without --resolve, the command goes through the internal DNS and does nothing to test the WAN path.

6. Secrets, if any¶

The values live in /opt/docker/monservice/.env on the host, never in the vault in plain text, never in the Compose file.
The encrypted version goes in 00-infra/secrets/vm-docker/monservice.enc.env (SOPS + age). The tree is indexed by host: three stack names exist on two or more hosts, and without this qualification the second overwrites the first – silently.
Only the public key age is needed to encrypt. No private key needs to exist on other hosts.
All API tokens are stored in Vaultwarden immediately, following the convention <hôte>/<stack>·<NOM_DE_VARIABLE>.

The ciphertext pushed is permanent

These files are also available on GitHub. The repository is private, but anything that has been pushed there remains: rotating a secret does not make previously published ciphertext unreadable. Rotating means rotating upstream, not simply rewriting the file.

Never put secrets in a URL

The error log records the resolved URL, including the token. Similarly, Caddy automatically expunges only three headers — Authorization, Cookie, Set-Cookie. A custom header (X-Mon-Token) is sent in plaintext in access.log, which is read by CrowdSec. Any new header containing a secret must be added to the log filter at the same time it is created.

7. Documenting what has been done¶

Entry dated in 10-stacks/monservice/decisions.md: reason, and how to revert.
Updating the master stack table.
Committing the vault and pushing to both remote repositories.

Summary¶

Dernière vérification de cette page : 2026-09-08

Publishing a new service on *.savethemall.eu¶