Resolve Traefik CA Chain Issue

I obtained a wildcard certificate to be applied within Traefik. The files are:

ChainCA1.crt
ChainCA2.crt
RootCA.crt
domain.key
domain.pem

For this example, we will configure our Traefik with the following requirements:

Use subdomain.
Implement TLS.
Redirect HTTP to HTTPS.
Accessible via the subdirectory path /traefik
Implement Basic Auth.

docker-compose.yml

services:
  traefik:
    image: traefik:v2.11.0
    restart: always
    ports:
      - 80:80
      - 443:443
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik:/traefik
    command:
      - --log.level=DEBUG
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --providers.file.filename=/traefik/traefik.yml
      - --entrypoints.http.address=:80
      - --entrypoints.https.address=:443
      - --accesslog
      - --log
      - --api
    labels:
      - traefik.enable=true
 
      # middlewares
      - traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
      - traefik.http.middlewares.https-redirect.redirectscheme.permanent=true
      - traefik.http.middlewares.traefik-auth.basicauth.users=${USERNAME?Variable not set}:${HASHED_PASSWORD?Variable not set}
      - traefik.http.middlewares.strip-traefik.stripprefix.prefixes=/traefik
 
      # http entrypoint
      - traefik.http.routers.traefik-dashboard-http.entrypoints=http
      - traefik.http.routers.traefik-dashboard-http.rule=Host(`subdomain.${DOMAIN?Variable not set}`) && (PathPrefix(`/traefik`) || PathPrefix(`/api`))
      - traefik.http.routers.traefik-dashboard-http.service=api@internal
      - traefik.http.routers.traefik-dashboard-http.middlewares=https-redirect,traefik-auth,strip-traefik
 
      # https entrypoint
      - traefik.http.routers.traefik-dashboard-https.entrypoints=https
      - traefik.http.routers.traefik-dashboard-https.rule=Host(`subdomain.${DOMAIN?Variable not set}`) && (PathPrefix(`/traefik`) || PathPrefix(`/api`))
      - traefik.http.routers.traefik-dashboard-https.tls=true
      - traefik.http.routers.traefik-dashboard-https.service=api@internal
      - traefik.http.routers.traefik-dashboard-https.middlewares=traefik-auth,strip-traefik

traefik.yml

tls:
  certificates:
    - certFile: "/traefik/certificates/domain.pem"
      keyFile: "/traefik/certificates/domain.key"

.env

DOMAIN="[DOMAIN]"
USERNAME="[USERNAME]"
HASHED_PASSWORD="[PASSWORD]"

We are using a combination of static configuration using labels and dynamic configuration using traefik.yml for the service.

Dynamic configuration is necessary because Traefik can only read certificates from it.

You can generate the HASHED_PASSWORD using this command, and don’t forget to escape $ into $$ from the output before putting it in the .env file:

$ openssl passwd -apr1 YOUR-PASSWORD
$apr1$jUNT1Xnu$HDmoA16Iv7Q7ugWJIG3nY.

After everything is set up, simply run docker compose up -d.

Now, the moment of truth arrives as we open the URL https://subdomain.example.com… and encounter an error:

Unable to verify the first certificate

The logs appear as follows:

level=debug msg="Adding route for subdomain.example.com with TLS options default" entryPointName=https
level=debug msg="Adding certificate for domain(s) *.example.com,example.com"
level=debug msg="http: TLS handshake error from 49.12.5.160:40796: EOF"

Time for debugging, we can use run this command:

$ openssl s_client -connect subdomain.example.com:443
 
CONNECTED(00000003)
depth=0 CN = *.example.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = *.example.com
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:CN = *.example.com
   i:C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA
---

Oops! It seems the CA chain was not properly set up!

After some investigation, it turns out that domain.pem is the culprit. It doesn’t contain the rest of the .crt files.

To fix this, we only need to merge the content of these files into domain.pem:

ChainCA1.crt
ChainCA2.crt
RootCA.crt

After restarting Traefik and using the same command, we get the following:

$ openssl s_client -connect subdomain.example.com:443
CONNECTED(00000003)
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA
verify return:1
depth=0 CN = *.example.com
verify return:1
---
Certificate chain
 0 s:CN = *.example.com
   i:C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA
 1 s:C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA
   i:C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
 2 s:C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
   i:C = GB, ST = Greater Manchester, L = Salford, O = Comodo CA Limited, CN = AAA Certificate Services
---

Yes, now our browser URL bar is green 🟢.