You are not logged in.

Pages: 1

#1 2026-03-14 10:12:15

eselmeister5
Member
Registered: 2025-07-29
Posts: 5

Go/Rust based docker images non working DNS on Arch, Debian is fine

Hello Arch users,

I am currently trying to migrate a docker compose file from a Debian host to an Arch host.
But I am expieriencing problems with DNS in two containers which are Go and Rust binaries.

On Debian all 4 containers can resolve external and internal hosts just fine.

On Arch only Traefik and certs-dumper are able to resolve internally and externally.
rathole and foward-auth are not able to resolve any names (internal or external)

I tried running the base image distroless-debug and manually pinged different internal and external hosts. Everything worked.

services:
  app:
    image: traefik:3
    restart: unless-stopped
    dns:
      - 1.1.1.1
    command:
      - --log.level=TRACE
      - --api
      - --......
    ports:
      - 443:443/tcp
      - 443:443/udp
    volumes:
      - ./certs:/letsencrypt
      - /var/run/docker.sock:/var/run/docker.sock:ro
    extra_hosts:
      - host.docker.internal:host-gateway
    labels:
      - traefik.enable=true
      - traefik.http.routers.traefik.rule=Host(`traefik.domain.tld`)
      - traefik.http.routers.traefik.entrypoints=websecure
      - traefik.http.routers.traefik.service=api@internal
      - traefik.http.routers.traefik.middlewares=forward-auth
    networks:
      - default
      - public

  forward-auth:
    image: ghcr.io/italypaleale/traefik-forward-auth:4
    restart: unless-stopped
    volumes:
      - ./forward_auth_config:/etc/traefik-forward-auth
    labels:
      - traefik.enable=true
      - traefik.http.routers.forward-auth.rule=Host(`auth.domain.tld`)
      - traefik.http.routers.forward-auth.entrypoints=websecure
      #- traefik.http.routers.traefik-forward-auth.tls=true
      #- traefik.http.routers.forward-auth.middlewares=forward-auth
      - traefik.http.services.forward-auth.loadbalancer.server.port=4181
      - traefik.http.middlewares.forward-auth.forwardauth.trustForwardHeader=true
      - traefik.http.middlewares.forward-auth.forwardauth.address=http://forward-auth:4181/portals/main
      - traefik.http.middlewares.forward-auth.forwardauth.authResponseHeaders=X-Forwarded-User,X-Forwarded-Displayname,X-Authenticated-User

  rathole:
    image: ghcr.io/rathole-org/rathole:dev
    command: --client /app/client.toml
    restart: unless-stopped
    volumes:
      - ./rathole/client.toml:/app/client.toml:ro

  certs-dumper:
    image: ldez/traefik-certs-dumper:latest
    entrypoint: sh -c ' apk add jq ; while ! [ -e /data/acme.json ] || ! [ `jq ".[]
      | .Certificates | length" /data/acme.json` != 0 ]; do sleep 1 ; done &&
      traefik-certs-dumper file --version v2 --watch --source /data/acme.json
      --dest /data/certs'
    volumes:
      - ./certs:/data

networks:
  public:
    external: false
  default:
    external: false

Details:

forward-auth never becomes healthy because it is trying to resolve localhost via an non existing dns server:

{"Start":"2026-03-14T10:56:44.694300051+01:00","End":"2026-03-14T10:56:44.721621668+01:00","ExitCode":1,"Output":"2026/03/14 09:56:44 ERROR Failed to perform request app=traefik-forward-auth version=4.8.0 error=\"Get \\\"http://localhost:4181/healthz\\\": dial tcp: lookup localhost on [::1]:53: read udp [::1]:37530->[::1]:53: read: connection refused\" url=http://localhost:4181/healthz ms=0\n"}

rathole is not able to resolve any hostnames external or internal, most likely also because it is using a wrong resolver. But I do not have the logs to prove it.

2026-03-14T10:03:46.460176Z ERROR handle{service=https_home}: rathole::client: Failed to run the control channel: failed to lookup address information: Temporary failure in name resolution. Retry in 1.007788s...


/etc/resolv.conf file inside the container:

# Generated by Docker Engine.
# This file can be edited; Docker Engine will not make further changes once it
# has been modified.

nameserver 127.0.0.11
search .
options edns0 trust-ad ndots:0

# Based on host file: '/etc/resolv.conf' (internal resolver)
# ExtServers: [host(127.0.0.53)]
# Overrides: []
# Option ndots from: internal

/etc/hosts inside container:

127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::  ip6-localnet
ff00::  ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.21.0.4      6481c121961d

/etc/nsswitch.conf inside container:

# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.

passwd:         compat
group:          compat
shadow:         compat
gshadow:        files

hosts:          files dns
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis

rathole config

[client]
remote_addr = "rat.domain.tld:48484"
[client.services.https_home]
token = "TOKEN"
local_addr = "app:443"

Offline

#2 2026-03-14 14:03:10

eselmeister5
Member
Registered: 2025-07-29
Posts: 5

Re: Go/Rust based docker images non working DNS on Arch, Debian is fine

I also extracted the files from inside the container running on debian.
The only difference I could find is the search domain in resolv.conf
seach lan vs search .


I could confirm that the issue is the same for both containers, Go and Rust based.
Boith try to talk to a DNS resolver at localhost:53, but ofcourse nothing is running there.

14:30:40.863075 lo    In  IP localhost > localhost: ICMP localhost udp port 53 unreachable, length 72

The Docker DNS resolver is running at 127.0.0.11

I still dont now wwhy this is only happening on Arch.



/etc/resolv.conf

# Generated by Docker Engine.
# This file can be edited; Docker Engine will not make further changes once it
# has been modified.

nameserver 127.0.0.11
search lan
options edns0 trust-ad ndots:0

# Based on host file: '/etc/resolv.conf' (internal resolver)
# ExtServers: [host(127.0.0.53)]
# Overrides: []
# Option ndots from: internal

/etc/hosts

127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::  ip6-localnet
ff00::  ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.19.0.3      177aad718957

/etc/nsswitch.conf

# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.

passwd:         compat
group:          compat
shadow:         compat
gshadow:        files

hosts:          files dns
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis

Last edited by eselmeister5 (2026-03-14 14:32:59)

Offline

#3 2026-03-14 17:24:24

eselmeister5
Member
Registered: 2025-07-29
Posts: 5

Re: Go/Rust based docker images non working DNS on Arch, Debian is fine

Switching to podman-compose "just works".
I guess I found a reason to switch.

my current assumption is, that the static binaries are trying IPv6 first, but in the resolv.conf there is no Docker local DNS resolver specified/listening.

Last edited by eselmeister5 (2026-03-14 17:25:20)

Offline

Pages: 1

Board footer

Powered by FluxBB