← Back to Tutorials

Debugging Environment Variable & Config Injection Issues in Dockerized Apps

dockerdocker-composeenvironment-variablesconfigurationdebuggingdevopscontainerssecrets

Debugging Environment Variable & Config Injection Issues in Dockerized Apps

Environment variables and configuration injection are the glue between a container image (immutable artifact) and a running container (environment-specific behavior). When something goes wrong—missing variables, wrong values, unexpected defaults, secrets not available, config files not mounted—you often get confusing symptoms: apps start with defaults, crash on boot, connect to the wrong database, or behave differently in CI vs local.

This tutorial is a practical, command-heavy guide to diagnosing and fixing environment/config injection issues in Dockerized applications. It focuses on real debugging workflows you can run locally or in CI.

Table of Contents

Mental Model: Where Config Can Come From

In Dockerized apps, configuration usually comes from one or more of these sources:

  1. Environment variables injected at runtime:

    • docker run -e KEY=value ...
    • docker run --env-file .env ...
    • docker compose environment: or env_file:
    • CI/CD injects env vars into the Docker process or Compose

  2. Files mounted into the container:

    • Bind mounts: -v $PWD/config:/app/config:ro
    • Named volumes
    • Docker secrets (Swarm) or Kubernetes secrets/configmaps (outside Docker scope, but similar concept)

  3. Defaults baked into the image:

    • ENV KEY=value in Dockerfile
    • default config files copied into the image
    • entrypoint scripts that generate config if missing

  4. Application-level config loading:

    • .env loaded by the app (e.g., dotenv) vs .env loaded by Docker/Compose (different!)
    • precedence rules in frameworks (Spring, Django, Node, etc.)

A key debugging principle: there are two separate systems:

You must verify both.

Common Failure Modes

Here are the most common causes of “env var not working” in containers:

Step 1: Confirm What the Container Actually Received

1.1 Inspect environment variables inside a running container

If the container is already running:

docker exec -it myapp sh -lc 'env | sort'

If it’s a minimal image without sh, try:

docker exec -it myapp /bin/sh -lc 'env | sort'

Or for distroless images (no shell), you may need to:

To check a specific variable:

docker exec -it myapp sh -lc 'printf "%s\n" "$DATABASE_URL"'

If it prints nothing, the variable is not set (or is empty).

1.2 Use docker inspect to see configured env vars

docker inspect shows the container’s configured environment:

docker inspect myapp --format '{{json .Config.Env}}' | jq -r '.[]'

Without jq:

docker inspect myapp --format '{{range .Config.Env}}{{println .}}{{end}}'

This is especially useful when you can’t exec into the container.

1.3 Confirm mounts and files exist inside the container

List mounts:

docker inspect myapp --format '{{json .Mounts}}' | jq

Inside container:

docker exec -it myapp sh -lc 'mount || true; ls -la /app; ls -la /app/config || true'

If a config file is supposed to be mounted at /app/config/app.conf, verify:

docker exec -it myapp sh -lc 'ls -la /app/config/app.conf; sed -n "1,120p" /app/config/app.conf'

If the file isn’t there, the problem is on the Docker/Compose side, not the app.

Step 2: Inspect the Image for Defaults and Entrypoint Behavior

Sometimes the container did receive your env vars, but the image’s entrypoint or defaults override them.

2.1 Check Dockerfile-defined ENV

List image config:

docker image inspect myimage:tag --format '{{json .Config.Env}}' | jq -r '.[]'

If you see ENV DATABASE_URL=... baked into the image, note that runtime -e DATABASE_URL=... should override it, but entrypoint scripts might still do unexpected things.

2.2 Inspect entrypoint and command

docker image inspect myimage:tag --format 'Entrypoint={{json .Config.Entrypoint}} Cmd={{json .Config.Cmd}}'

If the entrypoint is a script (e.g., /docker-entrypoint.sh), view it by running a shell in the image:

docker run --rm -it --entrypoint sh myimage:tag -lc 'ls -la /; ls -la /docker-entrypoint.sh; sed -n "1,200p" /docker-entrypoint.sh'

Look for patterns like:

2.3 Run the container with a debug entrypoint

To bypass the normal entrypoint and inspect the filesystem and env:

docker run --rm -it --entrypoint sh -e DATABASE_URL='postgres://x' myimage:tag -lc 'env | sort; ls -la; sleep 1'

If DATABASE_URL shows here but not in the normal run, your entrypoint or orchestration is the culprit.

Step 3: Debug docker run -e and .env Issues

3.1 Confirm your shell expanded variables the way you think

If you run:

docker run --rm -e API_KEY=$API_KEY myimage:tag

and $API_KEY is not set in your shell, Docker receives an empty value. Confirm:

echo "API_KEY=$API_KEY"

A safer pattern is to fail fast in your shell:

: "${API_KEY:?API_KEY is required}"
docker run --rm -e API_KEY="$API_KEY" myimage:tag

3.2 Use --env-file correctly

Create a file named app.env:

cat > app.env <<'EOF'
DATABASE_URL=postgres://user:pass@db:5432/app
LOG_LEVEL=debug
FEATURE_X=true
EOF

Run:

docker run --rm --env-file app.env myimage:tag

Debug what Docker injected:

docker run --rm --env-file app.env --entrypoint sh myimage:tag -lc 'env | sort | sed -n "1,120p"'

Common pitfalls with --env-file

file app.env
# If it says CRLF, convert:
sed -i 's/\r$//' app.env

Step 4: Debug Docker Compose Environment Injection

Compose adds two distinct concepts that are often conflated:

  1. Compose interpolation: ${VAR} in compose.yaml is substituted using environment variables from:
    • your shell environment
    • a .env file in the project directory (default behavior)
  2. Container environment: variables passed into the container via environment: or env_file:

These are not the same.

4.1 See the final rendered Compose configuration

This is the single most useful Compose debugging command:

docker compose config

It outputs the fully-resolved configuration after interpolation. If ${DATABASE_URL} became empty, you’ll see it here.

To focus on a service:

docker compose config | sed -n '/services:/,$p'

If you suspect a missing .env, print what Compose sees:

docker compose config --environment

(If your Compose version doesn’t support --environment, rely on docker compose config plus checking your shell and .env file.)

4.2 Verify what env vars are inside the running Compose container

docker compose exec myservice sh -lc 'env | sort'

If exec fails because the container is restarting, inspect logs:

docker compose logs -f --tail=200 myservice

4.3 Recreate containers after config changes

Compose does not always apply changes to existing containers unless they are recreated:

docker compose up -d --force-recreate

If you changed env vars and want a clean slate:

docker compose down
docker compose up -d

4.4 environment: vs env_file: precedence and behavior

When both are used, environment: typically wins for conflicting keys.

To demonstrate and debug:

# Start with a clean state
docker compose down -v

# Bring up
docker compose up -d

# Check env
docker compose exec myservice sh -lc 'printf "A=%s\nB=%s\n" "$A" "$B"'

If values are not what you expect, run:

docker compose config

and verify the resolved values.

4.5 .env file confusion: Compose .env vs application .env

Debug by checking inside container:

docker compose exec myservice sh -lc 'ls -la; ls -la .env || true; pwd'

If your app expects .env but it’s missing, either:

Step 5: Debug File-Based Config (Bind Mounts & Volumes)

Many apps read config from files (JSON, INI, YAML, TOML, .properties, etc.). Docker mounts can fail silently in ways that look like “the app ignored my config.”

5.1 Confirm the mount exists and points to the right host path

Run the container with a bind mount:

docker run --rm -it \
  -v "$PWD/config/app.conf:/app/config/app.conf:ro" \
  myimage:tag

If $PWD/config/app.conf doesn’t exist, Docker may create a directory or an empty file depending on context, leading to confusing results.

Check on the host:

ls -la "$PWD/config/app.conf"

Check in the container:

docker run --rm -it \
  -v "$PWD/config/app.conf:/app/config/app.conf:ro" \
  --entrypoint sh myimage:tag -lc 'ls -la /app/config; file /app/config/app.conf; sed -n "1,80p" /app/config/app.conf'

5.2 Beware: mounting a directory hides image contents

If the image contains /app/config/defaults.json and you mount a host directory onto /app/config, the container will no longer see the image’s /app/config/* files.

Debug:

docker run --rm -it --entrypoint sh myimage:tag -lc 'ls -la /app/config'

Then compare with mount:

docker run --rm -it \
  -v "$PWD/config:/app/config:ro" \
  --entrypoint sh myimage:tag -lc 'ls -la /app/config'

If defaults disappeared, you may need to mount individual files rather than the whole directory, or ensure the host directory contains all required files.

5.3 Permissions issues (especially with non-root containers)

If the container runs as a non-root user, it may not be able to read mounted files.

Check user:

docker exec -it myapp sh -lc 'id'

Check file permissions inside container:

docker exec -it myapp sh -lc 'ls -la /run/secrets /app/config || true'

Fix on host:

chmod 644 config/app.conf

Or run container with a matching UID/GID (advanced), or adjust image to run with appropriate permissions.

Step 6: Debug Secrets and “It Works Locally” Problems

Secrets often differ between local runs, CI, and production. Common issues include missing secret files, wrong paths, or environment variables not set in the runtime environment.

6.1 Detect whether a secret is expected as env var or file

Many systems support both patterns:

If your app supports _FILE convention, verify:

docker exec -it myapp sh -lc 'env | grep -E "API_KEY|API_KEY_FILE" || true; ls -la /run/secrets || true'

If a secret file is expected at /run/secrets/api_key, verify:

docker exec -it myapp sh -lc 'ls -la /run/secrets/api_key; wc -c /run/secrets/api_key; head -c 5 /run/secrets/api_key; echo'

(Use wc -c to confirm it’s not empty without printing the whole secret.)

6.2 CI/CD gotcha: env vars exist in CI job but not in docker build

Environment variables at runtime are different from build-time args.

If you accidentally rely on runtime variables during build, you’ll see missing values.

Check whether your Dockerfile uses ARG vs ENV:

grep -nE '^(ARG|ENV)\b' Dockerfile

Build with explicit args:

docker build --build-arg COMMIT_SHA="$GIT_SHA" -t myimage:tag .

Then confirm inside image:

docker run --rm myimage:tag sh -lc 'env | grep COMMIT_SHA || true'

If you used ARG, it won’t persist unless you copy it into ENV during build.

Step 7: Debug Variable Expansion and Quoting

A surprising number of config bugs are just quoting/expansion issues.

7.1 Shell quoting: avoid accidental expansion or word splitting

Bad (word splitting, globbing issues):

docker run --rm -e DATABASE_URL=$DATABASE_URL myimage:tag

Better:

docker run --rm -e DATABASE_URL="$DATABASE_URL" myimage:tag

If the value contains $, you might need single quotes to prevent shell expansion:

docker run --rm -e PASSWORD='p@$$w0rd' myimage:tag

7.2 Compose interpolation: ${VAR} vs literal $

In Compose files, ${VAR} is interpolated. If you want a literal $ passed to the container, you often need to escape it as $$ in Compose contexts (depending on where it appears).

A symptom: passwords like pa$$word become pa + unexpected interpolation behavior.

Debug by rendering:

docker compose config

If the rendered value is wrong, fix escaping/quoting.

7.3 Empty vs unset matters

Some scripts treat empty as “set” and won’t apply defaults:

: "${FOO:=default}"   # default only if unset or empty in many shells

But other logic might check only “is it set”:

if [ -n "${FOO+x}" ]; then
  echo "FOO is set (even if empty)"
fi

To debug, print with markers:

docker exec -it myapp sh -lc 'printf "FOO=[%s]\n" "${FOO}" ; env | grep "^FOO=" || echo "FOO not in env"'

Step 8: Debug App-Level Config Loading (12-factor vs frameworks)

Even if Docker injected the env var correctly, the application may not read it (or may override it).

8.1 Confirm the app reads the variable you think it reads

Search in code (host side):

rg -n 'DATABASE_URL|DB_URL|DATABASEURI|CONNECTION_STRING' .

If the app expects DB_HOST, DB_USER, etc., but you set DATABASE_URL, it will ignore it.

8.2 Print config at startup (safely)

Add a startup log that prints which keys are set without printing secrets. For example, in a shell entrypoint:

echo "DB_HOST=${DB_HOST:-<unset>}"
echo "DB_USER=${DB_USER:-<unset>}"
echo "DB_PASSWORD is ${DB_PASSWORD:+<set>}${DB_PASSWORD:-<unset>}"

In running containers, you can sometimes enable debug logging via env var:

docker run --rm -e LOG_LEVEL=debug myimage:tag

Then check logs:

docker logs -f myapp

8.3 Framework precedence surprises

Examples of common precedence rules (varies by framework):

To detect, check the app’s effective config output if available (many frameworks have a “print config” command). If not, add a diagnostic endpoint or log the resolved config keys (again, avoid printing secrets).

Step 9: Minimal Repro & Binary Search Strategy

When debugging config injection, reduce complexity until the problem becomes obvious.

9.1 Replace the app with env temporarily

If you suspect Docker/Compose is not injecting variables, replace the command with env:

With docker run:

docker run --rm --env-file app.env alpine:3.20 env | sort | sed -n '1,120p'

With Compose, you can temporarily override the command:

docker compose run --rm myservice sh -lc 'env | sort | sed -n "1,160p"'

If the variables show up here, injection works; the app is the issue.

9.2 Use an interactive shell to inspect runtime state

docker run --rm -it --env-file app.env --entrypoint sh myimage:tag

Then inside:

env | sort
ls -la
cat /app/config/settings.json 2>/dev/null || true

9.3 Compare “expected” vs “actual” as artifacts

Capture env from container:

docker exec myapp sh -lc 'env | sort' > actual.env

Capture expected env (from your .env or CI variables), then diff:

diff -u expected.env actual.env | sed -n '1,200p'

This makes missing/overridden keys obvious.

Hardening: Prevent These Bugs in the First Place

1) Fail fast on missing required config

In entrypoint scripts:

: "${DATABASE_URL:?DATABASE_URL must be set}"
: "${PORT:?PORT must be set}"

This turns silent defaults into clear errors.

2) Provide a --print-config or --diagnostics mode

Implement a mode that prints:

Then you can run:

docker run --rm --env-file app.env myimage:tag --print-config

3) Standardize variable names and document them

Maintain a canonical list:

Also provide an example env file:

cp .env.example .env

4) Avoid baking environment-specific config into the image

Prefer:

Keep images portable across environments.

5) Add automated checks in CI

Example: ensure required variables are present in Compose config output:

docker compose config > rendered.txt
grep -n 'DATABASE_URL' rendered.txt

Or run a smoke test container that asserts required vars exist:

docker compose run --rm myservice sh -lc ': "${DATABASE_URL:?missing}"; : "${API_KEY:?missing}"; echo OK'

Quick Reference Commands

Inspect container environment and mounts

docker inspect myapp --format '{{range .Config.Env}}{{println .}}{{end}}'
docker inspect myapp --format '{{json .Mounts}}' | jq
docker exec -it myapp sh -lc 'env | sort'
docker exec -it myapp sh -lc 'ls -la /app /app/config || true'

Debug image entrypoint and defaults

docker image inspect myimage:tag --format '{{json .Config.Env}}' | jq -r '.[]'
docker image inspect myimage:tag --format 'Entrypoint={{json .Config.Entrypoint}} Cmd={{json .Config.Cmd}}'
docker run --rm -it --entrypoint sh myimage:tag -lc 'sed -n "1,200p" /docker-entrypoint.sh 2>/dev/null || true'

Compose debugging

docker compose config
docker compose up -d --force-recreate
docker compose exec myservice sh -lc 'env | sort'
docker compose logs -f --tail=200 myservice

Env file hygiene

file app.env
sed -i 's/\r$//' app.env

Closing Checklist (Use This When You’re Stuck)

  1. Does the container have the variable?
    • docker exec ... env | sort
    • docker inspect ... .Config.Env
  2. Does the container have the config file mounted where the app expects it?
    • ls -la inside container
    • docker inspect ... .Mounts
  3. Is Compose rendering what you think?
    • docker compose config
  4. Did you recreate containers after changes?
    • docker compose up -d --force-recreate
  5. Is the app reading the right variable name and honoring precedence?
    • search code, add diagnostics, inspect entrypoint scripts
  6. Are quoting/expansion issues corrupting values?
    • check docker compose config, avoid unquoted shell expansions
  7. Are permissions preventing reading mounted secrets/config?
    • id, ls -la, adjust permissions or user

If you want, share:

Related Tutorials

Fast AWS Lambda Deployments: CI/CD and Tuning Tips How to Speed Up Data Analysis with pandas How to Fix Slow Queries with Advanced SQL