12. Debugging and Troubleshooting Containers | The Complete Docker Handbook.

12. Debugging and Troubleshooting Containers | The Complete Docker Handbook.
Photo by iamateapot on Unsplash
8 mins read
0 Like
7 Views

Welcome to Article 12 of The Complete Docker Handbook.

In Article 11, we automated our workflow with CI/CD pipelines. You now have a professional setup that builds and deploys your containers automatically. But let's be realistic: things will break.

A container might exit immediately. A network connection might time out. Your disk might fill up unexpectedly. When this happens, panic is not a strategy. You need a systematic approach to debugging.

In this article, we will turn you into a Docker detective. You will learn how to interpret logs, inspect container metadata, debug common error codes, and manage system resources effectively.


1. The First Line of Defense: Logs

When a container misbehaves, the first place to look is the standard output (stdout) and standard error (stderr) streams.

Basic Logging

Bash
docker logs

Advanced Logging Flags

  • -f (Follow): Stream logs in real-time (like tail -f).
    Plain Text
    1bash
    2docker logs -f
  • --tail : Show only the last n lines. Great for noisy apps.
    Plain Text
    1bash
    2docker logs --tail 100
  • --timestamps: Show when each log entry occurred. Crucial for correlating events.
    Plain Text
    1bash
    2docker logs --timestamps
  • --since: Show logs since a specific time (e.g., 10m, 1h, 2023-01-01).
    Plain Text
    1bash
    2docker logs --since 10m

> πŸ’‘ Pro Tip: If your logs are empty, your application might be logging to a file inside the container instead of stdout. You will need to exec into the container to check those files.


2. Going Inside the Container

Sometimes logs aren't enough. You need to inspect the filesystem, check environment variables, or test network connectivity from within the container.

The exec Command

Bash
docker exec -it
  • -i: Interactive.
  • -t: TTY (gives you a prompt).

Common Debugging Commands

  1. Open a Shell:

    Bash
    docker exec -it  bash

    Note: If using Alpine images, bash might not exist. Use sh instead.

    Bash
    docker exec -it  sh
  2. Check Environment Variables:

    Bash
    docker exec  env

    Verify your secrets and config loaded correctly.

  3. Test Network Connectivity:

    Bash
    1docker exec  ping google.com
    2docker exec  curl http://database:5432

    Verify DNS resolution and firewall rules.

  4. Check Filesystem:

    Bash
    docker exec  ls -la /app

    Verify your code was copied correctly.


3. Deep Inspection with docker inspect

docker inspect returns low-level metadata about the container in JSON format. It's overwhelming but powerful.

Basic Inspection

Bash
docker inspect

Extracting Specific Data

You don't need to read the whole JSON. Use the --format flag with Go templates to extract specific fields.

Get Container IP Address:

Bash
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}'

Check Mounts (Volumes):

Bash
docker inspect -f '{{json .Mounts}}'

Check Exit Code:

Bash
docker inspect -f '{{.State.ExitCode}}'
  • 0: Success.
    • 1: General error.
    • 137: Out of Memory (OOM).
    • 125: Docker daemon error.
    • 126: Command invoked cannot execute.
    • 127: Command not found.

4. Common Errors & Solutions

Here are the most frequent errors you will encounter and how to fix them.

❌ Error: Exit Code 137 (OOMKilled)

  • Meaning: The container was killed because it exceeded its memory limit.
  • Cause: Memory leak in app or limit set too low.
  • Fix:
    1. Increase memory limit: docker update --memory=1g .
    2. Optimize application code.
    3. Check logs for "OutOfMemoryError".

❌ Error: Port Already Allocated

  • Meaning: Bind for 0.0.0.0:80 failed: port is already allocated.
  • Cause: Another process (or container) is using the host port you want.
  • Fix:
    1. Find the process: lsof -i :80 (Mac/Linux) or netstat -ano | findstr :80 (Windows).
    2. Stop the conflicting container.
    3. Or change your port mapping (e.g., -p 8080:80).

❌ Error: Permission Denied (Volumes)

  • Meaning: Error: Permission denied @ dir_initialize - /data.
  • Cause: The user inside the container doesn't have write access to the mounted host folder (common on Linux).
  • Fix:
    1. Change host folder ownership: sudo chown -R $USER:$USER ./data.
    2. Or ensure the container user UID matches the host folder UID.

❌ Error: Cannot Connect to Docker Daemon

  • Meaning: Cannot connect to the Docker daemon at unix:///var/run/docker.sock.
  • Cause: Docker service is not running, or your user lacks permissions.
  • Fix:
    1. Start Docker Desktop.
    2. On Linux: sudo systemctl start docker or add user to docker group (see Article 2).

❌ Error: Image Not Found

  • Meaning: pull access denied for ... repository does not exist.
  • Cause: Typo in image name, or trying to pull a private image without logging in.
  • Fix:
    1. Check spelling.
    2. Run docker login.
    3. Check if the image is public or private.

5. Monitoring Resource Usage

Is your container slowing down your server? Use docker stats to see real-time resource consumption.

Bash
docker stats

Output:

Plain Text
1CONTAINER ID   NAME      CPU %     MEM USAGE / LIMIT   MEM %     NET I/O
2a1b2c3d4e5f6   web-api   0.05%     150MiB / 512MiB     29.30%    1.5kB / 0B
  • CPU %: How much processor time the container is using.
  • MEM USAGE: Current memory consumption.
  • NET I/O: Network traffic in/out.

Press Ctrl+C to exit the live stream.


6. Cleaning Up and Disk Space

Over time, Docker can consume significant disk space with unused images, containers, and build caches.

Check Disk Usage

Bash
docker system df

Shows how much space is used by images, containers, volumes, and build cache.

Prune Unused Data

Safe Cleanup: Removes stopped containers and dangling images.

Bash
docker system prune

Aggressive Cleanup: Removes all unused images (not just dangling).

Bash
docker system prune -a

> ⚠️ Warning: -a will delete any image not currently used by a running container. You will have to pull them again next time you need them.

Remove Specific Volumes

Volumes are not removed by prune by default (to protect data).

Bash
docker volume prune

7. Debugging Build Issues

Sometimes the problem isn't running the container, but building it.

  1. Cache Issues: If you changed a file but Docker uses the cached layer, force a rebuild.
    Plain Text
    1bash
    2docker build --no-cache -t my-app .
  2. Context Issues: If your build is slow, check your build context size.
    Plain Text
    1bash
    2docker build -t my-app .  # Watch the "Sending build context to Docker daemon" size
    If it's huge (e.g., 500MB), check your .dockerignore file.
  3. Syntax Errors: Use hadolint (a Dockerfile linter) to catch best practice violations before building.
    Plain Text
    1bash
    2hadolint Dockerfile

Summary Checklist

By the end of this article, you should be able to:

  • Retrieve and filter logs using docker logs.
  • Execute commands inside running containers with docker exec.
  • Extract specific metadata using docker inspect --format.
  • Diagnose common exit codes (137, 1, 125).
  • Monitor real-time resources with docker stats.
  • Clean up disk space safely with docker system prune.

What's Next?

You have now mastered the core Docker ecosystem. You can build, run, network, secure, automate, and debug containers. You are officially a Docker practitioner.

But Docker is just one piece of the cloud-native puzzle.

  • What happens when you need to manage containers across multiple servers?
  • What if you need to orchestrate thousands of containers?
  • What are the alternatives to Docker?

In Article 13 (The Final Article), we will look Beyond Docker. We will discuss Kubernetes, Docker Swarm, Podman, and the future of containerization.

Link: Read Article 13: Beyond Docker (The Ecosystem)


Challenge: Intentionally break a container (e.g., set a memory limit too low). Use docker stats and docker inspect to diagnose the OOM kill. Then fix it and document the steps you took.

Next Up: Beyond Docker (The Ecosystem)

Share:

Comments

0
Join the conversation

Sign in to share your thoughts and connect with other readers

No comments yet

Be the first to share your thoughts!