How to Debug Kubernetes Pods: A Complete Guide

Debugging in Kubernetes (K8s) can feel overwhelming at first. With containers, networking layers, persistent volumes, and multiple services running across clusters, figuring out why something isn’t working requires a systematic approach. In this blog, we’ll explore step-by-step methods to debug Kubernetes pods, tools you can use, and common troubleshooting scenarios.

💡 By the end, you’ll know how to confidently investigate and resolve pod-related issues in Kubernetes.

📌 What is a Pod in Kubernetes?

Before debugging, let’s recap:

• A Pod is the smallest deployable unit in Kubernetes.
• It can run one or more containers that share the same network and storage.
• Pods are managed by higher-level controllers like Deployments, ReplicaSets, or StatefulSets.

When a pod fails, it usually indicates:

• Misconfigured resources (CPU, memory, environment variables)
• Networking/DNS issues
• CrashLoopBackOff due to application errors
• Node-level resource shortages

🛠️ Step 1: Check Pod Status

Start by checking the pod’s state:

kubectl get pods -n <namespace>

Example output:

NAME                       READY   STATUS             RESTARTS   AGE
web-app-54d8f4c5f8-bn2d7   0/1     CrashLoopBackOff   5          3m
db-service-74f4cd5d7-xyz   1/1     Running            0          10m

• Running → Pod is healthy
• Pending → Pod is waiting for scheduling (maybe insufficient resources)
• CrashLoopBackOff → Container repeatedly fails
• ImagePullBackOff → Kubernetes cannot pull the container image

🛠️ Step 2: Describe the Pod

The describe command gives detailed information:

kubectl describe pod <pod-name> -n <namespace>

This reveals:

• Node where the pod is scheduled
• Events (warnings, errors, scheduling issues)
• Resource limits
• Volume mounts

Look for sections like:

Events:
  Type     Reason     Age                From              Message
  ----     ------     ----               ----              -------
  Warning  Failed     2m (x5 over 10m)   kubelet           Failed to pull image "myapp:v1": image not found

👉 This shows a misconfigured container image.

🛠️ Step 3: View Pod Logs

Logs help you understand what’s happening inside the container:

kubectl logs <pod-name> -n <namespace>

For pods with multiple containers:

kubectl logs <pod-name> -c <container-name> -n <namespace>

If a pod has restarted multiple times, you can check previous logs:

kubectl logs --previous <pod-name>

🛠️ Step 4: Debug Inside the Pod

You can run commands inside the container with:

kubectl exec -it <pod-name> -- /bin/sh

or

kubectl exec -it <pod-name> -- /bin/bash

Useful commands inside a pod:

• env → Check environment variables
• cat /etc/resolv.conf → Verify DNS resolution
• curl <service-name> → Test connectivity to other services
• df -h → Check disk usage

🛠️ Step 5: Use kubectl debug (Ephemeral Containers)

From Kubernetes v1.18+, you can use ephemeral containers for debugging.

kubectl debug <pod-name> -it --image=busybox

This starts a temporary container inside the same pod’s network namespace. Great for troubleshooting networking or filesystem issues without modifying the original pod.

🛠️ Step 6: Check Node & Events

Sometimes, pods fail because the node is under pressure.

kubectl describe node <node-name>

Check events for cluster-wide issues:

kubectl get events --sort-by=.metadata.creationTimestamp

🛠️ Step 7: Common Issues & Fixes

1. CrashLoopBackOff

Cause: Application error, missing config, or failing health check.
Fix:

• Check logs for stack traces.
• Verify readiness/liveness probes.

2. ImagePullBackOff

Cause: Wrong image name, tag, or missing registry secret.
Fix:

• Check image: in YAML.
• Add registry secret: kubectl create secret docker-registry myregistrykey \ --docker-server=<registry> \ --docker-username=<username> \ --docker-password=<password>

3. Pending Pods

Cause: No available node resources.
Fix:

• Scale up nodes.
• Reduce resource requests in deployment YAML.

4. DNS Resolution Issues

Cause: CoreDNS misconfiguration.
Fix:

• Restart CoreDNS: kubectl rollout restart deployment coredns -n kube-system

🛠️ Step 8: Advanced Debugging Tools

• Lens IDE → Visual Kubernetes dashboard
• Stern → Stream logs from multiple pods
• K9s → Terminal-based Kubernetes management
• Prometheus + Grafana → Metrics and alerts
• Jaeger → Distributed tracing

📖 Best Practices for Avoiding Debugging Nightmares

• Always use readiness/liveness probes.
• Set resource requests & limits properly.
• Enable logging & monitoring (ELK, EFK, Prometheus).
• Use ConfigMaps & Secrets instead of hardcoding values.
• Version control your YAML manifests with GitOps tools (ArgoCD, Flux).

✅ Conclusion

Debugging Kubernetes pods requires a structured approach:

1. Check pod status
2. Describe the pod
3. View logs
4. Exec into the container
5. Use ephemeral debug containers
6. Inspect node & events

With practice, you’ll develop intuition to quickly identify whether the issue is in the application, Kubernetes config, or infrastructure.