OpenParable
Skip to content
← Back to skills

K8S Debug Pods

Testing & Quality
k8s-debug-pods

K8S Debug Pods is an Agent Skill for testing & quality published in the kurtosis-tech/kurtosis repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Debug Kurtosis pods on Kubernetes. Diagnose why pods are Pending, CrashLoopBackOff, ImagePullBackOff, or Evicted. Check node taints, tolerations, resource pressure, and pod events. Use when kurtosis engine start fails or pods aren't coming online.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/kurtosis-tech/kurtosis/blob/HEAD/skills/k8s-debug-pods/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/k8s-debug-pods, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

K8s Debug Pods

Diagnose and fix issues with Kurtosis pods on Kubernetes.

Quick triage

# See all kurtosis-related pods across namespaces
kubectl get pods -A | grep kurtosis

# Check for problem pods (not Running)
kubectl get pods -A | grep kurtosis | grep -v Running

# Get events for a specific pod
kubectl describe pod <POD_NAME> -n <NAMESPACE> | tail -30

Common pod states and fixes

Pending — Unschedulable

The pod can't be scheduled because of node taints, resource pressure, or affinity rules.

# Check node taints
kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints

# Check node conditions (DiskPressure, MemoryPressure, etc.)
kubectl get nodes -o custom-columns=NAME:.metadata.name,CONDITIONS:.status.conditions[*].type

Fix: Add tolerations to the kurtosis config at ~/Library/Application Support/kurtosis/kurtosis-config.yml or fix the node condition.

ImagePullBackOff

The image tag doesn't exist on the registry.

# Check which image is failing
kubectl describe pod <POD_NAME> -n <NAMESPACE> | grep -A5 "Image:"

# Verify image exists on Docker Hub
docker manifest inspect <IMAGE>:<TAG>

Fix: Push the correct image tag, or fix the image reference in the code.

CrashLoopBackOff

The container starts but crashes immediately.

# Check container logs
kubectl logs <POD_NAME> -n <NAMESPACE>
kubectl logs <POD_NAME> -n <NAMESPACE> --previous

Evicted

The node evicted the pod due to resource pressure.

# Check which nodes have pressure
kubectl get nodes -o custom-columns=NAME:.metadata.name,STATUS:.status.conditions[-1].type

# Clean up evicted pods
kubectl get pods -A | grep Evicted | awk '{print $2 " -n " $1}' | xargs -L1 kubectl delete pod

Kurtosis-specific pod types

Pod patternComponentImage source
kurtosis-engine-*Engine serverengine/server/Dockerfile
kurtosis-api (in kt-* namespaces)API Container (APIC)core/server/Dockerfile
kurtosis-logs-collector-*Fluentbit DaemonSetPulled from registry
kurtosis-logs-aggregator-*Vector deploymentPulled from registry
remove-dir-pod-*Fluentbit cleanup podsbusybox
files-artifact-expander (init container)Files artifactscore/files_artifacts_expander/Dockerfile

Engine start failures

If kurtosis engine start fails:

  1. Check if old kurtosis namespaces exist: kubectl get ns | grep kurtosis
  2. Delete them: kubectl get ns | grep kurtosis | awk '{print $1}' | xargs -r kubectl delete ns
  3. Retry engine start

Logs collector issues

The logs collector is a DaemonSet that runs on every node. If some nodes are unhealthy:

# Check DaemonSet status
kubectl get ds -A | grep kurtosis

# See which pods are not running
kubectl get pods -A | grep logs-collector | grep -v Running

Nodes with DiskPressure or other taints may not schedule collector pods — this is expected and the engine should start with a warning about partially degraded collection.