Readiness probe failed
This error occurs when a container's readiness health check fails, preventing it from receiving network traffic from Services.
Usually happens because:
- ☑ Container starts up too slowly and fails initial probe timings
- ☑ Wrong HTTP path, port value, or query schema in probe setup
- ☑ Downstream database connection timeouts during health check queries
🔍 Quick Checklist:
What is Readiness probe failed?
A 'Readiness probe failed' error indicates that the container is running but is not yet ready to accept incoming client connections. When a readiness probe fails, the Kubelet removes the Pod's IP address from the Endpoints list of all matching Services. In Kubernetes v1.36, readiness probes support HTTP, TCP, gRPC, and Exec action hooks, and failure events are registered within the Kubelet state logs.
Common Causes
- Slow application startup: The container starts up slowly, and the readiness probe executes before the application service is fully initialized.
- Incorrect probe path or port: The probe configuration references a wrong HTTP path (e.g. '/health') or a wrong network port.
- Database or backend dependency offline: The application's health check endpoint tests downstream database connections that are temporarily unreachable.
| Cause | Frequency |
|---|---|
| Application startup is slower than probe initial delay | ⭐⭐⭐⭐⭐ |
| Typo in HTTP health check path or container port | ⭐⭐⭐⭐ |
| Unreachable downstream database or external API dependencies | ⭐⭐⭐ |
Common Mistakes
- Setting `timeoutSeconds` value lower than the average response latency of the downstream database checks, causing healthy containers to be marked unhealthy during query spikes.
- Using liveness probes configurations instead of readiness probes for slow-starting daemons, triggering infinite container restart loops.
How to Fix
Kubernetes Operations & Verification
Configure native gRPC readiness checks in Kubernetes v1.36 and protect bootstrapping using a startupProbe.
apiVersion: v1
kind: Pod
metadata:
name: grpc-service-pod
spec:
containers:
- name: grpc-app
image: grpc-server:v1.36.0
ports:
- containerPort: 50051
# 1. Startup probe shields app until boot finishes
startupProbe:
grpc:
port: 50051
failureThreshold: 30
periodSeconds: 10
# 2. Readiness probe runs only after startupProbe succeeds
readinessProbe:
grpc:
port: 50051
periodSeconds: 5Platform Specific Fixes
Query endpoint listings to see if the Pod IP is removed from Services list during readiness failures.
# 1. Inspect Service active endpoints
kubectl get endpoints web-server-service
# 2. Test the app port from inside a test cluster container
kubectl run curl-test --image=curlimages/curl --rm -it -- restart=Never -- curl -I http://pod-ip:80/healthzBest Practices
- Design light-weight health check endpoints (e.g. `/healthz`) that return quick HTTP 200 without executing expensive database writes.
- Adopt startup probes to handle slow cold-starts in large microservices architectures.
Frequently Asked Questions (FAQ)
Q: What is a readiness probe?
A readiness probe is a diagnostic check run by the Kubelet to determine if a container is ready to accept network traffic. If it fails, the Pod is removed from Kubernetes Service endpoints.
Q: How does a readiness probe differ from a liveness probe?
If a readiness probe fails, Kubernetes stops sending traffic to the Pod but keeps it running. If a liveness probe fails, Kubernetes kills and restarts the container.
Q: How do I fix failures caused by slow startup?
Increase the 'initialDelaySeconds' in your YAML, or configure a 'startupProbe' (available since v1.18 and recommended in v1.36) to shield the readiness probe until bootstrap completes.
Q: How do I write a gRPC readiness probe?
In Kubernetes v1.36, gRPC probes are natively supported: 'grpc: port: 8080'. Ensure your gRPC server implements the standard gRPC Health Checking Protocol.