Hubble errors occur when network observability components fail to start or cannot communicate. Fix by verifying Hubble deployment, checking Hubble Relay connectivity, and reviewing certificate configuration.
Cilium Hubble provides network observability by collecting flow events from eBPF. When Hubble fails, operators cannot see network traffic patterns, policy violations, or connection issues. Errors typically involve Hubble Relay unable to connect to agents, TLS issues, or service startup failures.
Check if Hubble is configured:
kubectl get daemonset -n kube-system cilium -o yaml | grep -A 5 "hubble"
kubectl get deployment -n kube-system cilium-ui
kubectl get deployment -n kube-system hubble-relayVerify Relay is running:
kubectl get pods -n kube-system -l app=hubble-relay
kubectl logs -n kube-system -l app=hubble-relay --tail=50If not running, check deployment for errors.
From Hubble Relay pod, test connection to agents:
kubectl exec -it -n kube-system deployment/hubble-relay -- \
hubble-relay-cli nodes -o jsonThis lists all connected Cilium nodes.
Verify port 4244 is listening on agents:
kubectl exec -it -n kube-system ds/cilium -- \
ss -tlnp | grep 4244If not listening, Hubble API not started on agent.
Check certificate secrets:
kubectl get secret -n kube-system | grep hubble
kubectl describe secret -n kube-system hubble-relay-client-certsEnsure certificates are valid and properly mounted.
Ensure network policies do not block Hubble:
kubectl get networkpolicy --all-namespaces
# Test if policies allow port 4244
kubectl exec -it -n kube-system ds/cilium -- \
nc -zv localhost 4244Verify service is properly configured:
kubectl get svc -n kube-system hubble-relay
kubectl describe svc -n kube-system hubble-relay
kubectl get endpoints -n kube-system hubble-relayIf Hubble is disabled, enable it:
# Via Helm
helm upgrade cilium cilium/cilium \
--set hubble.enabled=true \
--set hubble.relay.enabled=true
# Or edit ConfigMap
kubectl edit cm -n kube-system cilium-configFor production observability, deploy Hubble Relay separately from agents and give it dedicated resources. Integrate Hubble with Prometheus for metrics collection. Use Hubble UI for interactive debugging of network issues. Set up alerts on flow patterns indicating potential security issues. Hubble has minimal performance impact when enabled but use metrics filtering to avoid overwhelming database. Archive flow logs for compliance and forensics.
No subnets found for EKS cluster
How to fix "eks subnet not found" in Kubernetes
unable to compute replica count
How to fix "unable to compute replica count" in Kubernetes HPA
error: context not found
How to fix "error: context not found" in Kubernetes
default backend - 404
How to fix "default backend - 404" in Kubernetes Ingress
serviceaccount cannot list resource
How to fix "serviceaccount cannot list resource" in Kubernetes