Kubernetes Networking — Broken Labs & Incident Response

Source: Dev.to | Jan 09, 2026 (UTC)

How to Think Like DevOps in Production

When traffic fails, never guess.
Always follow this order:

Ingress ↓ Service ↓ Endpoints ↓ Pod ↓ Container

Enter fullscreen mode

Exit fullscreen mode

If one layer fails, everything above it fails.

LAB 1 — ClusterIP Service (Most Common Production Failure)

Scenario

Broken Setup

Deployment

apiVersion: apps/v1 kind: Deployment metadata: name: api spec: replicas: 2 selector: matchLabels: app: api template: metadata: labels: app: api-v1 # ❌ wrong label spec: containers: - name: app image: hashicorp/http-echo:0.2.3 args: - "-listen=:8080" - "-text=API OK" ports: - containerPort: 8080

Enter fullscreen mode

Exit fullscreen mode

Service

apiVersion: v1 kind: Service metadata: name: api-svc spec: selector: app: api # ❌ mismatch ports: - port: 80 targetPort: 8080

Enter fullscreen mode

Exit fullscreen mode

Symptoms

kubectl get pods kubectl get svc kubectl get endpoints api-svc

Enter fullscreen mode

Exit fullscreen mode

Output:

ENDPOINTS: <none>

Enter fullscreen mode

Exit fullscreen mode

Root Cause

Service selector does not match Pod labels.

Fix

kubectl edit deployment api

Enter fullscreen mode

Exit fullscreen mode

Change:

labels: app: api

Enter fullscreen mode

Exit fullscreen mode

Verify:

kubectl get endpoints api-svc

Enter fullscreen mode

Exit fullscreen mode

DevOps Interview Answer

Q: Service exists but no traffic, pods running. What do you check?
A: Endpoints. Empty endpoints indicate selector or readiness issues.

When to Use ClusterIP

Pros / Cons

Pros

Cons

LAB 2 — NodePort (Why It’s Dangerous)

Scenario

Setup

apiVersion: v1 kind: Service metadata: name: node-svc spec: type: NodePort selector: app: api ports: - port: 80 targetPort: 8080 nodePort: 30080

Enter fullscreen mode

Exit fullscreen mode

Symptoms

Root Cause

DevOps Fix

Interview Answer

Q: Why is NodePort rarely used in production?
A: It exposes every node, lacks security, routing, and doesn’t scale.

When NodePort Is Acceptable

LAB 3 — LoadBalancer Service (Cloud Reality)

Scenario

Setup

apiVersion: v1 kind: Service metadata: name: lb-svc spec: type: LoadBalancer selector: app: api ports: - port: 80 targetPort: 8080

Enter fullscreen mode

Exit fullscreen mode

Symptoms

kubectl get svc

Enter fullscreen mode

Exit fullscreen mode

Troubleshooting

kubectl describe svc lb-svc kubectl get endpoints lb-svc

Enter fullscreen mode

Exit fullscreen mode

Check cloud:

Root Cause

Cloud LB health check fails because:

DevOps Fix

Interview Answer

Q: Why not use LoadBalancer for every service?
A: Cost, lack of routing, and limited flexibility compared to Ingress.

LAB 4 — Ingress (Most Interviewed Topic)

Scenario

Broken Ingress

apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: app-ingress spec: rules: - http: paths: - path: /app pathType: Prefix backend: service: name: wrong-svc # ❌ wrong name port: number: 80

Enter fullscreen mode

Exit fullscreen mode

Symptoms

Troubleshooting

kubectl describe ingress kubectl get svc kubectl get pods -n ingress-nginx

Enter fullscreen mode

Exit fullscreen mode

Root Cause

Ingress routes to non-existent service.

Fix

Correct backend service name.

Interview Answer

Q: Ingress returns 404, where do you check first?
A: Ingress rules, service name, service port, and controller logs.

LAB 5 — DNS Failure (Hidden Killer)

Scenario

Test

kubectl run test --rm -it --image=busybox -- sh nslookup api-svc

Enter fullscreen mode

Exit fullscreen mode

Root Cause

Fix

kubectl get pods -n kube-system | grep dns

Enter fullscreen mode

Exit fullscreen mode

Restart if needed.

Interview Answer

Q: How do Pods discover services?
A: Via Kubernetes DNS resolving Service names to ClusterIP.

INCIDENT RESPONSE PLAYBOOK (Real DevOps)

Step-by-Step

  1. Check Ingress
  2. Check Service
  3. Check Endpoints
  4. Check Pod readiness
  5. Check container logs

Never skip steps.

FINAL DECISION MATRIX (Very Important)

Requirement

Use

Internal traffic

ClusterIP

External production traffic

Ingress

Cloud simple exposure

LoadBalancer

Debug only

NodePort

INTERVIEW RAPID FIRE (Must Memorize)

Q: Empty endpoints means?
A: Selector or readiness failure

Q: Most used service in prod?
A: ClusterIP

Q: Why Ingress?
A: Routing, TLS, cost efficiency

Q: NodePort in prod?
A: Avoid

REAL DEVOPS TRUTH

Networking issues are:

The difference between struggling and solving fast is methodical thinking.

For further actions, you may consider blocking this person and/or reporting abuse

Залишити відповідь

Ваша e-mail адреса не оприлюднюватиметься. Обов’язкові поля позначені *

Читайте також