K8S-Pod: initContainer

In this article, we will explore the initContainer when creating a pod, the initContainer must be ready before the running of the main container, let’s create a yaml file like this, which will block the creating of initContainer, and the main container will never be ready:

apiVersion: v1
kind: Pod
metadata:
  name: pod-init-test
  labels:
    app: test
spec:
  containers:
  - name: myapp-container
    image: busybox:latest
    imagePullPolicy: Never
    command: ['sh', '-c', 'echo The app is running! && sleep 3600']
  initContainers:
  - name: init-myservice
    image: busybox:latest
    imagePullPolicy: Never
    command: ['sh', '-c', "until nslookup myservice.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for myservice; sleep 2; done"]

Now, apply the yaml file, and check pod status, as you can see, it is not ready as initContainer failed to start:

$ kubectl describe pod pod-init-test
Name:             pod-init-test
Namespace:        default
Priority:         0
Service Account:  default
Node:             kind-worker/172.21.0.2
Start Time:       Fri, 07 Feb 2025 09:41:55 +0000
Labels:           app=test
Annotations:      <none>
Status:           Pending
IP:               10.244.2.2
IPs:
  IP:  10.244.2.2
Init Containers:
  init-myservice:
    Container ID:  containerd://5ba3e4bd5c5b5e783c76d4e6d0737021c7a776087fda0902858b7794a7432f13
    Image:         busybox:latest
    Image ID:      docker.io/library/import-2025-02-07@sha256:1fdddfbd8993e5144336c01f789aa15306b9877fe293f9bc16c9c7a18974d3d0
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
      until nslookup myservice.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for myservice; sleep 2; done
    State:          Running
      Started:      Fri, 07 Feb 2025 09:41:56 +0000
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-rkcxt (ro)
Containers:
  myapp-container:
    Container ID:
    Image:         busybox:latest
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
      echo The app is running! && sleep 3600
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-rkcxt (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True
  Initialized                 False
  Ready                       False
  ContainersReady             False
  PodScheduled                True
Volumes:
  kube-api-access-rkcxt:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  67s   default-scheduler  Successfully assigned default/pod-init-test to kind-worker
  Normal  Pulled     67s   kubelet            Container image "busybox:latest" already present on machine
  Normal  Created    67s   kubelet            Created container: init-myservice
  Normal  Started    67s   kubelet            Started container init-myservice

Let’s check the log of initContainer, it is the nslookup issue!

$ kubectl logs pod-init-test -c init-myservice

waiting for myservice
Server:         10.96.0.10
Address:        10.96.0.10:53

** server can't find myservice.default.svc.cluster.local: NXDOMAIN

** server can't find myservice.default.svc.cluster.local: NXDOMAIN

Let’s try to fix this issue by creating a service:

$ cat create_myservice.yaml
apiVersion: v1
kind: Service
metadata:
  name: myservice
spec:
  ports:
  - protocol: TCP
    port: 80
    targetPort: 9376

$ kubectl apply -f create_myservice.yaml
service/myservice created

$ kubectl get service
NAME         TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1     <none>        443/TCP   24h
myservice    ClusterIP   10.96.50.91   <none>        80/TCP    7s

# We fixed this!
$ kubectl get pod
NAME            READY   STATUS    RESTARTS   AGE
pod-init-test   1/1     Running   0          76m

Check the initContainer log now, we can see that the domain name of myservice is resolved:

$ kubectl logs pod-init-test -c init-myservice | tail  -n10
** server can't find myservice.default.svc.cluster.local: NXDOMAIN

waiting for myservice
Server:         10.96.0.10
Address:        10.96.0.10:53


Name:   myservice.default.svc.cluster.local
Address: 10.96.50.91