Setup Kibana with Elastic cluster Authentication (X-Pack Security) Enabled - Part 2

In this tutorial, we will setup Kibana with X-Pack security enabled to use basic authentication for accessing Kibana UI. We have already setup Elasticsearch cluster with X-Pack Security enabled and you must follow that tutorial step-by-step before going ahead with this one.

This tutorial is the second part of the 3 part series:

1. Setup Elasticsearch cluster with X-Pack security Enabled

2. Setup Fluent Bit with Elasticsearch and Kibana

You can find the complete code for the complete EFK setup on my Github repository for EFK setup on Kubernetes.

Let's start with the Kibana setup. We will first define the configmap in which we will specify properties like Elasticsearch host, Elasticsearch username, and password which we will define as environment variables in the Kibana deployment YAML file.

kibana-configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: logging
  name: kibana-config
  labels:
    app: kibana
data:
  kibana.yml: |-
    server.host: 0.0.0.0
    elasticsearch:
      hosts: ${ELASTICSEARCH_URL}
      username: ${ELASTICSEARCH_USER}
      password: ${ELASTICSEARCH_PASSWORD}

Then comes the Kibana service in which we will specify the HTTP port as 5601.

kibana-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: kibana
  namespace: logging
  labels:
    component: kibana
spec:
  selector:
    app: kibana
  ports:
  - name: http
    port: 5601
    protocol: TCP

To access the Kibana UI we will define an Ingress to use the AWS ELB(load balancer). Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster.

Here is the YAML file for ingress.

ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: kibana
  namespace: logging
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80}]'
spec:
  rules:
    - http:
        paths:
         - path: /*
           backend:
             serviceName: kibana
             servicePort: 5601

Now we will define the deployment of the Kibana service in which we will specify the container docker image, docker image version (7.3.0 - this should be same as elasticsearch service version), environment variables, and we will use the secret that we created during setup of Elasticsearch in the previous part.

kibana-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: logging
  name: kibana
  labels:
    app: kibana
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kibana
  template:
    metadata:
      labels:
        app: kibana
    spec:
      containers:
      - name: kibana
        image: docker.elastic.co/kibana/kibana:7.3.0
        env:
        - name: ELASTICSEARCH_URL
          value: "http://elasticsearch-client:9200"
        - name: ELASTICSEARCH_USER
          value: "elastic"
        - name: ELASTICSEARCH_PASSWORD
          valueFrom:
            secretKeyRef:
              name: elasticsearch-pw-elastic
              key: password
        resources:
          limits:
            cpu: 2
            memory: 1.5Gi
          requests:
            cpu: 0.5
            memory: 1Gi
        ports:
        - containerPort: 5601
          name: kibana
          protocol: TCP
        volumeMounts:
        - name: config
          mountPath: /usr/share/kibana/config/kibana.yml
          readOnly: true
          subPath: kibana.yml
      volumes:
      - name: config
        configMap:
          name: kibana-config

Now we will use the kubectl command to apply the above configurations and start the Kibana service.

Run the following kubectl command (change the names of the YAML files):

kubectl apply  -f kibana-configmap.yaml \
-f kibana-service.yaml \
-f kibana-deployment.yaml \
-f ingress.yaml

Once the services start, you can get the URL for ingress to access the Kibana UI using the following command:

kubectl get ingress -n logging

Use the following URL for accessing Kibana UI: <INGRESS_URL>/app/kibana

You will see the following screen in the browser and you can use the elastic user along with the password saved in the previous tutorial to access the Kibana UI.

Once you login you will see the Kibana home screen.

Once we configure Fluent Bit to start collecting logs and saving into Elasticsearch, then we can use Kibana UI to see the logs by creating a new index pattern in the Kibana.

Some points for Troubleshooting

It's mandatory that you use the same version of Elasticsearch service and Kibana, else you will get an error. Also, the URL provided in Kibana deployment for Elasticsearch service should be the name of the client node of elasticsearch cluster. If you change the name of the cluster nodes, then you will have to change the same in Kibana deployment too.

If you face some other issues use the kubectl logs command to see the logs for Kibana pod.

You may also like:

• What is Helm in the Kubernetes World?

• Kubernetes Applying Namespace Resource Quota and Limits for Pods

• How to Delete all the Evicted Pods in Kubernetes?

• How to List all Resources in a Kubernetes Namespace