Elasticsearch

Advance configuration information for Elasticsearch

1: Plugins Management
2: Keystore Integration

1 - Plugins Management

Plugins related configuration for elasticsearch

Plugins Installation

Plugins installation has been simplified using the Logging Operator. To install the plugins inside the elasticsearch, we just need to define the list of plugins inside the esPlugins section of elasticsearch CRD.

For example:-

---
apiVersion: logging.logging.opstreelabs.in/v1beta1
kind: Elasticsearch
metadata:
  name: elasticsearch
spec:
  esClusterName: "prod"
  esVersion: "7.16.0"
  esPlugins: ["repository-s3"]

Validation of plugin installation can be done using elasticsearch-plugin or curl command.

$ kubectl exec -it elasticsearch-master-0 -n ot-operators -- ./bin/elasticsearch-plugin list
...
repository-s3

$ kubectl exec -it elasticsearch-master-0 -n ot-operators -- curl http://localhost:9200/_cat/plugins
...
elasticsearch-master-1 repository-s3 7.16.0
elasticsearch-master-2 repository-s3 7.16.0
elasticsearch-master-0 repository-s3 7.16.0

Helm Configuration

Plugin installation can also be done using helm chart of elasticsearch. We just need to define the plugins list in the values file of elasticsearch helm chart.

https://github.com/OT-CONTAINER-KIT/helm-charts/blob/main/charts/elasticsearch/values.yaml#L8

$ helm upgrade elasticsearch ot-helm/elasticsearch --namespace ot-operators \
  --set esMaster.storage.storageClass=do-block-storage \
  --set esData.storage.storageClass=do-block-storage --install

2 - Keystore Integration

Keystore integration configuration for elasticsearch

Keystore integation

Keystore is a recommended way of integrating different credentials like:- AWS, GCP, Azure and Slack, etc. to elasticsearch cluster. We simply need to create a Kubernetes secret and the operator can take care the integration of Kubernetes secret to elasticsearch keystore.

$ kubectl create secret generic slack-hook \
--from-literal=xpack.notification.slack.account.monitoring.secure_url='https://hooks.slack.com/services/asdasdasd/asda'

or yaml file is also one of the way for creating the secret.

---
apiVersion: v1
kind: Secret
metadata:
  name: encryption-key
data:
  xpack.notification.slack.account.monitoring.secure_url: aHR0cHM6Ly9ob29rcy5zbGFjay5jb20vc2VydmljZXMvYXNkYXNkYXNkL2FzZGFzZGFzL2FzZGFzZA==
# other secrets key value pairs can be defined here
type: Opaque

Then simply we can define the keystore secret name in CRD definition.

---
apiVersion: logging.logging.opstreelabs.in/v1beta1
kind: Elasticsearch
metadata:
  name: elasticsearch
spec:
  esClusterName: "prod"
  esVersion: "7.16.0"
  esKeystoreSecret: encryption-key

Validation of keystore can be done using elasticsearch-keystore command.

$ kubectl exec -it elasticsearch-master-0 -n ot-operators -- ./bin/elasticsearch-keystore list
...
keystore.seed
xpack.notification.slack.account.monitoring.secure_url

Helm Configuration

Keystore integration can also be done using helm chart of elasticsearch. We just need to define the keystore secret name in the values file of elasticsearch helm chart.

https://github.com/OT-CONTAINER-KIT/helm-charts/blob/main/charts/elasticsearch/values.yaml#L9

$ helm upgrade elasticsearch ot-helm/elasticsearch --namespace ot-operators \
  --set esMaster.storage.storageClass=do-block-storage \
  --set esData.storage.storageClass=do-block-storage --install