Installing elasticsearch in k8s cluster1.28.

Elastic search:

Elasticsearch is a distributed, RESTful search and analytics engine designed for

horizontal scalability, reliability, and real-time search capabilities. When deployed in a
Kubernetes environment, Elasticsearch is typically used for various purposes such as
log aggregation, full-text search, monitoring, and analytics.

Uses: Uses:

● Log Aggregation: Elasticsearch can ingest and index large volumes of logs from
various sources, making it suitable for log aggregation and analysis.
● Full-text Search: Elasticsearch provides powerful full-text search capabilities,
enabling users to search and analyze textual data efficiently.
● Monitoring and Analytics: Elasticsearch can be used to store and analyze metrics
and monitoring data from applications and infrastructure components.
● Data Analysis: Elasticsearch supports complex queries, aggregations, and
analytics, making it useful for performing data analysis tasks on large datasets.

Architecture:

● Node: In Elasticsearch, a node is a single running instance of Elasticsearch,

which stores data and participates in the cluster's indexing and search
capabilities. Each node can be configured to perform specific roles such as data
node, master-eligible node, or coordinating node.
● Cluster: A cluster is a collection of one or more Elasticsearch nodes that work
together to store and index data, handle search requests, and ensure cluster
stability and resilience. Nodes within a cluster communicate with each other to
replicate data, distribute indexing and search tasks, and elect a master node to
coordinate cluster operations.
● Index: An index is a logical namespace that maps to one or more primary shards
and their corresponding replica shards. Each index stores a collection of
documents with similar characteristics and can be queried independently.
Elasticsearch indexes are highly distributed and horizontally scalable, allowing
data to be distributed across multiple nodes in a cluster.
● Shard: Elasticsearch indexes are divided into multiple shards, which are
individual segments of data that can be distributed across different nodes in the
cluster. Sharding allows Elasticsearch to distribute data and workload across
 nodes, improving performance and scalability. Each shard can be replicated to
provide redundancy and high availability.
● Replica: A replica is a copy of a primary shard that provides redundancy and fault
tolerance. Elasticsearch automatically creates replicas for each primary shard
and distributes them across nodes in the cluster. Replicas are used to handle
read requests and can also serve as failover targets in case of node failures.
● Coordinating Node: Coordinating nodes are lightweight nodes that act as a proxy
for client requests, routing search and indexing operations to the appropriate
data nodes. Coordinating nodes help distribute query load evenly across the
cluster and improve overall system performance.
● Data Node: Data nodes are responsible for storing and indexing data in
Elasticsearch. They hold primary and replica shards and handle indexing and
search requests. Data nodes are typically configured with ample storage capacity
and high-performance hardware to handle large volumes of data and high query
throughput.
● Master Node: Master nodes are responsible for cluster-wide coordination and
management tasks such as index creation, shard allocation, and cluster state
management. They maintain a complete view of the cluster state and handle
cluster-wide administrative operations such as adding or removing nodes,
rebalancing shards, and managing cluster metadata.

In a Kubernetes environment, Elasticsearch can be deployed as a StatefulSet to

ensure stable pod identities and persistent storage for data nodes. Coordinating nodes
can be deployed as regular Deployment resources to handle client requests efficiently.
Proper configuration of resources such as memory, CPU, and storage is crucial for
optimizing Elasticsearch performance and scalability in Kubernetes

Additionally, integration with other Kubernetes-native tools such as Prometheus for

monitoring and Fluentd for log collection can enhance the observability and operational
efficiency of Elasticsearch deployments.
To install Elasticsearch on a Kubernetes cluster using Helm, you can use the official
Elasticsearch Helm chart provided by Elastic. Below are the steps to install
Elasticsearch using Helm.

Add Helm Repository: If you haven't already added the Elastic Helm repository, you can
do so using the following command:

helm repo add elastic https://helm.elastic.co

helm repo update

Install Elasticsearch: Once the repository is added, you can install Elasticsearch
using the Helm chart.

helm install elasticsearch elastic/elasticsearch

Values.yaml:

# values.yaml
imageTag: "7.16.1"
replicas: 3
esJavaOpts: "-Xmx2g -Xms2g"
To install Elasticsearch using the custom values file.
helm install elasticsearch elastic/elasticsearch -f values.yaml

To port forward Elasticsearch to localhost:

kubectl port-forward service/elasticsearch-master 9200:9200

The other way of deploying the elatci search is deploy using manifest file:

Elastic-search-deployment .yaml

# elasticsearch-deployment.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

name: elasticsearch

spec:

replicas: 1

selector:

matchLabels:

app: elasticsearch

template:

metadata:

labels:
 app: elasticsearch

spec:

containers:

- name: elasticsearch

image: docker.elastic.co/elasticsearch/elasticsearch:7.0.0

ports:

- containerPort: 9200

name: http

- containerPort: 9300

name: transport

resources:

requests:

memory: "2Gi" # Adjust based on your resource requirements

cpu: "500m" # Adjust based on your resource requirements

env:

- name: discovery.type

value: single-node # For single-node deployment

elasticsearch-svc.yaml

# elasticsearch-service.yaml

apiVersion: v1

kind: Service

metadata:

name: elasticsearch

spec:
selector:

app: elasticsearch

ports:

- protocol: TCP

port: 9200

targetPort: 9200

name: http

- protocol: TCP

port: 9300

targetPort: 9300

name: transport

type: NodePort

To deploy:

Kubectl apply -f elasticsearch-deployment.yaml

Kubectl apply -f elasticsearch-svc.yaml

To view the pods deployed:

Kubectl get pods -A | grep elasticsearch

Output:
http://34.28.186.178:32264