Skip to content
This repository was archived by the owner on Mar 22, 2024. It is now read-only.

Commit 74a941b

Browse files
naemonoframsouzapebrc
authored
ECK Resources Helm Chart (elastic#5781)
* Adding initial revision of eck-stack, and eck-kibana charts. * Update Chart.yaml to have kubeversion. Add README. Add notes for post-installtion. Update values with more docs * Update values file with more information. * adding es resource * Making things consistent across charts. * Adding newlines Adding top yaml markers * Add public documentation for eck-stack helm chart. Update example values to work properly. * Adding helm test script. Adding minimal tests for eck-stack helm chart. * add newline * Add makefile entry. remove unused bits from script. Fix naming in helm chart. * Add helm test to ci checks * make chart.yaml consistent * Adding newlines Updating wording in documentation. * Update newline * Update readme for eck-stack * remove todo from eck-stack readme * quote things to passify spellcheck * Quote more vars. Use $() instead of `` * Fix sc2045 * Updates from first round of reviews. * Update the documentation. Run the dependencies prior to linting. * Fix shellcheck * Add note that ECK stack helm charts are enterprise * Update license note in docs/orchestrating-elastic-stack-applications/stack-helm-chart.asciidoc Co-authored-by: Peter Brachwitz <[email protected]> * Update verbiage deploy/eck-stack/README.md Co-authored-by: Peter Brachwitz <[email protected]> * Update verbiage deploy/eck-stack/README.md Co-authored-by: Peter Brachwitz <[email protected]> * Add 'the' to documentation in docs/orchestrating-elastic-stack-applications/stack-helm-chart.asciidoc Co-authored-by: Peter Brachwitz <[email protected]> * add 'the' again to docs in docs/orchestrating-elastic-stack-applications/stack-helm-chart.asciidoc Co-authored-by: Peter Brachwitz <[email protected]> * Comment out affinity section, as it won't work by default. Adjust kibana elasticsearcRef to be default for release name 'quickstart'. Adjust wording for reference. * Update public documentation to be more clear. * Add dedicated master nodes to dedicated examples * Fix elasticsearch test in eck-elasticsearch chart comment out namespace in elasticsearchref in kibana example. adjust kibana tests to tests for specifying namespace for elasticsearchref. s/fullNameOverride/fullnameOverride remove eck-stack examples with only ES or Kibana. Add eck-stack example with both ES and Kibana customizations. Adjust eck-stack tests for new example. * Fix missing default labels in Elasticsearch. Fix random '{}' in annotatinos in ES/Kibana. * Make default quickstart elasticsearchRef work by default. * Removing logic that inserts namespace for elasticsearchRef, as operator interally handles it. * Move to referencing https://helm.elastic.co helm repository. * Fix helm tests * Add helm test to pr ci pipeline * Add helm to ci-tools container. * Handling Review comments * Adjust default readme in deploy directory Adjust default values for eck-elasticsearch and eck-kibana. Debug helm unittest script in CI * Debugging helm unittest * Fix shellcheck complaining about debugging messages. * Try and run helm unittest directly * Install the correct plugin, with static version * Adjust default values for eck-elasticsearch, and eck-kibana for the eck-stack chart. * Fix test for default elasticsearch name * Remove debugging from helm test script. * Adjust chart.yaml back to using helm.elastic.co repository. * Use sed to force local changes during helm testing, and not charts in helm.elastic.co. * Making shellcheck happy * Add helm documentation to dev-setup.md. Co-authored-by: framsouza <[email protected]> Co-authored-by: Peter Brachwitz <[email protected]>
1 parent 17e01bc commit 74a941b

35 files changed

+1330
-8
lines changed

‎.ci/Dockerfile

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ ENV DOCKER_BUILDX_VERSION=0.8.2
1111
ENV GOTESTSUM_VERSION=1.8.0
1212
ENV EKSCTL_VERSION=0.74.0
1313
ENV ECK_DIAG_VERSION=1.1.0
14+
ENV HELM_VERSION=3.9.1
1415

1516
# golangci-lint
1617
RUN curl -sfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh \
@@ -82,6 +83,13 @@ RUN curl -fsSLO "https://github.com/weaveworks/eksctl/releases/download/v${EKSCT
8283
mv eksctl /usr/local/bin/eksctl && \
8384
rm eksctl_Linux_amd64.tar.gz
8485

86+
RUN curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 && \
87+
chmod 700 get_helm.sh && \
88+
./get_helm.sh -v v${HELM_VERSION} --no-sudo && \
89+
rm get_helm.sh
90+
91+
RUN helm plugin install https://github.com/quintush/helm-unittest --version 0.2.8
92+
8593
# Cache ECK Go dependencies in this Docker image
8694
WORKDIR /go/src/github.com/elastic/cloud-on-k8s
8795
COPY ["go.mod", "go.sum", "./"]

‎.ci/pipelines/pr-gke.Jenkinsfile

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,11 @@ pipeline {
107107
sh 'make -C .ci TARGET=reattach-pv ci'
108108
}
109109
}
110+
stage("helm-test") {
111+
steps {
112+
sh 'make -C .ci TARGET=helm-test ci'
113+
}
114+
}
110115
stage("unit-tests") {
111116
steps {
112117
script {

‎.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,3 +67,6 @@ pkg/controller/common/license/zz_generated.pubkey.go
6767

6868
# ignore validator for Jenkins pipelines
6969
hack/pipeline-validator/validator
70+
71+
# ignore helm chart dependencies in eck-stack chart
72+
deploy/eck-stack/charts/*

‎Makefile

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,9 @@ unit: clean
156156
unit-xml: clean
157157
ECK_TEST_LOG_LEVEL=$(LOG_VERBOSITY) gotestsum --junitfile unit-tests.xml -- -cover ./pkg/... ./cmd/... $(TEST_OPTS)
158158

159+
helm-test:
160+
@hack/helm/test.sh
161+
159162
integration: GO_TAGS += integration
160163
integration: clean generate-crds-v1
161164
ECK_TEST_LOG_LEVEL=$(LOG_VERBOSITY) go test -tags='$(GO_TAGS)' ./pkg/... ./cmd/... -cover $(TEST_OPTS)

‎deploy/README.md

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,15 @@
1-
# ECK Operator Helm Chart
1+
# ECK Operator, and ECK Resources Helm Charts
22

33
[![Artifact HUB](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/elastic)](https://artifacthub.io/packages/search?repo=elastic)
44

5-
This directory contains the Helm chart for deploying the ECK operator.
5+
This directory contains the Helm chart for deploying the ECK operator, and charts for deploying any resource in the Elastic Stack individually, or as a group.
66

7-
See also the [ECK Helm install guide](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-install-helm.html).
8-
9-
## Usage
7+
## ECK Operator Helm Chart Usage
108

119
Install the CRDs and deploy the operator with cluster-wide permissions to manage all namespaces.
1210

1311
```sh
14-
helm install elastic-operator elastic/eck-operator -n elastic-system --create-namespace
12+
helm install elastic-operator eck-operator -n elastic-system --create-namespace
1513
```
1614

1715
Install the operator restricted to a single namespace.
@@ -21,7 +19,7 @@ Install the operator restricted to a single namespace.
2119
helm install elastic-operator-crds ./eck/charts/eck-operator-crds
2220

2321
# This step can be done by any user with full access to the my-namespace namespace.
24-
helm install elastic-operator elastic/eck-operator -n my-namespace --create-namespace \
22+
helm install elastic-operator eck-operator -n my-namespace --create-namespace \
2523
--set=installCRDs=false \
2624
--set=managedNamespaces='{my-namespace}' \
2725
--set=createClusterScopedResources=false \
@@ -31,7 +29,19 @@ helm install elastic-operator elastic/eck-operator -n my-namespace --create-name
3129
View the available settings for customizing the installation.
3230

3331
```sh
34-
helm show values elastic/eck-operator
32+
helm show values eck-operator
3533
```
3634

35+
## ECK Stack Helm Chart Usage
36+
37+
Install a quickstart Elasticsearch and Kibana resource in a cluster controlled by the ECK Operator.
3738

39+
```sh
40+
helm install es-kb-quickstart elastic/eck-stack -n elastic-stack --create-namespace
41+
```
42+
43+
To see all resources installed by the helm chart
44+
45+
```sh
46+
kubectl get elastic -l "app.kubernetes.io/instance"=es-kb-quickstart -n elastic-stack
47+
```

‎deploy/eck-elasticsearch/.helmignore

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Patterns to ignore when building packages.
2+
# This supports shell glob matching, relative path matching, and
3+
# negation (prefixed with !). Only one pattern per line.
4+
.DS_Store
5+
# Common VCS dirs
6+
.git/
7+
.gitignore
8+
.bzr/
9+
.bzrignore
10+
.hg/
11+
.hgignore
12+
.svn/
13+
# Common backup files
14+
*.swp
15+
*.bak
16+
*.tmp
17+
*.orig
18+
*~
19+
# Various IDEs
20+
.project
21+
.idea/
22+
*.tmproj
23+
.vscode/
24+
templates/tests

‎deploy/eck-elasticsearch/Chart.yaml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
apiVersion: v2
2+
name: eck-elasticsearch
3+
description: A Helm chart to deploy Elasticsearch managed by the ECK Operator.
4+
kubeVersion: ">= 1.20.0-0"
5+
type: application
6+
version: 0.1.0
7+
sources:
8+
- https://github.com/elastic/cloud-on-k8s
9+
- https://github.com/elastic/elasticsearch/
10+
icon: https://helm.elastic.co/icons/elasticsearch.png

‎deploy/eck-elasticsearch/examples/hot-warm-cold.yaml

Lines changed: 198 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,198 @@
1+
---
2+
nodeSets:
3+
- name: masters
4+
count: 1
5+
config:
6+
node.roles: ["master"]
7+
# Comment out when setting the vm.max_map_count via initContainer, as these are mutually exclusive.
8+
# For production workloads, it is strongly recommended to increase the kernel setting vm.max_map_count to 262144
9+
# and leave node.store.allow_mmap unset.
10+
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/master/k8s-virtual-memory.html
11+
#
12+
node.store.allow_mmap: false
13+
podTemplate:
14+
spec:
15+
containers:
16+
- name: elasticsearch
17+
resources:
18+
limits:
19+
memory: 8Gi
20+
cpu: 2
21+
# Affinity/Anti-affinity settings for controlling the 'spreading' of Elasticsearch
22+
# pods across existing hosts.
23+
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
24+
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-advanced-node-scheduling.html#k8s-affinity-options
25+
#
26+
# affinity:
27+
# nodeAffinity:
28+
# requiredDuringSchedulingIgnoredDuringExecution:
29+
# nodeSelectorTerms:
30+
# - matchExpressions:
31+
# - key: beta.kubernetes.io/instance-type
32+
# operator: In
33+
# # This should be adjusted to the instance type according to your setup
34+
# #
35+
# values:
36+
# - highio
37+
# Volume Claim settings.
38+
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/2.2/k8s-volume-claim-templates.html
39+
#
40+
volumeClaimTemplates:
41+
- metadata:
42+
name: elasticsearch-data
43+
spec:
44+
accessModes:
45+
- ReadWriteOnce
46+
resources:
47+
requests:
48+
storage: 1Ti
49+
# Adjust to your storage class name
50+
#
51+
# storageClassName: local-storage
52+
- name: hot
53+
count: 1
54+
config:
55+
node.roles: ["data_hot", "data_content", "ingest"]
56+
# Comment out when setting the vm.max_map_count via initContainer, as these are mutually exclusive.
57+
# For production workloads, it is strongly recommended to increase the kernel setting vm.max_map_count to 262144
58+
# and leave node.store.allow_mmap unset.
59+
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/master/k8s-virtual-memory.html
60+
#
61+
node.store.allow_mmap: false
62+
podTemplate:
63+
spec:
64+
containers:
65+
- name: elasticsearch
66+
resources:
67+
limits:
68+
memory: 16Gi
69+
cpu: 4
70+
# Affinity/Anti-affinity settings for controlling the 'spreading' of Elasticsearch
71+
# pods across existing hosts.
72+
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
73+
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-advanced-node-scheduling.html#k8s-affinity-options
74+
#
75+
# affinity:
76+
# nodeAffinity:
77+
# requiredDuringSchedulingIgnoredDuringExecution:
78+
# nodeSelectorTerms:
79+
# - matchExpressions:
80+
# - key: beta.kubernetes.io/instance-type
81+
# operator: In
82+
# # This should be adjusted to the instance type according to your setup
83+
# #
84+
# values:
85+
# - highio
86+
# Volume Claim settings.
87+
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/2.2/k8s-volume-claim-templates.html
88+
#
89+
volumeClaimTemplates:
90+
- metadata:
91+
name: elasticsearch-data
92+
spec:
93+
accessModes:
94+
- ReadWriteOnce
95+
resources:
96+
requests:
97+
storage: 1Ti
98+
# Adjust to your storage class name
99+
#
100+
# storageClassName: local-storage
101+
- name: warm
102+
count: 1
103+
config:
104+
node.roles: ["data_warm"]
105+
# Comment out when setting the vm.max_map_count via initContainer, as these are mutually exclusive.
106+
# For production workloads, it is strongly recommended to increase the kernel setting vm.max_map_count to 262144
107+
# and leave node.store.allow_mmap unset.
108+
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/master/k8s-virtual-memory.html
109+
#
110+
node.store.allow_mmap: false
111+
podTemplate:
112+
spec:
113+
containers:
114+
- name: elasticsearch
115+
resources:
116+
limits:
117+
memory: 16Gi
118+
cpu: 2
119+
# Affinity/Anti-affinity settings for controlling the 'spreading' of Elasticsearch
120+
# pods across existing hosts.
121+
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
122+
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-advanced-node-scheduling.html#k8s-affinity-options
123+
#
124+
# affinity:
125+
# nodeAffinity:
126+
# requiredDuringSchedulingIgnoredDuringExecution:
127+
# nodeSelectorTerms:
128+
# - matchExpressions:
129+
# - key: beta.kubernetes.io/instance-type
130+
# operator: In
131+
# # This should be adjusted to the instance type according to your setup
132+
# #
133+
# values:
134+
# - highstorage
135+
# Volume Claim settings.
136+
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/2.2/k8s-volume-claim-templates.html
137+
#
138+
volumeClaimTemplates:
139+
- metadata:
140+
name: elasticsearch-data
141+
spec:
142+
accessModes:
143+
- ReadWriteOnce
144+
resources:
145+
requests:
146+
storage: 10Ti
147+
# Adjust to your storage class name
148+
#
149+
# storageClassName: local-storage
150+
- name: cold
151+
count: 1
152+
config:
153+
node.roles: ["data_cold"]
154+
# Comment out when setting the vm.max_map_count via initContainer, as these are mutually exclusive.
155+
# For production workloads, it is strongly recommended to increase the kernel setting vm.max_map_count to 262144
156+
# and leave node.store.allow_mmap unset.
157+
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/master/k8s-virtual-memory.html
158+
#
159+
node.store.allow_mmap: false
160+
podTemplate:
161+
spec:
162+
containers:
163+
- name: elasticsearch
164+
resources:
165+
limits:
166+
memory: 8Gi
167+
cpu: 2
168+
# Affinity/Anti-affinity settings for controlling the 'spreading' of Elasticsearch
169+
# pods across existing hosts.
170+
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
171+
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-advanced-node-scheduling.html#k8s-affinity-options
172+
#
173+
# affinity:
174+
# nodeAffinity:
175+
# requiredDuringSchedulingIgnoredDuringExecution:
176+
# nodeSelectorTerms:
177+
# - matchExpressions:
178+
# - key: beta.kubernetes.io/instance-type
179+
# operator: In
180+
# # This should be adjusted to the instance type according to your setup
181+
# #
182+
# values:
183+
# - highstorage
184+
# Volume Claim settings.
185+
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/2.2/k8s-volume-claim-templates.html
186+
#
187+
volumeClaimTemplates:
188+
- metadata:
189+
name: elasticsearch-data
190+
spec:
191+
accessModes:
192+
- ReadWriteOnce
193+
resources:
194+
requests:
195+
storage: 20Ti
196+
# Adjust to your storage class name
197+
#
198+
# storageClassName: local-storage

‎deploy/eck-elasticsearch/templates/NOTES.txt

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
2+
1. Check Elasticsearch resource status
3+
$ kubectl get es {{ include "elasticsearch.fullname" . }} -n={{ .Release.Namespace }}
4+
5+
2. Check Elasticsearch pod status
6+
$ kubectl get pods --namespace={{ .Release.Namespace }} -l elasticsearch.k8s.elastic.co/cluster-name={{ include "elasticsearch.fullname" . }}

‎deploy/eck-elasticsearch/templates/_helpers.tpl

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
{{/*
2+
Expand the name of the chart.
3+
*/}}
4+
{{- define "elasticsearch.name" -}}
5+
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
6+
{{- end }}
7+
8+
{{/*
9+
Create a default fully qualified app name.
10+
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
11+
If release name contains chart name it will be used as a full name.
12+
*/}}
13+
{{- define "elasticsearch.fullname" -}}
14+
{{- if .Values.fullnameOverride }}
15+
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
16+
{{- else }}
17+
{{- $name := default .Chart.Name .Values.nameOverride }}
18+
{{- if contains $name .Release.Name }}
19+
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
20+
{{- else }}
21+
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
22+
{{- end }}
23+
{{- end }}
24+
{{- end }}
25+
26+
{{/*
27+
Create chart name and version as used by the chart label.
28+
*/}}
29+
{{- define "elasticsearch.chart" -}}
30+
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
31+
{{- end }}
32+
33+
{{/*
34+
Common labels
35+
*/}}
36+
{{- define "elasticsearch.labels" -}}
37+
helm.sh/chart: {{ include "elasticsearch.chart" . }}
38+
{{ include "elasticsearch.selectorLabels" . }}
39+
app.kubernetes.io/managed-by: {{ .Release.Service }}
40+
{{- if .Values.labels }}
41+
{{ .Values.labels }}
42+
{{- end }}
43+
{{- end }}
44+
45+
{{/*
46+
Selector labels
47+
*/}}
48+
{{- define "elasticsearch.selectorLabels" -}}
49+
app.kubernetes.io/name: {{ include "elasticsearch.name" . }}
50+
app.kubernetes.io/instance: {{ .Release.Name }}
51+
{{- end }}

0 commit comments

Comments
 (0)