目录
一 helm简介
- Helm是Kubernetes 应用的包管理工具,主要用来管理 Charts,类似Linux系统的yum。
- Helm Chart是用来封装Kubernetes原生应用程序的一系列YAML文件。可以在你部署应用的时候自定义应用程序的一些 Metadata,以便于应用程序的分发。
- 对于应用发布者而言
- 通过Helm打包应用、管理应用依赖关系、管理应用版本并发布应用到软件仓库。
- 对于使用者而言
- 使用Helm后可以以简单的方式在Kubernetes上查找、安装、升级、回滚、卸载应用程序
二 部署helm
2.1 官网与资源
官网: Helm | 快速入门指南
软件资源: https://github.com/helm/helm/releases
2.2部署helm
2.2.1 安装helm
[root@K8s-master ~]# mkdir helm
[root@K8s-master ~]# cd helm/
[root@k8s-master helm]# tar zxf helm-v3.15.4-linux-amd64.tar.gz
[root@k8s-master helm]# ls
helm-v3.15.4-linux-amd64.tar.gz linux-amd64
[root@k8s-master helm]# cd linux-amd64/
[root@k8s-master linux-amd64]# ls
helm LICENSE README.md
[root@k8s-master linux-amd64]# cp -p helm /usr/local/bin/
2.2.2 配置helm命令补齐
[root@k8s-master linux-amd64]# echo "source <(helm completion bash)" >> ~/.bashrc
[root@k8s-master linux-amd64]# source ~/.bashrc
[root@k8s-master linux-amd64]# helm version
version.BuildInfo{Version:"v3.15.4", GitCommit:"fa9efb07d9d8debbb4306d72af76a383895aa8c4", GitTreeState:"clean", GoVersion:"go1.22.6"}
三 helm常用操作
命令 | 描述 |
create | 创建一个 chart 并指定名字 |
dependency | 管理 chart 依赖 |
get | 下载一个 release。可用子命令:all、hooks、manifest、notes、values |
history | 获取 release 历史 |
install | 安装一个 chart |
list | 列出 release |
package | 将 chart 目录打包到 chart 存档文件中 |
pull | 从远程仓库中下载 chart 并解压到本地 # helm pull stable/mysql -- untar |
repo | 添加,列出,移除,更新和索引 chart 仓库。可用子命令:add、index、 list、remove、update |
rollback | 从之前版本回滚 |
search | 根据关键字搜索 chart。可用子命令:hub、repo |
show | 查看 chart 详细信息。可用子命令:all、chart、readme、values |
status | 显示已命名版本的状态 |
template | 本地呈现模板 |
uninstall | 卸载一个 release |
upgrade | 更新一个 release |
version | 查看 helm 客户端版本 |
2.3.1 查询官方应用中心
[root@k8s-master helm]# helm search hub nginx #在官方仓库中搜索
[root@k8s-master helm]# helm search repo nginx #在本地仓库中搜索
2.3.2 管理第三方repo源
- 阿里云仓库:https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
- bitnami仓库: https://charts.bitnami.com/bitnami
- 微软仓库:Index of /kubernetes/charts/
- 官方仓库: https://hub.kubeapps.com/charts/incubator
#添加阿里云仓库,
[root@k8s-master helm]# helm repo add aliyun https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
"aliyun" has been added to your repositories
#添加bitnami仓库
[root@k8s-master helm]# helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories
#查看仓库信息
[root@k8s-master helm]# helm repo list
NAME URL
aliyun https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
bitnami https://charts.bitnami.com/bitnami
#查看仓库存储helm清单
[root@k8s-master helm]# helm search repo aliyun
NAME CHART VERSION APP VERSION DESCRIPTION #应用名称 封装版本 软件版本 软件描述
aliyun/acs-engine-autoscaler 2.1.3 2.1.1 Scales worker nodes within agent pools
aliyun/aerospike 0.1.7 v3.14.1.2 A Helm chart for Aerospike in Kubernetes
#删除第三方存储库
[root@k8s-master helm]# helm repo list
NAME URL
aliyun https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
bitnami https://charts.bitnami.com/bitnami
[root@k8s-master helm]# helm repo remove aliyun
"aliyun" has been removed from your repositories
[root@k8s-master helm]# helm repo list
NAME URL
bitnami https://charts.bitnami.com/bitnami
2.3.3 helm的使用方法
1 查找chart
[root@k8s-master helm]# helm search repo nginx
NAME CHART VERSION APP VERSION DESCRIPTION
bitnami/nginx 18.1.11 1.27.1 NGINX Open Source is a web server that can be a...
bitnami/nginx-ingress-controller 11.4.1 1.11.2 NGINX Ingress Controller is an Ingress controll...
bitnami/nginx-intel 2.1.15 0.4.9 DEPRECATED NGINX Open Source for Intel is a lig...
2 查看chart信息
[root@k8s-master helm]# helm show chart bitnami/nginx
annotations:
category: Infrastructure
images: |
- name: git
image: docker.io/bitnami/git:2.46.0-debian-12-r0
- name: nginx
image: docker.io/bitnami/nginx:1.27.1-debian-12-r2
- name: nginx-exporter
image: docker.io/bitnami/nginx-exporter:1.3.0-debian-12-r2
licenses: Apache-2.0
3 安装chart 包
[root@k8s-master ~]# helm install timinglee bitnami/nginx
[root@k8s-master ~]# helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
timinglee default 1 2024-09-10 12:08:12.3165525 +0800 CST deployed nginx-18.1.11 1.27.1
[root@k8s-master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
timinglee-nginx-8598578bcf-qjwwq 1/1 Running 0 41s
#查看项目的发布状态
[root@k8s-master ~]# helm status timinglee
NAME: timinglee
LAST DEPLOYED: Tue Sep 10 12:08:12 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: nginx
CHART VERSION: 18.1.11
APP VERSION: 1.27.1
#卸载项目
[root@k8s-master nginx]# helm uninstall timinglee
release "timinglee" uninstalled
[root@k8s-master nginx]# kubectl get pods
No resources found in default namespace.
[root@k8s-master nginx]# helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
2.3.4 安装项目前预定义项目选项
#拉取项目
[root@k8s-master helm]# helm pull bitnami/nginx
[root@k8s-master helm]# ls
nginx-18.1.11.tgz
[root@k8s-master helm]# tar zxf nginx-18.1.11.tgz
[root@k8s-master helm]# ls
nginx nginx-18.1.11.tgz
[root@k8s-master helm]# cd nginx/
[root@k8s-master nginx]# ls
Chart.lock charts Chart.yaml README.md templates values.schema.json values.yaml
[root@k8s-master nginx]# ls templates/ #项目模板
deployment.yaml hpa.yaml NOTES.txt serviceaccount.yaml
extra-list.yaml ingress-tls-secret.yaml pdb.yaml servicemonitor.yaml
health-ingress.yaml ingress.yaml prometheusrules.yaml svc.yaml
_helpers.tpl networkpolicy.yaml server-block-configmap.yaml tls-secret.yaml
[root@k8s-master nginx]# vim values.yaml #项目变量文件
13 imageRegistry: "reg.harbor.org"
#上传项目所需要镜像到仓库
[root@K8s-master helm]# docker load -i nginx-1.27.1-debian-12-r2.tar
30f5b1069b7f: Loading layer 190.1MB/190.1MB
Loaded image: bitnami/nginx:1.27.1-debian-12-r2
[root@K8s-master helm]# docker tag bitnami/nginx:1.27.1-debian-12-r2 reg.harbor.org/bitnami/nginx:1.27.1-debian-12-r2
[root@K8s-master helm]# docker push reg.harbor.org/bitnami/nginx:1.27.1-debian-12-r2
The push refers to repository [reg.harbor.org/bitnami/nginx]
30f5b1069b7f: Pushed
1.27.1-debian-12-r2: digest: sha256:6825a4d52b84873dd08c26d38dccce3d78d4d9f470b7555afdc4edfb4de7e595 size: 529
#安装本地项目
[root@K8s-master nginx]# pwd
/root/helm/nginx
[root@K8s-master nginx]# helm install zhang /root/helm/nginx
NAME: zhang
LAST DEPLOYED: Wed Sep 11 11:47:33 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: nginx
CHART VERSION: 18.1.11
APP VERSION: 1.27.1
[root@K8s-master nginx]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 10h
zhang-nginx LoadBalancer 10.98.48.25 <pending> 80:31015/TCP,443:30546/TCP 12s
[root@K8s-master nginx]# kubectl get pods
NAME READY STATUS RESTARTS AGE
zhang-nginx-b869d78bf-ps78b 1/1 Running 0 57s
#更新项目
[root@k8s-master nginx]# vim values.yaml #更新变量文件
624 type: ClusterIP
751 enabled: true
763 hostname: myapp.timinglee.org
783 ingressClassName: "nginx"
[root@k8s-master nginx]# helm upgrade timinglee .
Release "timinglee" has been upgraded. Happy Helming!
NAME: timinglee
LAST DEPLOYED: Tue Sep 10 15:31:19 2024
NAMESPACE: default
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
CHART NAME: nginx
CHART VERSION: 18.1.11
APP VERSION: 1.27.1
[root@k8s-master nginx]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 6d21h
timinglee-nginx ClusterIP 10.107.92.71 <none> 80/TCP,443/TCP 7m34s
[root@k8s-master nginx]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
timinglee-nginx nginx myapp.timinglee.org 172.25.254.20 80 68s
[root@k8s-master nginx]# curl myapp.timinglee.org
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
#查看历史
[root@k8s-master nginx]# helm history timinglee
REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
1 Tue Sep 10 15:24:40 2024 superseded nginx-18.1.11 1.27.1 Install complete
2 Tue Sep 10 15:31:19 2024 deployed nginx-18.1.11 1.27.1
#删除项目
[root@k8s-master nginx]# helm uninstall timinglee
release "timinglee" uninstalled
[root@k8s-master nginx]# helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
四 构建helm中的chart包
4.1 Helm Chart目录结构
[root@K8s-master helm]# helm create zhang
Creating zhang
[root@K8s-master helm]# tree zhang
zhang/
├── charts #目录里存放这个chart依赖的所有子chart。
├── Chart.yaml #用于描述这个 Chart 的基本信息
#包括名字、描述信息以及版本等。
├── templates #目录里面存放所有 yaml 模板文件。
│ ├── deployment.yaml
│ ├── _helpers.tpl #放置模板助手的地方,可以在整个 chart 中重复使用
│ ├── hpa.yaml
│ ├── ingress.yaml
│ ├── NOTES.txt
│ ├── serviceaccount.yaml
│ ├── service.yaml
│ └── tests
│ └── test-connection.yaml
└── values.yaml #用于存储 templates 目录中模板文件中用到变量的值。
3 directories, 10 files
4.2 构建方法
[root@K8s-master helm]# cd zhang/
[root@K8s-master zhang]# vim Chart.yaml
apiVersion: v2
name: zhang
description: A Helm chart for Kubernetes
type: application
version: 0.1.0 #项目版本
appVersion: "v1" #软件版本
[root@K8s-master zhang]# vim values.yaml
image:
repository: myapp
pullPolicy: IfNotPresent
tag: "v1"
ingress:
enabled: true
className: "nginx"
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
hosts:
- host: myapp.zhang.org
paths:
- path: /
pathType: ImplementationSpecific
#语法检测
[root@K8s-master zhang]# helm lint .
==> Linting .
[INFO] Chart.yaml: icon is recommended
1 chart(s) linted, 0 chart(s) failed
#项目打包
[root@K8s-master helm]# helm package zhang/
Successfully packaged chart and saved it to: /root/helm/zhang-0.1.0.tgz
#项目可以通过各种分享方式发方为任何人后部署即可
[root@K8s-master helm]# helm install zhang zhang-0.1.0.tgz
NAME: zhang
LAST DEPLOYED: Wed Sep 11 23:28:57 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=zhang,app.kubernetes.io/instance=zhang" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT
[root@K8s-master helm]# kubectl get pods
NAME READY STATUS RESTARTS AGE
zhang-7fc7c869c5-2hkmw 1/1 Running 0 19s
[root@k8s-master timinglee]# vim Chart.yaml
apiVersion: v2
name: timinglee
description: A Helm chart for Kubernetes
type: application
version: 0.1.0 #项目版本
appVersion: "v1" #软件版本
[root@k8s-master timinglee]# vim values.yaml
image:
repository: myapp
pullPolicy: IfNotPresent
tag: "v1"
ingress:
enabled: true
className: "nginx"
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
hosts:
- host: myapp.zhang.org
paths:
- path: /
pathType: ImplementationSpecific
#语法检测
[root@k8s-master timinglee]# helm lint .
==> Linting .
[INFO] Chart.yaml: icon is recommended
1 chart(s) linted, 0 chart(s) failed
#项目打包
[root@k8s-master timinglee]# cd ..
[root@k8s-master helm]# helm package timinglee/
Successfully packaged chart and saved it to: /root/helm/timinglee-0.1.0.tgz
[root@k8s-master helm]# ls
timinglee timinglee-0.1.0.tgz
#项目可以通过各种分享方式发方为任何人后部署即可
[root@k8s-master helm]# helm install timinglee timinglee-0.1.0.tgz
NAME: timinglee
LAST DEPLOYED: Tue Sep 10 15:54:05 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
http://myapp.timinglee.org/
[root@k8s-master helm]# kubectl get pods
NAME READY STATUS RESTARTS AGE
timinglee-8889c6bf5-cn257 1/1 Running 0 17s
[root@k8s-master helm]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
timinglee nginx myapp.timinglee.org 80 22s
五 构建helm仓库
5.1 在harbor仓库中构建一个公开的项目
5.2 安装helm push插件
官方网址:https://github.com/chartmuseum/helm-push
5.2.1 在线安装
如果网络没问题情况下直接安装即可
[root@k8s-master helm]# dnf install git -y
[root@k8s-master helm]# helm plugin install https://github.com/chartmuseum/helm-push
5.2.2 离线安装
#创建helm plugin的存放目录
[root@k8s-master helm]# mkdir ~/.local/share/helm/plugins/helm-push -p
#解压push插件包到指定目录
[root@k8s-master helm]# tar zxf helm-push_0.10.4_linux_amd64.tar.gz -C ~/.local/share/helm/plugins/helm-push
[root@k8s-master helm]# ls ~/.local/share/helm/plugins/helm-push
bin LICENSE plugin.yaml
#查看helm调用命令是否成功
[root@k8s-master helm]# helm cm-push --help
Helm plugin to push chart package to ChartMuseum
Examples:
$ helm cm-push mychart-0.1.0.tgz chartmuseum # push .tgz from "helm package"
$ helm cm-push . chartmuseum # package and push chart directory
$ helm cm-push . --version="1.2.3" chartmuseum # override version in Chart.yaml
$ helm cm-push . https://my.chart.repo.com # push directly to chart repo URL
5.3 上传项目到仓库
5.3.1 添加仓库
[root@K8s-master helm]# helm repo add timinglee https://reg.harbor.org/chartrepo/zhang1
#添加仓库时报错,因为我们用的是加密访问
Error: looks like "https://reg.harbor.org/chartrepo/zhang1" is not a valid chart repository or cannot be reached: Get "https://reg.harbor.org/chartrepo/zhang/index.yaml": tls: failed to verify certificate: x509: certificate signed by unknown authority
#为helm添加证书
[root@K8s-master helm]# cp /etc/docker/certs.d/reg.harbor.org/ca.crt /etc/pki/ca-trust/source/anchors/
#更新本地ca认证库
[root@K8s-master helm]# update-ca-trust
#再次添加仓库
[root@K8s-master helm]# helm repo add zhang https://reg.harbor.org/chartrepo/zhang1
"zhang" has been added to your repositories
5.3.2 上传本地项目
#命令执行格式
helm cm-push <项目名称> <仓库名称> -u admin -p redhat
[root@K8s-master helm]# helm cm-push zhang-0.1.0.tgz zhang -u admin -p redhat
Pushing zhang-0.1.0.tgz to zhang...
Done.
#查看项目上传情况
[root@k8s-master helm]# helm search repo zhang #上传后数据未更新
No results found
#更新仓库
[root@K8s-master helm]# helm repo update zhang
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "zhang" chart repository
Update Complete. ⎈Happy Helming!⎈
#再次查看
[root@K8s-master helm]# helm search repo zhang
NAME CHART VERSION APP VERSION DESCRIPTION
zhang/zhang 0.1.0 v1 A Helm chart for Kubernetes
#安装项目
[root@K8s-master helm]# helm search repo zhang
NAME CHART VERSION APP VERSION DESCRIPTION
zhang/zhang 0.1.0 v1 A Helm chart for Kubernetes
[root@K8s-master helm]# helm install zhang zhang/zhang
NAME: zhang
LAST DEPLOYED: Wed Sep 11 14:50:27 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=zhang,app.kubernetes.io/instance=zhang" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT
[root@K8s-master helm]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 13h
zhang ClusterIP 10.104.105.249 <none> 80/TCP 42s
[root@K8s-master helm]# curl 10.104.105.249
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
六 helm的版本迭代
6.1 从新构建新版本项目
[root@k8s-master helm]# vim zhang/Chart.yaml
version: 0.2.0
appVersion: "v2"
[root@k8s-master helm]# vim timinglee/values.yaml
tag: "v2"
[root@K8s-master helm]# helm package zhang
Successfully packaged chart and saved it to: /root/helm/zhang-0.2.0.tgz
6.2 上传项目到helm仓库中
[root@k8s-master helm]# helm cm-push timinglee-0.2.0.tgz timinglee -u admin -p lee
Pushing timinglee-0.2.0.tgz to timinglee...
Done.
[root@k8s-master helm]# helm cm-push timinglee-0.2.0.tgz timinglee -u admin -p lee
Pushing timinglee-0.2.0.tgz to timinglee...
Done.
[root@k8s-master helm]# helm repo update timinglee
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "timinglee" chart repository
Update Complete. ⎈Happy Helming!⎈
[root@k8s-master helm]# helm search repo
[root@k8s-master helm]# helm search repo timinglee -l
NAME CHART VERSION APP VERSION DESCRIPTION
timinglee/timinglee 0.2.0 v2 A Helm chart for Kubernetes
timinglee/timinglee 0.1.0 v1 A Helm chart for Kubernetes
6.3 更新应用
#1.更新
[root@K8s-master helm]# helm upgrade zhang zhang/zhang
Release "zhang" has been upgraded. Happy Helming!
NAME: zhang
LAST DEPLOYED: Wed Sep 11 14:57:51 2024
NAMESPACE: default
STATUS: deployed
REVISION: 2
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=zhang,app.kubernetes.io/instance=zhang" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT
[root@k8s-master helm]# curl http://myapp.timinglee.org/
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
#显示项目版本
[root@K8s-master helm]# helm history zhang
REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
1 Wed Sep 11 14:50:27 2024 superseded zhang-0.1.0 v1 Install complete
2 Wed Sep 11 14:57:51 2024 deployed zhang-0.2.0 v2 Upgrade complete
#2.应用回滚
[root@K8s-master helm]# helm rollback zhang
Rollback was a success! Happy Helming!
[root@K8s-master helm]# helm history zhang
REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
1 Wed Sep 11 14:50:27 2024 superseded zhang-0.1.0 v1 Install complete
2 Wed Sep 11 14:57:51 2024 superseded zhang-0.2.0 v2 Upgrade complete
3 Wed Sep 11 15:00:20 2024 deployed zhang-0.1.0 v1 Rollback to 1
[root@K8s-master helm]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
zhang-7fc7c869c5-7c7v7 1/1 Running 0 21s 10.224.36.81 k8s-node1 <none> <none>
[root@K8s-master helm]# curl 10.224.36.81
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>