文章目录
🚀 本文内容：在 Helm 中部署 Redis。
Redis Chart
Helm 提供了两个 Redis Chart：
- 单机版、哨兵版：https://artifacthub.io/packages/helm/bitnami/redis
- Cluster 版：https://artifacthub.io/packages/helm/bitnami/redis-cluster
部署 Redis 单机版
Chart：redis 17.9.4 · bitnami/bitnami (artifacthub.io)
单机版非常简单，只跑一个 master 节点，指定密码，再指定 nodePort 暴露到集群外即可。
对应的 values.yaml 如下：
master:
service:
type: "NodePort"
nodePorts:
redis: "30567"
auth:
# redis密码
password: "Redis@pass123"
部署 Redis 哨兵版
https://artifacthub.io/packages/helm/bitnami/redis/17.13.2
当前环境：Helm 3.3.x、K8s 1.18
- Chart 版本：17.13.2（Redis 为 7.10.x 版本）
- 目标架构：1 主 2 从 3 哨兵
第 1 步：准备 values.yaml 配置文件
# 架构
architecture: replication
# 密码
auth:
enabled: true
sentinel: true
password: Redis@pass123
# 主节点 1 个
master:
count: 1
# 从节点 2 个
replica:
# 这个应该是有误，实际上加上 master 才 3 个
replicaCount: 3
# 哨兵
sentinel:
enabled: true
service:
type: "NodePort"
nodePorts:
# 暴露 Redis
redis: 32700
# 暴露 Sentinel
sentinel: 32710
第 2 步：安装 bitnami/redis
helm install --values values.yaml my-redis bitnami/redis --version=17.13.2
# NAME: my-redis
# LAST DEPLOYED: Wed May 22 18:53:52 2024
# NAMESPACE: default
# STATUS: deployed
# REVISION: 1
# TEST SUITE: None
# NOTES:
# CHART NAME: redis
# CHART VERSION: 17.13.2
# APP VERSION: 7.0.12
#
# ** Please be patient while the chart is being deployed **
#
# Redis® can be accessed via port 6379 on the following DNS name from within your cluster:
#
# my-redis.default.svc.cluster.local for read only operations
#
# For read/write operations, first access the Redis® Sentinel cluster, which is available in port 26379 using the same domain name above.
#
#
#
# To get your password run:
#
# export REDIS_PASSWORD=$(kubectl get secret --namespace default my-redis -o jsonpath="{.data.redis-password}" | base64 -d)
#
# To connect to your Redis® server:
#
# 1. Run a Redis® pod that you can use as a client:
#
# kubectl run --namespace default redis-client --restart='Never' --env REDIS_PASSWORD=$REDIS_PASSWORD --image docker.io/bitnami/redis:7.0.12-debian-11-r2 --command -- sleep infinity
#
# Use the following command to attach to the pod:
#
# kubectl exec --tty -i redis-client \
# --namespace default -- bash
#
# 2. Connect using the Redis® CLI:
# REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h my-redis -p 6379 # Read only operations
# REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h my-redis -p 26379 # Sentinel access
#
# To connect to your database from outside the cluster execute the following commands:
#
# export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
# export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services my-redis)
# REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h $NODE_IP -p $NODE_PORT
# No need to upgrade, ports and nodeports have been set from values
😂 注意：如果最后一行提示如下信息，则需要进行升级。
#!#!#!#!#!#!#!# IMPORTANT #!#!#!#!#!#!#!#
YOU NEED TO PERFORM AN UPGRADE FOR THE SERVICES AND WORKLOAD TO BE CREATED
则需要执行 helm upgrade 升级：
helm upgrade --values values.yaml my-redis bitnami/redis --version=17.13.2
第 3 步：解决 Pod Pending 问题
Helm 安装后，查看 pod 状态，发现都是 Pending 状态，本质上是因为 PVC 没有绑定到 PV。
查看 Pod 状态
查看 Pod 状态：提示为 Pending
kubectl get pods
# NAME READY STATUS RESTARTS AGE
# my-redis-node-0 0/2 Pending 0 68s
查看 Pod 详细信息：
kubectl describe pod my-redis-node-0
# Name: my-redis-node-0
# Namespace: default
# Priority: 0
# Node: <none>
# Labels: app.kubernetes.io/component=node
# app.kubernetes.io/instance=my-redis
# app.kubernetes.io/managed-by=Helm
# app.kubernetes.io/name=redis
# controller-revision-hash=my-redis-node-7945f545f7
# helm.sh/chart=redis-17.13.2
# statefulset.kubernetes.io/pod-name=my-redis-node-0
# Annotations: checksum/configmap: b9d43febb642396f5ab9c8ab09bb5b4157fc3a46937ac846136d2858f0e4b023
# checksum/health: 134265b0a0a8aa6b15f7cda3672c0cf8f5c17c817a3061d22ccbfcc74a11454d
# checksum/scripts: 898e9e1f07fdbef56a8d034b9d85b1c169d7a4cced31918a92d5a6d606139bc2
# checksum/secret: fb9156fa8445c3bf10d1c751ac7641ab5d9a7e61929cd25238f6b69d22b44c79
# Status: Pending
# IP:
# IPs: <none>
# Controlled By: StatefulSet/my-redis-node
# Containers:
# redis:
# Image: docker.io/bitnami/redis:7.0.12-debian-11-r2
# Port: 6379/TCP
# Host Port: 0/TCP
# Command:
# /bin/bash
# Args:
# -c
# /opt/bitnami/scripts/start-scripts/start-node.sh
# Liveness: exec [sh -c /health/ping_liveness_local.sh 5] delay=20s timeout=5s period=5s #success=1 #failure=5
# Readiness: exec [sh -c /health/ping_readiness_local.sh 1] delay=20s timeout=1s period=5s #success=1 #failure=5
# Startup: exec [sh -c /health/ping_liveness_local.sh 5] delay=10s timeout=5s period=10s #success=1 #failure=22
# Environment:
# BITNAMI_DEBUG: false
# REDIS_MASTER_PORT_NUMBER: 6379
# ALLOW_EMPTY_PASSWORD: no
# REDIS_PASSWORD: <set to the key 'redis-password' in secret 'my-redis'> Optional: false
# REDIS_MASTER_PASSWORD: <set to the key 'redis-password' in secret 'my-redis'> Optional: false
# REDIS_TLS_ENABLED: no
# REDIS_PORT: 6379
# REDIS_SENTINEL_TLS_ENABLED: no
# REDIS_SENTINEL_PORT: 26379
# REDIS_DATA_DIR: /data
# Mounts:
# /data from redis-data (rw)
# /health from health (rw)
# /opt/bitnami/redis/etc from redis-tmp-conf (rw)
# /opt/bitnami/redis/mounted-etc from config (rw)
# /opt/bitnami/scripts/start-scripts from start-scripts (rw)
# /tmp from tmp (rw)
# /var/run/secrets/kubernetes.io/serviceaccount from my-redis-token-6nqr8 (ro)
# sentinel:
# Image: docker.io/bitnami/redis-sentinel:7.0.12-debian-11-r1
# Port: 26379/TCP
# Host Port: 0/TCP
# Command:
# /bin/bash
# Args:
# -c
# /opt/bitnami/scripts/start-scripts/start-sentinel.sh
# Liveness: exec [sh -c /health/ping_sentinel.sh 5] delay=20s timeout=5s period=10s #success=1 #failure=6
# Readiness: exec [sh -c /health/ping_sentinel.sh 1] delay=20s timeout=1s period=5s #success=1 #failure=6
# Startup: exec [sh -c /health/ping_sentinel.sh 5] delay=10s timeout=5s period=10s #success=1 #failure=22
# Environment:
# BITNAMI_DEBUG: false
# REDIS_PASSWORD: <set to the key 'redis-password' in secret 'my-redis'> Optional: false
# REDIS_SENTINEL_TLS_ENABLED: no
# REDIS_SENTINEL_PORT: 26379
# Mounts:
# /data from redis-data (rw)
# /health from health (rw)
# /opt/bitnami/redis-sentinel/etc from sentinel-data (rw)
# /opt/bitnami/redis-sentinel/mounted-etc from config (rw)
# /opt/bitnami/scripts/start-scripts from start-scripts (rw)
# /var/run/secrets/kubernetes.io/serviceaccount from my-redis-token-6nqr8 (ro)
# Conditions:
# Type Status
# PodScheduled False
# Volumes:
# redis-data:
# Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
# ClaimName: redis-data-my-redis-node-0
# ReadOnly: false
# start-scripts:
# Type: ConfigMap (a volume populated by a ConfigMap)
# Name: my-redis-scripts
# Optional: false
# health:
# Type: ConfigMap (a volume populated by a ConfigMap)
# Name: my-redis-health
# Optional: false
# config:
# Type: ConfigMap (a volume populated by a ConfigMap)
# Name: my-redis-configuration
# Optional: false
# sentinel-data:
# Type: EmptyDir (a temporary directory that shares a pod's lifetime)
# Medium:
# SizeLimit: <unset>
# redis-tmp-conf:
# Type: EmptyDir (a temporary directory that shares a pod's lifetime)
# Medium:
# SizeLimit: <unset>
# tmp:
# Type: EmptyDir (a temporary directory that shares a pod's lifetime)
# Medium:
# SizeLimit: <unset>
# my-redis-token-6nqr8:
# Type: Secret (a volume populated by a Secret)
# SecretName: my-redis-token-6nqr8
# Optional: false
# QoS Class: BestEffort
# Node-Selectors: <none>
# Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
# node.kubernetes.io/unreachable:NoExecute for 300s
# Events:
# Type Reason Age From Message
# ---- ------ ---- ---- -------
# Warning FailedScheduling 89s default-scheduler running "VolumeBinding" filter plugin for pod "my-redis-node-0": pod has unbound immediate PersistentVolumeClaims
# Warning FailedScheduling 89s default-scheduler running "VolumeBinding" filter plugin for pod "my-redis-node-0": pod has unbound immediate PersistentVolumeClaims
最后面，提示“running “VolumeBinding” filter plugin for pod “my-redis-node-0”: pod has unbound immediate PersistentVolumeClaims”，所以没法被调度。
查看 PVC：发现也是 Pending 状态
kubectl get pvc
# NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
# redis-data-my-redis-node-0 Pending 4m1s
查看 PVC 详细信息：
kubectl describe pvc redis-data-my-redis-node-0
# Name: redis-data-my-redis-node-0
# Namespace: default
# StorageClass:
# Status: Pending
# Volume:
# Labels: app.kubernetes.io/component=node
# app.kubernetes.io/instance=my-redis
# app.kubernetes.io/name=redis
# Annotations: <none>
# Finalizers: [kubernetes.io/pvc-protection]
# Capacity:
# Access Modes:
# VolumeMode: Filesystem
# Mounted By: my-redis-node-0
# Events:
# Type Reason Age From Message
# ---- ------ ---- ---- -------
# Normal FailedBinding 13s (x19 over 4m31s) persistentvolume-controller no persistent volumes available for this claim and no storage class is set
😂 绑定失败原因：针对这个 Claim 没有可以的 PV，而且也没有设置 StorageClass，也就没法使用 StorageClass 去匹配 PV。
😊 在 values.yaml 中可以使用全局指定 storageClass，也可以针对 master、replica、sentinel 单独指定 storageClass 哦。
添加 PV 持久化卷
这里会用到三个 PV，创建步骤如下。
创建 pv1
mkdir -p /csp/local-pv/redis1
chmod 777 -R /csp/local-pv/redis1
kubectl apply -f pv1.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: local-pv-redis1
spec:
capacity:
storage: 8Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
hostPath:
path: "/csp/local-pv/redis1"
创建 pv2
mkdir -p /csp/local-pv/redis2
chmod 777 -R /csp/local-pv/redis2
kubectl apply -f pv2.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: local-pv-redis2
spec:
capacity:
storage: 8Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
hostPath:
path: "/csp/local-pv/redis2"
创建 pv3
mkdir -p /csp/local-pv/redis3
chmod 777 -R /csp/local-pv/redis3
kubectl apply -f pv3.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: local-pv-redis3
spec:
capacity:
storage: 8Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
hostPath:
path: "/csp/local-pv/redis3"
查看 Pod 状态
再次查看 Pod 状态：发现正常 Running 了，等待几分钟后，所有 Pod 创建完成
kubectl get pods -w
# NAME READY STATUS RESTARTS AGE
# my-redis-node-0 0/2 Running 0 20m
# my-redis-node-0 1/2 Running 0 22m
# my-redis-node-0 2/2 Running 0 22m
# my-redis-node-1 0/2 Pending 0 0s
# my-redis-node-1 0/2 ContainerCreating 0 0s
# my-redis-node-1 0/2 Running 0 7s
# my-redis-node-1 1/2 Running 0 28s
# my-redis-node-1 2/2 Running 0 29s
# my-redis-node-2 0/2 Pending 0 0s
# my-redis-node-2 0/2 ContainerCreating 0 2s
# my-redis-node-2 0/2 Running 0 10s
# my-redis-node-2 1/2 Running 0 29s
# my-redis-node-2 2/2 Running 0 31s
第 4 步：连接到 Redis
查看端口
Redis 端口：
- 32700：访问下面三个之一【应该是轮询/随机】
- 32701：master，my-redis-node-0.default
- 32702：slave，my-redis-node-1.default
- 32703：slave，my-redis-node-2.default
Sentinel 端口：
- 32710：访问下面三个之一【应该是轮询/随机】
- 32711：run_id:24eee4e4b30f448bd267fca21ef41956b1ad00a8
- 32712：run_id:cf059f9de8325872c8c6bf71ea7ee7a8ecfb88c9
- 32713：run_id:6aca574c82d5402b12e1b9b6692328f4b719ec26
netstat -tunlp | grep kube-proxy | grep 327
# tcp 0 0 0.0.0.0:32700 0.0.0.0:* LISTEN 6243/kube-proxy
# tcp 0 0 0.0.0.0:32701 0.0.0.0:* LISTEN 6243/kube-proxy
# tcp 0 0 0.0.0.0:32702 0.0.0.0:* LISTEN 6243/kube-proxy
# tcp 0 0 0.0.0.0:32703 0.0.0.0:* LISTEN 6243/kube-proxy
# tcp 0 0 0.0.0.0:32710 0.0.0.0:* LISTEN 6243/kube-proxy
# tcp 0 0 0.0.0.0:32711 0.0.0.0:* LISTEN 6243/kube-proxy
# tcp 0 0 0.0.0.0:32712 0.0.0.0:* LISTEN 6243/kube-proxy
# tcp 0 0 0.0.0.0:32713 0.0.0.0:* LISTEN 6243/kube-proxy
使用 redis-cli 连接
# 1 进入容器内
kubectl exec -it my-redis-node-0 bash
# 2 使用 redis-cli 连接 sentinel
redis-cli -h 172.24.4.246 -p 32710
# 2.1 密码认证
172.24.4.246:32710> auth Redis@pass123
# OK
# 2.2、查看信息
172.24.4.246:32710> info
# Server
# redis_version:7.0.12
# ...
# redis_mode:sentinel
# ...
# process_id:1
# run_id:cf059f9de8325872c8c6bf71ea7ee7a8ecfb88c9
# tcp_port:26379
# ...
# executable:/redis-server
# config_file:/opt/bitnami/redis-sentinel/etc/sentinel.conf
# ...
# # Sentinel
# sentinel_masters:1
# ...
# master0:name=mymaster,status=ok,address=my-redis-node-0.default:32701,slaves=2,sentinels=3
# 2.3 查看 master 信息
172.24.4.246:32710> sentinel get-master-addr-by-name mymaster
# 1) "my-redis-node-0.default"
# 2) "32701"
# 2.4 查看 master 的 replia 节点
172.24.4.246:32710> sentinel replicas mymaster
# my-redis-node-1.default:32702
# my-redis-node-2.default:32703
使用 Redis Desktop 连接
配置 /etc/hosts
172.24.4.246 为集群 IP 地址，需要将上述 k8s 中的主机名称，映射为 IP 地址访问。
在 /etc/hosts 添加如下配置：
172.24.4.246 my-redis-node-0.default my-redis-node-1.default my-redis-node-2.default
配置 Redis 连接信息
使用 32710 连接到任意一个 Sentinel，然后 Sentinel 会返回 master 地址（my-redis-node-0.default:32701），本地 hosts 解析此主机名后，即可访问成功！
存在的疑问及尝试
Redis Sentinel 返回的 master 地址为 my-redis-node-0.default:32701。
能不能让 Redis Sentinel 直接返回 172.24.4.246:32701 这样的 master 地址呢？
尝试 1：修改 replica、sentinel 配置文件，sentinel 配置中直接 announce-ip 为 172.24.4.246
# 从节点
replica:
replicaCount: 3
configuration: |-
replica-announce-ip 172.24.4.246
# 哨兵
sentinel:
enabled: true
service:
type: "NodePort"
nodePorts:
sentinel: 32710
redis: 32700
configuration: |-
SENTINEL resolve-hostnames no
SENTINEL announce-hostnames no
SENTINEL announce-ip 172.24.4.246
配置添加后，redis-cli 登录 sentinel，执行 info 命令，发现配置文件是 config_file:/opt/bitnami/redis-sentinel/etc/sentinel.conf。
但是，这个配置文件在 Pod 容器中找不到哇。所以依然是默认配置。
能找的 sentinel.conf 是 /opt/bitnami/redis/mounted-etc/sentinel.conf。里面的配置还真就是我们上面配置的，可惜了，不会生效。
尝试 2：使用 useHostnames 为 false
# Use hostnames internally when announcing replication
useHostnames: false
修改后，发现只是不用主机名了，用的 IP 是内部 Pod IP。
127.0.0.1:26379> sentinel get-master-addr-by-name mymaster
1) "192.168.103.247"
2) "32701"
完。
相关博文
1.第 1 篇 Helm 简介及安装
2.第 2 篇 Helm 部署 MySQL【入门案例】
3.第 3 篇 Helm 命令、环境变量、相关目录
4.第 4 篇 Chart 仓库详解
5.第 5 篇 Chart 文件结构详解
6.第 6 篇 自定义 Helm Chart
7.第 7 篇 Helm 部署 Nacos【详细步骤】
8.第 8 篇 Chart 修改入门示例：Nacos
9.第 9 篇 Helm 部署 Seata Server
10.第 10 篇 Chart 修改完美示例：Seata Server
11.第 11篇 Helm 部署 RabbitMQ
12.第 12 篇 Helm 部署 Redis
13.第13 篇 Helm 部署 ElasticSearch