参考：在 K8S 中快速部署 Redis Cluster & Redisinsight
参考：Redis 中文文档（一）
helm redis-cluster
[root@k8s-master01 redis-cluster]# helm search repo redis-cluster
NAME CHART VERSION APP VERSION DESCRIPTION
bitnami/redis-cluster 9.1.3 7.2.3 Redis(R) is an open source, scalable, distribut...
[root@k8s-master01 redis-cluster]# helm pull bitnami/redis-cluster解压，查看使用到的镜像文件，提前下载并上传私有仓库
[root@k8s-master01 redis-cluster]# cat Chart.yaml
annotations:
category: Database
images: |
- name: os-shell
image: docker.io/bitnami/os-shell:11-debian-11-r91
- name: redis-cluster
image: docker.io/bitnami/redis-cluster:7.2.3-debian-11-r1
- name: redis-exporter
image: docker.io/bitnami/redis-exporter:1.55.0-debian-11-r2配置私有仓库
docker tag docker.io/bitnami/redis-exporter:1.55.0-debian-11-r2 harbor.david.org/bitnami/redis-exporter:1.55.0-debian-11-r2
docker tag docker.io/bitnami/redis-cluster:7.2.3-debian-11-r1 harbor.david.org/bitnami/redis-cluster:7.2.3-debian-11-r1
docker tag docker.io/bitnami/os-shell:11-debian-11-r91 harbor.david.org/bitnami/os-shell:11-debian-11-r91
docker tag docker.io/redislabs/redisinsight:latest harbor.david.org/redislabs/redisinsight:latesthelm安装redis-cluster
使用之前的storyageClass
StorageClass
helm install redis-cluster bitnami/redis-cluster -n develop \
--values values.yaml \
--set persistence.storageClass=nfs-sc \
--set image.registry=harbor.david.org \
--set volumePermissions.image.registry=harbor.david.org \
--set metrics.image.registry=harbor.david.org \
--set sysctlImage.image.registry=harbor.david.org安装完成
NAME: redis-cluster
LAST DEPLOYED: Fri Nov 24 14:05:41 2023
NAMESPACE: develop
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: redis-cluster
CHART VERSION: 9.1.3
APP VERSION: 7.2.3** Please be patient while the chart is being deployed **
To get your password run:
export REDIS_PASSWORD=$(kubectl get secret --namespace "develop" redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d)
You have deployed a Redis® Cluster accessible only from within you Kubernetes Cluster.INFO: The Job to create the cluster will be created.To connect to your Redis® cluster:
1. Run a Redis® pod that you can use as a client:
kubectl run --namespace develop redis-cluster-client --rm --tty -i --restart='Never' \
--env REDIS_PASSWORD=$REDIS_PASSWORD \
--image harbor.david.org/bitnami/redis-cluster:7.2.3-debian-11-r1 -- bash
2. Connect using the Redis® CLI:
redis-cli -c -h redis-cluster -a $REDIS_PASSWORD
查看密码
[root@k8s-master01 redis-cluster]# kubectl get secret --namespace "develop" redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d
导出所有节点的地址为ClusterIp（如果需要）
参考： VMware Tanzu Application Catalog (Applications Tutorials) Documentation
helm upgrade redis-cluster --set "password=xxxxxxxxxx,cluster.externalAccess.enabled=true bitnami/redis-cluster -n develop[root@k8s-master01 redis]# helm upgrade redis-cluster bitnami/redis-cluster --set cluster.externalAccess.enabled=true --set password=An1y6Hh0Cv -n develop
Release "redis-cluster" has been upgraded. Happy Helming!
NAME: redis-cluster
LAST DEPLOYED: Sun Nov 26 18:10:15 2023
NAMESPACE: develop
STATUS: deployed
REVISION: 3
TEST SUITE: None
NOTES:
CHART NAME: redis-cluster
CHART VERSION: 9.1.3
APP VERSION: 7.2.3** Please be patient while the chart is being deployed **
To get your password run:
export REDIS_PASSWORD=$(kubectl get secret --namespace "develop" redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d)
To connect to your Redis® server from outside the cluster check the following information:
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
Watch the status with: 'kubectl get svc --namespace develop -w redis-cluster'
You will have a different external IP for each Redis® node. Get the external ip from `-external` suffixed services: `kubectl get svc`.
Redis® port: 6379
Once the LoadBalancerIPs are ready, you need to provide them and perform a Helm Upgrade:
helm upgrade --namespace develop redis-cluster --set "cluster.externalAccess.enabled=true,cluster.externalAccess.service.type=LoadBalancer,cluster.externalAccess.service.loadBalancerIP[0]=load-balancerip-0,cluster.externalAccess.service.loadBalancerIP[1]=load-balancerip-1,cluster.externalAccess.service.loadBalancerIP[2]=load-balancerip-2,cluster.externalAccess.service.loadBalancerIP[3]=load-balancerip-3,cluster.externalAccess.service.loadBalancerIP[4]=load-balancerip-4,cluster.externalAccess.service.loadBalancerIP[5]=load-balancerip-5" oci://registry-1.docker.io/bitnamicharts/redis-cluster
Where loadbalancer-ip-i are the LoadBalancerIPs provided by the cluster.helm upgrade ingress-nginx导出redis 6379端口
参考：K8s and Redis; a tale of Layer 4 Ingress
helm upgrade ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx \
--values values.yaml \
--set tcp.26379="develop/redis-cluster:6379" --debug修改端口为你想要的端口
[root@k8s-master01 ~]# kubectl edit svc -n ingress-nginx ingress-nginx-controller
service/ingress-nginx-controller edited
[root@k8s-master01 ~]# kubectl get svc -n ingress-nginx也可以使用port-forward进行临时转发端口，不过得一直挂着
参考： 使用端口转发来访问集群中的应用
参考： kubectl port-forward 踩坑记录
[root@k8s-master01 redis]# kubectl port-forward service/redis-cluster -n develop 30009:6379 --address 0.0.0.0
Forwarding from 0.0.0.0:30009 -> 6379
Handling connection for 30009
使用ingress-nginx进行转发（集群方式下不推荐）
因为集群会涉及到不同的key在不同的slot上，
Redis会进行重定向，将内网IP返回给客户端
但客户端并不能访问到内网地址
参考： 浅谈Redis Cluster 集群的MOVED、ASK重定向
ingress-nginx导出TCP端口
配置redis-cluster-proxy代理（外网客户端及程序可连接）
参考： 【Redis】——引入redis-cluster-proxy使得Redis Cluster对Kubernetes外部可提供服务
参考： Redis系列(十五)、Redis6新特性之集群代理(Cluster Proxy)
linux上安装redis-cluster-proxy
依赖gcc需要提前安装
gcc version 9.3.1
参考： CentOS/Ubuntu安装最新的gcc-9
参考：
【已解决】Error: Failed to download metadata for repo ‘appstream‘: Cannot prepare internal mirrorlist
或者参考下面的dockerfile
make redis-cluster-proxy
# Make Install
git clone https://github.com/artix75/redis-cluster-proxy
cd redis-cluster-proxy
make PREFIX=/usr/local/redis_cluster_proxy install
dockerfile
在目录/usr/local/redis_cluster_proxy/bin/ 下复制redis-cluster-proxy到当前目录，然后创建dockerfile 内容如下
redis_cluster_proxy使用C编写，需要gcc version 9.3.1环境
https://blog.csdn.net/weixin_43252521/article/details/124409151
[root@master01 redis-cluster-proxy]# vi dockerfile
FROM centos:8
WORKDIR /data
RUN sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
RUN sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
RUN yum makecache
#RUN yum update -y
RUN rpm -ivh https://cbs.centos.org/kojifiles/packages/centos-release-scl-rh/2/3.el7.centos/noarch/centos-release-scl-rh-2-3.el7.centos.noarch.rpm
RUN rpm -ivh https://cbs.centos.org/kojifiles/packages/centos-release-scl/2/3.el7.centos/noarch/centos-release-scl-2-3.el7.centos.noarch.rpm
RUN yum install -y centos-release-scl
RUN yum install -y devtoolset-9-gcc devtoolset-9-gcc-c++
RUN echo "scl enable devtoolset-9 bash" >> /etc/profile
RUN echo "source /opt/rh/devtoolset-9/enable" >> /etc/profile
RUN source /etc/profile
RUN yum install -y wget
RUN wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-8.repo
RUN sed -i 's/$releasever/8/g' /etc/yum.repos.d/CentOS-Base.repo
RUN yum repolist
RUN yum install -y gcc
RUN gcc -v
ADD redis-cluster-proxy /usr/local/bin/
EXPOSE 7777
docker build . -t redis-cluster-proxy:v1.0.0 [root@k8s-master01 ~]# docker tag redis-cluster-proxy:v1.0.0 harbor.david.org/redislabs/redis-cluster-proxy:v1.0.0
[root@k8s-master01 ~]# docker push harbor.david.org/redislabs/redis-cluster-proxy:v1.0.0配置redis-cluster-proxy configmap
[root@master01 redis-cluster-proxy]# vi configMap.yaml
---
# Redis-Proxy Config
apiVersion: v1
kind: ConfigMap
metadata:
name: redis-cluster-proxy
namespace: develop
data:
proxy.conf: |
cluster redis-cluster:6379 # 配置为Redis Cluster Service
bind 0.0.0.0
port 7777 # redis-cluster-proxy 对外暴露端口
threads 8 # 线程数量
daemonize no
enable-cross-slot yes
auth An1y6Hh0Cv # 配置Redis Cluster 认证密码
log-level error
[root@master01 redis-cluster-proxy]#
kubectl apply -f configMap.yaml -n develop配置deployment.yaml
[root@master01 redis-cluster-proxy]# vi deployment.yaml
---
# Redis-Proxy NodePort
apiVersion: v1
kind: Service
metadata:
name: redis-cluster-proxy
namespace: develop
spec:
type: NodePort # 对K8S外部提供服务
ports:
- name: redis-cluster-proxy
nodePort: 30009 # 对外提供的端口
port: 7777
protocol: TCP
targetPort: 7777
selector:
app: redis-cluster-proxy
---
# Redis-Proxy Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis-cluster-proxy
namespace: develop
spec:
replicas: 1
selector:
matchLabels:
app: redis-cluster-proxy
template:
metadata:
labels:
app: redis-cluster-proxy
spec:
#imagePullSecrets:
# - name: harbor
containers:
- name: redis-cluster-proxy
image: harbor.david.org/redislabs/redis-cluster-proxy:v1.0.0
imagePullPolicy: Always
command: ["redis-cluster-proxy"]
args:
- -c
- /data/proxy.conf # 指定启动配置文件
ports:
- name: redis-7777
containerPort: 7777
protocol: TCP
volumeMounts:
- name: redis-proxy-conf
mountPath: /data/
volumes: # 挂载proxy配置文件
- name: redis-proxy-conf
configMap:
name: redis-cluster-proxy[root@master01 redis-cluster-proxy]#
kubectl apply -f deployment.yaml -n develop
[root@master01 redis-cluster-proxy]# kubectl get pods -n develop
NAME READY STATUS RESTARTS AGE
redis-cluster-proxy-6679cb8f78-nbgr4 1/1 Running 0 167m
[root@master01 redis-cluster-proxy]# kubectl get svc -n develop |grep redis
redis-cluster ClusterIP 10.102.144.124 <none> 6379/TCP 5d21h
redis-cluster-headless ClusterIP None <none> 6379/TCP,16379/TCP 5d21h
redis-cluster-proxy NodePort 10.103.13.20 <none> 7777:30009/TCP 167m安装redisinsight
参考：redisinsight
打开网页右上角的 install
helm repo add redisinsight https://spy86.github.io/redisinsight
[root@k8s-master01 ~]# helm search hub redisinsight -o json
[{"url":"https://artifacthub.io/packages/helm/redisinsight/redisinsight","version":"0.1.0","app_version":"0.1.0","description":"A Helm chart for Redis Insight.","repository":{"url":"https://spy86.github.io/redisinsight","name":"redisinsight"}},{"url":"https://artifacthub.io/packages/helm/truecharts/redisinsight","version":"3.0.13","app_version":"1.14.0","description":"GUI for Redis.","repository":{"url":"https://charts.truecharts.org/","name":"truecharts"}},{"url":"https://artifacthub.io/packages/helm/riftbit/redisinsight","version":"0.1.0","app_version":"v1.0.0","description":"RedisInsight - The GUI for Redis","repository":{"url":"https://charts.riftbit.com/","name":"riftbit"}},{"url":"https://artifacthub.io/packages/helm/heywood8-helm-charts/redisinsight","version":"0.4.5","app_version":"2.28.0","description":"A Redisinsight Helm chart for kubernetes","repository":{"url":"https://heywood8.github.io/helm-charts/","name":"heywood8-helm-charts"}},{"url":"https://artifacthub.io/packages/helm/eximiait/openshift-secured-redisinsight","version":"0.9.0","app_version":"","description":"A secured version of RedisInsight for OpenShift","repository":{"url":"https://charts.eximiait.com.ar","name":"eximiait"}}]
[root@k8s-master01 ~]#
[root@k8s-master01 ~]# helm search repo redisinsight
NAME CHART VERSION APP VERSION DESCRIPTION
redisinsight/redisinsight 0.1.0 0.1.0 A Helm chart for Redis Insight.
[root@k8s-master01 ~]#[root@k8s-master01 ~]# helm pull redisinsight/redisinsight
解压，查看values.yaml
helm install redisinsight redisinsight/redisinsight -n develop \
--values values.yaml \
--set namespace.name=develop \
--set image.repository=harbor.david.org/redislabs/redisinsight
NAME: redisinsight
LAST DEPLOYED: Fri Nov 24 14:36:26 2023
NAMESPACE: develop
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace develop -l "app.kubernetes.io/name=redisinsight,app.kubernetes.io/instance=redisinsight" -o jsonpath="{.items[0].metadata.name}")
echo "Visit http://127.0.0.1:8001 to use your application"
kubectl --namespace develop port-forward $POD_NAME 8001:8001配置ingress-nginx转发
vi ingress-nginx.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: redisinsight-ingress
namespace: develop
spec:
ingressClassName: nginx
rules: # 一个ingress可以配置多个rules
- host: redisinsight.david.org # 域名配置，可以不写，匹配*，或者写 *.bar.com
http:
paths: # 相当于nginx的location，同一个host可以配置多个path
- pathType: Prefix
backend:
service:
name: redisinsight # 代理到哪个svc
port:
number: 80 # svc的端口
path: /
应用
kubectl apply -f ingress-nginx.yaml
配置hosts
192.168.221.132 redisinsight.david.org
访问测试
http://redisinsight.david.org