1. 使用ingress
[kubeadm@server2 manifest]$ cat ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-demo
spec:
rules:
- host: www1.westos.org
http:
paths:
- path: /
backend:
serviceName: myservice
servicePort: 80
[kubeadm@server2 manifest]$ kubectl apply -f ingress.yaml
[kubeadm@server2 manifest]$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-demo <none> www1.westos.org 10.106.5.122 80 41h
[kubeadm@server2 manifest]$ kubectl -n ingress-nginx get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx NodePort 10.106.5.122 <none> 80:31188/TCP,443:31595/TCP 42h
[kubeadm@server2 manifest]$ kubectl -n ingress-nginx get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx NodePort 10.106.5.122 <none> 80:31188/TCP,443:31595/TCP 42h
[kubeadm@server2 manifest]$ kubectl -n ingress-nginx describe svc ingress-nginx
Name: ingress-nginx
Namespace: ingress-nginx
Labels: app.kubernetes.io/name=ingress-nginx
app.kubernetes.io/part-of=ingress-nginx
Annotations: Selector: app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx
Type: NodePort
IP: 10.106.5.122
Port: http 80/TCP
TargetPort: 80/TCP
NodePort: http 31188/TCP
Endpoints: 10.244.1.32:80
Port: https 443/TCP
TargetPort: 443/TCP
NodePort: https 31595/TCP
Endpoints: 10.244.1.32:443
Session Affinity: None
External Traffic Policy: Cluster
真机访问：
[kiosk@foundation60 Downloads]$ curl 172.25.60.4:31188
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.17.8</center>
</body>
</html>
[kiosk@foundation60 Downloads]$ curl www1.westos.org:31188
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
2. 用DaemonSet结合nodeselector来部署ingress-controller到特定的node上
用DaemonSet结合nodeselector来部署ingress-controller到特定的node上，然后使用HostNetwork直接把该pod与宿主机node的网络打通，直接使用宿主机的80/433端口就能访问服务。
优点是整个请求链路最简单，性能相对NodePort模式更好。
缺点是由于直接利用宿主机节点的网络和端口，一个node只能部署一个ingress-controller pod。
较适合大并发的生产环境使用。
[kubeadm@server2 manifest]$ kubectl get nodes --show-labels
NAME STATUS ROLES AGE VERSION LABELS
server2 Ready master 6d v1.18.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server2,kubernetes.io/os=linux,node-role.kubernetes.io/master=
server3 Ready <none> 5d23h v1.18.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server3,kubernetes.io/os=linux
server4 Ready <none> 5d23h v1.18.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server4,kubernetes.io/os=linux
设置ingress controller节点的标签
[kubeadm@server2 manifest]$ kubectl label nodes server4 ingress=nginx
node/server4 labeled
[kubeadm@server2 manifest]$ kubectl get nodes --show-labels
NAME STATUS ROLES AGE VERSION LABELS
server2 Ready master 6d v1.18.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server2,kubernetes.io/os=linux,node-role.kubernetes.io/master=
server3 Ready <none> 6d v1.18.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=server3,kubernetes.io/os=linux
server4 Ready <none> 6d v1.18.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress=nginx,kubernetes.io/arch=amd64,kubernetes.io/hostname=server4,kubernetes.io/os=linux
[kubeadm@server2 manifest]$ vim mandatory.yaml
200 kind: DaemonSet
208 # replicas: 1
222 hostNetwork: true
226 nodeSelector:
227 ingress: nginx
[kubeadm@server2 manifest]$ kubectl -n ingress-nginx get deployment.apps
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-ingress-controller 1/1 1 1 44h
[kubeadm@server2 manifest]$ kubectl -n ingress-nginx delete deployment.apps nginx-ingress-controller
deployment.apps "nginx-ingress-controller" deleted
[kubeadm@server2 manifest]$ kubectl apply -f mandatory.yaml
[kubeadm@server2 manifest]$ kubectl -n ingress-nginx get daemonsets.apps
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
nginx-ingress-controller 1 1 1 1 1 ingress=nginx 111s
[kubeadm@server2 manifest]$ kubectl -n ingress-nginx get pod
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-rdd9j 1/1 Running 0 2m13s
查看server4宿主机
[root@server4 ~]# netstat -antlpe|grep :80
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 101 319840 11140/nginx: master
tcp6 0 0 :::80 :::* LISTEN 101 319833 11140/nginx: master
[root@server4 ~]# netstat -antlpe|grep :443
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 101 319842 11140/nginx: master
tcp 0 0 10.96.0.1:54496 10.96.0.1:443 ESTABLISHED 0 122242 7233/flanneld
tcp 0 0 10.96.0.1:54610 10.96.0.1:443 ESTABLISHED 101 319647 11112/nginx-ingress
tcp6 0 0 :::443 :::* LISTEN 101 319835 11140/nginx: master
真机访问：不用使用端口访问
[kiosk@foundation60 Downloads]$ curl www1.westos.org/hostname.html
nginx-deployment-b97c4ff7f-9zxxf
[kiosk@foundation60 Downloads]$ curl www1.westos.org/hostname.html
nginx-deployment-b97c4ff7f-krq5z
3. ingress tls配置
[kubeadm@server2 manifest]$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-demo <none> www1.westos.org 10.106.5.122 80 44h
[kubeadm@server2 manifest]$ kubectl describe ingress ingress-demo
Name: ingress-demo
Namespace: default
Address: 10.106.5.122
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
www1.westos.org
/ myservice:80 (10.244.1.35:80,10.244.2.32:80)
Annotations: Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 31m nginx-ingress-controller Ingress default/ingress-demo
Normal UPDATE 30m nginx-ingress-controller Ingress default/ingress-demo
创建证书
[kubeadm@server2 manifest]$ mkdir ingress
[kubeadm@server2 manifest]$ cd ingress/
[kubeadm@server2 ingress]$ mkdir certs
[kubeadm@server2 ingress]$ cd certs/
[kubeadm@server2 certs]$ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginxsvc/O=nginxsvc"
Generating a 2048 bit RSA private key
.....+++
..........................................................................................................+++
writing new private key to 'tls.key'
[kubeadm@server2 certs]$ ls
tls.crt tls.key
将证书放到secret中
[kubeadm@server2 certs]$ kubectl create secret tls tls-secret --key tls.key --cert tls.crt
secret/tls-secret created
[kubeadm@server2 certs]$ kubectl get secrets
NAME TYPE DATA AGE
default-token-64lq2 kubernetes.io/service-account-token 3 6d
tls-secret kubernetes.io/tls 2 58s
[kubeadm@server2 certs]$ kubectl describe secrets tls-secret
Name: tls-secret
Namespace: default
Labels: <none>
Annotations: <none>
Type: kubernetes.io/tls
Data
====
tls.crt: 1143 bytes
tls.key: 1704 bytes
[kubeadm@server2 ingress]$ cat ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-demo
spec:
tls:
- hosts:
- www1.westos.org
secretName: tls-secret
rules:
- host: www1.westos.org
http:
paths:
- path: /
backend:
serviceName: myservice
servicePort: 80
[kubeadm@server2 ingress]$ kubectl apply -f ingress.yaml
ingress.networking.k8s.io/ingress-demo configured
[kubeadm@server2 ingress]$ kubectl describe ingress ingress-demo
Name: ingress-demo
Namespace: default
Address: 10.106.5.122
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
tls-secret terminates www1.westos.org
Rules:
Host Path Backends
---- ---- --------
www1.westos.org
/ myservice:80 (10.244.1.35:80,10.244.2.32:80)
Annotations: Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 47m nginx-ingress-controller Ingress default/ingress-demo
Normal UPDATE 26s (x2 over 46m) nginx-ingress-controller Ingress default/ingress-demo
基础认证：https://kubernetes.github.io/ingress-nginx/examples/auth/basic/
[kubeadm@server2 ingress]$ sudo yum install -y httpd-tools
[kubeadm@server2 ingress]$ htpasswd -c auth zjy
New password:
Re-type new password:
Adding password for user zjy
[kubeadm@server2 ingress]$ ls
auth certs ingress2.yaml ingress3.yaml ingress.yaml
[kubeadm@server2 ingress]$ cat auth
zjy:$apr1$Ape6yblZ$E1Z3Ebvlxkw8caz8ITu/W.
[kubeadm@server2 ingress]$ kubectl create secret generic basic-auth --from-file=auth
secret/basic-auth created
[kubeadm@server2 ingress]$ kubectl describe secrets basic-auth
Name: basic-auth
Namespace: default
Labels: <none>
Annotations: <none>
Type: Opaque
Data
====
auth: 42 bytes
[kubeadm@server2 ingress]$ kubectl get secret basic-auth -o yaml
apiVersion: v1
data:
auth: emp5OiRhcHIxJEFwZTZ5YmxaJEUxWjNFYnZseGt3OGNhejhJVHUvVy4K
kind: Secret
metadata:
creationTimestamp: "2020-04-23T13:51:00Z"
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:data:
.: {}
f:auth: {}
f:type: {}
manager: kubectl
operation: Update
time: "2020-04-23T13:51:00Z"
name: basic-auth
namespace: default
resourceVersion: "608602"
selfLink: /api/v1/namespaces/default/secrets/basic-auth
uid: 9a2f694c-feda-4f4a-ab84-160ef47ca2f0
type: Opaque
[kubeadm@server2 ingress]$ cat ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-demo
annotations:
# type of authentication
nginx.ingress.kubernetes.io/auth-type: basic
# name of the secret that contains the user/password definitions
nginx.ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropriate context why the authentication is required
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - zjy'
spec:
tls:
- hosts:
- www1.westos.org
secretName: tls-secret
rules:
- host: www1.westos.org
http:
paths:
- path: /
backend:
serviceName: myservice
servicePort: 80
[kubeadm@server2 ingress]$ kubectl apply -f ingress.yaml
ingress.networking.k8s.io/ingress-demo configured
[kubeadm@server2 ingress]$ kubectl describe ingress ingress-demo
Name: ingress-demo
Namespace: default
Address: 10.106.5.122
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
tls-secret terminates www1.westos.org
Rules:
Host Path Backends
---- ---- --------
www1.westos.org
/ myservice:80 (10.244.1.35:80,10.244.2.32:80)
Annotations: nginx.ingress.kubernetes.io/auth-realm: Authentication Required - zjy
nginx.ingress.kubernetes.io/auth-secret: basic-auth
nginx.ingress.kubernetes.io/auth-type: basic
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal UPDATE 51s (x3 over 68m) nginx-ingress-controller Ingress default/ingress-demo
[kiosk@foundation60 Downloads]$ curl -v www1.westos.org -Lk -u "zjy:redhat"
<
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
* Connection #1 to host www1.westos.org left intact
重写：https://kubernetes.github.io/ingress-nginx/examples/rewrite/
[kubeadm@server2 ingress]$ kubectl delete -f ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: rewrite-example
annotations:
nginx.ingress.kubernetes.io/app-root: /hostname.html
spec:
rules:
- host: www1.westos.org
http:
paths:
- path: /
backend:
serviceName: myservice
servicePort: 80
[kubeadm@server2 ingress]$ kubectl apply -f ingress3.yaml
ingress.networking.k8s.io/rewrite-example created
[kiosk@foundation60 Downloads]$ curl www1.westos.org -L # 支持重定向
nginx-deployment-b97c4ff7f-krq5z
[kubeadm@server2 ingress]$ cat ingress3.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: rewrite-example
annotations:
#nginx.ingress.kubernetes.io/app-root: /hostname.html
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: www1.westos.org
http:
paths:
- path: /demo(/|$)(.*)
backend:
serviceName: myservice
servicePort: 80
[kubeadm@server2 ingress]$ kubectl apply -f ingress3.yaml
ingress.networking.k8s.io/rewrite-example configured
[kubeadm@server2 ingress]$ kubectl -n ingress-nginx exec -it nginx-ingress-controller-rdd9j -- bash
bash-5.0$ cd /etc/nginx/
bash-5.0$ ls
fastcgi.conf geoip mime.types nginx.conf scgi_params uwsgi_params.default
fastcgi.conf.default koi-utf mime.types.default nginx.conf.default scgi_params.default win-utf
fastcgi_params koi-win modsecurity opentracing.json template
fastcgi_params.default lua
annotations参数
