部署结构
- 172.16.7.71 Master节点
- 172.16.7.72-73 node节点
- 172.16.7.71、72、73ETCD集群
ETCD
1、将etcd、etcdctl拷贝到/usr/bin目录下
2、新增/usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/bin/etcd --name=\"${ETCD_NAME}\" --data-dir=\"${ETCD_DATA_DIR}\" --listen-client-urls=\"${ETCD_LISTEN_CLIENT_URLS}\""
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
3、新增配置文件/etc/etcd/etcd.conf
[member]
ETCD_NAME=etcd3
ETCD_DATA_DIR="/var/lib/etcd/etcd3"
#ETCD_WAL_DIR=""
#ETCD_SNAPSHOT_COUNT="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
ETCD_LISTEN_PEER_URLS="http://172.16.7.73:2380"
ETCD_LISTEN_CLIENT_URLS="http://172.16.7.73:2379,http://127.0.0.1:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://172.16.7.73:2380"
# if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
ETCD_INITIAL_CLUSTER="etcd1=http://172.16.7.71:2380,etcd2=http://172.16.7.72:2380,etcd3=http://172.16.7.73:2380"
ETCD_INITIAL_CLUSTER_STATE="new33"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster33"
ETCD_ADVERTISE_CLIENT_URLS="http://172.16.7.73:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_SRV=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_STRICT_RECONFIG_CHECK="false"
#ETCD_AUTO_COMPACTION_RETENTION="0"
#
#[proxy]
#ETCD_PROXY="off"
#ETCD_PROXY_FAILURE_WAIT="5000"
#ETCD_PROXY_REFRESH_INTERVAL="30000"
#ETCD_PROXY_DIAL_TIMEOUT="1000"
#ETCD_PROXY_WRITE_TIMEOUT="5000"
#ETCD_PROXY_READ_TIMEOUT="0"
#
#[security]
#ETCD_CERT_FILE=""
#ETCD_KEY_FILE=""
#ETCD_CLIENT_CERT_AUTH="false"
#ETCD_TRUSTED_CA_FILE=""
#ETCD_AUTO_TLS="false"
#ETCD_PEER_CERT_FILE=""
#ETCD_PEER_KEY_FILE=""
#ETCD_PEER_CLIENT_CERT_AUTH="false"
#ETCD_PEER_TRUSTED_CA_FILE=""
#ETCD_PEER_AUTO_TLS="false"
#
#[logging]
#ETCD_DEBUG="false"
# examples for -log-package-levels etcdserver=WARNING,security=DEBUG
#ETCD_LOG_PACKAGE_LEVELS=""
注意ETCD_NAME、ETCD_DATA_DIR每个节点配置成不同,
ETCD_INITIAL_CLUSTER_STATE、ETCD_INITIAL_CLUSTER_TOKEN每个集群最好配置成不同
4、systemctl daemon-reload 然后 systemctl start etcd即可,注意要多个节点同时运行,否则集群初始化会失败
5、验证
[root@localhost ~]# etcdctl member list
855390a4542535be: name=etcd3 peerURLs=http://172.16.7.73:2380 clientURLs=http://172.16.7.73:2379 isLeader=false
973d9cab4aa5f1ba: name=etcd1 peerURLs=http://172.16.7.71:2380 clientURLs=http://172.16.7.71:2379 isLeader=true
9ecaf535a6810487: name=etcd2 peerURLs=http://172.16.7.72:2380 clientURLs=http://172.16.7.72:2379 isLeader=false
master->apiServer
1、把二进制文件都拷贝到/usr/local/bin目录下,并创建日志目录/var/logs/kubernetes
2、添加配置文件/usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Service
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
After=etcd.service
[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/apiserver
ExecStart=/usr/local/bin/kube-apiserver $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBE_ETCD_SERVERS $KUBE_API_ADDRESS $KUBE_API_PORT $KUBELET_PORT $KUBE_ALLOW_PRIV $KUBE_SERVICE_ADDRESSES $KUBE_ADMISSION_CONTROL $KUBE_API_ARGS
Restart=on-failure
Type=notify
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
3、创建配置文件 /etc/kubernetes/config
###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
# kube-apiserver.service
# kube-controller-manager.service
# kube-scheduler.service
# kubelet.service
# kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=false --log-dir=/var/log/kubernetes"
# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"
# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=true"
# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://172.16.7.71:8080"
4、创建配置文件/etc/kubernetes/apiserver
##
# kubernetes system config
#
# The following values are used to configure the kube-apiserver
##
# The address on the local server to listen to.
KUBE_API_ADDRESS="--advertise-address=172.16.7.1 --insecure-bind-address=0.0.0.0"
# The port on the local server to listen on.
KUBE_API_PORT="--port=8080"
# Port minions listen on
KUBELET_PORT="--kubelet-port=10250"
# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd-servers=http://172.16.7.1:2379,http://172.16.7.2:2379,http://172.16.7.3:2379"
# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.96.0.0/12"
# default admission control policies
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,LimitRanger,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota"
# Add your own!
KUBE_API_ARGS="--storage-backend=etcd2 --enable-swagger-ui=true --apiserver-count=3 --audit-log-maxage=30 --audit-log-maxbackup=3 --audit-log-maxsize=100 --audit-log-path=/var/lib/audit.log --event-ttl=1h"
5、把kubectl二进制文件拷贝到/usr/bin目录下
6、启动kube-apiserver
# systemctl daemon-reload
# systemctl enable kube-apiserver
# systemctl start kube-apiserver
7、验证
kubectl get componentstatuses
[root@localhost kubernetes]# curl 172.16.7.71:8080
{
"paths": [
"/api",
。。。。。。
"/version"
]
}
master->controller-manager
1、将二进制文件拷贝到/usr/local/bin目录下
2、创建文件/usr/lib/systemd/system/kube-controller-manager.service
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/controller-manager
ExecStart=/usr/local/bin/kube-controller-manager $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBE_MASTER $KUBE_CONTROLLER_MANAGER_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
3、创建配置文件 /etc/kubernetes/controller-manager
###
# The following values are used to configure the kubernetes controller-manager
# defaults from config and apiserver should be adequate
# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS="--master= --address=127.0.0.1 --service-cluster-ip-range=10.96.0.0/12 --cluster-name=kubernetes --leader-elect=true"
4、启动
# systemctl daemon-reload
# systemctl enable kube-apiserver
# systemctl start kube-apiserver
master->scheduler
1、将二进制文件拷贝到/uar/local/bin目录下
2、创建配置文件/usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler Plugin
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/scheduler
ExecStart=/usr/local/bin/kube-scheduler $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBE_MASTER $KUBE_SCHEDULER_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
3、创建配置文件/etc/kubernetes/scheduler
###
# kubernetes scheduler config
# default config should be adequate
# Add your own!
KUBE_SCHEDULER_ARGS="--leader-elect=true --address=127.0.0.1"
4、启动
# systemctl daemon-reload
# systemctl enable kube-scheduler
# systemctl start kube-scheduler
5、验证
[root@localhost kubernetes]# kubectl get componentstatuses
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-0 Healthy {"health": "true"}
etcd-1 Healthy {"health": "true"}
etcd-2 Healthy {"health": "true"}
node->kubelet
1、将kubelet放到/usr/local/bin中,确认docker已经安装完毕(最好是overlayfs驱动)
2、添加配置文件/usr/lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service
[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/kubelet
ExecStart=/usr/local/bin/kubelet $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBELET_API_SERVER $KUBELET_ADDRESS $KUBELET_PORT $KUBELET_HOSTNAME $KUBE_ALLOW_PRIV $KUBELET_POD_INFRA_CONTAINER $KUBELET_ARGS
Restart=on-failure
[Install]
WantedBy=multi-user.target
3、创建目录mkdir -p /etc/kubernetes/ /var/lib/kubelet/ /var/logs/kubernetes
4、创建配置文件/etc/kubernetes/config
###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
# kube-apiserver.service
# kube-controller-manager.service
# kube-scheduler.service
# kubelet.service
# kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=false --log-dir=/var/log/kubernetes"
# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"
# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=true"
# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://172.16.7.71:8080"
5、创建配置文件/etc/kubernetes/kubelet
##
# kubernetes kubelet (minion) config
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"
# The port for the info server to serve on
KUBELET_PORT="--port=10250"
# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname-override=172.16.7.72"
## location of the api-server
KUBELET_API_SERVER="--api-servers=http://172.16.7.71:8080"
# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"
# Add your own!
KUBELET_ARGS="--cgroup-driver=systemd --allow-privileged=true"
6、启动kubelet
# systemctl daemon-reload
# systemctl enable kubelet
# systemctl start kubelet
7、验证:在master启动kubectl get no即可查看
node->kube-proxy
1、拷贝二进制文件到/usr/local/bin
2、新增配置文件/usr/lib/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/proxy
ExecStart=/usr/local/bin/kube-proxy $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBE_MASTER $KUBE_PROXY_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
3、新增配置文件/etc/kubernetes/proxy
###
# kubernetes proxy config
# default config should be adequate
# Add your own!
KUBE_PROXY_ARGS="--master=http://172.16.7.1:8080 --proxy-mode=iptables --logtostderr=false --log-dir=/var/log/kubernetes --alsologtostderr=false --v=2 "
4、启动
# systemctl daemon-reload
# systemctl enable kube-proxy
# systemctl start kube-proxy