Ubuntu16.04手动部署Kubernetes(3)——Dashboard和KubeDNS部署

今天继续接着前文:

继续来部署Kubernetes。今天主要来部署Dashboard和KubeDNS,主要是后者。因为Kubernetes在每个Pod内都会起一个很小的“系统”容器pause-amd64:3.0来实现Pod内的网络,而这个容器默认会去从Google的Registry拉,但国内如果没有梯子访问不了。所以我pull了一个,传到了国内的镜像仓库,我们改下每个Node节点上的kubelet的配置,增加--pod_infra_container_image选项,这个选项可以指定从哪里pull这个基础镜像,我修改后的配置如下:

KUBELET_ARGS="--api-servers=http://192.168.56.101:8080 --cluster-dns=169.169.0.2 --cluster-domain=cluster.local --hostname-override=192.168.56.101 --logtostderr=false --log-dir=/var/log/kubernetes --v=2 --pod_infra_container_image=hub.c.163.com/allan1991/pause-amd64:3.0"

然后重启Kubelet:systemctl restart kubelet.service.另外,

除了这个镜像外,还有很多镜像也会有这个问题,所以我都传到了国内的镜像仓库,本文介绍时都使用国内我自己传的。大家没有梯子的话,直接使用我用的这个就可以了。OK,下面开始今天的主题。

部署Dashboard

Kubernetes提供了一个基础的图形化界面,就是这个Dashboard。基本上使用kubectl可以实现的功能现在都可以通过界面去做了,本质上它俩都是调用Kubernetes API。而部署这个Dashboard也非常简答,就是创建一个Pod和一个Service。最新的创建dashboard的yaml文件可以去查看官方:https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml。我的yaml文档如下:

# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Configuration to deploy release version of the Dashboard UI.
#
# Example usage: kubectl create -f <this_file>

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  labels:
    app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kubernetes-dashboard
  template:
    metadata:
      labels:
        app: kubernetes-dashboard
      # Comment the following annotation if Dashboard must not be deployed on master
      annotations:
        scheduler.alpha.kubernetes.io/tolerations: |
          [
            {
              "key": "dedicated",
              "operator": "Equal",
              "value": "master",
              "effect": "NoSchedule"
            }
          ]
    spec:
      containers:
      - name: kubernetes-dashboard
        image: hub.c.163.com/allan1991/kubernetes-dashboard-amd64:v1.5.1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9090
          protocol: TCP
        args:
          # Uncomment the following line to manually specify Kubernetes API server Host
          # If not specified, Dashboard will attempt to auto discover the API server and connect
          # to it. Uncomment only if the default does not work.
          - --apiserver-host=http://192.168.56.101:8080
        livenessProbe:
          httpGet:
            path: /
            port: 9090
          initialDelaySeconds: 30
          timeoutSeconds: 30
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 9090
  selector:
    app: kubernetes-dashboard

这里需要注意的是--apiserver-host默认是注释掉的,我有次部署时虽然找到了APIServer,但是连接被决绝,但是显式的去掉这个注释,写上APIServer的地址后就好了,所以我推荐还是写上比较好。然后使用kubectl create -f <this_file>就可以创建了。待这个DeploymentService创建成功后,执行kubectl --namespace=kube-system get svc命令查看映射的端口号,当然我们也可以在yaml文件里面使用nodePort指定为我们想要的端口,这里我没有指定,让系统自己选择,因为选择好以后就会固定,不会再变了。我的创建好有如下:

Master➜  kube-system kubectl --namespace=kube-system get svc
NAME                   CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
kubernetes-dashboard   169.169.162.93   <nodes>       80:10977/TCP    3d

这里需要注意执行命令时要加上--namespace=kube-system。前面我们也已经介绍过了,Kubernetes部署好后默认有两个namespace:defaultkube-system。用户自己的默认在default下,kubectl默认也使用这个namespace;Kubernetes自己的系统模块在kube-system下面。这样我们的Dashboard已经创建好了,通过APIServer的IP加上面的端口号10977就可以访问界面了:

Kubernetes_Dashboard.png

界面还是非常的清爽的,使用起来也很方便和简单,所以这里就不详细介绍了,可以参考官方图文教程:https://kubernetes.io/docs/user-guide/ui/

部署KubeDNS

在这之前我们先要搞清楚为什么需要DNS?是为了实现服务发现。而且DNS其实是Kubernetes 1.3开始才内置的功能,在这之前是利用Linux环境的方式来做的服务发现。这里我们举例说明一下利用环境变量做服务发现的过程:比如我们部署了几个服务,这些服务需要找到彼此。利用环境变量的方式就是在创建Pod的时候将Service的一些信息(主要是IP+端口)以环境变量的方式(环境变量按照一定的规则命名)注入到Pod上的容器内,这样就做到了服务发现。这种方式虽然简单,但显然有两个不足:

  1. 随着服务越来越多,环境变量会越来越多。

  2. Pod必须在Service之后创建。

所以后来就出现了DNS,利用这种方式去做服务发现就不会有上面所说的限制了。那DNS又是怎么做服务发现的呢,再举个例子:比如我们在Kubernetes的叫bar的namespace下面创建了一个Service叫foo。那么在bar下运行的Pod只需要做一个foo DNS查询便可获取到这个服务的IP和Port。如果是运行在其他namespace下的Pod想要查询foo这个服务,则只需要查询foo.bar即可。当然一般还会有一个域名(domain)。更详细的信息请参阅https://kubernetes.io/docs/admin/dns/

起初,Kubernetes的DNS是使用SkyDNS来实现的,网上现在的教程也基本是基于此的。但是新版本的Kubernetes已经不需要SkyDNS了,它有了自己的DNS模块——KubeDNS。这也是本文要介绍的。SkyDNS依赖于etcd,而KubeDNS不需要etcd。KubeDNS的部署也很简单,官方已经提供了基础的yaml文件:https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/dns。下面是我的yaml文件:

kubedns-deployment.yaml:

# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# TODO - At some point, we need to rename all skydns-*.yaml.* files to kubedns-*.yaml.*
# Should keep target in cluster/addons/dns-horizontal-autoscaler/dns-horizontal-autoscaler.yaml
# in sync with this file.

# Warning: This is a file generated from the base underscore template file: skydns-rc.yaml.base

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: kube-dns
  namespace: kube-system
  labels:
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
spec:
  # replicas: not specified here:
  # 1. In order to make Addon Manager do not reconcile this replicas parameter.
  # 2. Default is 1.
  # 3. Will be tuned in real time if DNS horizontal auto-scaling is turned on.
  strategy:
    rollingUpdate:
      maxSurge: 10%
      maxUnavailable: 0
  selector:
    matchLabels:
      k8s-app: kube-dns
  template:
    metadata:
      labels:
        k8s-app: kube-dns
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: ''
        scheduler.alpha.kubernetes.io/tolerations: '[{"key":"CriticalAddonsOnly", "operator":"Exists"}]'
    spec:
      containers:
      - name: kubedns
        image: hub.c.163.com/allan1991/kubedns-amd64:1.9
        resources:
          # TODO: Set memory limits when we've profiled the container for large
          # clusters, then set request = limit to keep this container in
          # guaranteed class. Currently, this container falls into the
          # "burstable" category so the kubelet doesn't backoff from restarting it.
          limits:
            memory: 170Mi
          requests:
            cpu: 100m
            memory: 70Mi
        livenessProbe:
          httpGet:
            path: /healthz-kubedns
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 60
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 5
        readinessProbe:
          httpGet:
            path: /readiness
            port: 8081
            scheme: HTTP
          # we poll on pod startup for the Kubernetes master service and
          # only setup the /readiness HTTP server once that's available.
          initialDelaySeconds: 3
          timeoutSeconds: 5
        args:
        - --kube-master-url=http://192.168.56.101:8080
        - --domain=cluster.local.
        - --dns-port=10053
        - --config-map=kube-dns
        # This should be set to v=2 only after the new image (cut from 1.5) has
        # been released, otherwise we will flood the logs.
        - --v=0
        # {{ pillar['federations_domain_map'] }}
        env:
        - name: PROMETHEUS_PORT
          value: "10055"
        ports:
        - containerPort: 10053
          name: dns-local
          protocol: UDP
        - containerPort: 10053
          name: dns-tcp-local
          protocol: TCP
        - containerPort: 10055
          name: metrics
          protocol: TCP
      - name: dnsmasq
        image: hub.c.163.com/allan1991/kube-dnsmasq-amd64:1.4
        livenessProbe:
          httpGet:
            path: /healthz-dnsmasq
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 60
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 5
        args:
        - --cache-size=1000
        - --no-resolv
        - --server=127.0.0.1#10053
        - --log-facility=-
        ports:
        - containerPort: 53
          name: dns
          protocol: UDP
        - containerPort: 53
          name: dns-tcp
          protocol: TCP
        # see: https://github.com/kubernetes/kubernetes/issues/29055 for details
        resources:
          requests:
            cpu: 150m
            memory: 10Mi
      # - name: dnsmasq-metrics
      #   image: gcr.io/google_containers/dnsmasq-metrics-amd64:1.0
      #   livenessProbe:
      #     httpGet:
      #       path: /metrics
      #       port: 10054
      #       scheme: HTTP
      #     initialDelaySeconds: 60
      #     timeoutSeconds: 5
      #     successThreshold: 1
      #     failureThreshold: 5
      #   args:
      #   - --v=2
      #   - --logtostderr
      #   ports:
      #   - containerPort: 10054
      #     name: metrics
      #     protocol: TCP
      #   resources:
      #     requests:
      #       memory: 10Mi
      - name: healthz
        image: hub.c.163.com/allan1991/exechealthz-amd64:1.2
        resources:
          limits:
            memory: 50Mi
          requests:
            cpu: 10m
            # Note that this container shouldn't really need 50Mi of memory. The
            # limits are set higher than expected pending investigation on #29688.
            # The extra memory was stolen from the kubedns container to keep the
            # net memory requested by the pod constant.
            memory: 50Mi
        args:
        - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1 >/dev/null
        - --url=/healthz-dnsmasq
        - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1:10053 >/dev/null
        - --url=/healthz-kubedns
        - --port=8080
        - --quiet
        ports:
        - containerPort: 8080
          protocol: TCP
      dnsPolicy: Default  # Don't use cluster DNS.

这里我把dnsmasq-metrics这个container注释掉了,因为我没把这个镜像下载下来...不过没了这个也没什么影响,这个也是后来才新增的,原来是没有的。

kubedns-svc.yaml:

# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This file should be kept in sync with cluster/images/hyperkube/dns-svc.yaml

# TODO - At some point, we need to rename all skydns-*.yaml.* files to kubedns-*.yaml.*

# Warning: This is a file generated from the base underscore template file: skydns-svc.yaml.base

apiVersion: v1
kind: Service
metadata:
  name: kube-dns
  namespace: kube-system
  labels:
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: "KubeDNS"
spec:
  selector:
    k8s-app: kube-dns
  clusterIP: 169.169.0.2
  ports:
  - name: dns
    port: 53
    protocol: UDP
  - name: dns-tcp
    port: 53
    protocol: TCP

这里需要注意下clusterIP,你可以不设置,让系统自己去选择。如果显式的设置的话,指定的IP必须在APIServer的--service-cluster-ip-range参数指定的网段内。我们可以用以下命令去检查是否创建成功:

Master➜  ~ kubectl --namespace=kube-system get deployment
NAME                   DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
kube-dns               1         1         1            1           1d
kubernetes-dashboard   1         1         1            1           3d
Master➜  ~ kubectl --namespace=kube-system get pod
NAME                                    READY     STATUS    RESTARTS   AGE
kube-dns-2528035289-r7vrq               3/3       Running   0          19h
kubernetes-dashboard-1077846755-lj7sc   1/1       Running   0          19h
Master➜  ~ kubectl --namespace=kube-system get svc
NAME                   CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
kube-dns               169.169.0.2      <none>        53/UDP,53/TCP   1d
kubernetes-dashboard   169.169.162.93   <nodes>       80:10977/TCP    3d

Service、Deployment都OK后,我们需要在每个Node的Kubelet配置中加入DNS的信息,主要是--cluster-dns--cluster-domain,比如我的配置如下:

KUBELET_ARGS="--api-servers=http://192.168.56.101:8080 --cluster-dns=169.169.0.2 --cluster-domain=cluster.local --hostname-override=192.168.56.101 --logtostderr=false --log-dir=/var/log/kubernetes --v=2 --pod_infra_container_image=hub.c.163.com/allan1991/pause-amd64:3.0"

这里我们使用的是默认的Domain cluster.local,当然你可以在kubedns-deployment.yaml将所有的cluster.local换为你想要的,比如time-track.cn

最后我们来验证一下,首先创建一个Pod,里面运行一个busybox容器,主要是为了使用里面的nslookup命令:

busybox.yaml

apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
spec:
  containers:
  - image: hub.c.163.com/library/busybox
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
    name: busybox
  restartPolicy: Always

然后我们再创建一个服务,比如mysql吧:

mysql.yaml

apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  ports:
    - port: 3306
  selector:
    app: mysql

然后我们在先创建的busybox里面通过DNS查询后创建的mysql服务,看是否可以查到:

Master➜  ~ kubectl exec busybox -c busybox -- nslookup mysql
Server:    169.169.0.2
Address 1: 169.169.0.2 kube-dns.kube-system.svc.cluster.local

Name:      mysql
Address 1: 169.169.96.10 mysql.default.svc.cluster.local


Master➜  ~ kubectl describe svc mysql
Name:            mysql
Namespace:        default
Labels:            <none>
Selector:        app=mysql
Type:            ClusterIP
IP:            169.169.96.10
Port:            <unset>    3306/TCP
Endpoints:        <none>
Session Affinity:    None
No events.

可以看到,成功的查到了。这样通过DNS变做到了服务发现,而且没有先后顺序的限制了。


添加新评论

选择表情 captcha

友情提醒:不填或错填验证码会引起页面刷新,导致已填的评论内容丢失。

|