Tanzu Kubernetes Grid(TKGm) 1.2.1 にIstio をインストールする
サービスメッシュを色々と触ってみたいと思ったので、サービスメッシュとしては一番知られているIstio をTanzu Kubernetes Grid(TKG) にインストールしてみました。この記事では、Istio のGetting Started に従ってインストールしました。
前提
- TKGm on vSphere 環境(TKGm v1.2.1)
type: LoadBalancerが使える事(この記事ではmetalLBを利用しています)
手順
Workload Cluster のデプロイ
TKG のWorkload Cluster をデプロイします。
今回は該当しないかもしれませんが、こちらの手順を実施した上でWorkload Cluster をデプロイします。※
StatefulSet は利用していなかったので、この手順は実施しないでも大丈夫そうです。$ tkg get cluster --include-management-cluster
NAME NAMESPACE STATUS CONTROLPLANE WORKERS KUBERNETES ROLES
devsecops default running 1/1 1/1 v1.18.10+vmware.1 <none>
tkgm-mylab tkg-system running 1/1 1/1 v1.19.1+vmware.2 management
~/.tkg/providers/infrastructure-vsphere/v0.7.1/ytt$ ls
add_csi.yaml base-template.yaml csi.lib.yaml csi-vsphere.lib.txt overlay.yaml
~/.tkg/providers/infrastructure-vsphere/v0.7.1/ytt$ vim csi.lib.yaml
この環境においては、
~/.tkg/config.yaml にてWorker ノードのディスクが128GB に設定されています。テスト用なので、今回デプロイするIstio 用クラスタに関しては、64GB に設定した上でデプロイします。$ export VSPHERE_WORKER_DISK_GIB=64
Workload Cluster をデプロイします。
$ tkg create cluster istio --plan dev --vsphere-controlplane-endpoint-ip xxx.xxx.xxx.xxx --kubernetes-version=v1.18.10+vmware.1 -v 6
Logs of the command execution can also be found at: /tmp/tkg-20210321T054219990398081.log
...(SNIP)...
Checking cluster reachability...
Waiting for addons installation...
Waiting for resources type *v1alpha3.ClusterResourceSetList to be up and running
Waiting for resource antrea-controller of type *v1.Deployment to be up and running
Workload cluster 'istio' created$ tkg get clusters
NAME NAMESPACE STATUS CONTROLPLANE WORKERS KUBERNETES ROLES
devsecops default running 1/1 1/1 v1.18.10+vmware.1 <none>
istio default running 1/1 1/1 v1.18.10+vmware.1 <none>$ tkg get credentials istio
Credentials of workload cluster 'istio' have been saved
You can now access the cluster by running 'kubectl config use-context istio-admin@istio'$ kubectl config use-context istio-admin@istio
Switched to context "istio-admin@istio".この後、このWorkload Cluster 上で
metalLB をデプロイし、type: LoadBalancer でEXTERNAL-IP が払い出せる様にしておきます。Istio のデプロイ
「Istio / Getting Started」に従ってデプロイしていきます。
$ curl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.9.1 TARGET_ARCH=x86_64 sh -
$ cd istio-1.9.1
$ ls
bin LICENSE manifests manifest.yaml README.md samples tools
$ cd bin
$ ./istioctl install --set profile=demo
Detected that your cluster does not support third party JWT authentication. Falling back to less secure first party JWT. See https://istio.io/v1.9/docs/ops/best-practices/security/#configure-third-party-service-account-tokens for details.
This will install the Istio 1.9.1 demo profile with ["Istio core" "Istiod" "Ingress gateways" "Egress gateways"] components into the cluster. Proceed? (y/N) y
✔ Istio core installed
✔ Istiod installed
✔ Ingress gateways installed
✔ Egress gateways installed
✔ Installation complete
暫くすると、Namespace istio-system にリソースがデプロイされている事を確認出来ます。
$ kubectl get all -n istio-system
NAME READY STATUS RESTARTS AGE
pod/istio-egressgateway-7f9744768f-8pg2r 1/1 Running 0 2m59s
pod/istio-ingressgateway-7c89f4dbb-jsgkt 1/1 Running 0 2m59s
pod/istiod-5749997fb9-zw7s5 1/1 Running 0 3m19s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/istio-egressgateway ClusterIP 100.70.91.13 <none> 80/TCP,443/TCP,15443/TCP 2m59s
service/istio-ingressgateway LoadBalancer 100.64.163.178 xxx.xxx.xxx.xxx 15021:32500/TCP,80:31272/TCP,443:30877/TCP,31400:30786/TCP,15443:30756/TCP 2m59s
service/istiod ClusterIP 100.68.18.243 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 3m19s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/istio-egressgateway 1/1 1 1 2m59s
deployment.apps/istio-ingressgateway 1/1 1 1 2m59s
deployment.apps/istiod 1/1 1 1 3m19s
NAME DESIRED CURRENT READY AGE
replicaset.apps/istio-egressgateway-7f9744768f 1 1 1 2m59s
replicaset.apps/istio-ingressgateway-7c89f4dbb 1 1 1 2m59s
replicaset.apps/istiod-5749997fb9 1 1 1 3m19s
「Istio / Getting Started」ではNamespace は
default を利用していますが、こちらの記事ではisito-test というNamespace を利用し実施したいと思いますので、Namespace を作成し、このNamespace にデプロイされたアプリケーションはIstio のサービスメッシュの機能が使える様に設定しておきます。$ kubectl create ns istio-test
namespace/istio-test created$ kubectl label namespace istio-test istio-injection=enabled
namespace/istio-test labeled$ kubectl describe namespaces istio-test
Name: istio-test
Labels: istio-injection=enabled
Annotations: <none>
Status: Active
No resource quota.
No LimitRange resource.
サンプルアプリケーションのデプロイ
Istio を利用するサンプルアプリケーションをデプロイします。
$ kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml -n istio-test
service/details created
serviceaccount/bookinfo-details created
deployment.apps/details-v1 created
service/ratings created
serviceaccount/bookinfo-ratings created
deployment.apps/ratings-v1 created
service/reviews created
serviceaccount/bookinfo-reviews created
deployment.apps/reviews-v1 created
deployment.apps/reviews-v2 created
deployment.apps/reviews-v3 created
service/productpage created
serviceaccount/bookinfo-productpage created
deployment.apps/productpage-v1 created
$ kubectl get svc -n istio-test
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
details ClusterIP 100.68.144.112 <none> 9080/TCP 2m50s
productpage ClusterIP 100.71.188.179 <none> 9080/TCP 2m50s
ratings ClusterIP 100.71.193.208 <none> 9080/TCP 2m50s
reviews ClusterIP 100.67.101.161 <none> 9080/TCP 2m50s
$ kubectl get pods -n istio-test
NAME READY STATUS RESTARTS AGE
details-v1-66b6955995-sk7p7 2/2 Running 0 7m30s
productpage-v1-5d9b4c9849-v82mh 2/2 Running 0 7m30s
ratings-v1-fd78f799f-65mgt 2/2 Running 0 7m30s
reviews-v1-6549ddccc5-cdtb8 2/2 Running 0 7m30s
reviews-v2-76c4865449-rg57k 2/2 Running 0 7m30s
reviews-v3-6b554c875-tp2mm 2/2 Running 0 7m30s
ちゃんとデプロイされたか確認してみます。
$ kubectl exec "$(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}' -n istio-test)" -c ratings -n istio-test -- curl -sS productpage:9080/productpage | grep -o "<title>.*</title>"
<title>Simple Bookstore App</title>大丈夫そうですね。
外部からもアクセス出来るように設定しておきます。
$ kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml -n istio-test
gateway.networking.istio.io/bookinfo-gateway created
virtualservice.networking.istio.io/bookinfo created問題無いかどうか確認します。大丈夫そうですね。
~/istio-1.9.1/bin$ ./istioctl analyze -n istio-test
✔ No validation issues found when analyzing namespace: istio-test.istio-ingressgateway にアクセスするためのIPアドレス等を確認します。それを用いてWeb ブラウザからアクセスしてみます。$ export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
$ export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
$ export SECURE_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].port}')
Web ブラウザからアクセスしてみます。
モニタリングツールのインストール
「Istio / Getting Started」に従って、モニタリングツールも導入します。
$ kubectl apply -f samples/addons
serviceaccount/grafana created
configmap/grafana created
service/grafana created
deployment.apps/grafana created
configmap/istio-grafana-dashboards created
configmap/istio-services-grafana-dashboards created
deployment.apps/jaeger created
service/tracing created
service/zipkin created
service/jaeger-collector created
customresourcedefinition.apiextensions.k8s.io/monitoringdashboards.monitoring.kiali.io created
serviceaccount/kiali created
configmap/kiali created
clusterrole.rbac.authorization.k8s.io/kiali-viewer created
clusterrole.rbac.authorization.k8s.io/kiali created
clusterrolebinding.rbac.authorization.k8s.io/kiali created
role.rbac.authorization.k8s.io/kiali-controlplane created
rolebinding.rbac.authorization.k8s.io/kiali-controlplane created
service/kiali created
deployment.apps/kiali created
serviceaccount/prometheus created
configmap/prometheus created
clusterrole.rbac.authorization.k8s.io/prometheus created
clusterrolebinding.rbac.authorization.k8s.io/prometheus created
service/prometheus created
deployment.apps/prometheus created
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"
unable to recognize "samples/addons/kiali.yaml": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"エラーが出力されますが、
If there are errors trying to install the addons, try running the command again. There may be some timing issues which will be resolved when the command is run again とドキュメントにあるので、再度実施します。$ kubectl apply -f samples/addons
serviceaccount/grafana unchanged
configmap/grafana unchanged
service/grafana unchanged
deployment.apps/grafana configured
configmap/istio-grafana-dashboards configured
configmap/istio-services-grafana-dashboards configured
deployment.apps/jaeger unchanged
service/tracing unchanged
service/zipkin unchanged
service/jaeger-collector unchanged
customresourcedefinition.apiextensions.k8s.io/monitoringdashboards.monitoring.kiali.io unchanged
serviceaccount/kiali unchanged
configmap/kiali unchanged
clusterrole.rbac.authorization.k8s.io/kiali-viewer unchanged
clusterrole.rbac.authorization.k8s.io/kiali unchanged
clusterrolebinding.rbac.authorization.k8s.io/kiali unchanged
role.rbac.authorization.k8s.io/kiali-controlplane unchanged
rolebinding.rbac.authorization.k8s.io/kiali-controlplane unchanged
service/kiali unchanged
deployment.apps/kiali unchanged
monitoringdashboard.monitoring.kiali.io/envoy created
monitoringdashboard.monitoring.kiali.io/go created
monitoringdashboard.monitoring.kiali.io/kiali created
monitoringdashboard.monitoring.kiali.io/micrometer-1.0.6-jvm-pool created
monitoringdashboard.monitoring.kiali.io/micrometer-1.0.6-jvm created
monitoringdashboard.monitoring.kiali.io/micrometer-1.1-jvm created
monitoringdashboard.monitoring.kiali.io/microprofile-1.1 created
monitoringdashboard.monitoring.kiali.io/microprofile-x.y created
monitoringdashboard.monitoring.kiali.io/nodejs created
monitoringdashboard.monitoring.kiali.io/quarkus created
monitoringdashboard.monitoring.kiali.io/springboot-jvm-pool created
monitoringdashboard.monitoring.kiali.io/springboot-jvm created
monitoringdashboard.monitoring.kiali.io/springboot-tomcat created
monitoringdashboard.monitoring.kiali.io/thorntail created
monitoringdashboard.monitoring.kiali.io/tomcat created
monitoringdashboard.monitoring.kiali.io/vertx-client created
monitoringdashboard.monitoring.kiali.io/vertx-eventbus created
monitoringdashboard.monitoring.kiali.io/vertx-jvm created
monitoringdashboard.monitoring.kiali.io/vertx-pool created
monitoringdashboard.monitoring.kiali.io/vertx-server created
serviceaccount/prometheus unchanged
configmap/prometheus unchanged
clusterrole.rbac.authorization.k8s.io/prometheus unchanged
clusterrolebinding.rbac.authorization.k8s.io/prometheus unchanged
service/prometheus unchanged
deployment.apps/prometheus configuredリソースがちゃんと稼働しているか確認します。
$ kubectl get all -n istio-system
NAME READY STATUS RESTARTS AGE
pod/grafana-94f5bf75b-96nwf 1/1 Running 0 34s
pod/istio-egressgateway-7f9744768f-8pg2r 1/1 Running 0 23m
pod/istio-ingressgateway-7c89f4dbb-jsgkt 1/1 Running 0 23m
pod/istiod-5749997fb9-zw7s5 1/1 Running 0 24m
pod/jaeger-5c7675974-z4vd2 1/1 Running 0 34s
pod/kiali-d4fdb9cdb-zpbvc 1/1 Running 0 34s
pod/prometheus-7d76687994-zw9m5 2/2 Running 0 34s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/grafana ClusterIP 100.70.13.60 <none> 3000/TCP 34s
service/istio-egressgateway ClusterIP 100.70.91.13 <none> 80/TCP,443/TCP,15443/TCP 23m
service/istio-ingressgateway LoadBalancer 100.64.163.178 xxx.xxx.xxx.xxx 15021:32500/TCP,80:31272/TCP,443:30877/TCP,31400:30786/TCP,15443:30756/TCP 23m
service/istiod ClusterIP 100.68.18.243 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 24m
service/jaeger-collector ClusterIP 100.68.245.63 <none> 14268/TCP,14250/TCP 34s
service/kiali ClusterIP 100.64.102.154 <none> 20001/TCP,9090/TCP 34s
service/prometheus ClusterIP 100.69.42.164 <none> 9090/TCP 34s
service/tracing ClusterIP 100.69.146.45 <none> 80/TCP 34s
service/zipkin ClusterIP 100.65.162.115 <none> 9411/TCP 34s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/grafana 1/1 1 1 34s
deployment.apps/istio-egressgateway 1/1 1 1 23m
deployment.apps/istio-ingressgateway 1/1 1 1 23m
deployment.apps/istiod 1/1 1 1 24m
deployment.apps/jaeger 1/1 1 1 34s
deployment.apps/kiali 1/1 1 1 34s
deployment.apps/prometheus 1/1 1 1 34s
NAME DESIRED CURRENT READY AGE
replicaset.apps/grafana-94f5bf75b 1 1 1 34s
replicaset.apps/istio-egressgateway-7f9744768f 1 1 1 23m
replicaset.apps/istio-ingressgateway-7c89f4dbb 1 1 1 23m
replicaset.apps/istiod-5749997fb9 1 1 1 24m
replicaset.apps/jaeger-5c7675974 1 1 1 34s
replicaset.apps/kiali-d4fdb9cdb 1 1 1 34s
replicaset.apps/prometheus-7d76687994 1 1 1 34sダッシュボードを起動してみます。
~/istio-1.9.1/bin$ ./istioctl dashboard kiali --address xxx.xxx.xxx.xxx
http://localhost:20001/kiali以下の通りWeb ブラウザからアクセス出来ました。これで準備が整いました。
まとめ
無事TKG 上にIstio をデプロイする事が出来ました。この環境を利用して、Istio の機能を確認していきたいと思います。
