Managing Talend microservices with Istio service mesh on Google Kubernetes Engine

Overview

The Talend Community Knowledge Base (KB) article, Managing Talend microservices with Istio service mesh on Kubernetes, shows you how to connect, secure, control, and observe Talend microservices leveraging Istio.

 

This article describes the installation and platform-specific steps for managing Talend microservices using Istio service mesh on Google Kubernetes Engine (GKE).

 

For more information on installing Talend microservices using Istio on other cloud providers, see the following Talend Community KB articles:

 

Prerequisites

 

IAM Roles and API access

 

Grant users additional IAM roles for GKE

From the Navigation menu, click IAM & admin, grant Kubernetes Engine Admin role for the user who is going to install Istio. For more information on the Predefined GKE Roles read the Google Kubernetes Engine documentation.

Iam_permissions.jpg

 

Enable Google Kubernetes Engine API

  1. From the Navigation menu, click APIs & Services, click Library, then search for Kubernetes Engine API.

    Enable_aPIS.jpg

     

  2. Enable Kubernetes Engine API.

    eNABLE_api.jpg

     

Launch a standard Google Kubernetes Engine (GKE)

  1. Log in to the Google cloud console and launch a Kubernetes cluster (GKE) by performing the following steps.

    Note: Follow the steps in the Google Kubernetes Engine, Creating a cluster, documentation to create a GKE cluster using the gcloud command-line tool.

  2. Launch a Standard GKE cluster with two nodes, in the Master version drop-down list, select the latest stable version of Kubernetes.

    create_cluster.jpg

     

  3. In the default-pool configuration, click the More button and configure the following:

    1. Set the Number of nodes to 2.

    2. Select the Enable auto-scaling check box.

    3. Set Minimum number of nodes to 2.

    4. Set Maximum number of nodes to 3.

    defaultpool_autoscaling.jpg

     

  4. Leave all other settings with the default values, then click Create cluster.

  5. After the cluster is created, click the check box next to the cluster name to select it, then click Activate Cloud Shell.

    Activate_Cloudshell.jpg

     

  6. From the cloud shell, enter the kubectl command, and verify there are two running nodes in the cluster.

    kubectl get nodes

    Cloudshell_getnodes.jpg

     

  7. Your GKE cluster is ready!

     

Preparing the GKE cluster for Istio

Create the necessary RBAC rules for Istio, assign a ClusterRole (cluster-admin) to the current user.

# Execute the below command from Cloud shell or gcloud SDK
kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole=cluster-admin \
--user=$(gcloud config get-value core/account)

 

Installing Istio with Helm on GKE

In GKE with the installation wizard, Istio can be installed by enabling the Anthos feature. However, in this section, you'll install it manually to ensure that you're using the latest version of Istio.

 

Helm is a package manager for Kubernetes. Helm Charts help you define, install, and upgrade even the most complex Kubernetes application.

 

Istio provides customizable Helm templates for installation on a Kubernetes cluster. For more information on other installation options, see the Istio Installation Guides.

 

Install Helm client

Install Helm client using one of the three following approaches, for example, Google Cloud shell / Linux / Windows OS.

 

Using Cloud Shell (GKE)

Helm client is preinstalled on Google Cloud shell.

helm version

helm_client.jpg

 

Using shell script (Linux)

Skip this step if you use Google Cloud shell. Helm client can be installed on Linux OS using shell commands.

Connect to a Cloud shell, then install Helm client.

curl -LO https://git.io/get_helm.sh
chmod 700 get_helm.sh
./get_helm.sh

 

From Chocolatey (Windows)

Skip this step if you use Google Cloud shell. The Helm package can be installed on Windows using Chocolatey

  1. Launch a command prompt and execute the following command:

    choco

    Note: If you get an error command not found choco, then follow the instructions in this link and Install Chocolatey on Windows.

  2. Install Helm using the choco install command:

    #Install helm client
    choco install kubernetes-helm

     

Customizing the Istio installation with Helm

From the cloud shell or command prompt:

  1. Create an istio_install directory.

    # create directory istio_install
    mkdir istio_install (Windows / Linux)
    
     # move into the istio_install
    cd istio_install
  2. Download an Istio release and then execute the steps in the guide Customizable Install with Helm.

    # Download the installation file using the curl command
    curl -Ls -O https://github.com/istio/istio/releases/download/1.3.4/istio-1.3.4-linux.tar.gz (Linux)
    curl -Ls -O https://github.com/istio/istio/releases/download/1.3.4/istio-1.3.4-win.zip (Windows)
    
    # gunzip and untar the file
    gunzip istio-1.3.4-linux.tar.gz 
    tar -xvf istio-1.3.4-linux.tar
    
    # move into the directory istio-1.3.4
    cd istio-1.3.4
    
    # Configure the PATH environment variable
    export PATH=$PWD/bin:$PATH (Linux)
    set PATH=%CD%/bin;%PATH% (Windows)
    
    # Initialize helm
    helm init
    #Add Istio to the helm repository
    helm repo add istio.io https://storage.googleapis.com/istio-release/releases/1.3.4/charts/
    
    #Create namespace istio-system
    kubectl create namespace istio-system
    
    #Install all the istio CRDs
    helm template install/kubernetes/helm/istio-init --name istio-init --namespace istio-system | kubectl apply -f -
  3. Install Istio using one of the Installation Configuration Profiles.

    Note: This article uses the demo_auth profile.

    # Execute the helm template with the predefined settings in the demo_auth profile
    helm template install/kubernetes/helm/istio --name istio --namespace istio-system \
    --values install/kubernetes/helm/istio/values-istio-demo-auth.yaml | kubectl apply -f -
  4. Execute the following command, then verify that you’ve installed all of the Istio components in the demo-auth configuration profile.

    kubectl get svc -n istio-system

    Note:

    • Ingress traffic to application pods or microservices is only made possible by using the External / Public IP assigned to the istio-ingressgateway service.

    • An external Load-balancer is launched on GKE automatically, and its IP is assigned to the istio-ingressgateway service.

     

Launching Istio monitoring dashboards for GKE

With Istio demo-auth profile, add-ons for dashboards such as Prometheus, Grafana, Kiali, and Jaeger are enabled by default.

  • Prometheus is a web-based graphical user interface for querying the Istio metric values.

  • Grafana is a web-based graphical user interface that provides a global view of the mesh along with services and their workloads.

  • Kiali is a web-based graphical user interface to view service graphs of the mesh and Istio configuration objects. Different graph types such as App, Versioned App, Workload, and Service are available to view the services in the mesh.

  1. Open the gcloud command-line tool from your Windows machine and execute the following commands:

    # Initialize the gcloud SDK
    gcloud init
    
    gcloud container clusters get-credentials <cluster-name> --zone <zone> --project <project-id>
    
    # Note: The above command generates the kubeconfig file to the windows USER_HOME/.kube folder.
  2. Launch the dashboards from your Windows machine using local port forwarding, as shown below:

    # Execute the below port-forward command and access Prometheus using local port forwarding
    kubectl port-forward svc/prometheus 9090:9090 -n istio-system
    
    # Execute the below port-forward command and access Grafana using local port forwarding
    kubectl port-forward svc/grafana 3000:3000 -n istio-system
    
    # Execute the below port-forward command and access Kiali using local port forwarding
    kubectl port-forward svc/kiali 20001:20001 -n istio-system
    
    # Execute the below port-forward command and access Jaeger using local port forwarding
    kubectl port-forward svc/jaeger 16686:16686 -n istio-system

     

Publish Talend microservices to Google Container Registry

Before you publish the Talend microservice container images to GCR, you need to determine the registry name and the Authentication methods supported by GCR. This section shows you how to authenticate with GCR using Access tokens.

  1. Connect to the gcloud shell, execute the following command, and generate an access token.

    gcloud auth print-access-token
  2. In Talend Studio, publish the Customer microservice to GCR.

    1. In the Repository view, expand Job Designs > Standard > Customer > Microservices > right-click Customers, then select Publish.

    2. Select Microservice (Spring Boot) For ESB Docker Image from the Export Type pull-down menu. Click Next.
    3. Provide your Registry URL, for example, gcr.io/ProjectId.

    4. Define the username as oauth2accesstoken.

    5. Set the password with the access token you generated in Step 1.

    6. Click Finish.

    pub_completejpg_1.jpg

  3. Repeat Step 2 and publish the Orders microservice to GCR.

  4. Make a note of the URLs of the microservice images published to GCR.

    pub_gcr_versions.jpg

     

Authentication with Google Container Registry from GKE

GKE has read access to the container images in Google Cloud Registry. For more information, see Using Google Container Registry

 

Conclusion

This article shows you how to launch a Google Kubernetes Engine, install Istio manually using Helm, and how to publish Talend Microservices on Google Container Registry.

Version history
Revision #:
37 of 37
Last update:
‎12-23-2019 08:17 AM
Updated by:
 
Contributors