Azure Kubernetes Service with Launchpad

This chapter discusses preparing Azure Kubernetes Service (AKS) for use with Launchpad.

Overview

You can use Azure Kubernetes Service (AKS) to run Stardog and interact with Launchpad.

Our goal in this tutorial is to do the following:

1. Show the commands needed to prepare Azure Kubernetes Service for use with Stardog Launchpad.

2. Show the commands help to show when automating in a CI/CD pipeline.

This page only covers how to set up Azure Kubernetes Service via the CLI.

Prerequisites

Commands:

• Linux-based environment
• Azure CLI
• jq

Dependencies:

• Azure Container Registry

Setup

Setting up the environment

Log in to Azure using the instructions found here.

We need to set the following environment variables:

Environment Example

Example variables for a 1-node cluster:

export AKS_NAME=dev-aks
export AKS_RESOURCE_GROUP=dev-stardog-aks-rg
export LOCATION=eastus
export ACR_NAME=devsdacr 
export SYSTEM_POOL_NODES=2 
export SYSTEM_POOL_VM=Standard_D2_v2
export STARDOG_POOL_VM=Standard_D4s_v4 #adjust based on capacity analysis
export STARDOG_POOL_NODES=1

For a 3-node cluster, replace the last line with:

export STARDOG_POOL_NODES=3
export ZK_POOL_NODES=3
export ZK_POOL_VM=3

Creating a resource group

If you do not have a resource group, you can create one with the following command:

az group create  --name $AKS_RESOURCE_GROUP --location $LOCATION

You can check if a resource group already exists with:

az group list | jq -r --arg name $AKS_RESOURCE_GROUP 'any(.[]; .name == $name)'

This command does the following:

• az group list: Gets a list of all the resource groups for the logged-in account.
• jq -r --arg name $AKS_RESOURCE_GROUP 'any(.[]; .name == $name)': Checks if there is a resource group whose name matches $AKS_RESOURCE_GROUP.

If you have not already register with Azure monitoring providers, you can do so with the following commands:

az provider register --namespace Microsoft.OperationsManagement
az provider register --namespace Microsoft.OperationalInsights

These commands are idempotent, so they can be safely rerun.

Creating the pools

Creating the AKS cluster - System pool

You can create a system pool in an AKS cluster using the following command:

az aks create \ 
--resource-group $AKS_RESOURCE_GROUP \
--name $AKS_NAME \
--nodepool-name system \
--vm-set-type VirtualMachineScaleSets \
--load-balancer-sku standard \
--node-count $SYSTEM_POOL_NODES \
--node-vm-size $SYSTEM_POOL_VM \
--enable-addons monitoring \
--generate-ssh-keys \
--attach-acr $ACR_NAME

If you are not using ACR, simply drop the --attach-acr argument.

You can check whether the cluster already exists with:

az aks list | jq -r --arg aks_name $AKS_NAME '.[] | select(.name == $aks_name)'

This command does the following:

• az aks list: Lists all the clusters in our Azure subscription.
• jq -r --arg aks_name $AKS_NAME '.[] | select(.name == $aks_name)': Checks to see if any of the clusters have a name that matches $AKS_NAME, which is what we named our cluster in the previous step.

Creating the Stardog pool

We recommend creating a dedicated pool for Stardog servers. You can do so with this command:

az aks nodepool add \
--resource-group $AKS_RESOURCE_GROUP \
--cluster-name $AKS_NAME \
--name sdpool \
--enable-cluster-autoscaler \
--node-count $STARDOG_POOL_NODES \
--node-vm-size $STARDOG_POOL_VM \
--node-taints workload=stardog:NoSchedule \
--labels target=stardog

The options --node-taints and --labels ensure only Stardog workloads are assigned to these nodes.

You can check whether the node pool already exists with:

az aks nodepool list --resource-group $RESOURCE_GROUP --cluster-name $AKS_NAME | jq -r --arg pool 'sdpool' '.[] | select(.name == $pool)'

This command does the following:

• az aks nodepool list --resource-group $RESOURCE_GROUP --cluster-name $AKS_NAME: Lists all the node pools in our AKS cluster.
• jq -r --arg pool 'sdpool' '.[] | select(.name == $pool): Checks to see if any of the pools have the name “sdpool”, which is what we named our pool in the previous step.

Creating the ZooKeeper pool

This step is only required if you’re running a Stardog cluster of 2 or more nodes. If you start with a single-node deployment and later upgrade to a cluster, you can simply add the ZooKeeper pool when you do so.

Like the Stardog pool, we recommend creating a dedicated pool for the ZooKeeper cluster.

az aks nodepool add \
--resource-group $AKS_RESOURCE_GROUP \
--cluster-name $AKS_NAME \
--name zkpool \
--enable-cluster-autoscaler \
--node-count $ZK_POOL_NODES \
--node-vm-size $ZK_POOL_VM \
--node-taints workload=zookeeper:NoSchedule \
--labels target=zookeeper

The options --node-taints and --labels ensure only ZooKeeper workloads are assigned to these nodes.

You can check whether the node pool already exists with:

az aks nodepool list --resource-group $RESOURCE_GROUP --cluster-name $AKS_NAME | jq -r --arg pool 'zkpool' '.[]|select(.name == $pool)'

Getting the AKS credentials

The last thing to do is to get the credentials for AKS so you can use kubectl and helm when installing other components. You can do so with the following command:

az aks get-credentials --resource-group $AKS_RESOURCE_GROUP --name $AKS_NAME --overwrite-existing

Cleaning up

You can now delete your Azure resource group via the following command:

az group delete --resource-group $AKS_RESOURCE_GROUP -y