Azure Key Vault with Launchpad

This chapter discusses preparing Azure Key Vault (AKV) for use with Launchpad.

Page Contents

Overview
Prerequisites
Setup
1. Setting up the environment
Setting up the AKV
1. Creating an AKV
2. Granting AKS access to AKV
Certificates
Troubleshooting
Conclusion
Cleaning up

Overview

You can use Azure Key Vault (AKV) to store certifactes and securely communicate with Launchpad.

Our goal in this tutorial is to do the following:

Show the commands needed to prepare Azure Key Vault for use with Stardog Launchpad.
Show the commands help to show when automating in a CI/CD pipeline.

This page only covers how to set up Azure Key Vault via the CLI.

Prerequisites

Commands:

Linux-based environment
Azure CLI
helm
jq
az-acme.sh (OPTIONAL - only needed if you want to use Let’s Encrypt)
socat (OPTIONAL - only needed if you want to use Let’s Encrypt)

Dependencies:

AKS Cluster is created
Azure DNS Zone is created (OPTIONAL - only needed if you want to use Let’s Encrypt)

Setup

Setting up the environment

Set the environment variables $AKS_NAME, $RESOURCE_GROUP and $LOCATION to the ones you used when creating your AKS Cluster. Set $AKV_NAME to the name you want to use for your Key Vault.

Example variables:

export AKS_NAME=stardog-aks
export AKV_RESOURCE_GROUP=dev-stardog-akv-rg
export LOCATION=eastus
export AKV_NAME=dev-stardog-akv 

# OPTIONAL - if you want to use Let’s Encrypt
export DNS_ZONE=dev.sd-testlab.com
export DNS_ZONE_RESOURCE=testlab-servicedomain
export SUBSCRIPTION=$(az account subscription list | jq -r '.[].subscriptionId')

See the documentation for az account subscription list, used in the last line of the above code block, here.

Setting up the AKV

Creating an AKV

You can create a new Azure Key Vault with the following command:

az keyvault create --resource-group $AKV_RESOURCE_GROUP --name $AKV_NAME --location $LOCATION

Your AKV name must be globally unique. You can check if your AKV already exists with the following command:

az keyvault list | jq -r --arg name $AKV_NAME '.[] | select(.name == $name)'

This command does the following:

az keyvault list: Gets a list of all the Key Vaults for the logged-in account.
jq -r: Processes the output as raw strings. --arg name $AKV_NAME sets a variable inside jq’s environment. '.[] iterates over the entire list of Key Vaults returned by az keyvault list.
select (...): Uses a filter to match the Key Vault whose name we stored in .name (which is set to the value of $AKV_NAME).

Granting AKS access to AKV

First, you must retrieve your Managed Identity Object ID:

AKS_MANAGE_IDENTITY_OBJECT_ID=$(az aks show -n $AKS_NAME -g $AKV_RESOURCE_GROUP | jq .identityProfile.kubeletidentity.objectId -r)

This command does the following:

az aks show -n $AKS_NAME -g $AKV_RESOURCE_GROUP: Shows the details of our AKS cluster.
jq .identityProfile.kubeletidentity.objectId -r: Extracts the Object ID of the kubelet identity from our AKS cluster.

Then apply the policy to allow AKS to get the certificates, secrtes, and key:

for pol in 'certificate' 'secret' 'key' ; do 
 az keyvault set-policy --name $AKV_NAME --object-id $AKS_MANAGE_IDENTITY_OBJECT_ID --$pol-permissions get 
done

See the documentation for az keyvault set-policy here. This documentation shows you other was to identify your Key Vault if Object ID does not conform to your company’s security policies.

Certificates

Importing Certificates

Now we’ll load our certificates into AKV. This can be done through the Azure Portal, or with the following command:

az keyvault certificate import --file sparql.pem --name sparql_stardog_dev --vault-name $AKV_NAME

Depending on your strategy, you may want to load a single certificate for all 3 services (SPARQL, BI, and Launchpad), or one for each service. For the sake of consistency across our documentation, we recommend create the following 3 entries:

sparql_stardog_dev
launchpad_stardog_dev
bi_stardog_dev

You can also import .pfx files.

Often when you are developing, you do not yet have your certificate(s). You may have also decided to use a dynamic certificate. In the next section, we will cover how this can be achieved with Let’s Encrypt.

Dynamic Certificates

There are multiple ways to set up dynamic certificates. One way is to use az-acme, like so:

az-acme register \
        --server https://acme-v02.api.letsencrypt.org/directory \
        --key-vault-uri https://dev-stardog-akv.vault.azure.net/ \
        --account-secret az-amce-registration \
        --email person@example.com \
        --agree-tos

You only need to perform this action once.

Next, you can order your certificate:

for cert in 'sparql' 'bi' 'launchpad' ; do 
  az-acme order \
    --server https://acme-v02.api.letsencrypt.org/directory \
    --key-vault-uri https://$AKV_NAME.vault.azure.net/ \
    --certificate $cert-stardog-dev \
    --subject $cert.dev.sd-testlab.com \
    --account-secret az-amce-registration \
    --dns-provider Azure \
    --azure-dns-zone /subscriptions/$SUBSCRIPTION/resourceGroups/$DNS_RESOURCE_GROUP/providers/Microsoft.Network/dnszones/$DNS_ZONE\
    --renew-within-days 80 \
    --verbose
done

This command will order the certificate, validate the domain, and push it to AKV.

Since Let’s Encrypt certificates are good for 90 days, you can easily rotate the certificates every 80 days by deploying the command above in a cron job.

If all of your services use Let’s Encrypt, we suggest using Cert-Manager instead. It easier to manage, and there are fewer things that could go wrong. Let’s Encrypt certificates are ideal for keeping costs down in a development environment and even an internal production system, without any compromise on security. Since Let’s Encrypt does not serve Extended Validation (EV) SSL Certificates, they are usually not used for public-facing services. However, if your production system uses AKV, it’s better to keep all of your environments consistent and use AKV on your development system, as well.

akv2k8s

To synchronize our certificates to our cluster, we will use akv2k8s. You can install it with the following commands:

kubectl create namespace akv2k8s  --dry-run=client -o yaml | kubectl apply  -f -
helm repo add spv-charts https://charts.spvapi.no
helm repo update
helm upgrade --install akv2k8s spv-charts/akv2k8s --namespace akv2k8s --set global.metrics.enabled=true

These commands do the following:

kubectl create namespace akv2k8s --dry-run=client -o yaml | kubectl apply -f -:
- kubectl create namespace akv2k8s: Creates a new Kubernetes namespace named akv2k8s.
  - --dry-run=client: Generates the command output without actually applying the changes.
  - -o yaml: Outputs the result in YAML format.
- kubectl apply -f -: Applies the configuration.
  - -f -: Tells kubectl to read the input from the previous command in the pipeline.
helm repo add spv-charts https://charts.spvapi.no:
- helm repo add spv-charts: Adds a new Helm chart repository named spv-charts to our Helm configuration.
- https://charts.spvapi.no: The URL of the repository.
helm repo update: Updates our local Helm chart repository cache, ensuring we have the latest information about charts available from the repositories we’ve added.
helm upgrade --install akv2k8s spv-charts/akv2k8s --namespace akv2k8s --set global.metrics.enabled=true:
- helm upgrade --install akv2k8s: Upgrades the Helm release to akv2k8s or installs it if it doesn’t exist.
- spv-charts/akv2k8s: Specifies which chart to use for the release; in this case, the akv2k8s chart from the spv-charts repository.
- --namespace akv2k8s: Specifies the Kubernetes namespace where the release should be deployed. It uses the namespace we created in the first command.
- --set global.metrics.enabled=true: Sets a value in the chart’s configuration. This particular setting enables metrics globally in the akv2k8s chart.

Running helm upgrade when nothing has changed will work, but it takes a nontrivial amount of time. You can use this command to see if it has already been installed:

helm list -A -o json | jq -r --arg chart akv2k8s 'any(.name == $chart)'

helm list -A -o json: Gets a list of all the Helm releases in your Kubernetes cluster.
jq -r --arg chart akv2k8s 'any(.name == $chart)'': Checks if there is a chart whose name matches akv2k8s.

An alternative to akv2k8s is Azure Key Vault CSI Driver.

Troubleshooting

To see the state of the sync:

$ kubectl -n stardog-ns get akvs
NAME          VAULT          VAULT OBJECT   SECRET NAME                       SYNCHED
secret-sync   stardog-ns     my-secret      certificate-sync-bi-stardog-dev   True

If you do not have a Secret Name, there is something wrong. Common errors are a bad AKV name, 401, 403.

Irrespective of the issue, looking at the logs is useful when troubleshooting. You can find them using the following command:

kubectl -n akv2k8s logs deployment/akv2k8s-controller

For more information on logging in akv2aks, see here.

Conclusion

Now we have Azure Key Vault with the certificates loaded and the mechanism to synchronize them to the K8S secret in place.

The next step is to apply rules to synchronize them, but that will be done as part of the Stardog deployment.

Cleaning up

You can now delete your Azure resource group via the following command:

az group delete --resource-group $AKV_RESOURCE_GROUP -y

Overview
Prerequisites
Setup
- Setting up the environment
Setting up the AKV
- Creating an AKV
- Granting AKS access to AKV
Certificates
Troubleshooting
Conclusion
Cleaning up