Solr cloud in AKS (dev)

Jan 27, 2018

 solr containers azure  indexing  docker  kubernetes

This example describes how to run Solr cloud cluster + zookeeper ensemble in Kubernetes on Azure (AKS). Since Solr cloud relies on Zookeeper to manage configuration and data storage, two containers need to be used in this example: Zookeeper (Zk) and Solr.

Provided example was tested with AKS cluster version 1.8.2, Solr 6.6.2 and Zookeeper 3.4.11.

TL;TR

Scripts and configs:

Create Kubernetes cluster

Follows instructions in [AKS tutorial][aks-tutorial] or my previous post. I used AKS cluster version 1.8.2.

Build Zookeeper Docker image

Using Dockerfile Zk build Zk Docker image and upload to your Docker image repository that is accessible to AKS. I used Azure Container Registry (ACR).

[Optional] Build Solr Docker image

This example uses solr:6.6-alpine image from official solr repo image. If you want to pack custom configuration into Solr image, you need to build a custom image and upload into the image registry (e.g. ACR). See example in in flexible solr container article.

Configure AKS secret to pull images

If you store images in a private registry, you need to create a Kubernetes secret with credentials that will be used to pull images.

[Optional] Create service principal to pull image from ACR

If you use ACR to host Docker images, you may want to create a service principle account that would be used to pull images from the registry. See full example at AKS docs.

ACR_REGISTRY_ID=$(az acr show --name myAcrName --query id --output tsv)
SP_PASSWORD=$(az ad sp create-for-rbac --name my-acr-sp --role Reader --scopes $ACR_REGISTRY_ID --query password --output tsv)
CLIENT_ID=$(az ad sp show --id http://my-acr-sp --query appId --output tsv)
echo "Service principal ID: $CLIENT_ID"
echo "Service principal password: $SP_PASSWORD"

Record CLIENT_ID and SP_PASSWORD values as they need to be used to create a Kubernetes secret in order to pull images from ACR.

Create secret

Replace variables wrapped in <> in the following command with corresponding values from your environment.

kubectl create secret docker-registry acr-auth --docker-server <acr-login-server> --docker-username <CLIENT_ID> --docker-password <SP_PASSWORD> --docker-email <email-address>

Verify that your acr-auth secret got created in AKS.

kubectl get secret

Create Zookeeper ensemble

Use zookeeper.yaml to create Zk ensemble in AKS.

kubectl apply -f zookeeper.yaml

Check whether Zk pods got created.

kubectl get pods

Create Solr cloud cluster

Use solrcloud.yaml to create Solr cluster in AKS.

kubectl apply -f solrcloud.yaml

Verify that pods got created and running.

Test Solr cloud access

Make sure sure solrsvc service of type LoadBalancer got created and assigned an external IP. It may take a few minutes for Azure to allocate and assign an IP to the service.

kubectl get svc

Use load balancer IP to access Solr instance: http://<LB_IP>:8983/solr

Creating indexes

To make sure everything is running fine, you can create a sample index using one of default configsets supplied with Solr dist (e.g. basic_configs).

Upload basic_configs configuration into Zk

In order to create an index in Solr cloud, you must have a configuration uploaded into Zk.

Get Solr and Zk pods names

kubectl get pods

For example, pod names are solrapp-0 and zk-0.

Get Zk instance full name

kubectl exec zk-0 -- hostname
kubectl exec zk-0 -- sh -c 'echo $(hostname -d)'

Upload configset from a known path into Zk

Each Solr distro has a few predefined configsets located at /opt/solr/server/solr/configsets path.

kubectl exec solrapp-0 -- solr zk -upconfig -d /opt/solr/server/solr/configsets/basic_configs -n basic -z zk-0.zk.default.svc.cluster.local:2181

Create sample index

curl http://<solr-host>:8983/solr/admin/collections?action=CREATE -d name=basic -d numShards=3 -d replicationFactor=2 -d maxShardsPerNode=2 -d collection.configName=basic

Tips & tricks

Usefull commands and troubleshooting tips.

Create short alias for kubectl command

To minimize number of key strokes to execute AKS commands, consider creating a shorter alias for kubectl command.

alias k=kubectl
k get pods

Assign existing service principal to Azure resource

ACR_ID=$(az acr show --name isregistry --query id -o tsv)
$ az role assignment create --assignee <SP_ID> --role 'Reader' --scope $ACR_ID

Verify service principal is assigned to the ACR

ACR_ID=$(az acr show --name isregistry --query id -o tsv)
az role assignment list --scope $ACR_ID -o table

Get events for specific pod

kubectl get ev | grep <podName>

Get Zk container mode

kubectl exec <podName> -- bin/zkServer.sh status

Get container hostname and DNS

kubectl exec <podName> -- sh -c hostname
kubectl exec <podName> -- sh -c 'echo $(hostname -d)'

Resources

 Toc
 Tags
 Related