Few months before, I have written a Kubernetes .net controller that uses Azure Kubernetes Service as a platform to host Azure pipeline agents. It allows elastic scale-out model by continuously watching the job queue in Azure DevOps. You can read about it here. I have received quite a few messages from others who are using it. Most of them are running it with some modifications that applies to their circumstances without any issues. Few had issues though. In this update I want to share some updates that I accumulated during the past months.
Following are the list of updates that I made in recent days:
- The project has been migrated to DOTNET 6
- The pod watcher implementation has changed to adopt a very lean version of Kubernetes Informer. Inspired from YARP.
- Added support for Azure Application Insights to monitor the agent responses from controller. See the updated YAML example with new environment variables for Application Insights connection strings.
apiVersion: apps/v1 kind: Deployment metadata: name: octolamp-agent-controller namespace: octolamp-system spec: replicas: 1 selector: matchLabels: run: octolamp-agent-controller template: metadata: labels: run: octolamp-agent-controller spec: serviceAccountName: octolamp-service-account containers: - env: - name: AZDO_ORG_URI value: https://dev.azure.com/<Organization Name> - name: AZDO_TOKEN value: <A PAT token that can manage agent pool> - name: AZDO_POOLNAME value: "k8s-pool" - name: TARGET_NAMESPACE value: "azdo-agents" - name: STANDBY_AGENT_COUNT value: "2" - name: MAX_AGENT_COUNT value: "25" - name: APPINSIGHT_CONN_STR value: < Application Insight connection string (not instrumentation key!!)> image: moimhossain/octolamp-agent-controller:net6-v1.0.0 imagePullPolicy: Always name: octolamp-agent-controller resources: limits: cpu: 100m memory: 100Mi
Some of you noticed that after running it for a while, there too many jobs in queue and idle agents but the jobs are still waiting for a long time before gets queued. This is most likely because you have not purchased parallel jobs in Azure Jobs. Often the default is 3, hence the 4th job will be waiting even though the controller has created agents for them.
The entire code can be found in GitHub. This source code is MIT licensed, provided as-is (without any warranties) and you can use, modify without issues. However, I would appreciate if you acknowledged the author. Also, more than welcome to contribute directly to GitHub.