Last time, I created an empty Azure Container Apps Environment using Bicep. I also created some required components like Key vaults, Log analytics workspace, Application Insights and Computer vision.
Application Architecture
Today I want to make progress on that. As I have briefly explained before, I want to have a frontend where users will upload photos and the frontend will store the image into the storage account as a blob. And here I want to draw my microservice boundary. I want another service to be triggered every time a new blob has been added to that storage account.
This particular scenario is relatively common (I alone been into this kind of setup multiple time in past few years), where we want to get a shoulder tap when a blob has been uploaded (or modified). Many often deal such cases in Azure Functions (or Web Jobs) and go with the blob binding feature.
Few years before, I learned a lesson not to use blob triggers in any serious application, Azure Function’s blob trigger feature scans the container to track changes and if you have a large blob container these scanning quickly becomes expensive. Hence, my preferred method in such cases is to leverage Azure Event Grid Topic and redirect the events to either Azure Storage Queue or Azure service bus queue or topics. In this learning exercise, I will be using service bus topics. Then my backend microservice can consume that service bus topic for listening any blob changes in the blob container.
There are quite a few things we need then:
- Storage Account
- A blob container in that account
- Event grid topic that listens to the storage account
- Managed Identity will be used by the Event grid topic (so I won’t have to spread the storage account keys)
- Service Bus Namespace and a topic
- Configure Event grid topic to dispatch the messages to the service bus topic.
- Assign roles to the Event grid managed identity so it can access both the storage account and the service bus.
Seems like I will be writing some Bicep again. Let’s do it.
Storage Account and Container
Here’re the interesting bits of the Bicep snippets, the entire source code can be found into the GitHub repository.
resource mainstorage 'Microsoft.Storage/storageAccounts@2021-02-01' = {
name: accountName
location: location
kind: 'StorageV2'
sku: {
name: 'Standard_LRS'
}
properties: {
minimumTlsVersion: 'TLS1_2'
supportsHttpsTrafficOnly: true
}
}
resource mainstoragecontainer 'Microsoft.Storage/storageAccounts/blobServices/containers@2021-02-01' = {
name: '${mainstorage.name}/default/${containerName}'
}
Event Grid Topic with Managed Identity
resource eventGridSystemTopic 'Microsoft.EventGrid/systemTopics@2022-06-15' = {
name: eventGridSystemTopicName
location: location
identity: {
type: 'SystemAssigned'
}
properties: {
source: storageAccount.id
topicType: 'Microsoft.Storage.StorageAccounts'
}
}
Service Bus Namespace and Topic
Let’s create our service bus namespace and topic now.
resource sbNamespace 'Microsoft.ServiceBus/namespaces@2022-01-01-preview' = {
name: serviceBusNamespace
location: location
sku: {
name: 'Standard'
tier: 'Standard'
}
properties: {
publicNetworkAccess: 'Enabled'
disableLocalAuth: false
zoneRedundant: false
minimumTlsVersion: '1.2'
}
}
resource topic 'Microsoft.ServiceBus/namespaces/topics@2022-01-01-preview' = {
name: '${serviceBusNamespace}/${serviceBusTopicName}'
dependsOn: [
sbNamespace
]
properties: {
maxMessageSizeInKilobytes: 256
defaultMessageTimeToLive: 'P14D'
maxSizeInMegabytes: 1024
requiresDuplicateDetection: false
duplicateDetectionHistoryTimeWindow: 'PT10M'
enableBatchedOperations: true
status: 'Active'
supportOrdering: true
autoDeleteOnIdle: 'P10675199DT2H48M5.4775807S'
enablePartitioning: false
enableExpress: false
}
}
Dispatch Event grid messages to Service Bus topic
I was impressed that Azure is embracing the Cloud event schema for the event payloads. CloudEvents simplifies interoperability by providing a common event schema for publishing, and consuming cloud-based events. This schema allows for uniform tooling, standard ways of routing & handling events, and universal ways of deserializing the outer event schema. With a common schema, you can more easily integrate work across platforms.
resource eventgridTopicSubscription 'Microsoft.EventGrid/systemTopics/eventSubscriptions@2022-06-15' = {
name: '${eventGridSystemTopic.name}/xenielsubscription'
properties: {
eventDeliverySchema: 'CloudEventSchemaV1_0'
destination: {
endpointType: 'ServiceBusTopic'
properties: {
resourceId: topic.id
}
}
filter: {
includedEventTypes: [
'Microsoft.Storage.BlobCreated'
'Microsoft.Storage.BlobDeleted'
]
enableAdvancedFilteringOnArrays: true
}
retryPolicy: {
maxDeliveryAttempts: 30
eventTimeToLiveInMinutes: 1440
}
}
}
Role Assignments
We will be assigning appropriate roles to the Event Grid’s managed identity so that I can speak to storage account and service bus.
@description('This is the built-in Contributor role. See https://docs.microsoft.com/azure/role-based-access-control/built-in-roles')
var roleDefinitionId = 'b24988ac-6180-42a0-ab88-20f7382dd24c'
resource roleDefinition 'Microsoft.Authorization/roleDefinitions@2022-04-01' existing = {
name: roleDefinitionId
}
resource roleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
scope: storageAccount
name: guid(eventGridSystemTopicName)
properties: {
roleDefinitionId: roleDefinition.id
principalId: eventGridSystemTopic.identity.principalId
}
}
resource roleAssignmentForSystemIdentity 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
scope: serviceBus
name: guid('${eventGridSystemTopicName}-managedidentity')
properties: {
roleDefinitionId: roleDefinition.id
principalId: eventGridSystemTopic.identity.principalId
}
}
At this point, I have the necessary infrastructure, where if I upload a blob into the container (from Azure portal) I will see a corresponding ‘event’ message into the service bus topic.
With that I can focus on my backend application that should listen for those service bus event messages. And here comes my first Dapr component.
Dapr Pub-sub
I will be using a PubSub component from Dapr that allows me to listen for messages in an Azure Service Bus topic. Generally, I used to create Dapr component as YAML syntax and they were just Kubernetes CRDs (Customer Resource Definition). However, in Azure Container Apps we need to put that in Bicep syntax, which is merely a trivial translation exercise. Here is what it looks like:
resource environment 'Microsoft.App/managedEnvironments@2022-03-01' existing = {
name: acaEnvName
}
resource storageQueueBindingComponent 'Microsoft.App/managedEnvironments/daprComponents@2022-03-01' = {
name: componentName
parent: environment
properties: {
componentType: 'pubsub.azure.servicebus'
version: 'v1'
ignoreErrors: false
initTimeout: '5s'
metadata: [
{
name: 'namespaceName'
// NOTE: Dapr expects just the domain name.
value: replace(replace(serviceBusEndpoint, 'https://', ''), ':443/', '')
}
{
name: 'azureClientId'
value: managedIdentityClientId
}
{
name: 'timeoutInSec'
value: '60'
}
]
scopes: appScopes
}
}
To be continued
That’s all for now. In the next part, I will be moving on configuring Dapr secret stores. The entire source code can be found in GitHub – if you want to take a look.
Thanks for reading.
Reverse Engineering 