Continuously deploy Blazor SPA to Azure Storage static web site

Lately I am learning ASP.net Blazor – the relatively new UI framework from Microsoft. Blazor is just awesome – the ability to write c# code both in server and client side is extremely productive for .net developers. From Blazor documentations:

Blazor lets you build interactive web UIs using C# instead of JavaScript. Blazor apps are composed of reusable web UI components implemented using C#, HTML, and CSS. Both client and server code is written in C#, allowing you to share code and libraries.

I wanted to write a simple SPA (Single Page Application) and run it as server-less. Azure Storage offers hosting static web sites for quite a while now. Which seems like a very nice option to run a Blazor SPA which executes into the user’s browser (within the same Sandbox as JavaScript does). It also a cheap way to run a Single Page application in Cloud.

I am using GitHub as my source repository for free (in a private repository). Today wanted to create a pipeline that will continuously deploy my Blazor app to the storage account. Azure Pipelines seems to have pretty nice integration with GitHub and it’s has a free tier as well . If either our GitHub repository or pipeline is private, Azure Pipeline still provide a free tier. In this tier, one can run one free parallel job that can run up to 60 minutes each time until we’ve used 1800 minutes per month. That’s pretty darn good for my use case.

I also wanted to build the project many times in my local machine (while developing) in the same way it gets built in the pipeline. Being a docker fan myself, that’s quite no-brainer. Let’s get started.

Pre-requisite

I have performed few steps before I ran after the pipeline – that are beyond the scope of this post.

  • I have created an Azure Subscription
  • Provisioned resource groups and storage account
  • I have created a Service Principal and granted Contributor role to the storage account

Publishing in Docker

I have created a docker file that will build the app, run unit tests and if all goes well, it will publish the app in a folder. All of these are standard dotnet commands.
Once, we have the application published in a folder, I have taken the content of that folder to a Azure CLI docker base image (where CLI is pre-installed) and thrown away the rest of the intermediate containers.

Here’s our docker file:

The docker file expects few arguments (basically the service principal ID, the password of the service principal and the Azure AD tenant ID – these are required for Azure CLI to sign-in to my Azure subscription). Here’s how we can build this image now:

Azure Pipeline as code

We have now the container, time to run it every time a commit has been made to the GitHub repository. Azure Pipeline has a yaml format to define pipeline-as-code – which is another neat feature of Azure Pipelines.

Let’s see how the pipeline-as-code looks like:

I have committed this to the same repository into the root folder.

Creating the pipeline

We need to login to Azure DevOps and create a project (if there’s none). From the build option we can create a new build definition.

ado

The steps to create build definition is very straightforward. It allows us to directly point to a GitHub repository that we want to build.
Almost there. We need to supply the service principal ID, password, tenant ID and storage account names to this pipeline – because both our docker file and the pipeline-as-code expected them as dependencies. However, we can’t just put their values and commit them to GitHub. They should be kept secret.

Azure Pipeline Secret variables

Azure Pipeline allows us to define secret variable for a pipeline. We need to open the build definition in “edit” mode and then go to the top-right ellipses button as below:

ado1

Now we can define the values of these secret and keep them hidden (there’s a lock icon there).

ad02

That’s all what we need. Ready to deploy the code to Azure storage via this pipeline. If we now go an make a change in our repository it will trigger the pipeline and sure enough it will build-test-publish-deploy to Azure storage as a Static SPA.

Thanks for reading!

Resilient Azure Data Lake Analytics (ADLA) Jobs with Azure Functions

Azure Data Lake Analytics is an on-demand analytics job service that allows writing queries to transform data and grab insights efficiently. The analytics service can handle jobs of any scale instantly by setting the dial for how much power you need.

JObs

In many organizations, these jobs could play a crucial role and reliability of these job executions could be business critical. Lately I have encountered a scenario where a particular USQL job has failed with following error message:

Usql – Job failed due to internal system error – NM_CANNOT_LAUNCH_JM

A bit of research on Google revealed, it’s a system error, which doesn’t leave a lot of diagnostic clue to reason out. Retrying this job manually (by button clicking on portal) yielded success! Which makes it a bit unpredictable and uncertain. However, uncertainty like this is sort of norm while developing Software for Cloud. We all read/heard about Chaos Monkeys of Netflix.

What is resiliency?

Resiliency is the capability to handle partial failures while continuing to execute and not crash. In modern application architectures — whether it be micro services running in containers on-premises or applications running in the cloud — failures are going to occur. For example, applications that communicate over networks (like services talking to a database or an API) are subject to transient failures. These temporary faults cause lesser amounts of downtime due to timeouts, overloaded resources, networking hiccups, and other problems that come and go and are hard to reproduce. These failures are usually self-correcting. (Source)
Today I will present an approach that mitigated this abrupt job failure.

The Solution Design

Basically, I wanted to have a job progress watcher, waiting to see a failed job and then resubmit that job as a retry-logic. Also, don’t want to retry more than once, which has potential to repeat a forever-failure loop. I can have my watcher running at a frequency – like every 5 minutes or so.

Azure Functions

Azure Functions continuously impressing me for its lightweight built and consumption-based pricing model. Functions can run with different triggers, among them time schedule trigger- that perfectly fits my purpose.

Prerequisites

The function app needs to retrieve failed ADLA jobs and resubmit them as needed. This can be achieved with the Microsoft.Azure.Management.DataLake.Analytics, Version=3.0.0.0 NuGet package. We will also require Microsoft.Rest. ClientRuntime.Azure.Authentication, Version=2.0.0.0 NuGet package for Access Token retrievals.

Configuration

We need a Service Principal to be able to interact with ADLA instance on Azure. Managed Service Identity (written about it before) can also be used to make it secret less. However, in this example I will use Service Principal to keep it easier to understand. Once we have our Service Principal, we need to configure them in Function Application Settings.

Hacking the function

[FunctionName("FN_ADLA_Job_Retry")]

public static void Run([TimerTrigger("0 0 */2 * * *")]TimerInfo myTimer, TraceWriter log)

{

var accountName = GetEnvironmentVariable("ADLA_NAME");

var tenantId = GetEnvironmentVariable("TENANT_ID");

var clientId = GetEnvironmentVariable("SERVICE_PRINCIPAL_ID");

var clientSecret = GetEnvironmentVariable("SERVICE_PRINCIPAL_SECRET");

 

ProcessFailedJobsAsync(tenantId, clientId, clientSecret, accountName).Wait();

}

That’s our Azure Function scheduled to be run every 2 hours. Once we get a trigger, we retrieve the AD tenant ID, Service Principal ID, secret and the account name of target ADLA.

Next thing we do, write a method that will give us a ADLA REST client – authenticated with Azure AD, ready to make a call to ADLA account.

private static async Task GetAdlaClientAsync(

string clientId, string clientSecret, string tenantId)

{

var creds = new ClientCredential(clientId, clientSecret);

var clientCreds = await ApplicationTokenProvider

.LoginSilentAsync(tenantId, creds);

 

var adlsClient = new DataLakeAnalyticsJobManagementClient(clientCreds);

return adlsClient;

}

The DataLakeAnalyticsJobManagementClient class comes from Microsoft.Azure.Management.DataLake.Analytics, Version=3.0.0.0 NuGet package that we have already installed into our project.

Next, we will write a method that will get us all the failed jobs,

private static async Task<Microsoft.Rest.Azure.IPage>

GetFailedJobsAsync(string accountName, DataLakeAnalyticsJobManagementClient client)

{

// We are ignoring the data pages that has older jobs

// If that's important to you, use CancellationToken to retrieve those pages

return await client.Job

.ListAsync(accountName,

new ODataQuery(job => job.Result == JobResult.Failed));

}

We have now the capability to retrieve failed jobs, great! Now we should write the real logic that will check for failed jobs that never been retried and resubmit them.

private const string RetryJobPrefix = "RETRY-";

public static async Task ProcessFailedJobsAsync(

string tenantId, string clientId, string clientSecret, string accountName)

{

var client = await GetAdlaClientAsync(clientId, clientSecret, tenantId);

 

var failedJobs = await GetFailedJobsAsync(accountName, client);

 

foreach (var failedJob in failedJobs)

{

// If it's a retry attempt we will not kick this off again.

if (failedJob.Name.StartsWith(RetryJobPrefix)) continue;

 

// we will retry this with a name prefixed with a RETRY

var retryJobName = $"{RetryJobPrefix}{failedJob.Name}";

 

// Before we kick this off again, let's check if we already have retried this before..

if (!(await HasRetriedBeforeAsync(accountName, client, retryJobName)))

{

var jobDetails = await client.Job.GetAsync(accountName, failedJob.JobId.Value);

var newJobID = Guid.NewGuid();

 

var properties = new USqlJobProperties(jobDetails.Properties.Script);

var parameters = new JobInformation(

retryJobName,

JobType.USql, properties,

priority: failedJob.Priority,

degreeOfParallelism: failedJob.DegreeOfParallelism,

jobId: newJobID);

 

// resubmit this job now

await client.Job.CreateAsync(accountName, newJobID, parameters);

}

}

}

private async static Task HasRetriedBeforeAsync(string accountName,

DataLakeAnalyticsJobManagementClient client, string name)

{

var jobs = await client.Job

.ListAsync(accountName,

new ODataQuery(job => job.Name == name));

 

return jobs.Any();

}

This is it all!

Final thoughts!

We can’t avoid failures, but we can respond in ways that will keep our system up or at least minimize downtime. In this example, when one Job fails unpredictably, its effects can cause the system to fail.

We should build our own mitigation against these uncertain factors – with automation.