Part-1 Introduction and deployment of a webjob into Azure

In this article, we will learn how to deploy a web-job to the Azure platform using PowerShell script. Many of you might have already used other ways such as Local Git, FTP, OneDrive Sync, Azure DevOps (formerly VSTS online), Dropbox Sync etc.

You might also have used Visual Studio SDK and publish wizard which is a GUI based easy way that enables you to perform this task through several dialog windows. What if you have hundreds or thousands of web applications to be deployed on the cloud?

Well, you may not want to spend a lot of time on clicking dialog boxes, instead, you may want an easy script to run on few key presses or few clicks. So here we are !!

Microsoft Windows platform has already given us Windows PowerShell ISE desktop app to perform such tasks much similar to cmd prompt. We can also install Azure CLI tool which allows us to run easy short commands for deployment.

Here I am assuming that viewers are already aware of Azure portal services and .Net core or simple console or web application is written in .Net.

Before jump into the deployment process, you may want to understand what Azure app services, Web Apps and Web jobs are. Because these are the key points to deploy a webjob.

Azure App service

Azure App service is a hosted service and a fully manages PaaS (Platform as a service) or in simple terms, we can say It’s a service to host our multiple types of applications and APIs on the cloud, for example, Web Application, Rest APIs, Mobile backends, and we can use .Net, PHP, Python, Java, HTML both for Windows and Linux Operating Systems.

You can find more details on Microsoft documentation :

https://docs.microsoft.com/en-us/azure/app-service/

You can find App service by searching on the Azure portal as below:

If you click on Add service button, you will see several services provided by Azure to host your applications or projects.

Web Job

Web job is a feature in Azure app service that gives you the facility to run your program or script using the same infrastructure as a web app, API app or the mobile app. Web jobs are a great way to run your asynchronous process such as batch data. Therefore, it enables you to run a long-running process without worrying about it being terminated prematurely.

So basically, a web job is a service located within an Azure App service suite and allow us to carry out scheduled or continuous tasks just like an old windows service.

Triggered Jobs can be run one time or periodically until an expiration date. It can be executed through a request with a script, but it has to be triggered through an endpoint. It cannot be remotely debugged and always requires a VM or web app. Another type is a scheduled webjob, which is actually a triggered webjob with a schedule CRON expression. The schedule is defined in a settings.json file that sits alongside your webjob executable.

Continuous job starts immediately when the job is created. It can be debugged and always requires VM or WebApp. To keep the job from ending, the program or script typically does its work inside an endless loop.

WebApp

WebApp is a part of Azure App services, it is a platform which enables you to build and host applications on Azure without managing infrastructure with lots of advantages. If you click on webApp on Azure, you will see a better description as below :

Azure App Service Plan

App service plan is a container for WebApps , which means It provides a set of resources to run a webApp. When you create an app service plan in a region, a set of compute resources is also created for that plan in that region. One or more apps can be configured to run on the same computing resources or on the same App service plan. The pricing tier of an App Service plan determines what App Service features you get and how much you pay for the plan.

For more details please visit the following Microsoft documentation:

https://docs.microsoft.com/en-us/azure/app-service/azure-web-sites-web-hosting-plans-in-depth-overview

Creating a webjob from scratch

*First of all, we need an Azure account, if you do not have one, then you create from this URL: https://azure.microsoft.com/en-us/free/

*We require Windows PowerShell ISE, which is already there in our windows OS Apps, or we can use Azure CLI which we have to download and install. It can be downloaded from URL: https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest

Azure CLI is installed via an MSI Installer gives you access to the CLI through the Windows Command Prompt (CMD) or PowerShell.

*We also require Azure storage explorer tool which we will use to test our storage account for testing.

Setup before deploying a webjob

There are a number of activities to be performed before deploying a webjob. We will quickly understand this by the following diagram-

Let’s start the setup. Run these commands on Azure CLI tool–

1- Login Azure account

az login

This command will open a tab in the browser where you can log in to your Azure account and you can close the tab after successful login.

On successful log-in, you will see your Azure account and subscription details. Below the screenshot for this command :

2- Create a resource group

az group create --name TestAnuRG1 --location "WestEurope"

az group create -l WestEurope -n TestAnuRG1

This command will create a resource group with the name “TestAnuRG1” in WestEurope location.

We can verify this resource group on our Azure account, alternatively we can create this from azure portal in few clicks. Below is the screen in azure portal, yes, we got the newly created ResourceGroup.

3- Create a storage account

az storage account create -n anuvermastorage1 -g TestAnuRG1 -l westeurope --sku Standard_LRS

Give a unique name to the storage account with other parameters, which are - The name of the resource group and location with sku details. Storage account name must be between 3 and 24 characters in length and use numbers and lower-case letters only.

We can see, a storage account has been created on our Azure account

4- Create an app service plan, which will host our application

az appservice plan create -n AzureAnuVermaServicePlan -g ` TestAnuRG1 -l WestEurope

5- Create a webApp

az webapp create -n AnuVermaWebApp1 -g TestAnuRG1 -p AzureAnuVermaServicePlan

This command will create a webApp in the specified app service plan and resource group. It will create a default host for us. See below screenshot.

On Azure portal we can see, now we have AppService plan, webApp and a storage account as below:

6- Configure the AppService by setting webApp as always-on

az webapp config set --always-on true --php-version Off -n AnuVermaWebApp1 -g TestAnuRG1

7- Create the configuration connection string

Set Application Setting, Connection string AzureWebJobsStorage and AzureWebJobsDashboard

Currently, we have an empty connection string. We have to create it.

As per Microsoft MSDN document :

These next two settings can be tricky if you don’t understand how Azure App Services handles settings.

The default templates for .NET projects can and do utilize App.Config files AND they set these values to empty strings. With that in mind, you make the changes in App.Config or delete the entries (because they override the portal settings) and set these in the dashboard. In other words, if you define this in the Azure Portal, but do not take the settings out in the App.Config, you are overriding these settings with blank values in the App.Config probably without realizing it.
Connection string AzureWebJobsDashboard might be optional.

Note: If you do not add this setting and this is a project you created in Visual Studio using the WebJobs SDK, your WebJob will fail to run. If your project is NOT created and deployed as a WebJob, your WebJob may still run however you will get a 404-error page when you try to visit the WebJob logs page: https://portal.azure.com/Error/UE_404?aspxerrorpath=/null&shown=true.

You can still get to the portal however if you navigate to the WebJobs Log in Kudu:

https://WEBAPPNAMECONTAINGWEBJOBHERE.scm.azurewebsites.net/azurejobs/#/jobs

Connection string AzureWebJobsStorage is optional but if you are using the Azure Web Jobs SDK, this is required. This is how the SDK does the magic of wiring up Queues, Tables and Blobs for use in your WebJob.

(i) Get the Connection String

az storage account show-connection-string -n anuvermastorage1 -g TestAnuRG1

(ii) Create the Configurations in the Web App

In previous command(i) You will get a connection string value.

Syntax:

az webapp config connection-string set --connection-string-type Custom --settings AzureWebJobsDashboard=<connection string> AzureWebJobsStorage=<connection string> -g <resourcegroupName> -n <WebAppName>

I ran the following command using my connection string :

az webapp config connection-string set --connection-string-type Custom –settings AzureWebJobsDashboard="DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=anuvermastorage1;AccountKey=3Iyn2SfIdgPvzCyXpBBN4WFL+OyLWndGrQT1VRPEOkTe+UnCGzecO209fsFdaRyjtS6Fizyb9Qjh1oRmA7hfxw==" AzureWebJobsStorage="DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=anuvermastorage1;AccountKey=3Iyn2SfIdgPvzCyXpBBN4WFL+OyLWndGrQT1VRPEOkTe+UnCGzecO209fsFdaRyjtS6Fizyb9Qjh1oRmA7hfxw==" -g TestAnuRG1 -n AnuVermaWebApp1

Below is the screenshot of how my command worked:

So now we have successfully created both the connection strings. As we can see on the portal:

Where is our WebJobs stored on Azure?

A WebJob is stored under the following directory in your site on Azure:

site\wwwroot\App_Data\jobs\{job
type}\{job name}

Where {job type} can be either continuous for a job that is always running or triggered for a job that starts from an external trigger (on demand / scheduler).

And {job name} is your WebJob's name.

So if I named my continuous WebJob as myjob1, then it will be located on Kudu portal at :

site\wwwroot\App_Data\jobs\continuous\myjob1

So, if I named my triggered WebJob as myjob2, then it will be located on Kudu portal at :

site\wwwroot\App_Data\jobs\triggered\myjob2

Basically, the URL constructed by the name of your webApp followed by “scm.azurewebsites.net” as below:

https://<your


web app name>.scm.azurewebsites.net/

for example ,
in my case it is:

https://anuvermawebapp1.scm.azurewebsites.net/

Right now, I do not have any webjob in my webApp, we can see [] webjobs (empty array) on our portal by going on below options-

WebApp-> Go to Advanced Tool-> Click on Go-> Kudu service -> WebJobs->All

Here you see [] (empty array), means no web job deployed yet.

So now we are ready to create our web application. And deploy our project with above configured setup as a webjob on azure.

Part-2 Deploying a webjob with Windows PowerShell ISE

In this section, We will deploy our webjob using Windows Powershell ISE command. This command line utility already comes with the Windows Operating system. Just search by name, and open it.

Here, I have a simple .net console application “GetDesiredDate”, that I am going to deploy on Azure.

I have written some code to get a desired date on program.cs class file.

I placed this project in my E drive :

We need a resource group, webapp, and app service plan, which we have already created in previous demo part-1 of this article.

Step -1 Open the Windows PowerShell ISE app and login with the following command-

az login

This command will open a tab in the browser and we have to login on our azure account.

Step-2 Define all the variables

$resourceGroup = "TestAnuRG1"

$location = "West Europe"

$appName = "AnuVermaWebApp1"

$planName = "AzureAnuVermaServicePlan"

Step-3 Get the credentials for deployment

# get the credentials for deployment

$user = az webapp deployment list-publishing-profiles -n $appName -g ` $resourceGroup --query "[?publishMethod=='MSDeploy'].userName" -o tsv

$pass = az webapp deployment list-publishing-profiles -n $appName -g ` $resourceGroup --query "[?publishMethod=='MSDeploy'].userPWD" -o tsv

Step-4 Build and publish our project

Store the path of your project in a variable and publish it using dotnet publish command in release mode as below:

$publishFolder = "E:\test code\GetDesiredDate\GetDesiredDate"

dotnet publish $publishFolder -c Release

This command will create an app.publish profile in release folder as below:

Note: In case of .Net Core Application this path will be something like:

E:\test code\coreApplication1\coreApplication1\bin\Release\netcoreapp2.1\publish

In such cases, while deployment we have to provide the path of this publish folder.

Step-5 Deploy our first webjob using kudu webjob API

Provide the path of this app.publish folder created in the previous step.

We simply need to provide AUTHORIZATION and CONTENT-DISPOSITION header.

# make the zip for main publish folder

$task1zip = "task1.zip"

if(Test-path $task1zip) {Remove-item $task1zip}

[io.compression.zipfile]::CreateFromDirectory("E:\test code\GetDesiredDate\GetDesiredDate\bin\Release\app.publish\", $task1zip)

$ZipHeaders = @{

Authorization = $basicAuthValue

"Content-Disposition" = "attachment; filename=run.cmd"

}

Deployment on Azure-

1- Triggered webjob

We will create a triggered WebJob using kudu API as follows, here we will use the PUT verb to upload the content on Kudu.

# upload the job using the Kudu WebJobs API

Invoke-WebRequest -Uri https://$appName.scm.azurewebsites.net/api/triggeredwebjobs/Webjob1 -Headers ` $ZipHeaders -InFile $task1zip -ContentType "application/zip" -Method Put

We got the status OK, now we can check the same on our Azure portal :

2- Similarly, we can create a continuous Webjob using the following command:

Invoke-WebRequest -Uri https://$appName.scm.azurewebsites.net/api/continuouswebjobs/anuWebJob1 -Headers $ZipHeaders -InFile $task1zip -ContentType "application/zip" -Method Put

To check it worked, you can visit the Kudu portal and explore the contents of the app_data folder or look at the web jobs page.

# launch Kudu portal

Start-Process https://$appName.scm.azurewebsites.net

It will open the following screen in the web browser.

To verify the deployment on PowerShell prompt, we can run the following command to get all triggered webjobs inside our webApp using GET verb as follows:

# get triggered jobs

Invoke-RestMethod -Uri https://$appName.scm.azurewebsites.net/api/triggeredwebjobs -Headers $Headers -Method Get

Similarly, we can search our Continuous webjob-

#Get Continuous Webjobs

Invoke-RestMethod -Uri https://$appName.scm.azurewebsites.net/api/continuouswebjobs -Headers $Headers -Method Get

So, in the above screen, we can verify that we have successfully deployed our webjobs.

Initial Status of the jobs-

Step- 6 Run the triggered web job

To run the webjob we can post to the run endpoint for this triggered webjob. And we can optionally pass arguments in the query string. Don't forget to provide the content type or you'll get a 403 error.

# run the job

$resp = Invoke-WebRequest -Uri "https://$appName.scm.azurewebsites.net/api/triggeredwebjobs/Webjob1/run?arguments=arg1" -Headers $Headers -Method Post -ContentType "multipart/form-data"

This command runs our triggered webjob.

Now we can see successfully deployed and running webjobs.

Step -7 Log History

If we want to know the log history, we can use the following command-

# output response includes a Location to get history:

if ($resp.RawContent -match "\nLocation\: (.+)\n")

{

$historyLocation = $matches[1]

$hist = Invoke-RestMethod -Uri $historyLocation -Headers $Headers -Method Get

# $hist has status, start_time, end_time, duration, error_url etc

# get the logs from output_url

Invoke-RestMethod -Uri $hist.output_url -Headers $Headers -Method Get

}

Step – 8 Run a scheduled Job – A simple .net application

Scheduled jobs are triggered jobs, but they are deployed to run periodically just like applications running from windows scheduler. We specify a CRON expression to tell the Azure system as to how to run and when to run the job.

we're using a regular .NET console app running on the regular .NET framework. Here's the code:

We'll build it with MSBuild and create a zip, very similar to what we did with the first webjob:

Locate your msbuild.exe utility in your file explorer -> Program Folder (x86) under Visual studio.

In my case I have VS community version, but you may have other versions for example Enterprise.

In my case I found it in following directory:

C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\msbuild.exe

Run the following command :

# build the regular .net webjob

$msbuild = "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\msbuild.exe"

. $msbuild "E:\test code\SimpleConsoleApp1\SimpleConsoleApp1\SimpleConsoleApp.csproj" /property:Configuration=Release

This build creates a Release folder :

Now run the below command to create a zip folder(task2.zip) of published files, Provide the source path as release folder’s path :

$task2zip = "task2.zip"

if(Test-path $task2zip) {Remove-item $task2.zip}

[io.compression.zipfile]::CreateFromDirectory("E:\test code\SimpleConsoleApp1\SimpleConsoleApp1\bin\Release\", $task2zip)

This creates a zip folder in C:\Users\AnuVerma\

You can give any other folder path as well, as we are creating a zip of release folder’s file so do not give the release folder path as a destination, otherwise, it may end up in error of [can’t access files used by other process]

Now deploy the job :

# upload the web job

$ZipHeaders = @{

Authorization = $basicAuthValue

"Content-Disposition" = "attachment; filename=SimpleConsoleApp1.exe"

}

#Delpoy the job

Invoke-WebRequest -Uri https://$appName.scm.azurewebsites.net/api/triggeredwebjobs/myScheduledJob1 -Headers $ZipHeaders -InFile $task2zip -ContentType "application/zip" -Method Put

We got a success message-

We successfully deployed our webjob, It’s in the Ready state, but it is not scheduled yet. We have to set a CRON expression to provide it a schedule to run periodically.

Here we haven’t provided setting.json with a cron expression in our zip file, but if we'd included this, then this would already be a scheduled job with nothing further to do.

But yet we don’t have it, so let us push the contents of the settings file, which we can use to set the schedule. Here we'll set up our third webjob to run every 1 minute, as follows:

Now, this myScheduledjob1 will run in every 1 minute.

In the above screen, you can see, the status is now running for myscheduledJob1.

In Logs we can see it’s running history, we can also toggle output by clicking on the running Job-

Navigating through the webjobs on Kudu

If we click on site , we can navigate to our deployed web jobs as below screenshot :

Step-9 Update and deleting webjobs:

Use the following command to update or delete the webjobs:

Invoke-WebRequest -Uri https://$appName.scm.azurewebsites.net/api/triggeredwebjobs/WebJob2 -Headers $Headers -Method Delete

For update-> -Method Update

For Delete -> -Method Delete

Note: Please use ` (tick symbol) in the end of the line in PowerShell ISE if there is a line break in the code. It’s a continuation character.

Happy Learning with Azure webjobs !! 😊 Good Luck !

Anu V.

Anu's Blog Post

Wednesday, 17 March 2021

Import data to Azure Cosmos DB using Azure DocumentDB Data Migration Tool

Tuesday, 20 November 2018

Introduction and deployment of a Webjob into Azure