THE SQL Server Blog Spot on the Web

Welcome to SQLblog.com - The SQL Server blog spot on the web Sign in | |
in Search

Jamie Thomson

This is the blog of Jamie Thomson, a data mangler in London working for Dunnhumby

Azure Automation Pain Points

In April of this year Microsoft announced a new offering on their Azure platform called Azure Automation (which I briefly talked about at Azure Automation – the beginning of cloud ETL on Azure?). Azure Automation allows one to run Powershell scripts on Azure.

I have been using Azure Automation for the past couple of months on my current project and, as the offering is still in preview, have been asked by the product team to share with them some experiences of using it, specifically the pain points. As is my habit I’d like to share those experiences here on my blog too and given that (a) anyone can sign up and use Azure Automation and (b) I’m not under NDA I see no reason not to do that. Hence, here is the transcript of an email I sent to a member of the Azure Automation team yesterday:


Hi,

Thank you for the invite to email you with my pain points regarding AA so far.

Biggest thing has to be the developer experience. I consider myself a developer rather than a sysadmin-type-person so the develop->deploy-> test-> fix cycle is important to me. As a developer its important that everything is source controlled and so I choose to develop my runbooks locally in my Powershell editor of choice and then upload to Azure Automation using scripts (which are, of course, written in Powershell). These same deployment scripts will be the ones that we eventually use to deploy to our production environment (when we reach that stage - we're not there yet) and are themselves source-controlled. Eventually we will get to the point where each time someone checks a change into the master branch we automatically push to the production environment (aka Continuous Deployment).

The problem I have is that Azure Automation is not well suited to this type of quick development cycle. Firstly, because there are activities in my runbooks that don't work outside of Azure Automation (i.e. Get/Set-AutomationVariable) I can't run my runbooks locally. Hence, each time I make a change to a runbook (of which I currently have 6) I run my deployment script so that I can test it. Deploying then publishing a runbook takes in the region of 20seconds, multiply that by 6 and I've got a long wait between making a change and getting to the stage where I can test that change.

Worse than that though is the time it takes to run a runbook. I know Azure Automation is currently in preview (and hence can kind of excuse what I'm about to say) but I still find the length of time that a runbook spends in the “starting” status inordinately long. I want to be able to fire off a runbook and have it start executing immediately - currently I have to wait a long time until the runbook actually starts executing (and I'm talking minutes, not seconds).

Put all that together and you've got a situation where testing just a single change to a runbook takes minutes. That is unacceptable to be honest - I need it to be seconds.

You might say that I could use the in-browser editing experience but that means I'm not editing the master copy of my runbook that goes into source control, and it doesn't solve the problem of the massive start up time.

Hope this is useful to you.

Regards
Jamie Thomson

 


 

I was asked to describe my pain points which is why that email comes over as rather derogatory however I should point out that I have found the overall capabilities of Azure Automation to be quite impressive. Its a very capable platform for automating the management of Azure resources and I am looking forward to it becoming generally available very soon.

@Jamiet

Published Tuesday, September 23, 2014 9:19 AM by jamiet
Filed under: ,

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

 

Dean said:

Hi Jamie, great article.  Did you get any feedback on this letter?  I was also wondering how the issue of debugging powershell code that only works on Azure can be achieved when they are generally modified/changed on-premise.  I'm a developer and the thought of making a change, deploying it, and then seeing what happens is just a bit scary after having lived in the various Visual Studio's over the past 10+ years....

December 2, 2014 8:45 PM
 

jamiet said:

Hi Dean,

They have been very responsive and in fact I have a conference call setup with a member of the AA team in a few weeks to discuss these issues and what they're doing to alleviate them.

"I was also wondering how the issue of debugging powershell code that only works on Azure can be achieved when they are generally modified/changed on-premise"

My tactic has been to modularise my code into many runbooks which then call each other. The "root" runbook does nothing more than call the AA specific cmdlets (i.e. Get-AzureAutomationVariable) and then passes the retrieved values into all the other runbooks which then do the actual work. This means that all of the runbooks except that root runbook are runnable from my development workstation.

JT

December 3, 2014 2:24 AM

Leave a Comment

(required) 
(required) 
Submit

This Blog

Syndication

Privacy Statement