Buck Woody : Development, Windows Azure

Cloud Computing - just get started already!

BuckWoody — Tue, 30 Oct 2012 14:43:27 GMT

OK - you've been hearing about "cloud" (I really dislike that term, but whatever) for over two years. You've equated it with just throwing some VM's in some vendor's datacenter - which is certainly part of it, but not the whole story. There's a whole world of - wait for it - *coding* out there that you should be working on. If you're a developer, this is just a set of servers with operating systems and the runtime layer (like.NET, Java, PHP, etc.) that you can deploy code to and have it run. It can expand in a horizontal way, allowing massive - and I really, honestly mean massive, not just marketing talk kind of scale. We see this every day.

If you're not a developer, well, now's the time to learn. Explore a little. Try it.

We'll help you. There's a free conference you can attend in November, and you can sign up for it now. It's all on-line, and the tools you need to code are free.

Put down Facebook and Twitter for a minute - go sign up. Learn. Do. :)

See you there. http://www.windowsazureconf.net/

Windows Azure End to End Examples

BuckWoody — Tue, 29 May 2012 13:45:59 GMT

I’m fascinated by the way people learn. I’m told there are several methods people use to understand new information, from reading to watching, from experiencing to exploring.

Personally, I use multiple methods of learning when I encounter a new topic, usually starting with reading a bit about the concepts. I quickly want to put those into practice, however, especially in the technical realm. I immediately look for examples where I can start trying out the concepts. But I often want a “real” example – not just something that represents the concept, but something that is real-world, showing some feature I could actually use.

And it’s no different with the Windows Azure platform – I like finding things I can do now, and actually use. So when I started learning Windows Azure, I of course began with the Windows Azure Training Kit – which has lots of examples and labs, presentations and so on. But from there, I wanted more examples I could learn from, and eventually teach others with. I was asked if I would write a few of those up, so here are the ones I use.

CodePlex

CodePlex is Microsoft’s version of an “Open Source” repository. Anyone can start a project, add code, documentation and more to it and make it available to the world, free of charge, using various licenses as they wish. Microsoft also uses this location for most of the examples we publish, and sample databases for SQL Server.

If you search in CodePlex for “Azure”, you’ll come back with a list of projects that folks have posted, including those of us at Microsoft. The source code and documentation are there, so you can learn using actual examples of code that will do what you need. There’s everything from a simple table query to a full project that is sort of a “Corporate Dropbox” that uses Windows Azure Storage.

The advantage is that this code is immediately usable. It’s searchable, and you can often find a complete solution to meet your needs. The disadvantage is that the code is pretty specific – it may not cover a huge project like you’re looking for. Also, depending on the author(s), you might not find the documentation level you want.

Link: http://azureexamples.codeplex.com/site/search?query=Azure&ac=8

Tailspin

Microsoft Patterns and Practices is a group here that does an amazing job at sharing standard ways of doing IT – from operations to coding. If you’re not familiar with this resource, make sure you read up on it. Long before I joined Microsoft I used their work in my daily job – saved a ton of time. It has resources not only for Windows Azure but other Microsoft software as well.

The Patterns and Practices group also publishes full books – you can buy these, but many are also online for free. There’s an end-to-end example for Windows Azure using a company called “Tailspin”, and the work covers not only the code but the design of the full solution. If you really want to understand the thought that goes into a Platform-as-a-Service solution, this is an excellent resource.

The advantages are that this is a book, it’s complete, and it includes a discussion of design decisions. The disadvantage is that it’s a little over a year old – and in “Cloud” years that’s a lot. So many things have changed, improved, and have been added that you need to treat this as a resource, but not the only one. Still, highly recommended.

Link: http://msdn.microsoft.com/en-us/library/ff728592.aspx

Azure Stock Trader

Sometimes you need a mix of a CodePlex-style application, and a little more detail on how it was put together. And it would be great if you could actually play with the completed application, to see how it really functions on the actual platform.

That’s the Azure Stock Trader application. There’s a place where you can read about the application, and then it’s been published to Windows Azure – the production platform – and you can use it, explore, and see how it performs.

I use this application all the time to demonstrate Windows Azure, or a particular part of Windows Azure.

The advantage is that this is an end-to-end application, and online as well. The disadvantage is that it takes a bit of self-learning to work through.

Links: Learn it: http://msdn.microsoft.com/en-us/netframework/bb499684 Use it: https://azurestocktrader.cloudapp.net/

Preparation is key to a successful cloud deployment

BuckWoody — Tue, 01 May 2012 13:09:16 GMT

If you want to be wise, watch the actions and outcomes of others. Emulate the successful actions, and avoid the actions that cause failure. That’s true in life in general - and in technology projects in specific.

I’ve worked with several clients who have created or migrated an application to “the cloud” - meaning using Microsoft Windows Azure or another provider. Although the statement in the title of this post is trite, I cannot over-emphasize how accurate it is. In every case of those who had a great experience with a distributed computing environment (which is thankfully the vast majority of my projects),

What kind of preparation do you need to do? Here are some tips I’ve learned in the successful (and not-so-successful) deployments I’ve seen:

Follow standard recommendations for successful projects in general

You and your organization have probably done a few projects before - this one should have the same general attributes: a well-defined goal, a small, motivated team, a realistic timeline, and an adequate budget. I know, I know, you *never* seem to get those things - but if you don’t, you’ll fail. Simple as that.

Educate yourself

Computing technology started out on a single set of hardware for a single purpose - and realizing the limits of the hardware at hand, systems designers quickly realized that scale-out and virtualization was key. No, that’s not new - mainframes almost always worked on the concept of scale-out and virtual machines. But we switched in the 1980’s to single-user systems again, and we’ve been there ever since. By that I mean you install an OS on the things you work on. Now we move back to distributed system concepts, and there are some real differences. You’ll need to learn how those work, and do things a new way. Hey, we’re IT - we LOVE learning new things, right?

Get a partner if needed

There are a few of us white-haired Gandalf’s around that remember how to work in a distributed system, but if it’s new to you, that’s completely OK. You can save yourself a world of trouble by working with someone who’s done this before - a partner you hire, someone from Microsoft Consulting, whatever.

And don’t forget support - who will handle each issue, what is the escalation model, who are your contacts at Microsoft, and what is your “light’s out” strategy?

“A new broom sweeps clean”, the old adage goes, but the old brooms know where the dirt is.

Build a model

Take some time to do a Proof of Concept on your local system and using your Azure hours from your MSDN account if you have one. Going through this build - and being willing to throw it away and try it a different way - is invaluable.

Test your theories

Three statisticians are walking in a field. They see a rabbit - the first guy raises his gun, firing far in front of the rabbit. The second guy simultaneously raises his gun and fires far behind the rabbit. The third guy yells “We got him!”

Not every theory is correct - not every attempt is the right one. Build in your success tests while you’re building your model. Then check them - don’t leave this step out.

Rinse, lather, repeat

This is advice from a shampoo bottle - which I’ve never used (I don’t really have that much hair - especially now). But in a “Cloud” project, it’s important. It’s an evolving system, that gains new improvements at an amazing rate. As soon as you deploy and stabilize you need to start the process over again. If you created your system in a Services model, with contracts for the APIs and abstracted code, this is far easier.

It’s not hard to do a cloud project right. But it’s really simple to do it wrong. Follow these guidelines and you’ll learn from the successes - and mistakes - of others.

Java Resources for Windows Azure

BuckWoody — Tue, 13 Mar 2012 00:45:00 GMT

Windows Azure is a Platform as a Service – a PaaS – that runs code you write. That code doesn’t just mean the languages on the .NET platform – you can run code from multiple languages, including Java. In fact, you can develop for Windows and SQL Azure using not only Visual Studio but the Eclipse Integrated Development Environment (IDE) as well.

Although not an exhaustive list, here are several links that deal with Java and Windows Azure:

Resource	Link
Windows Azure Java Development Center	http://www.windowsazure.com/en-us/develop/java/
Java Development Guidance	http://msdn.microsoft.com/en-us/library/hh690943(VS.103).aspx
Running a Java Environment on Windows Azure	http://blogs.technet.com/b/port25/archive/2010/10/28/running-a-java-environment-on-windows-azure.aspx
Running a Java Environment on Windows Azure	http://blogs.technet.com/b/port25/archive/2010/10/28/running-a-java-environment-on-windows-azure.aspx
Run Java with Jetty in Windows Azure	http://blogs.msdn.com/b/dachou/archive/2010/03/21/run-java-with-jetty-in-windows-azure.aspx
Using the plugin for Eclipse	http://blogs.msdn.com/b/craig/archive/2011/03/22/new-plugin-for-eclipse-to-get-java-developers-off-the-ground-with-windows-azure.aspx
Run Java with GlassFish in Windows Azure	http://blogs.msdn.com/b/dachou/archive/2011/01/17/run-java-with-glassfish-in-windows-azure.aspx
Improving experience for Java developers with Windows Azure	http://blogs.msdn.com/b/interoperability/archive/2011/02/23/improving-experience-for-java-developers-with-windows-azure.aspx
Java Access to SQL Azure via the JDBC Driver for SQL Server	http://blogs.msdn.com/b/brian_swan/archive/2011/03/29/java-access-to-sql-azure-via-the-jdbc-driver-for-sql-server.aspx
How to Get Started with Java, Tomcat on Windows Azure	http://blogs.msdn.com/b/usisvde/archive/2011/03/04/how-to-get-started-with-java-tomcat-on-windows-azure.aspx
Deploying Java Applications in Azure	http://blogs.msdn.com/b/mariok/archive/2011/01/05/deploying-java-applications-in-azure.aspx
Using the Windows Azure Storage Explorer in Eclipse	http://blogs.msdn.com/b/brian_swan/archive/2011/01/11/using-the-windows-azure-storage-explorer-in-eclipse.aspx
Windows Azure Tomcat Solution Accelerator	http://archive.msdn.microsoft.com/winazuretomcat
Deploying a Java application to Windows Azure with Command-line Ant	http://java.interoperabilitybridges.com/articles/deploying-a-java-application-to-windows-azure-with-command-line-ant
Video: Open in the Cloud: Windows Azure and Java	http://channel9.msdn.com/Events/PDC/PDC10/CS10
AzureRunMe	http://azurerunme.codeplex.com/
Windows Azure SDK for Java	http://www.interoperabilitybridges.com/projects/windows-azure-sdk-for-java
AppFabric SDK for Java	http://www.interoperabilitybridges.com/projects/azure-java-sdk-for-net-services
Information Cards for Java	http://www.interoperabilitybridges.com/projects/information-card-for-java
Apache Stonehenge	http://www.interoperabilitybridges.com/projects/apache-stonehenge
Channel 9 Case Study on Java and Windows Azure	http://www.microsoft.com/casestudies/Windows-Azure/Gigaspaces/Solution-Provider-Streamlines-Java-Application-Deployment-in-the-Cloud/400000000081

Application Lifecycle Management Overview for Windows Azure

BuckWoody — Tue, 07 Feb 2012 14:58:39 GMT

Developing in Windows Azure is at once not that much different from what you’re familiar with in on-premises systems, and different in significant ways. Because of these differences, developers often ask about the specific process to develop and deploy a Windows Azure application - more formally called an Application Lifecycle Management, or ALM.

There are specific resources you can use to learn more about various parts of ALM - I’ve referenced those at the end of this post. But ALM has multiple definitions, from the governance of code injection, domain upgrade, testing, process flow and more. Many developers are interested in the finer-grained information, like how do I develop and deploy an application? What tools do I need, and how do I get the code running somewhere that I can test?

I’ll cover the very high-level process here, and refer you to specifics at the end of each section, so that you can take it all in at one viewing, and then bookmark for more detail when you need more information. I won’t be covering processes like Continuous Integration or Agile and other methodologies in this post - I’ll blog those later.

Initial Development

You start with writing code. You have three ways to do this. You can use Visual Studio (even the Express Edition Works), Eclipse, or by leveraging the REST API format. You can do this in a standalone (non-connected) environment like your laptop.

Using Visual Studio is one of the simplest methods to create an Azure application, allowing you to combine the Azure components you want to leverage (Storage, Compute, SQL Azure, the Service Bus, etc.) along with the on-premises code you have now or are creating. Once you’ve installed and patched Visual Studio, just download and install the Windows Azure Software Development Kit (SDK) and you’ll have not only all the API’s you need to talk to Azure, but a fully functioning local environment to run and test your code before you deploy it. You’ll also get a robust set of samples. You can download what you need for all of that (free) here: http://www.windowsazure.com/en-us/develop/downloads/ . There’s a step-by-step process here: http://msdn.microsoft.com/en-us/magazine/ee336122.aspx

You can also use Eclipse to develop for Windows Azure. You won’t get the full runtime environment in just that kit alone, but you can use this successfully on a Linux system. I have several folks using this method. The downloads and documentation for that is here: http://www.windowsazure4e.org/

You can use REST API’s to hit Azure Assets and control them. Not my preferred method, but possible. There are REST API’s for various sections of Azure. You can find the main reference for that here: http://msdn.microsoft.com/en-us/library/windowsazure/ff800682.aspx

Note: We recently demonstrated using a Cloud-based Integrated Development Environment (IDE) for Node.js deployment to Windows Azure. More on that here: http://www.readwriteweb.com/cloud/2012/01/cloud9-ide-to-enable-nodejs-ap.php

Deploying to a Test Instance

After you write the code, you’ll need to test it somewhere. The Azure Emulator on your development laptop is for a single user on that laptop, and it also has some subtle differences from the production fabric as you might imagine. Normally you’ll set up a small subscription to run and test the application, just like you would have a set of test servers. Each subscription has its own management keys and certificates, so this assists in keeping the testing environment separate for billing and control.

More on that general information here: http://msdn.microsoft.com/en-us/library/ff803362.aspx

Deploying to Production

Once you have developed the code and tested it, you need to move it to a location where users can access it. In reality, there is no physical difference in the type of machines, fabric or any other component in “Production” Windows Azure accounts and the “Test” accounts, but you’ll most often pick smaller systems to deploy on in testing, and you’ll probably keep the URL in the plain format.

In the Production Windows Azure account, the team normally limits the access to the account for deployment to a separate set of developers. This ensures code flow and control. A DNS name is normally mapped to the longer, Microsoft-generated URL so that your users access the application or data the way you want them to.

More on setting up an account here: http://techinch.com/2010/06/14/setup-your-windows-azure-account/

Managing Code Change

With the application deployed, there are two broad tasks you need to consider. One is managing changes through the application, and the other involves management, monitoring and performance tuning for an application.

To make a code change, the standard ALM process is followed, just as above. You can use command-line tools to automate the process as you would with an on-premises system. A vide on that shows you how: http://www.microsoftpdc.com/2009/SVC25. Normally this is used with an “In-Place” upgrade into Production Account, since your testing is completed in a separate account. More on that process here: http://msdn.microsoft.com/en-us/library/windowsazure/ee517255.aspx

One difference is the “VIP Swap” process you can use for the final push to Production. In essence, this allows you to have two copies of the application running on the Production account, with a quick way to cut over and back when you’re ready. The process for that is detailed here: http://msdn.microsoft.com/en-us/library/windowsazure/ee517253.aspx

For monitoring, you have several options. You should enable the Windows Azure Diagnostics in your code - more on that here: http://archive.msdn.microsoft.com/WADiagnostics.

You can observe uptime and other information on the Windows Azure Service Dashboard, where you can also consume the uptime as an RSS feed: http://www.windowsazure.com/en-us/support/service-dashboard/

From there, you can also use System Center to monitor not only Windows Azure deployments but internal applications as well. The Management Pack and documentation for that is here: http://www.microsoft.com/download/en/details.aspx?id=11324.

There are also 3rd-party tools to manage Windows Azure. More on that here: http://www.bing.com/search?q=monitor+Windows+Azure&form=OSDSRC

Other References:

There is a lot more detail in this official reference: https://www.windowsazure.com/en-us/develop/net/fundamentals/deploying-applications/

Bryan Group explains the ramifications of the Secure Development Lifecycle (SDL) with lots of collateral you can review: http://blogs.msdn.com/b/bryang/archive/2011/04/26/applying-the-sdl-to-windows-azure.aspx

Team Foundation Server (TFS) in the Cloud - My Experience So Far

BuckWoody — Tue, 24 Jan 2012 12:45:13 GMT

I recently joined a software development project that involves not only myself and other internal Microsoft employees, but a partner and a customer as well. We are building a hybrid solution that uses assets on premises as well as Windows Azure for processing. When we put the team together we picked a methodology (Agile) for the project (we use multiple methodologies at Microsoft - whatever the project needs) and then we started talking about Source Control.

We’re all comfortable with various tools for check-in-check-out, branching, and so on. We have all used GIT, SVN, and TFS. Some of us have even used Source Safe in past, but that’s another post. Each company has a full set of Source Control systems in place. But using each other’s systems requires logins, firewalls and the like - so we decided to use the TFS Service Preview to run the entire project from “the cloud”. Here are my experiences with that.

The process was really simple. In fact, we talked about using the cloud TFS in the first SCRUM, and the team was working from the Work Items list that afternoon. The original account login provides a web interface to allow people to join the team. Each of us happened to have a Live.Com address, so we just invited those addresses to join and they got a link, like this:

projectname.tfspreview.com

I’m using Visual Studio, and it’s a requirement for TFS preview to have SP1 installed, and this patch: KB2581206

From there, I opened Visual Studio and navigated from the main menu to Team and then Connect to Team Foundation Server. I’m given this menu:

Selecting port 443 and HTTPS (for security) and then ensuring the lower link has the “tfs” appended as the location, I opened the project.

(This VSTS screenshot is of a project I did in my University of Washington class I teach - I never show client code or names in a blog post)

From there it’s a normal set of operations. Right now the preview doesn’t have some things I’d really like, such as an automated build or some of the testing tools, but you can read this blog entry to learn more about the entire sign-up process, and what the team has planned.

Each day I log in to the project, and I’m given this new sign-in option:

I click the option, and I open the environment, hit My Work Items query, and get to work. All in all, a seamless - although basic - experience. The speed at which we could set up and work on a project was really sweet. It’s remarkable how un-remarkable this is - I just do my work each day, everything is running and backed up in the cloud. I think that’s the point.

Developing a Cost Model for Cloud Applications

BuckWoody — Tue, 08 Nov 2011 15:30:49 GMT

Note - please pay attention to the date of this post. As much as I attempt to make the information below accurate, the nature of distributed computing means that components, units and pricing will change over time. The definitive costs for Microsoft Windows Azure and SQL Azure are located here, and are more accurate than anything you will see in this post: http://www.microsoft.com/windowsazure/offers/

When writing software that is run on a Platform-as-a-Service (PaaS) offering like Windows Azure / SQL Azure, one of the questions you must answer is how much the system will cost. I will not discuss the comparisons between on-premise costs (which are nigh impossible to calculate accurately) versus cloud costs, but instead focus on creating a general model for estimating costs for a given application.

You should be aware that there are (at this writing) two billing mechanisms for Windows and SQL Azure: “Pay-as-you-go” or consumption, and “Subscription” or commitment. Conceptually, you can consider the former a pay-as-you-go cell phone plan, where you pay by the unit used (at a slightly higher rate) and the latter as a standard cell phone plan where you commit to a contract and thus pay lower rates. In this post I’ll stick with the pay-as-you-go mechanism for simplicity, which should be the maximum cost you would pay. From there you may be able to get a lower cost if you use the other mechanism. In any case, the model you create should hold.

Developing a good cost model is essential. As a developer or architect, you’ll most certainly be asked how much something will cost, and you need to have a reliable way to estimate that. Businesses and Organizations have been used to paying for servers, software licenses, and other infrastructure as an up-front cost, and power, people to the systems and so on as an ongoing (and sometimes not factored) cost. When presented with a new paradigm like distributed computing, they may not understand the true cost/value proposition, and that’s where the architect and developer can guide the conversation to make a choice based on features of the application versus the true costs.

The two big buckets of use-types for these applications are customer-based and steady-state. In the customer-based use type, each successful use of the program results in a sale or income for your organization. Perhaps you’ve written an application that provides the spot-price of foo, and your customer pays for the use of that application. In that case, once you’ve estimated your cost for a successful traversal of the application, you can build that into the price you charge the user. It’s a standard restaurant model, where the price of the meal is determined by the cost of making it, plus any profit you can make.

In the second use-type, the application will be used by a more-or-less constant number of processes or users and no direct revenue is attached to the system. A typical example is a customer-tracking system used by the employees within your company. In this case, the cost model is often created “in reverse” - meaning that you pilot the application, monitor the use (and costs) and that cost is held steady. This is where the comparison with an on-premise system becomes necessary, even though it is more difficult to estimate those on-premise true costs. For instance, do you know exactly how much cost the air conditioning is because you have a team of system administrators? This may sound trivial, but that, along with the insurance for the building, the wiring, and every other part of the system is in fact a cost to the business.

There are three primary methods that I’ve been successful with in estimating the cost. None are perfect, all are demand-driven. The general process is to lay out a matrix of:

components
units
cost per unit

and then multiply that times the usage of the system, based on which components you use in the program. That sounds a bit simplistic, but using those metrics in a calculation becomes more detailed. In all of the methods that follow, you need to know your application. The components for a PaaS include computing instances, storage, transactions, bandwidth and in the case of SQL Azure, database size. In most cases, architects start with the first model and progress through the other methods to gain accuracy.

Simple Estimation

The simplest way to calculate costs is to architect the application (even UML or on-paper, no coding involved) and then estimate which of the components you’ll use, and how much of each will be used. Microsoft provides two tools to do this - one is a simple slider-application located here: http://www.microsoft.com/windowsazure/pricing-calculator/

The other is a tool you download to create an “Return on Investment” (ROI) spreadsheet, which has the advantage of leading you through various questions to estimate what you plan to use, located here: https://roianalyst.alinean.com/msft/AutoLogin.do?d=176318219048082115

You can also just create a spreadsheet yourself with a structure like this:

Program Element	Azure Component	Unit of Measure	Cost Per Unit	Estimated Use of Component	Total Cost Per Component	Cumulative Cost

Of course, the consideration with this model is that it is difficult to predict a system that is not running or hasn’t even been developed. Which brings us to the next model type.

Measure and Project

A more accurate model is to actually write the code for the application, using the Software Development Kit (SDK) which can run entirely disconnected from Azure. The code should be instrumented to estimate the use of the application components, logging to a local file on the development system. A series of unit and integration tests should be run, which will create load on the test system.

You can use standard development concepts to track this usage, and even use Windows Performance Monitor counters. The best place to start with this method is to use the Windows Azure Diagnostics subsystem in your code, which you can read more about here: http://blogs.msdn.com/b/sumitm/archive/2009/11/18/introducing-windows-azure-diagnostics.aspx This set of API’s greatly simplifies tracking the application, and in fact you can use this information for more than just a cost model.

After you have the tracking logs, you can plug the numbers into ay of the tools above, which should give a representative cost or in some cases a unit cost.

The consideration with this model is that the SDK fabric is not a one-to-one comparison with performance on the actual Windows Azure fabric. Those differences are usually smaller, but they do need to be considered. Also, you may not be able to accurately predict the load on the system, which might lead to an architectural change, which changes the model. This leads us to the next, most accurate method for a cost model.

Sample and Estimate

Using standard statistical and other predictive math, once the application is deployed you will get a bill each month from Microsoft for your Azure usage. The bill is quite detailed, and you can export the data from it to do analysis, and using methods like regression and so on project out into the future what the costs will be. I normally advise that the architect also extrapolate a unit cost from those metrics as well. This is the information that should be reported back to the executives that pay the bills: the past cost, future projected costs, and unit cost “per click” or “per transaction”, as your case warrants.

The challenge here is in the model itself - statistical methods are not foolproof, and the larger the sample (in this case I recommend the entire population, not a smaller sample) is key.

References and Tools

Articles:

http://blogs.msdn.com/b/patrick_butler_monterde/archive/2010/02/10/windows-azure-billing-overview.aspx

http://technet.microsoft.com/en-us/magazine/gg213848.aspx

http://blog.codingoutloud.com/2011/06/05/azure-faq-how-much-will-it-cost-me-to-run-my-application-on-windows-azure/

http://blogs.msdn.com/b/johnalioto/archive/2010/08/25/10054193.aspx

http://geekswithblogs.net/iupdateable/archive/2010/02/08/qampa-how-can-i-calculate-the-tco-and-roi-when.aspx

Other Tools:

http://cloud-assessment.com/

http://communities.quest.com/community/cloud_tools

Creating a Distributed Computing System Using a Windows Azure Queue

BuckWoody — Tue, 11 Oct 2011 13:12:42 GMT

The Windows Azure Queue component, like all Windows Azure components (Roles, Storage, App Fabric, SQL Azure) can be used by itself or with other Windows Azure components. That’s why I refer to Windows Azure as “Distributed Computing” rather than “cloud”.

Having a distributed off premise queue has a lot of use-cases. An interesting use-case is a company that wanted to harness the power of all of the PC’s and laptops in the company when they were not being used throughout the day. A developer wrote a screen-saver program that connected to an Azure Queue, pulling work off of the queue and placing an entry when it was done. In essence he had a partially connected distributed work relay system, and since he used a Windows Azure Queue, the system worked from anywhere in the world.

He uses an on-site central server (which was actually only a workstation-level system) that holds the computations in a scatter/gather paradigm. The computations are broken into less-than-8K chunks, so that it fits within a message. The server connects to a Windows Azure Queue, and places the message marked for computation. It also scrubs the Queue for completed work, and as part of the process puts that kind of message into a mapping function (queues are not guaranteed a message order).

The workstations that are not being used (even those systems at remote workers and travelers) connect to the same Windows Azure Queue when the system is not being used for a period of time, when the screen saver kicks in. It then takes one message from the queue, computes the information, and then sets a new message for the server to pick up with the answer. The workstation then deletes the message.

The Server picks up the completed work, processes it and then deletes that queue message. He also added logic to process messages for computation on the server as well, when the server function of adding work is not required.

There are a few caveats here. This works because of the mapping function on the head server. Order is not guaranteed, so he includes a number for the function step as part of the message body, which cuts the size a bit. Also, he’s careful to watch the encoding, since Azure will hand binary back in Base64 format.

He’s found that there are enough systems to ensure that the messages are cleared every few days – important, since the Windows Azure Queue ages out after seven days. Also, he’s careful to use the CloudQueue.PeekMessage function when he wants to monitor the system – that function ensures that the message status doesn’t reset as “read” when he accesses it.

This is a great example of using the “cloud” as what it is intended to be – a distributed architecture you can use as needed to solve a business problem. It’s not an “all or nothing” proposition, but instead it is simply another set of components to use where you need them.

Rip and Replace or Extend and Embrace?

BuckWoody — Tue, 13 Sep 2011 11:20:05 GMT

As most of you know, I don’t like the term “cloud” very
much. It isn’t defined, which means it can be anything. I prefer “distributed
computing”, which is more technically accurate and describes what you’re doing
in more concrete terms.

So when you think about Windows and SQL Azure, you don’t
have to think about an entire product – you can use parts of the system
together or independently to accomplish what you need to do. You can use the
computing functions, storage, and more and more I see folks leverage the
Service Bus to enable current applications to expose things to the web.

And that brings up the point of this post. Once you decide
that a distributed architecture works to solve a problem, you’re faced with a
decision: should you completely re-write your architecture to take advantage of
the current systems or should you just fold in new code that makes the data or
function available to the web?

Of course, the answer is always “it depends” on the situation
– and it does. But unless you’re fixing a problem with current code, I usually
advocate a migration approach. That means at the very least retaining the
business logic (again, unless it’s not currently working) and as much of the
code as you can. In fact, if you follow this paradigm, you’re on your way to
making a Service Bus out of the functions you currently have. You can expose
the results of a system rather than opening the system up. Let’s take an
example.

Assume for a moment that you have an order-taking system
on-premise. That system performs many functions, one of which might creating a
Purchase Order. Your system might be enclosed, meaning that it has an
application that talks to a middle-tier, and then from there to a database
system. A query is generated from a screen, and passed along to eventually
compute, store and return a Purchase Order Number, along with other
information. Imagine now that you wire up the code not only to return the PO
number to the client, but to make that number available on an endpoint –
actually really not that hard to do.

Now you can make that PO number available to the web using
Azure. You could restrict who can make that call to the system, or open it up
to a broader audience. Or instead of the PO Number, you could make a product
list available. And you can go further than that – EBay, for instance, uses the
OData protocol (which is very cool in and of itself) which you can query from
the web. You could compare your company’s product catalog to what is on EBay,
and list the items you have there if there are no competitors in that space.
And on and on it goes.

So the point is this – where you can, retain what works.
Fold in systems like Azure where they make sense. Extend and Embrace.

Plan for Diagnostics in Cloud Computing From the Git-Go

BuckWoody — Tue, 06 Sep 2011 13:11:22 GMT

“Git-Go” is something we say in the South that means “right at the start”. I’ve seen several applications for on-premise systems that don’t have much in the way of diagnostics - the developers rely on a debugger, the event logs on the server and client workstation, and most of all, the ability to watch the system from end-to-end.

This approach is a mistake for an on-premise system, and it’s definitely a problem for a distributed architecture. You simply do not own all of the components from end to end in a cloud environment, nor are you always able to attach a debugger or other remote monitoring tools to the various areas within the code path. So you need to make sure that from the very outset of your design that you build in diagnostics. My personal preference is to build a system such that a control file turns on deeper information gathering from the system, up to a minimal level.

When I do that, I set a high level of logging, a medium level, and a moderate level. I normally use the deepest level of information during the testing and acceptance phase of the deployment, then switch to moderate and then the least level of information gathering. Also in my design I often set an error condition to begin gathering the deeper information along with the exception, where possible.

There are decisions you need to make as to where to store the diagnostics (many operations in the cloud cost money), how often you collect them, and so on. You can get a quick overview on using the diagnostics that come with Windows Azure here: http://www.azuresupport.com/2010/03/getting-started-with-windows-azure-diagnostics-and-monitoring/ This is where you should start first. More detail on that: http://msdn.microsoft.com/en-us/library/gg433048.aspx

My friend Dave Pallman has a great tool he’s released for free: http://davidpallmann.blogspot.com/2009/03/azure-application-monitor-now-on.html

If the issue is in storage apps: http://social.msdn.microsoft.com/Forums/en-US/windowsazuredata/thread/d84ba34b-b0e0-4961-a167-bbe7618beb83

If you have System Center, this is the quickest and easiest way to implement the monitoring – really handy: http://pinpoint.microsoft.com/en-us/applications/windows-azure-application-monitoring-management-pack-release-candidate-12884907699

SQL Azure Use Case: Web-based Applications

BuckWoody — Tue, 19 Apr 2011 14:38:40 GMT

This is one in a series of posts on when and where to use a distributed architecture design in your organization's computing needs. You can find the main post here: http://blogs.msdn.com/b/buckwoody/archive/2011/01/18/windows-azure-and-sql-azure-use-cases.aspx

Description:

Some applications lend themselves for the entire architecture to be placed on an outside provider such as Azure. And in some cases, you’re interested in using a Relational Database Management System (RDBMS):

Web application with fast meta-data search requirements
Web application requiring high levels of consistency and/or atomicity
Common-use data, shared among multiple web applications
Durable data storage for stateless applications

Unless you need the data to be kept local (see the hybrid application use-case), SQL Azure makes a good fit.SQL Azure shares the same logical “backbone” as Windows Azure, so an Azure application that needs structured storage, either exclusively or alongside Blob or Windows Azure Table storage (Key/Value pair, more akin to NoSQL in architecture than RDBMS).

Implementation:

This is actually one of the easiest concepts to display for a SQL Azure architecture. It’s logically the same as keeping the application completely local, with the exception that you don’t have to install or maintain anything. Note that although Windows Azure applications are a common use, you can use any web program to access the SQL Azure database:

Considerations here include the decisions on when to use structured storage for a datum or some other storage. In many configurations, you might want multiple storage paradigms. Here is one such example architecture, although many others are possible:

In this diagram I’m indicating a simple shopping-cart application. A Windows Azure Web Role provides a “front end” or presentation layer to the client. A Worker Role provides computation functions, and the Queue maintains the state information so that the application is scalable. SQL Azure stores meta-data about the items in a catalogue of items a user can purchase, such as name, size, price and so on. This provides fast lookup, and allows re-use of code that existed on an on-premise SQL Server.

Once the item is located, a reference in a SQL Azure column (from a standard SQL query) locates the GUID for the object’s picture, stored in Windows Azure Blob storage, and displays that to the user. The Worker Role moves the information for the customer’s order from the Queue to a Windows Azure Table object.

Of course, you could architect all of these data elements into only one or another kind of storage. In this case, the cost, performance and other characteristics of each data requirement dictated this selection.

Resources:

Storage and their abstractions: http://blogs.msdn.com/b/windowsazurestorage/archive/2010/05/10/windows-azure-storage-abstractions-and-their-scalability-targets.aspx