Buck Woody : Best Practices, Windows Azure, Cloud Computing

Book Review (Book 11) - Applied Architecture Patterns on the Microsoft Platform

BuckWoody — Tue, 15 May 2012 16:50:34 GMT

This is a continuation of the books I challenged myself to read to help my career - one a month, for year. You can read my first book review here, and the entire list is here. The book I chose for April 2012 was: Applied Architecture Patterns on the Microsoft Platform. I was traveling at the end of last month so I’m a bit late posting this review here.

Why I chose this book:

I actually know a few of the authors on this book, so when they told me about it I wanted to check it out. The premise of the book is exactly as it states in the title - to learn how to solve a problem using products from Microsoft.

What I learned:

I liked the book - a lot. They've arranged the content in a "Solution Decision Framework", that presents a few elements to help you identify a need and then propose alternate solutions to solve them, and then the rationale for the choice. But the payoff is that the authors then walk through the solution they implement and what they ran into doing it.

I really liked this approach. It's not a huge book, but one I've referred to again since I've read it. It's fairly comprehensive, and includes server-oriented products, not things like Microsoft Office or other client-side tools. In fact, I would LOVE to have a work like this for Open Source and other vendors as well - would make for a great library for a Systems Architect. This one is unashamedly aimed at the Microsoft products, and even if I didn't work here, I'd be fine with that. As I said, it would be interesting to see some books on other platforms like this, but I haven't run across something that presents other systems in quite this way.

And that brings up an interesting point - This book is aimed at folks who create solutions within an organization. It's not aimed at Administrators, DBA's, Developers or the like, although I think all of those audiences could benefit from reading it. The solutions are made up, and not to a huge level of depth - nor should they be. It's a great exercise in thinking these kinds of things through in a structured way.

The information is a bit dated, especially for Windows and SQL Azure. While the general concepts hold, the cloud platform from Microsoft is evolving so quickly that any printed book finds it hard to keep up with the improvements.

I do have one quibble with the text - the chapters are a bit uneven. This is always a danger with multiple authors, but it shows up in a couple of chapters. I winced at one of the chapters that tried to take a more conversational, humorous style. This kind of academic work doesn't lend itself to that style.

I recommend you get the book - and use it. I hope they keep it updated - I'll be a frequent customer. :)

Preparation is key to a successful cloud deployment

BuckWoody — Tue, 01 May 2012 13:09:16 GMT

If you want to be wise, watch the actions and outcomes of others. Emulate the successful actions, and avoid the actions that cause failure. That’s true in life in general - and in technology projects in specific.

I’ve worked with several clients who have created or migrated an application to “the cloud” - meaning using Microsoft Windows Azure or another provider. Although the statement in the title of this post is trite, I cannot over-emphasize how accurate it is. In every case of those who had a great experience with a distributed computing environment (which is thankfully the vast majority of my projects),

What kind of preparation do you need to do? Here are some tips I’ve learned in the successful (and not-so-successful) deployments I’ve seen:

Follow standard recommendations for successful projects in general

You and your organization have probably done a few projects before - this one should have the same general attributes: a well-defined goal, a small, motivated team, a realistic timeline, and an adequate budget. I know, I know, you *never* seem to get those things - but if you don’t, you’ll fail. Simple as that.

Educate yourself

Computing technology started out on a single set of hardware for a single purpose - and realizing the limits of the hardware at hand, systems designers quickly realized that scale-out and virtualization was key. No, that’s not new - mainframes almost always worked on the concept of scale-out and virtual machines. But we switched in the 1980’s to single-user systems again, and we’ve been there ever since. By that I mean you install an OS on the things you work on. Now we move back to distributed system concepts, and there are some real differences. You’ll need to learn how those work, and do things a new way. Hey, we’re IT - we LOVE learning new things, right?

Get a partner if needed

There are a few of us white-haired Gandalf’s around that remember how to work in a distributed system, but if it’s new to you, that’s completely OK. You can save yourself a world of trouble by working with someone who’s done this before - a partner you hire, someone from Microsoft Consulting, whatever.

And don’t forget support - who will handle each issue, what is the escalation model, who are your contacts at Microsoft, and what is your “light’s out” strategy?

“A new broom sweeps clean”, the old adage goes, but the old brooms know where the dirt is.

Build a model

Take some time to do a Proof of Concept on your local system and using your Azure hours from your MSDN account if you have one. Going through this build - and being willing to throw it away and try it a different way - is invaluable.

Test your theories

Three statisticians are walking in a field. They see a rabbit - the first guy raises his gun, firing far in front of the rabbit. The second guy simultaneously raises his gun and fires far behind the rabbit. The third guy yells “We got him!”

Not every theory is correct - not every attempt is the right one. Build in your success tests while you’re building your model. Then check them - don’t leave this step out.

Rinse, lather, repeat

This is advice from a shampoo bottle - which I’ve never used (I don’t really have that much hair - especially now). But in a “Cloud” project, it’s important. It’s an evolving system, that gains new improvements at an amazing rate. As soon as you deploy and stabilize you need to start the process over again. If you created your system in a Services model, with contracts for the APIs and abstracted code, this is far easier.

It’s not hard to do a cloud project right. But it’s really simple to do it wrong. Follow these guidelines and you’ll learn from the successes - and mistakes - of others.

Pay in the future should make you think in the present

BuckWoody — Tue, 10 Apr 2012 13:53:50 GMT

Distributed Computing - and more importantly “-as-a-Service” models of computing have a different cost model. This is something that sounds obvious on the surface but it’s often forgotten during the design and coding phase of a project.

In on-premises computing, we’re used to purchasing a server and all of the hardware infrastructure and software licenses needed not only for one project, but several. This is an up-front or “sunk” cost that we consume by running code the organization needs to perform its function. Using a direct connection over wires you’ve already paid for, we don’t often have to think about bandwidth, hits on the data store or the amount of compute we use - we just know more is better. In a pay-as-you-go model, however, each of these architecture decisions has a potential cost impact. The amount of data you store, the number of times you access it, and the amount you send back all come with a charge. The offset is that you don’t buy anything at all up-front, so that sunk cost is freed up. And financial professionals know that money now is worth more than money later. Saving that up-front cost allows you to invest it in other things.

It’s not just that you’re using things that now cost money - it’s that the design itself in distributed computing has a cost impact. That can be a really good thing, such as when you dynamically add capacity for paying customers. If you can tie back the cost of a series of clicks to what a user will pay to do so, you can set a profit margin that is easy to track.

Here’s a case in point: Assume you are using a large instance in Windows Azure to compute some data that you retrieve from a SQL Azure database. If you don’t monitor the path of the application, you may not know what you are really using. Since you’re paying by the size of the instance, it’s best to maximize it all the time. Recently I evaluated just this situation, and found that downsizing the instance and adding another one where needed, adding a caching function to the application, moving part of the data into Windows Azure tables not only increased the speed of the application, but reduced the cost and more closely tied the cost to the profit.

The key is this: from the very outset - the design - make sure you include metrics to measure for the cost/performance (sometimes these are the same) for your application. Windows Azure opens up awesome new ways of doing things, so make sure you study distributed systems architecture before you try and force in the application design you have on premises into your new application structure.

“I could use a little help here” or “I can do it myself, thank you” for Cloud Projects

BuckWoody — Tue, 03 Apr 2012 13:20:55 GMT

Windows Azure allows you to write code in languages within the .NET stack, you can use Java, C++, PHP, NodeJS and others. Code is code - other than keeping things stateless, using a Web or Worker Role in Azure is not all that different from working with an on-premises system.

However….

Working in a scalable, component-based stateless architecture that can use federated security is not all that common for many developers. Some are used to owning the server, scaling up, and state-full paradigms that have a single security domain. Making the transition whilst trying to create a new software application or even port a previous one can be daunting. Sure, we have absolutely tons of free training, kits, videos, online books and more to learn on your own, but some things like architecture can be pivotal as you move along.

So the question is, should you just strike out on your own for a Cloud project, or get Microsoft Consulting Services or another partner to work with you on your first one? I use a few decision points to help guide the projects I assist in.

Note: I’m a huge fan of having help that ends up giving you training and leaves you in charge. If you do engage with someone to help you, make sure you keep this clear and take more and more ownership yourself as the project progresses.

How much time do you have?

Usually the first thing I ask is about the timeline for the project. It doesn’t matter how skilled you are, if you have a short window to get things done it’s better to get help - especially if this is your first cloud project. Having someone that knows the platform well can save you amazing amounts of time. If you have longer, then start with the training in the link above and once you feel confident, jump in.

How complex is the project?

If there are a lot of moving parts, it’s best to engage a partner. The reason is that certain interactions - particularly things like Service Bus or Data Integration - can be quite different than what you may have encountered before.

How many people do you have?

I have a “pizza rule” about projects I’ve used in my career - if it takes over two pizzas to feed everyone on the project, it’s too big and will fail. That being said, one developer and a one-week deadline does not a good project make, usually. It’s best to have at least one architect (or someone in that role) guiding the project along, and at least two developers to work on a cloud project. That’s a generalization of course, since I’ve seen great software on Azure with one developer writing code all by herself, but for more complex projects, more (to a point) is better. The nice thing about bringing on a partner is that you don’t have to hire them full time - they help you and then they go away.

How critical is the project?

There’s no shame in using some help. If the platform is new, if the project is large and complex, and if it is critical to the business, you should engage a partner. That’s regardless of Cloud or anything else - get some help. You don’t want to hit your company’s bottom line in a negative way, but you have to innovate and get them a competitive advantage. Do your research, make sure the partner is qualified to help you, and get it done.

Don’t let these questions scare you off. There are lots of projects you can implement on Windows and SQL Azure with nothing other than the Software Development Kit (SDK) that you get for free with Windows Azure. And assistance comes in many forms - sometimes just phone support, a friend you can ask. Microsoft Consulting Services or any of our great partners. You can get help on just the architecture piece or have them show you how to write the code. They’ll get involved as little or as much as you like.

Why do I need two Instances in Windows Azure?

BuckWoody — Tue, 20 Mar 2012 13:23:00 GMT

Windows Azure as a Platform as a Service (PaaS) means that there are various components you can use in it to solve a problem:

Compute “Roles” - Computers running an OS and optionally IIS - you can have more than one "Instance" of a given Role
Storage - Blobs, Tables and Queues for Storage
Other Services - Things like the Service Bus, Azure Connection Services, SQL Azure and Caching

It’s important to understand that some of these services are Stateless and others maintain State. Stateless means (at least in this case) that a system might disappear from one physical location and appear elsewhere. You can think of this as a cashier at the front of a store. If you’re in line, a cashier might take his break, and another person might replace him. As long as the order proceeds, you as the customer aren’t really affected except for the few seconds it takes to change them out. The cashier function in this example is stateless.

The Compute Role Instances in Windows Azure are Stateless. To upgrade hardware, because of a fault or many other reasons, a Compute Role's Instance might stop on one physical server, and another will pick it up. This is done through the controlling fabric that Windows Azure uses to manage the systems.

It’s important to note that storage in Azure does maintain State. Your data will not simply disappear - it is maintained - in fact, it’s maintained three times in a single datacenter and all those copies are replicated to another for safety. Going back to our example, storage is similar to the cash register itself. Even though a cashier leaves, the record of your payment is maintained.

So if a Compute Role Instance can disappear and re-appear, the things running on that first Instance would stop working. If you wrote your code in a Stateless way, then another Role Instance simply re-starts that transaction and keeps working, just like the other cashier in the example.

But if you only have one Instance of a Role, then when the Role Instance is re-started, or when you need to upgrade your own code, you can face downtime, since there’s only one. That means you should deploy at least two of each Role Instance not only for scale to handle load, but so that the first “cashier” has someone to replace them when they disappear. It’s not just a good idea - to gain the Service Level Agreement (SLA) for our uptime in Azure it’s a requirement. We point this out right in the Management Portal when you deploy the application:

(Click to enlarge)

When you deploy a Role Instance you can also set the “Upgrade Domain”. Placing Roles on separate Upgrade Domains means that you have a continuous service whenever you upgrade (more on upgrades in another post) - the process looks like this for two Roles. This example covers the scenario for upgrade, so you have four roles total - One Web and one Worker running the "older" code, and one of each running the new code. In all those Roles you want at least two instances, and this example shows that you're covered for High Availability and upgrade paths:

The take-away is this - always plan for forward-facing Roles to have at least two copies. For Worker Roles that do background processing, there are ways to architect around this number, but it does affect the SLA if you have only one.

Application Lifecycle Management Overview for Windows Azure

BuckWoody — Tue, 07 Feb 2012 14:58:39 GMT

Developing in Windows Azure is at once not that much different from what you’re familiar with in on-premises systems, and different in significant ways. Because of these differences, developers often ask about the specific process to develop and deploy a Windows Azure application - more formally called an Application Lifecycle Management, or ALM.

There are specific resources you can use to learn more about various parts of ALM - I’ve referenced those at the end of this post. But ALM has multiple definitions, from the governance of code injection, domain upgrade, testing, process flow and more. Many developers are interested in the finer-grained information, like how do I develop and deploy an application? What tools do I need, and how do I get the code running somewhere that I can test?

I’ll cover the very high-level process here, and refer you to specifics at the end of each section, so that you can take it all in at one viewing, and then bookmark for more detail when you need more information. I won’t be covering processes like Continuous Integration or Agile and other methodologies in this post - I’ll blog those later.

Initial Development

You start with writing code. You have three ways to do this. You can use Visual Studio (even the Express Edition Works), Eclipse, or by leveraging the REST API format. You can do this in a standalone (non-connected) environment like your laptop.

Using Visual Studio is one of the simplest methods to create an Azure application, allowing you to combine the Azure components you want to leverage (Storage, Compute, SQL Azure, the Service Bus, etc.) along with the on-premises code you have now or are creating. Once you’ve installed and patched Visual Studio, just download and install the Windows Azure Software Development Kit (SDK) and you’ll have not only all the API’s you need to talk to Azure, but a fully functioning local environment to run and test your code before you deploy it. You’ll also get a robust set of samples. You can download what you need for all of that (free) here: http://www.windowsazure.com/en-us/develop/downloads/ . There’s a step-by-step process here: http://msdn.microsoft.com/en-us/magazine/ee336122.aspx

You can also use Eclipse to develop for Windows Azure. You won’t get the full runtime environment in just that kit alone, but you can use this successfully on a Linux system. I have several folks using this method. The downloads and documentation for that is here: http://www.windowsazure4e.org/

You can use REST API’s to hit Azure Assets and control them. Not my preferred method, but possible. There are REST API’s for various sections of Azure. You can find the main reference for that here: http://msdn.microsoft.com/en-us/library/windowsazure/ff800682.aspx

Note: We recently demonstrated using a Cloud-based Integrated Development Environment (IDE) for Node.js deployment to Windows Azure. More on that here: http://www.readwriteweb.com/cloud/2012/01/cloud9-ide-to-enable-nodejs-ap.php

Deploying to a Test Instance

After you write the code, you’ll need to test it somewhere. The Azure Emulator on your development laptop is for a single user on that laptop, and it also has some subtle differences from the production fabric as you might imagine. Normally you’ll set up a small subscription to run and test the application, just like you would have a set of test servers. Each subscription has its own management keys and certificates, so this assists in keeping the testing environment separate for billing and control.

More on that general information here: http://msdn.microsoft.com/en-us/library/ff803362.aspx

Deploying to Production

Once you have developed the code and tested it, you need to move it to a location where users can access it. In reality, there is no physical difference in the type of machines, fabric or any other component in “Production” Windows Azure accounts and the “Test” accounts, but you’ll most often pick smaller systems to deploy on in testing, and you’ll probably keep the URL in the plain format.

In the Production Windows Azure account, the team normally limits the access to the account for deployment to a separate set of developers. This ensures code flow and control. A DNS name is normally mapped to the longer, Microsoft-generated URL so that your users access the application or data the way you want them to.

More on setting up an account here: http://techinch.com/2010/06/14/setup-your-windows-azure-account/

Managing Code Change

With the application deployed, there are two broad tasks you need to consider. One is managing changes through the application, and the other involves management, monitoring and performance tuning for an application.

To make a code change, the standard ALM process is followed, just as above. You can use command-line tools to automate the process as you would with an on-premises system. A vide on that shows you how: http://www.microsoftpdc.com/2009/SVC25. Normally this is used with an “In-Place” upgrade into Production Account, since your testing is completed in a separate account. More on that process here: http://msdn.microsoft.com/en-us/library/windowsazure/ee517255.aspx

One difference is the “VIP Swap” process you can use for the final push to Production. In essence, this allows you to have two copies of the application running on the Production account, with a quick way to cut over and back when you’re ready. The process for that is detailed here: http://msdn.microsoft.com/en-us/library/windowsazure/ee517253.aspx

For monitoring, you have several options. You should enable the Windows Azure Diagnostics in your code - more on that here: http://archive.msdn.microsoft.com/WADiagnostics.

You can observe uptime and other information on the Windows Azure Service Dashboard, where you can also consume the uptime as an RSS feed: http://www.windowsazure.com/en-us/support/service-dashboard/

From there, you can also use System Center to monitor not only Windows Azure deployments but internal applications as well. The Management Pack and documentation for that is here: http://www.microsoft.com/download/en/details.aspx?id=11324.

There are also 3rd-party tools to manage Windows Azure. More on that here: http://www.bing.com/search?q=monitor+Windows+Azure&form=OSDSRC

Other References:

There is a lot more detail in this official reference: https://www.windowsazure.com/en-us/develop/net/fundamentals/deploying-applications/

Bryan Group explains the ramifications of the Secure Development Lifecycle (SDL) with lots of collateral you can review: http://blogs.msdn.com/b/bryang/archive/2011/04/26/applying-the-sdl-to-windows-azure.aspx

Rip and Replace or Extend and Embrace?

BuckWoody — Tue, 13 Sep 2011 11:20:05 GMT

As most of you know, I don’t like the term “cloud” very
much. It isn’t defined, which means it can be anything. I prefer “distributed
computing”, which is more technically accurate and describes what you’re doing
in more concrete terms.

So when you think about Windows and SQL Azure, you don’t
have to think about an entire product – you can use parts of the system
together or independently to accomplish what you need to do. You can use the
computing functions, storage, and more and more I see folks leverage the
Service Bus to enable current applications to expose things to the web.

And that brings up the point of this post. Once you decide
that a distributed architecture works to solve a problem, you’re faced with a
decision: should you completely re-write your architecture to take advantage of
the current systems or should you just fold in new code that makes the data or
function available to the web?

Of course, the answer is always “it depends” on the situation
– and it does. But unless you’re fixing a problem with current code, I usually
advocate a migration approach. That means at the very least retaining the
business logic (again, unless it’s not currently working) and as much of the
code as you can. In fact, if you follow this paradigm, you’re on your way to
making a Service Bus out of the functions you currently have. You can expose
the results of a system rather than opening the system up. Let’s take an
example.

Assume for a moment that you have an order-taking system
on-premise. That system performs many functions, one of which might creating a
Purchase Order. Your system might be enclosed, meaning that it has an
application that talks to a middle-tier, and then from there to a database
system. A query is generated from a screen, and passed along to eventually
compute, store and return a Purchase Order Number, along with other
information. Imagine now that you wire up the code not only to return the PO
number to the client, but to make that number available on an endpoint –
actually really not that hard to do.

Now you can make that PO number available to the web using
Azure. You could restrict who can make that call to the system, or open it up
to a broader audience. Or instead of the PO Number, you could make a product
list available. And you can go further than that – EBay, for instance, uses the
OData protocol (which is very cool in and of itself) which you can query from
the web. You could compare your company’s product catalog to what is on EBay,
and list the items you have there if there are no competitors in that space.
And on and on it goes.

So the point is this – where you can, retain what works.
Fold in systems like Azure where they make sense. Extend and Embrace.

Plan for Diagnostics in Cloud Computing From the Git-Go

BuckWoody — Tue, 06 Sep 2011 13:11:22 GMT

“Git-Go” is something we say in the South that means “right at the start”. I’ve seen several applications for on-premise systems that don’t have much in the way of diagnostics - the developers rely on a debugger, the event logs on the server and client workstation, and most of all, the ability to watch the system from end-to-end.

This approach is a mistake for an on-premise system, and it’s definitely a problem for a distributed architecture. You simply do not own all of the components from end to end in a cloud environment, nor are you always able to attach a debugger or other remote monitoring tools to the various areas within the code path. So you need to make sure that from the very outset of your design that you build in diagnostics. My personal preference is to build a system such that a control file turns on deeper information gathering from the system, up to a minimal level.

When I do that, I set a high level of logging, a medium level, and a moderate level. I normally use the deepest level of information during the testing and acceptance phase of the deployment, then switch to moderate and then the least level of information gathering. Also in my design I often set an error condition to begin gathering the deeper information along with the exception, where possible.

There are decisions you need to make as to where to store the diagnostics (many operations in the cloud cost money), how often you collect them, and so on. You can get a quick overview on using the diagnostics that come with Windows Azure here: http://www.azuresupport.com/2010/03/getting-started-with-windows-azure-diagnostics-and-monitoring/ This is where you should start first. More detail on that: http://msdn.microsoft.com/en-us/library/gg433048.aspx

My friend Dave Pallman has a great tool he’s released for free: http://davidpallmann.blogspot.com/2009/03/azure-application-monitor-now-on.html

If the issue is in storage apps: http://social.msdn.microsoft.com/Forums/en-US/windowsazuredata/thread/d84ba34b-b0e0-4961-a167-bbe7618beb83

If you have System Center, this is the quickest and easiest way to implement the monitoring – really handy: http://pinpoint.microsoft.com/en-us/applications/windows-azure-application-monitoring-management-pack-release-candidate-12884907699

Should All Data Be Encrypted By Default?

BuckWoody — Tue, 09 Aug 2011 13:45:04 GMT

Recently several IT industry information outlets have reported that there has been a 10-year concentrated, organized effort on breaking through computer security at some of the largest companies in the world. Government sites have also been attacked in multiple countries. Add to this the regular loss of data by banking and other industries, and the fear of “the cloud” as a storage location, and it seems to beg the question asked in the title in this post: “should all data, everywhere, be encrypted by default?”

If you’re new to encryption, there’s an excellent video and overview here: http://blogs.msdn.com/b/plankytronixx/archive/2010/10/23/crypto-primer-understanding-encryption-public-private-key-signatures-and-certificates.aspx

If all data were encrypted, the break-in to websites would still continue, but the value would be lessened for some types of “orthogonal” attacks that only seek the pure stream of data.

Data States

Computing has two major components - static program elements and data. The program doesn’t change (until it is updated, of course) over the course of a transaction between a user and the ultimate data store. Data is classified as anything that is manipulated by the program. That implies three states of the data interchange: Creation, Transmission, and Storage. In on-premise systems, many times none of these states are encrypted. The entire system from user to data store is viewed as “secure”, which of course evidence has proved it is not. In some cases, even laptops are viewed as part of an on-premise system, and so is left unprotected. If all data were treated as “publicly viewable”, that mindset would lead to encrypting the data at all states, even for on-premise systems.

Creation

In this phase, a user, device or other input program creates data to send to the program. This can be entries on a web form, input from a weather sensor, or one service (program) sending information to another service. There are multiple ways to encrypt data at this state, most notably using client-side libraries such as the Windows Crypto API, hardware encryption and others. The reference for the Crypto API is here: http://msdn.microsoft.com/en-us/library/ms867086.aspx

Transmission

After the data is created, it needs to be transmitted to the processing and storage system. the references above explain how to secure the communications channel between the client systems and the various components used within the system. In the case of Windows Azure, the session can be protected with a secure session, and all communications within the Azure datacenters are encrypted. The key is that the transmission of data, regardless of method, should be considered to be “in the clear”, and treated as such. Without the decryption algorithm, it’s much harder to get to the ultimate goal.

Storage (data at rest)

It follows that f the data is encrypted at the source, and the decryption method is retained only with the code that processes the data, then the data “at rest” if obtained is less accessible. If the data is not encrypted at the source, then this step should be put into place at a minimum. In many cloud systems, including Windows and SQL Azure, the data is not encrypted at rest. There are various reasons for this, including performance, physical and logical security already in place, and the fact that the encryption process would expose customer data to the provider while it is being encrypted. In this case, the key is to encrypt the data before it is transmitted and stored, so that it is encrypted ahead of time.

Considerations

Encrypting data is a separate process, and must be factored into the original codebase. This means additional effort, and more CPU power for the encryption process (although many systems have security hardware included which help with this) and of course protecting the keys. If the keys are accessed, the data is considered unencrypted from then on, and all previous encryption with that particular key is now vulnerable. Key rotation and protection is essential. Even so, the benefits of treating all data as being at risk outweighs the efforts.

You can learn more about general encryption here: http://msdn.microsoft.com/en-us/library/aa380255(VS.85).aspx

Windows Azure Security Review

BuckWoody — Tue, 02 Aug 2011 13:24:50 GMT

Current as of 08/01/2011 - Check the Resources listed below for more up-to-date information on this topic

Background:

Security for any computing platform involves three primary areas:

Principals (users or programmatic access to an asset or other program)
Securables (objects, data or programs that can be accessed)
Channels (methods of access by Principals to Securables)

On-premise systems normally use a central system to control security. In a Windows operating system-based environment, this is often accomplished with Active Directory or other systems that provide sign-on and user identity information. While other networking security paradigms have different terminology, all involve the three areas defined above.

In addition to the names and passwords for a user, Active Directory (like other security mechanisms) store other information about Principals - called Claims. These claims can include any custom fields the provider allows. In many networks, these fields are not used heavily, because applications that eventually need to secure the assets they control are not always deployed on the same platforms everywhere.

In a single environment, security is often quite simple. A Principal is created such as a user or group, and then the Principal is granted access to a Securable such as a a folder, database or other asset. Permissions or Rights (or both) combine to allow a particular Principal to read, write, delete or edit data, or to access or run a particular program.

Figure 1 - On-premise security environment example

The simplicity of this arrangement is due to a single, homogenous boundary. Even if more than one location is used, the Principals and Securables are grouped into a single logical boundary that is managed from one location.

This background serves as the starting point for the Federating Security topic below.

Windows Azure Security Boundaries

Windows Azure is a series of resources - servers, data and service buses, in addition to other features. Developers write code, and the deploy that to the Azure environment.

Figure 2 - Azure Components

The code or data can be deployed to use one or more of the services. In other words, the Web Role in Windows Azure might host a simple website, and no other component need be used.

Figure 3 - Simple Azure Web Role Application - only one feature used

Or, a complex mix of Web, Worker and Data Services, along with a Service Bus, RDBS and even on-site systems can be grouped into a much larger program.

Figure 4 - Complex Windows and SQL Azure Application With Multiple Interactions

For a more basic introduction to Windows and SQL Azure, see this link: http://channel9.msdn.com/Events/TechEd/Europe/2010/COS322

Windows Azure, like any web-based property, has three general layers of security:

Physical Access
Operating Environment (Including the Operating System itself)
Data and Programmatic Security

Each of these layers have additional layers within themselves, and this forms the basis of a secure experience for the end user or program. Some of these layers are the responsibility of Microsoft; others are the responsibility of the architect and developer; others are a joint or shared responsibility of both Microsoft and the client.

Layer One: Physical Access

The first layer of security within a web property such as Windows or SQL Azure is a secure facility. the following data points are important to understand for the worldwide facilities that host Windows and SQL Azure:

Microsoft Global Foundation Services (GFS) is responsible for the physical security of the datacenters located worldwide for Windows and SQL Azure. Information on Microsoft datacenters can be found here: http://www.globalfoundationservices.com/
The address and exact locations facilities are not commonly documented for security reasons.
Microsoft runs it’s own data centers and does not contract this function out.
The GFS controlled facilities hold an ISO/IEC 27001:2005 certification, and are audited to SAS level II.
Standard secure operations protocols are in place, including least-privilege access.

Layer Two: Operating Environment

Windows Azure and SQL Azure do not currently hold certifications. Microsoft does not comment on the security certifications being pursued for Windows or SQL Azure. That being said, the Windows Azure environment is based on a modified Windows 2008 R2 Enterprise environment, developed using the Trustworthy Computing Initiative (TCI).

The system controlling the host machines and their guest environments that ultimately hold the Web and Worker Roles within Windows Azure is called the Fabric - not to be confused with the Application Fabric feature. The Fabric is not accessible by client code - it controls the inner workings of Windows Azure, including Load-balancing, system restarts, maintenance and monitoring.

Within the host machines that house the Web and Worker Roles, special networking constructs broker all conversations between Virtual Machines. Virtual Machines - even ones configured to communicate with each other - move through this network. Direct-machine to machine communication is not allowed, protecting one application from another or one data construct from another.

Figure 5 - Windows Azure Fabric

Windows and SQL Azure support only TCP-based communications. Ports commonly used are:

80 - Default public port used for Web Roles - can be enabled/disabled per configuration
443 - Default secure port used for Web roles - can be enabled/disabled per configuration
9350-9353 - These ports are used by the Windows Azure AppFabric service bus bindings. Refer to http://msdn.microsoft.com/en-us/library/ee732535.aspx for more details
1433 - SQL Azure
3389 - This port is used for RDP access to VM-based roles, only if enabled

Layer Three: Data and Programmatic Security

All internal access through use of keys only. Without the proper key, code or data will not transfer. Storage Accounts have individual keys, so in this manner different security layers may be applied not only programmatically but at the account layer.

Figure 6 - Windows Azure communications between components

Calls to Windows Azure are made using standard SOAP, XML or REST-based protocols. The communications channel can be encrypted between the client and Windows Azure or allow it to remain unencrypted based on security needs.

SQL Azure uses the standard SQL Server Tabular Data Stream (TDS) protocol, but only allows encrypted communications.

Data is unencrypted within Windows Azure Blob or Table Storage - but is only accessible via the key for a storage account. Data can be encrypted client-side and stored in Windows Azure in an encrypted fashion. Microsoft does not inspect internal data for validity or encryption enforcement. The key is that the data is client-side encrypted and decrypted.

Figure 7 - Example data at rest encryption scenario

Alternatively, a hybrid solution can store sensitive data locally and non-sensitive data in Azure Storage. The data can be coalesced at the client level such that the data is never transferred over any channel not owned or controlled by the organization.

Federating Security:

In the case of a single security boundary for Windows Azure, multiple security options are available. Users can be anonymously authorized, such as in the case of a public website for advertisement or informational purposes.

Another option is to create an Internet Information Services (IIS) Internal Security Store. This is not a best-practice (although still possible) approach since the Fabric services within Windows Azure may recycle an instance and the session may sever between a given role and a client. Architecting stateless applications is a preferred approach.

Using Claims-Based Authentication is a better solution. In this approach, the Principal is authenticated through a trusted party, such as Active Directory, OpenID, OpenAuthentication, or LiveID. Many web-properties use these methods, such as Microsoft, Google, Yahoo and Facebook to name a few. After authenticating with one of these services, the client is issued Claims using the WS-Federation (WS-Fed) or Security Assertion Markup Language (SAML) that are passed to Windows Azure. At no time does Windows Azure store, transfer or interrogate the Principal’s security token. Claims can be anything from a group or role membership to location or any other settable attribute. Assets are then secured allowing only the Claim, without regard to the user’s location or access method. In this fashion a single security paradigm covers the Securables, with the Principals being controlled in any number of other mechanisms. This allows single-sign-on and/or federated security access from multiple providers.

The simplest mechanism for building this environment is the Access Control Services (ACS) feature found in the Windows Azure Application Fabric component. It is a federated authorization management service that simplifies user access authorization across organizations and ID providers and performs claims transformation to map identities with access levels.

ACS can:

Create and manage scopes such as URLs
Create and manage claim types
Create and manage signing and encryption keys
Create and manage rules within an application scope
Chain claims rules
Manage permissions on scopes or perform delegation

Figure 8 - Federated Security Example

Full information on the Access Control Service is available at this link: http://social.technet.microsoft.com/wiki/contents/articles/windows-identity-foundation-wif-and-azure-appfabric-access-control-service-acs-survival-guide.aspx?wa=wsignin1.0

Since the Web and Worker Roles within Windows Azure are designed to be stateless, Microsoft created a Certification Store within the Management area to hold Certificates that can be called from within code. An example of using the Certification Store is here: http://blogs.msdn.com/b/jnak/archive/2010/01/29/installing-certificates-in-windows-azure-vms.aspx

Additional Resources:

Official, authoritative security resource list: http://msdn.microsoft.com/en-us/library/ff934690.aspx
Technical Overview of the Security Features in the Windows Azure Platform: http://www.microsoft.com/online/legal/?langid=en-us&docid=11.
Windows Azure Security Overview: http://www.globalfoundationservices.com/security/documents/WindowsAzureSecurityOverview1_0Aug2010.pdf
Windows Azure Privacy: http://www.microsoft.com/online/legal/?langid=en-us&docid=11
Securing Microsoft Cloud Infrastructure: http://www.globalfoundationservices.com/security/documents/SecuringtheMSCloudMay09.pdf.
A list of other security resources is here: http://blogs.msdn.com/b/buckwoody/archive/2010/12/07/windows-azure-learning-plan-security.aspx

Image Attribution: David Pallmann: http://davidpallmann.blogspot.com/2011/07/windows-azure-design-patterns-part-1.html

The Conundrum of Cloud Computing Evidence

BuckWoody — Tue, 26 Jul 2011 12:56:08 GMT

“Who is using Windows Azure? How are they doing it?”

I get asked this all the time as I speak to our clients when we mention Windows or SQL Azure as a possible solutions to an architectural problem the company has. I completely understand the question. I’ve worked far longer outside of Microsoft than here, and one of my roles as a Systems Architect was to select solutions from a range of possibilities. When you’re faced with a decision like that, it saves a ton of time if you can find out where others have done the same thing and what problems and successes they had with this or that approach. I’m not a marketing person, so I work with clients directly all the time, and so folks know I’ve seen the way others do things. And they want to know what someone else has done before they try it.

But therein lies the rub.

Most folks aren’t willing to talk about their internal infrastructure much. We talk about how we do things here at Microsoft quite frequently, and if you’re not using this resource, definitely go research what we’ve done. In fact, I don’t think I’ve seen many other technology companies with this depth of information. For instance, here’s a quick list of the areas we talk about on our internal systems:

Microsoft Desktop Optimization Pack

Office System

System Center

Windows

And yes, Azure as you can see is part of that list - we’ve implemented it everywhere. But when I point out that we’re using IT resources just like any other company, I hear “yeah, but you’re Microsoft. You have lots of people to handle the systems, and you can just have the developer who wrote the software come over and fix it if it breaks.” Well, that’s not the way it works at all. When I got here, I met with some folks from Microsoft IT, and I was absolutely shocked at how few people manage our systems. From Systems Administrators all the way to the DBA’s, the ratio of technical resource to machines is really extraordinary, and I had far more folks managing systems for me in my other companies than we have here. And no, we don’t send developers to fix production systems “just for us”, although we will do that when we are using a beta of something that we haven’t even released to CTP yet.

But even with this evidence of ourselves, clients want to “see how others are doing things with Azure”. We do have the customer evidence site, and we have lots of case-studies there. But many of these lack technical details, again, because folks aren’t always willing to share that. That’s understandable - for instance, even though Microsoft does it, I’m not sure I would have the Visio diagrams for my company’s applications posted in a public place. There may even be legal or competitive reasons not to do that. So as you peruse that list, keep in mind these are only the companies that are willing to talk in public about their experience - we have far more on Azure than this list.

J.D. Meiers has a good blog post entry of a rundown of the companies that have released case studies on Azure. And be assured that we have a patterns and practices group that works with the teams here at Azure to make sure we apply what we learn as we help each client. And many of our partners that you use to help you implement technology are here at Microsoft this week, sharing customer stories (with permission, of course), which allows us to learn from them on how you’re doing things - what works and what doesn’t.

As time goes on I think you’ll see more patterns emerge from the engagements we’ve done. Right now those are sometimes a competitive advantage, so you’ll see us generalize the patterns into content we can release. For now, check out those case studies, and do make use of the Internal Microsoft IT sites. They really are quite detailed, and have a lot of information you can use.