Normally I try to put topics in the positive in other words "Do this" not "Don't do that". Sometimes its clearer to focus on what *not* to do. Popular development processes often start with screen mockups, or user input descriptions. In a scale-out pattern like Cloud Computing on Windows Azure, that's the wrong place to start.

Start with the Data

   Instead, I recommend that you start with the data that a process requires. That data might be temporary or persisted, but starting with the data and its requirements helps to define not only the storage engine you need but also drives everything from security to the integrity of the application. For instance, assume the requirements show that the user must enter their phone number, and that this datum is used in a contact management system further down the application chain. For that datum, you can determine what data type you need (U.S. only or International?) the security requirements, whether it needs ACID compliance, how it will be searched, indexed and so on. From one small data point you can extrapolate out your options for storing and processing the data. Here's the interesting part, which begins to break the patterns that we've used for decades: all of the data doesn't have the same requirements. The phone number might be best suited for a list, or an element, or a string, with either BASE or ACID requirements, based on how it is used. That means we don't have to dump everything into XML, an RDBMS, a NoSQL engine, or a flat file exclusively. In fact, one record might use all of those depending on the use-case requirements.

Next Is Data Management

 With the data defined, we can move on to how to store the data. Again, the requirements now dictate whether we need a full relational calculus or set-based operations, or we can choose another method based on the requirements for the data. And breaking another pattern its OK to store in more than once, in more than one location. We do this all the time for reporting systems and Business Intelligence systems, so this is a pattern we need to think about even for OLTP data.

Move to Data Transport

How does the data get around? We can use a connection-based method, sending the data along a transport to the storage engine, but in some cases we may want to use a cache, a queue, the Service Bus, or Complex Event Processing.

Finally, Data Processing

Most RDBMS engines, NoSQL, and certainly Big Data engines not only store data, but can process and manipulate it as well. Its doubtful that you'll calculate that phone number right? Well, if you're the phone company, you most certainly will. And so we see that even once we've chosen the data type, storage and engine, the same element can have different computing requirements based on how it is used.

Sure, We Need A Front-End At Some Point

Not all data is entered by human hands in fact most data isn't. We don't really need a Graphical User Interface (GUI) we need some way for a GUI to get data into and out of the systems listed earlier.  

But when we do need to allow users to enter or examine data, that should be left to the GUI that best fits the device the user has. Ever tried to use an application designed for a web browser on a phone? Or one designed for a tablet on a phone? Its usually quite painful. The siren song of "We'll just write one interface for all devices" is strong, and has beguiled many an unsuspecting architect. But they just don't work out.  

Instead, focus on the data, its transport and processing. Create API calls or a message system that allows for resilient transport to the device or interface, and let it do what it does best.

References

Microsoft Architecture Journal:   http://msdn.microsoft.com/en-us/architecture/bb410935.aspx
Patterns and Practices:   http://msdn.microsoft.com/en-us/library/ff921345.aspx
Windows Azure iOS, Android, Windows 8 Mobile Devices SDK: http://www.windowsazure.com/en-us/develop/mobile/tutorials/get-started-ios/
Windows Azure Facebook SDK: http://ntotten.com/2013/03/14/using-windows-azure-mobile-services-with-the-facebook-sdk-for-windows-phone/

Cloud computing is actually being largely driven by the “Consumerization of IT”. That phrase, as grammatically incorrect as it is, represents a fundamental change to the way businesses think about technology, and subsequently how the IT team provides it.

Years ago, technology was introduced by the office. No one owned a mainframe at home of course, and even in the early years of PC’s few people could afford to have them in their houses. Other than game consoles and hobbyists on small computers, most full-up “PC’s” were used for  work. 

That rapidly changed, with the lowering of costs and miniaturization of technology. PC’s and then laptops became ubiquitous in the home, and of course the “smart phone” ushered in an entire generation where the technology available to the consumer outpaced what is installed at the place of work. Many of us have laptops  that are more powerful than some of the servers the company uses in some applications.

IT as a department grew up in the era of the “office-first” technology. Modern users, especially those controlling the budget, are now more “home-first” technology buyers. In extreme cases, I’ve seen IT departments relegated to maintenance of legacy systems, with new IT projects being scoped, designed and run by business teams – usually on a Cloud Computing platform. The business wants to create a technical solution as quickly as they can download an app to their phone. They want the same level of speed and ease that they have on home technology in their business work.

However, this can be problematic if not thought through. As with any new technology, Cloud Computing provides both benefits and concerns. It’s true that almost anyone can quickly stand up a server or deploy an application quickly with nothing more than an e-mail address and a credit card. But business teams are not always aware of areas such as security or similar concerns that the IT teams solved through many hours of careful planning. Unfortunately, it’s often a matter of “Ready, Fire, Aim.”

So what is the business (who wants the agility of a smart phone and a single-click solution) to do? What about the need for security, strategic design, integration and all of the other functions that IT needs to handle? This is where I think Windows Azure (not to be too sales-y) handles the situation well.

If you’re using another cloud provider, by the way, that’s fine. The concepts here are the same.

Microsoft sells an on-premises operating system, and has done for many years. We’ve architected Windows Azure Virtual Machines, Active Directory Services, Platform-as-a-Service, and even the Hadoop and other offerings to work together – and with the tools that you use to manage them today, like System Center and PowerShell.

   To the business team, I say this:

• Work with your IT staff on projects, even if you’re designing the project and paying for it – the IT professionals can keep you out of danger. Most of them have made the mistakes you're going to make, and know what to do to avoid them.
• Plan for the future – “This is just a proof-of-concept” project becomes productions in a frighteningly quick period of time.
• Understand the cost model – a good architect can solve one problem in multiple ways, and cost is always a vector. The IT team can help you with this - they have the relationships with the vendors to consolidate and help you understand those costs.

     To the IT team, I have this advice:

• Don’t stand in the way of the business – they’ll just go around you. Work with them.  Enable the business to do what they need, when they need it, and they’ll work with you. I've seen both results when I witnessed the mainframe-to-the-PC transition, and I'm seeing it again in the PC-to-the-cloud transition. Change is inevitable - get on board or become irrelevant to the people who pay your salary.
• Learn the cloud. Talk to your vendor, get training, read up, ask questions. If this bothers the vendor, get a different one.
• Create a self-service portal. This point may be the most important one. Become your own “Cloud”, and your users won’t need to go elsewhere.  I’ll talk more about how to do this in another post.

In the end, the relationship between IT teams and Business is eerily similar to a marriage – it’s an amazing thing, it takes a lot of work to get right, and the "Consumerization of IT" is that cute person at the end of the bar.Work together or one of you will soon be with somebody new.

f you want to know more about Windows Azure, how it works, the components, or more about the entire platform, I’ve written about that here: http://blogs.msdn.com/b/buckwoody/archive/2012/06/13/windows-azure-write-run-or-use-software.aspx

But….

Maybe you just want to cut to the chase. Windows Azure. What do I *do* with it? How about...create some websites. Or website applications. Or both. For free. OK, ten of them are free, then you have to pay for more.

This week I wanted to set up a DotNetNuke Content Management System (More here if you don't know what that is: http://www.dotnetnuke.com/Products/DotNetNuke-Platform.aspx) for a charity I work with. DotNetNuke (DNN) is an open-source project, all ready to go and easy to manage place for web parts, content, blogs, whatever. With Windows Azure, you have the ability to quickly and easily create websites based on ASP or PHP code, for free. You also have the ability to use packages from a gallery, and one of those packages is DotNetNuke - both the community and the professional (pay for) use. I set this one up in 9 minutes:

It's easy to set these up. A simple website where you can deploy ASPX or PHP code is just a few clicks, but while I was setting up my site I figured I'd grab the screenshots and show them to you here.

First, you need an account - you can get one of those for free: https://www.windowsazure.com/en-us/pricing/free-trial/

After you sign up for the account, hit the http://windowsazure.com site and click the "Portal" button at the top right of the screen. Then click the second icon down, called "Web Sites":

Click the "Create a New Web Site" link on the screen and you're shown this menu:

If you want a quick web site, just click "Create Web Site". If you want another type, click "From Gallery" and make your choice:

I selected the Community Edition of DotNetNuke. That brings up a configuration panel that looks like this:

You'll have to pick a name that isn't already in use, and in my case I told the system to create and build a SQL Azure (Windows Azure SQL Database) to hold the data. You'll also need to pick a region. After you make those selections, you'll need to enter the information for the database server and database:

Write down the database name, database administrator name and password - you'll need those later.

After that, you'll see the system deploying the code, creating the database server and so on.

From there, you're all set.

Whenever you want to monitor the site's health, you can just click the name here in the Portal to get more information on it:

Write down the URL of the site so you can access it in a moment. But don't move off of this screen - Windows Azure is now all set up, but DotNetNuke needs a little info when you first log in.

Before you leave the Portal, click the "DB" icon, and click the name of the database server you created a moment ago (blanked out here on my graphic):

Write down the entire server name (looks like myserver.database.windows.net) and database name (looks like mydatabasename) from this panel.

Now open your new DotNetNuke URL in your browser, and DotNetNuke will take over. You'll be asked the name of the database server (type in the whole name with the .database.windows.net part) and database name, and the database admin name and password you wrote down earlier.

You'll be asked to name the first site, and create a DotNetNuke admin name and password. Write all that down too.

Now log in to your DotNetNuke site with the admin name and change the site to whatever you like! It defaults to "Awesome Cycles", but since you probably don't want that one, read up on what to do with DNN once you're here:

PS - Want to do more than just deploy a canned site? Write code! Do HTML5! BE the Web: https://www.windowsazure.com/en-us/develop/net/tutorials/get-started/ 

"DevOps" (Short for Developer Operations) is one of a group of new terms such as "Cloud", "Big Data" and "Data Scientist" - words that are somewhere between marketing and tasks we've actually had around in other forms for years.However, working in a Distributed Environment (Both on and off premises)  like Windows Azure does bring a new set of tasks to the operations we currently perform in Information Technology.

Before I offer some guidance here, I need to carefully define the term "DevOps" as I use it.There are other definitions that involve Application Lifecycle Management (ALM) and standard operations policies, and you're free to use those as well, but this is the definition I'll use for this post:  By DevOps I mean those tasks involved with deploying, managing and monitoring a Windows Azure (or hybrid) project.

Another caveat: This is a non-authoritative, non-comprehensive post. I'll include only an outline of the major tasks, not a complete manual on the topic. There's enough knowledge needed on this topic for at least a whitepaper or two, and perhaps even a book, but for the moment I wanted to get some information out to ensure you have something to work from until those come along.This is primarily a list of resources for a DevOps team.

With all of those caveats in mind, we'll start the discussion after the project is conceived and architected. In most cases the DevOps team (whether that is a dedicated team or simply part of what the current IT Ops team does) is also involved in the design, at least from an information point of view. There's a great overview of the entire process available in poster form here: http://www.microsoft.com/en-us/download/details.aspx?id=36837 And you should also read this complete manual in preparation here: http://msdn.microsoft.com/en-us/library/hh871440.aspx

Deployment

The first task after the design of the project is deployment. The deployment method depends on the type of solution; Windows Azure has the ability to run VM's, software code, or provide services that are already created (such as Active Directory). 

IaaS

Deploying Virtual Machines:

Manually from the Portal: http://go.microsoft.com/fwlink/?linkid=254427&clcid=0x409

Through Scripting: https://www.windowsazure.com/en-us/downloads/?fb=en-us, http://msdn.microsoft.com/en-us/library/ee460812.aspx

Copying your own VM's to Windows Azure: http://msdn.microsoft.com/en-us/library/windowsazure/gg465385.aspx 

Using System Center: http://www.techrepublic.com/blog/datacenter/deploy-an-on-premise-vm-to-windows-azure-with-app-controller/5919 

Virtual Networking: http://msdn.microsoft.com/en-us/library/windowsazure/jj156075.aspx, http://channel9.msdn.com/Shows/Cloud+Cover/Episode-88-Tips-and-Tricks-for-Windows-Azure-Virtual-Machines-and-Virtual-Networks,

PaaS

Through Visual Studio: http://www.microsoft.com/BizSpark/Azure/HowToDeployAzureApp.aspx

Using CSPack: http://msdn.microsoft.com/en-us/library/windowsazure/gg432988.aspx

Through Scripting: https://www.windowsazure.com/en-us/downloads/?fb=en-us, http://msdn.microsoft.com/en-us/library/ee460812.aspx

SaaS

Manually from the Portal:https://datamarket.azure.com/

Through Scripting: https://www.windowsazure.com/en-us/downloads/?fb=en-us

Monitoring

Monitoring the system after deployment involves watching the availability and uptime of the system, along with security intrusions and tracking access through code.

Health

Using MetricsHub: http://channel9.msdn.com/Shows/Cloud+Cover/Episode-102-Using-MetricsHub-to-Monitor-Your-Windows-Azure-Applications

Uptime and Availability through the Portal: http://www.windowsazure.com/en-us/support/service-dashboard/

Uptime and Availability through Third Party Vendors: http://www.paraleap.com/AzureWatch, http://blogs.msdn.com/b/buckwoody/archive/2012/07/03/management-and-monitoring-tools-for-windows-azure.aspx

Automatic Notification: http://www.codeproject.com/Articles/375892/Adding-SMS-notifications-to-your-Windows-Azure-pro

Performance

Performance Counters: http://msdn.microsoft.com/en-us/library/windowsazure/hh411520.aspx 

Logging Diagnostics PaaS: http://msdn.microsoft.com/en-us/library/windowsazure/gg433048.aspx

Internal Instrumentation for PaaS: http://msdn.microsoft.com/en-us/library/windowsazure/hh674491%28v=vs.103%29.aspx 

Third Party Performance Testing: http://www.neustar.biz/enterprise/web-performance, http://blogs.msdn.com/b/buckwoody/archive/2012/07/03/management-and-monitoring-tools-for-windows-azure.aspx

Costs

Understanding Costs: http://msdn.microsoft.com/en-us/library/ff803372.aspx, http://technet.microsoft.com/en-us/magazine/gg213848.aspx

Subscription Management: http://msdn.microsoft.com/en-us/library/windowsazure/gg465713.aspx 

System Center: http://technet.microsoft.com/en-us/library/hh221354.aspx

Third-Party Tools: http://blogs.msdn.com/b/buckwoody/archive/2012/07/03/management-and-monitoring-tools-for-windows-azure.aspx

Example of listing your deployments: http://msdn.microsoft.com/en-us/library/gg651127.aspx

Management

Managing the deployment involves Security, Upgrades, Troubleshooting, and High-Availability/Disaster Recovery.

Windows Azure Management Portal: http://www.windowsazure.com/en-us/ 

Management API's: https://www.windowsazure.com/en-us/downloads/?fb=en-us and http://msdn.microsoft.com/en-us/library/ee460812.aspx, http://www.packtpub.com/sites/default/files/2220-chapter-7-managing-hosted-services-with-the-service-management-api.pdf?utm_source=packtpub&utm_medium=free&utm_campaign=pdf

Security

Security Trust Center: http://www.windowsazure.com/en-us/support/trust-center/

Working with Windows Azure Active Directory: http://blogs.msdn.com/b/windowsazure/archive/2012/11/28/windows-azure-now-supports-federation-with-windows-server-active-directory.aspx

Windows Azure Authentication: http://www.asp.net/vnext/overview/fall-2012-update/windows-azure-authentication

Deploying a secure ASP.NET MVC application with OAuth: http://blogs.msdn.com/b/webdev/archive/2013/03/12/deploy-a-secure-asp-net-mvc-application-with-oauth-membership-and-sql-database.aspx

Upgrades

ALM Process for PaaS: http://sqlblog.com/blogs/buck_woody/archive/2011/01/25/windows-azure-use-case-agility.aspx 

Troubleshooting

Windows Azure Support: http://www.windowsazure.com/en-us/support/contact/

Upgrade and Fault Domains: http://blog.toddysm.com/2010/04/upgrade-domains-and-fault-domains-in-windows-azure.html

HADR

Load-Balancing Endpoints for IaaS: http://www.windowsazure.com/en-us/manage/windows/common-tasks/how-to-load-balance-virtual-machines/

Extending SQL Server HADR to Windows Azure: http://blogs.msdn.com/b/buckwoody/archive/2013/01/08/microsoft-windows-azure-disaster-recovery-options-for-on-premises-sql-server.aspx 

HADR for IaaS: http://www.visionsolutions.com/, http://blogs.technet.com/b/windowsserver/archive/2012/03/28/microsoft-online-backup-service.aspx

Multiple Instances for PaaS: http://msdn.microsoft.com/en-us/library/windowsazure/ee871996.aspx 

Business Continuity for Windows Azure: http://msdn.microsoft.com/en-us/library/windowsazure/hh873027.aspx, http://blogs.msdn.com/b/avkashchauhan/archive/2011/10/14/windows-azure-vm-downtime-due-to-host-and-guest-os-update-and-how-to-manage-it-in-multi-instance-windows-azure-application.aspx

Disposition

When the project is complete, you'll need to remove the VM's in IaaS, or data and code from PaaS and shut down the deployment. Prior to doing that, you should:

1. Copy all data from the deployment to a local repository
2. Document the process
3. Notify Microsoft of your intent to stop the project to work with your representative on billing matters

The primary tool for disposal is the Windows Azure Portal.

I've recently posted a blog on how cloud computing would change the Systems Architect’s role in an organization, another on how the cloud changes a Database Administrator's job, and the last post dealt with the Systems Administrator. In this post I'll cover the changes facing the Software Developer when using the cloud.

The software developer role was the earliest adopter of cloud computing. This makes perfect sense, because the software developer has always used computing "as a service" - they (most often) don't buy and configure servers, platforms and the like, they write code that runs on those platforms. And there's probably not a simpler definition of a software developer to be found, but as with all simple statements, you lose fidelity and detail.  I'll offer a more complete list in a moment.

Because the software developer's process involves designing, testing and writing code locally and then migrating it to a production environment, all of the paradigms in cloud computing - from IaaS to PaaS to SaaS - come naturally.

The Software Developer's Role

The software developer has evolved since the earliest days of programming.The software developer not only "writes code"  - there are far more tasks involved in modern systems development:

• Assisting the Business Role(s) in developing software specifications
  
• Planning software system components and modules
  
• Designing system components
  
• Working in teams writing classes, modules, interfaces and software endpoints
  
• Designing data layouts, architectures, access and other data controls
• Designing and implementing security, either programmatic, declarative, or referential
  
• Mixing and matching various languages, scripting and other constructs within the system
  
• Designing and implementing user and account security rights and restrictions
• Designing various software code tests - unit, functional, fuzz, integration, regression, performance and others
• Deploying systems
  
• Managing and maintaining code updates and changes
  

Like most of the previous roles, those tasks also unpacks into a larger set of tasks, and no single developer has exactly that same list. And like the DBA, the role is often more, or less of that list based on where the developer works. Smaller companies may include the development platform in the duties so that a developer is also a systems administrator. In larger organizations I've seen developers that specialized on User Interfaces, Engine Components, Data Controls or other specific areas.

How the Cloud Changes Things

The software developer role obviously has the same concerns and impacts of "the cloud" as the Systems Architect. They need to educate themselves on the options within this new option (Knowledge), try a few test solutions out (Experience) and of course work with others on various parts of the implementation (Coordination).

The big changes for a developer include three major areas: Hybrid Software Design, Security, and Distributed Computing.

Hybrid Software Design

After the PC revolution, software developers designed systems that ran primarily on a single computer. From there the industry moved to "client/server", where most of the code still lived on the user's workstation, and various levels of state (such as the data layer) moved to a server over fast connected lines. After than followed the Internet phase, which had less to do with HTML coding than it did with state-less architectures. While no architecture is truly stateless, there are ways of allowing the client to be in a different state than the server of the application at any one time - this is the way the Web works.

Even so, the developer often simply moved one the primary layers (such as Model, View or Controller) to the server, using the User Interface merely as the View or Presentation layer. While technically stateless, this doesn't require a great deal of architecture change - there are various software modules that run on a server, and perhaps that connects to a remote data server. In the end, it's still a single paradigm. 

We now have the ability to run IaaS (hardware abstraction), PaaS (hardware, operating system and runtime abstraction) and SaaS (everything abstracted, API calls only) in a single environment such as Windows Azure. A single application might have a Web-based Interface Server with federated processes  (using a PaaS set of roles), a database service (using a SaaS provider such as Windows Azure SQL Database), a specialized process in Linux (using an IaaS role in Windows Azure) and a translator API (from the Windows Azure Marketplace). This example involves only one vendor - Microsoft. I've seen applications that use multiple vendors in this same way.

Thinking this way opens up a great deal of flexibility - and complexity. Complexity isn't evil; it's how complicated things get done many times. The modern developer  needs to understand how to build hybrid software architectures.

Resources: Hybrid Architectures with step-by-step instructions and examples: http://msdn.microsoft.com/en-us/library/hh871440.aspx and Windows Azure Hybrid Systems: http://msdn.microsoft.com/en-us/library/hh871440.aspx?AnnouncementFeed 

Security

Having a single security boundary, such as "everyone who works in my company", is a relatively simple problem to solve. Normally the System Administrators configure and control a security provider, such as Active Directory, and developers can access that security layer programmatically.  That allows for good separation of duties and role-based control.

In modern applications, clients, managers, and users both internal and external need various levels of access to the same objects, code and data. A client should be able to enter an order, a store should be able to accept the order, the credit-card company should be able to check the order and authorize payment, and the managers should be able to report on the order or change it if needed. Using role-based security across multiple domains would be impossible to maintain.

Enter "claims-based" authentication. In this paradigm, the user logs in with whatever security they use - corporate or other Active Directory, Facebook, Google, whatever. The application (using Windows Identity Foundation or WIF) can accept a "claim" from that provider, and the developer can match whatever parts of that claim they wish to the objects, code and data. And example might be useful.

Buck logs in to his corporate Active Directory (AD), and attempts to use a program based in Windows Azure. Windows Azure rejects the login silently, and is configured to check with Buck's AD. Buck's AD says "yes, I know Buck, and he has been granted the following claims: "partner", "manager", "approver". The developer does not need to know about Buck's AD, Buck, his login, or anything else. She simply codes the proper data access to allow "approver" to approve a sale. 

This allows a lot of control, at a very fine level, without having to get into the details of each security provider. .

Resources: Overview of using claims-based Azure Security: http://adnanboz.wordpress.com/2011/02/06/claims-based-access-and-windows-azure/

Distributed Computing

Is there a difference between stateless computing, or even the hybrid programming I mentioned earlier, and "Distributed Computing"? Yes - the primary difference is latency. Even stateless code can have too small a tolerance for latency. 

Dealing with slow connectivity, or breaks in connections has many impacts. One method of dealing with this is to locate data and computing of that data as closely as possible, even if this means relaxing consistency or duplicating data. Another method is to go back to a great paradigm from the past that is possible underused today is a Service Oriented Architecture. The Windows Azure Service Bus is possibly one of the fastest and easiest way to adopt cloud computing without completely rearchitecting your application.

References: Great breakdown of the thought process around a distributed architecture: http://msdn.microsoft.com/en-us/magazine/jj553517.aspx and using a Windows Azure Relay Service: http://www.windowsazure.com/en-us/develop/net/how-to-guides/service-bus-relay/ 

I recently posted a blog entry on how cloud computing would change the Systems Architect’s role in an organization. In a way, the Systems Architect has the easiest transition to a new way of using computing technologies. In fact, that’s actually part of the job description. I mentioned that a Systems Architect has three primary vectors to think about for cloud computing, as it applies to what they should do:

1. Knowledge - Which options are available to solve problems, and what are their strengths and weaknesses.
2. Experience - What has the System Architect seen and worked with in the past.
3. Coordination - A system design is based on multiple factors, and one person can't make all the choices. There will need to be others involved at every level of the solution, and the Systems Architect will need to know who those people are and how to work with them.

The Database Administrator Role

But a Database Administrator (DBA) is probably one of the harder roles to think about when it comes to cloud computing. First, let’s define what a Database Administrator usually thinks about as part of their job:

• Planning, Installing and Configuring a Database Platform
• Planning, designing and creating databases
• Planning, designing and implementing High Availability and Disaster Recovery for each database (HADR) based on requirements for its workload
• Maintaining and monitoring the database platform
• Implementing performance tuning on the databases based on monitoring
• Re-balancing workloads across database servers based on monitoring
• Securing databases platforms and individual databases based on requirements and implementation

That’s just a short list, and each of those unpacks into a larger set of tasks.

The issue is that I’ve never actually met a DBA that does all of those things, or just all of those things. Many times they do much more, sometimes the systems are so large they specialize on just a few of them.

And as you can see from the list, some of these areas are shared with other roles. For instance, in some shops, the DBA plans, purchases, sets up and configures the hardware for database servers. In others that’s done
by the Infrastructure Team. In some shops the DBA designs databases from software requirements, and in others the developers do that – or perhaps it’s done as a joint effort. The same holds true for database code – sometimes the
DBA does it, other times the developer, and still others it’s a shared task.

In fact, you could argue that there are few other roles in IT where the roles are so intermixed. Also, the DBA works with software the company develops, and software the company buys. They work with hardware, networking, security and software. There are certain aspects of design and tuning that are outside the purview of some of those things, and inside the others.

With all of these variables, simply telling a DBA that they should “use the cloud” is not the proper approach.

How the Cloud Changes Things

To be sure, the DBA has the same vectors as the Systems Architect. They need to educate themselves on the options within this new option (Knowledge), try a few test solutions out (Experience) and of course work with others on various parts of the implementation (Coordination). But it goes beyond that.

There are three big buckets of cloud computing, dealing with simply using a Virtual Machine (IaaS) to writing code without worrying about the virtualization or even the operating system (PaaS) and using software that’s already written and being delivered via an Application Programming Interface (API). Each of these has so many options and configurations that it’s often better to think about the problem you’re trying to solve rather than all of the technology within a given area - although some of that is certainly necessary anyway. 

Database Platform Architecture

I’ll start with when the DBA should even consider cloud computing for a solution. Once again, it’s not an “all or nothing” paradigm, where you either run something on premises or in the cloud – it’s often a matter of selecting the right components to solve a problem.  In my design sessions with DBA’s I break these down into three big areas where they might want to consider the cloud –and then we talk about how to implement each one:

1. Audiences
2. HADR
3. Data Services

Audiences

If the users of your database systems all sit in the same facility, you own the servers and networking, and the application servers are separate from the database server, it doesn’t usually make sense to take that database workload and place it on Windows Azure – or any other cloud provider. The latency alone prevents a satisfactory performance profile, and in some cases won’t work at all. It doesn’t matter if the cloud solution is cheaper or easier – if you’re moving a lot of data every second between an on-premises system and the cloud it won’t work well.

However – if your users are in multiple locations, especially globally, or you have a mix of company and external customer users, it might make sense to evaluate a shared data location. You still need to consider the implications of how much data the application server pushes back and forth, but you may be able to locate both the application server and SQL Server in an IaaS role. Assuming the data sent to the final client will work across public Internet channels, there may be a fit. There are security implications, but unless you have point-to-point connections for your current solution you’re faced with the same security questions on both options.

Your audience might also be developers looking for a way to quickly spin up a server and then turn it down when they are done, paying for the time and not the hardware or licenses. This is also a prime case for evaluating IaaS. And there are others that you'll find in your own organization as you work through the requirements you have.  

Resources: Windows Azure Virtual Machines: http://www.windowsazure.com/en-us/manage/windows/tutorials/virtual-machine-from-gallery/ and Windows Azure SQL Server Virtual Machines: http://www.windowsazure.com/en-us/manage/windows/common-tasks/install-sql-server/

HADR

The next possible place to consider using cloud computing with SQL Server is as a part of your High Availability and Disaster Recovery plans. In fact, this is the most common use I see for cloud computing and the Database Administrator. The key is the Recovery Point Objective (RPO) and Recovery Time Objective (RTO). Based on each application’s requirements, you may find that using Windows Azure or even supplementing your current plan is
the right place to evaluate options. I’ve covered this use-case in more detail in another article.

References: SQL Server High Availability and Disaster Recovery options with Windows Azure: http://blogs.msdn.com/b/buckwoody/archive/2013/01/08/microsoft-windows-azure-disaster-recovery-options-for-on-premises-sql-server.aspx

Data Services

Windows Azure, along with other cloud providers, offers another way to design, create and consume data. In this use-case, however, the tasks DBA’s normally perform for sizing, ordering and configuring a system don’t apply.

With Windows Azure SQL Databases (the artist formerly known as SQL Azure), you can simply create a database and begin using it. There are places where this fits and others where it doesn’t, and there are differences, limitations and enhancements, so it isn’t meant as replacement for what you could do with “Full-up” SQL Server on a Windows Azure Virtual Machine or an on-premises Instance. If a developer needs an Relational Database Management
(RDBMS) data store for a web-based application, then this might be a perfect fit.

But there is more to data services than Windows Azure SQL Databases. Windows Azure also offers MySQL as a service, RIAK and MongoDB (among others) and even Hadoop for larger distributed data sets. In addition you can use Windows Azure Reporting Services, and also tap into datasets and data functions in the Windows Azure Marketplace.

The key for the DBA with this option is that you will have to do a little investigation this time, and potentially without a specific workload in mind this time. I think that’s acceptable thing to ask – DBA’s constantly keep up with data processing trends, and most will consider different ways to solve a problem.

References:

Windows Azure SQL Databases: http://www.windowsazure.com/en-us/home/features/data-management/

Windows Azure Reporting Services: http://www.windowsazure.com/en-us/manage/services/other/sql-reporting/

HDInsight Service (Hadoop on Azure): https://www.hadooponazure.com/

MongoDB Offerings on Windows Azure: http://www.windowsazure.com/en-us/manage/linux/common-tasks/mongodb-on-a-linux-vm/

Windows Azure Marketplace: http://www.windowsazure.com/en-us/store/overview/

One of the use-cases for a cloud solution is to serve as a Disaster Recovery option for your on-premises servers. I’ll explain one particular use-case in this entry, specifically using Windows Azure “IaaS” or Virtual Machines as a Recovery Solution for SQL Server (more detail here: http://www.windowsazure.com/en-us/home/features/virtual-machines/). In future installments I’ll explain options for other workloads such as Linux and Windows Servers, SharePoint and other solutions. Some architectures also allow for using Windows Azure SQL Database (Formerly SQL Azure) in recovery scenarios; I’ll cover that separately.

Using Azure as a Disaster Recovery site gives you a range of options, uses world-wide datacenters that you can pick from, and does not require traditional licensing and maintenance paths. You can also integrate the offsite data into other uses, such as reporting (in some cases) or to leverage within other applications.  However, the cost-model is different, so make sure you do your homework to ensure that it makes sense to use a cloud provider for safety. You may find that it is cheaper, more expensive, or that you require a mix of technologies and options to get the best solution.

NOTE: The Microsoft Windows Azure platform evolves constantly. That means new features and capabilities, as well as security, optimizations and more improve on a frequent basis. As with any cloud provider, ensure that you check the date of this post to ensure you are within six months or so. If the date is longer than that, then check each of the “Details” links to ensure you are working with the latest information.

The options you have range from simple off-site storage for database backups to systems that your users can access when your primary options are offline.  To select which options to use, evaluate the databases you want to protect, and then create your Recovery Point Objectives (RPO) and Recovery Time Objectives (RTO) for each workload. Those two vectors will provide the starting point for each choice you make.

NOTE: If you’re not familiar with RPO and RTO on a database system, learn those terms carefully before designing a recovery solution – on any platform. RPO and RTO are business/technology terms, and are not vendor or platform-specific. http://wikibon.org/wiki/v/Recovery_point_objective_-_recovery_time_objective_strategy 

The range of protection you have is very similar to the on-premises options for SQL Server (on-premises details here: http://msdn.microsoft.com/en-us/library/ms190202.aspx), with the primary limitation being bandwidth. While Microsoft has the largest connections we can get into our datacenters, depending on where your systems are and their connection to the Internet, you will need to consider how much data you transfer, and how often.  For backup files, a single, larger transfer is acceptable, using Log Shipping or Database Mirroring, smaller, more frequent transfers are preferable.

Another limitation is controlling the hardware on the Windows Azure Virtual Machine. That means hardware-based clustering isn’t possible, as of this writing. You’re also limited to the size of the Virtual Machines that Windows Azure (or any other cloud provider) offers. It’s important to keep in mind that you’re building a Disaster Recovery solution, not necessarily a full Highly-Available system. The difference is that in this case DR provides a means to recover and operate at a more limited fashion than a full on-premises HA (with matching hardware and licenses) involves. Storage, however, isn’t as affected. You can mount large amounts of storage on a Windows Azure Virtual Machine, so it’s more memory and CPU that you need to consider for your solution.

The final consideration is security. There are two aspects in security that you need to consider: data security and authentication and access. For the first consideration, the Windows Azure system does hold multiple certifications and attestations that you can find here:  . In some cases those certifications are agreements on the part of security each party will hold liability for; so it’s important to carefully read and understand what the agreement states. There are also methods of encrypting data (such as the backups) using your own certificates or hardware devices and then storing them externally. This means no one can easily un-encrypt your data.

For the authentication portion, you can create a secure “tunnel”  between your network and Windows Azure. This involves a certificate that is installed on your hardware firewall at your facility, and an agent that is enabled with the same certificate on Windows Azure. This gives you a “point to point” connection, encrypted but over a public connection. From there you can use Active Directory to connect the authentication for the systems involved in the DR solution.

Backups

The First and most simple DR solution using Windows Azure is to store your backup files (*.bak) in Windows Azure storage. Windows Azure Storage is triple-redundant across multiple fault-domains within a single datacenter, and then all three copies are replicated to a geographically separate (although data-sovereignty same) location. That translates to six copies of data stored remotely. In case of a disaster, you connect to storage, download the images, and restore them to a new server. The server can have the same name or different, and unless you’re using contained databases, you’ll need to re-create and re-authorize the security accounts needed for the database.

Note that you also have the option of using an “appliance”, which is a piece of hardware you install at your facility which will act as a backup device or share location (or both). The device handles the encryption, de-duplication and compression for the files, and then stores those files on Windows Azure. More information on that option is here: http://www.storsimple.com/

RPO: As of last backup

RTO: (Time of transfer from Windows Azure + Time of Restore to New System + Bringing System Online with User Accounts) - Time of Backup

References:

More detail on storing files on Windows Azure: http://sqlblog.com/blogs/sqlos_team/archive/2013/01/24/backup-and-restore-to-cloud-simplified-in-sql-server-2012-sp1-cu2.aspx 

Free Client: http://azurestorageexplorer.codeplex.com/

Database Mirroring

Database Mirroring is a deprecated feature in SQL Server, which means it will be removed in a future release. It is, however, still supported in SQL Server 2012, and it can be used between on-premises SQL Server Instances and Windows Azure VM’s.  Using connection strings and .NET languages, clients can actually point to the partner server automatically.

The granularity of this solution is at the individual database level.  Machines can retain their individual identities. You can use certificates to connect the systems, or you can use the point-to-point solution and Active Directory.

There are limitations, however. You won’t use a Listener in this configuration, and you’ll be using Asynchronous mode. If you are not running in the same Active Directory, you’ll also need to factor in the time to re-create and tie out those accounts when calculating the RTO value.

RPO: As of last good synchronization

RTO: (Time of failure + Time of client redirect to New System ) - Time of last good synchronization

References:

A complete tutorial on setting up this configuration is here: http://msdn.microsoft.com/en-us/library/jj870964.aspx

Log Shipping

Another feature available for DR in a Hybrid fashion is using Log Shipping, which also protects your system at a database level. Log shipping involves an automated log backup of your database, and the log is copied and then applied at the secondary server. Because the log file is copied to a Windows share, this solution requires both networking access and an Active Directory integration.

RPO: As of last good log backup application to the secondary system

RTO: (Time of failure + Time of manual client redirect to New System + Time of Manual Failover ) - Time of last good log backup

References:

Log Shipping information is here: http://technet.microsoft.com/en-us/library/ms187103.aspx

AlwaysOn Availability Groups

SQL Server 2012 introduces a new set of features called “AlwaysOn” that encompass many of the HA/DR features in previous releases. One feature within that set is called “Availability Groups”, and with certain caveats that feature is available for a Hybrid on-premises to Windows Azure VM solution.

AlwaysOn requires a Windows Cluster (WFSC), which is where the caveats come into play. You’re able to set up a  multi-subnet WSFC cluster, but you won’t have access to the Availability Group Listener function, so you need to consider the client reconnection.

RPO: As of last good synchronization

RTO: (Time of failure + Time of manual client redirect to New System + Time of Manual Failover ) - Time of last good log backup

References:

A complete tutorial on setting up this configuration is here: http://msdn.microsoft.com/en-us/library/jj870959.aspx

Other Solution Options

Taking an overview approach, you can use other data transfer mechanisms. While these involve more manual coding and architecture, you do have more control. For instance, you could copy the data to multiple locations, platforms and more, and allow reading and manipulations of the data at the destination. You can use code options, Windows Azure Data Sync (http://msdn.microsoft.com/en-us/library/windowsazure/hh456371.aspx), or even SQL Server Replication (blog on this process is here: http://tk.azurewebsites.net/2012/07/17/how-to-setup-peer-to-peer-replication-in-azure-iaas-sql-server-2012/)

RPO: Varies

RTO: Varies

References:

A whitepaper on the information I've discussed throughout this article and other options is available here: http://msdn.microsoft.com/en-us/library/jj870962.aspx

The “SQL AlwaysOn” Team Blog (where you may find more current information) is here: http://blogs.msdn.com/b/sqlalwayson/