Search results matching tag 'Concepts'

How Does the Cloud Change a Developer's Job?

BuckWoody — Tue, 12 Feb 2013 16:26:51 GMT

I've recently posted a blog on how cloud computing would change the Systems Architect’s role in an organization, another on how the cloud changes a Database Administrator's job, and the last post dealt with the Systems Administrator. In this post I'll cover the changes facing the Software Developer when using the cloud.

The software developer role was the earliest adopter of cloud computing. This makes perfect sense, because the software developer has always used computing "as a service" - they (most often) don't buy and configure servers, platforms and the like, they write code that runs on those platforms. And there's probably not a simpler definition of a software developer to be found, but as with all simple statements, you lose fidelity and detail. I'll offer a more complete list in a moment.

Because the software developer's process involves designing, testing and writing code locally and then migrating it to a production environment, all of the paradigms in cloud computing - from IaaS to PaaS to SaaS - come naturally.

The Software Developer's Role

The software developer has evolved since the earliest days of programming.The software developer not only "writes code" - there are far more tasks involved in modern systems development:

Assisting the Business Role(s) in developing software specifications
Planning software system components and modules
Designing system components
Working in teams writing classes, modules, interfaces and software endpoints
Designing data layouts, architectures, access and other data controls
Designing and implementing security, either programmatic, declarative, or referential
Mixing and matching various languages, scripting and other constructs within the system
Designing and implementing user and account security rights and restrictions
Designing various software code tests - unit, functional, fuzz, integration, regression, performance and others
Deploying systems
Managing and maintaining code updates and changes

Like most of the previous roles, those tasks also unpacks into a larger set of tasks, and no single developer has exactly that same list. And like the DBA, the role is often more, or less of that list based on where the developer works. Smaller companies may include the development platform in the duties so that a developer is also a systems administrator. In larger organizations I've seen developers that specialized on User Interfaces, Engine Components, Data Controls or other specific areas.

How the Cloud Changes Things

The software developer role obviously has the same concerns and impacts of "the cloud" as the Systems Architect. They need to educate themselves on the options within this new option (Knowledge), try a few test solutions out (Experience) and of course work with others on various parts of the implementation (Coordination).

The big changes for a developer include three major areas: Hybrid Software Design, Security, and Distributed Computing.

Hybrid Software Design

After the PC revolution, software developers designed systems that ran primarily on a single computer. From there the industry moved to "client/server", where most of the code still lived on the user's workstation, and various levels of state (such as the data layer) moved to a server over fast connected lines. After than followed the Internet phase, which had less to do with HTML coding than it did with state-less architectures. While no architecture is truly stateless, there are ways of allowing the client to be in a different state than the server of the application at any one time - this is the way the Web works.

Even so, the developer often simply moved one the primary layers (such as Model, View or Controller) to the server, using the User Interface merely as the View or Presentation layer. While technically stateless, this doesn't require a great deal of architecture change - there are various software modules that run on a server, and perhaps that connects to a remote data server. In the end, it's still a single paradigm.

We now have the ability to run IaaS (hardware abstraction), PaaS (hardware, operating system and runtime abstraction) and SaaS (everything abstracted, API calls only) in a single environment such as Windows Azure. A single application might have a Web-based Interface Server with federated processes (using a PaaS set of roles), a database service (using a SaaS provider such as Windows Azure SQL Database), a specialized process in Linux (using an IaaS role in Windows Azure) and a translator API (from the Windows Azure Marketplace). This example involves only one vendor - Microsoft. I've seen applications that use multiple vendors in this same way.

Thinking this way opens up a great deal of flexibility - and complexity. Complexity isn't evil; it's how complicated things get done many times. The modern developer needs to understand how to build hybrid software architectures.

Resources: Hybrid Architectures with step-by-step instructions and examples: http://msdn.microsoft.com/en-us/library/hh871440.aspx and Windows Azure Hybrid Systems: http://msdn.microsoft.com/en-us/library/hh871440.aspx?AnnouncementFeed

Security

Having a single security boundary, such as "everyone who works in my company", is a relatively simple problem to solve. Normally the System Administrators configure and control a security provider, such as Active Directory, and developers can access that security layer programmatically. That allows for good separation of duties and role-based control.

In modern applications, clients, managers, and users both internal and external need various levels of access to the same objects, code and data. A client should be able to enter an order, a store should be able to accept the order, the credit-card company should be able to check the order and authorize payment, and the managers should be able to report on the order or change it if needed. Using role-based security across multiple domains would be impossible to maintain.

Enter "claims-based" authentication. In this paradigm, the user logs in with whatever security they use - corporate or other Active Directory, Facebook, Google, whatever. The application (using Windows Identity Foundation or WIF) can accept a "claim" from that provider, and the developer can match whatever parts of that claim they wish to the objects, code and data. And example might be useful.

Buck logs in to his corporate Active Directory (AD), and attempts to use a program based in Windows Azure. Windows Azure rejects the login silently, and is configured to check with Buck's AD. Buck's AD says "yes, I know Buck, and he has been granted the following claims: "partner", "manager", "approver". The developer does not need to know about Buck's AD, Buck, his login, or anything else. She simply codes the proper data access to allow "approver" to approve a sale.

This allows a lot of control, at a very fine level, without having to get into the details of each security provider. .

Resources: Overview of using claims-based Azure Security: http://adnanboz.wordpress.com/2011/02/06/claims-based-access-and-windows-azure/

Distributed Computing

Is there a difference between stateless computing, or even the hybrid programming I mentioned earlier, and "Distributed Computing"? Yes - the primary difference is latency. Even stateless code can have too small a tolerance for latency.

Dealing with slow connectivity, or breaks in connections has many impacts. One method of dealing with this is to locate data and computing of that data as closely as possible, even if this means relaxing consistency or duplicating data. Another method is to go back to a great paradigm from the past that is possible underused today is a Service Oriented Architecture. The Windows Azure Service Bus is possibly one of the fastest and easiest way to adopt cloud computing without completely rearchitecting your application.

References: Great breakdown of the thought process around a distributed architecture: http://msdn.microsoft.com/en-us/magazine/jj553517.aspx and using a Windows Azure Relay Service: http://www.windowsazure.com/en-us/develop/net/how-to-guides/service-bus-relay/

How Does the Cloud Change a Systems Administrator's Job?

BuckWoody — Tue, 05 Feb 2013 15:46:02 GMT

I recently posted a blog entry on how cloud computing would change the Systems Architect’s role in an organization, and another on how the cloud changes a database administrator's job. This time I'll cover a few of the changes the cloud brings for the Systems Administrator.

The systems administrator shares some similarity with the database administrator, in that it's rare to find a single job description that fits all people in that role. There are some basic similarities among various organizations, so I'll use those as a starting point.

The Systems Administrator Role

The systems administrator role is perhaps one of the earliest in technology, at least as far as the implementation of a system goes. In the earliest days of computing, electronic technical professionals built prototype computers, and newly minted "programmers" wrote logical instructions for these systems. In time, the systems administration role owned the installation, configuration, operation and tuning of these systems once they went into production and use on a larger scale. A few of the tasks associated with the role are:

Planning, installing and configuring systems
Planning, designing and creating storage, networking and other system components
Planning, designing and implementing High Availability and Disaster Recovery for each system
Maintaining and monitoring systems
Implementing performance tuning systems based on monitoring
Re-balancing workloads across servers based on monitoring
Securing systems, networks and individual computers based on requirements and implementation
Planning, implementing and controlling user and account security rights and restrictions

Like the DBA, that’s just a short list, and each of those tasks also unpacks into a larger set of tasks. And like the DBA, the role is often more, or less of that list based on where the system administrator works. In smaller companies I've been a "systems administrator" that also ran the database and mail servers, web systems, front-line end-user support and made the coffee. In larger organization I was only able to spend the day on one or two parts of that list, since there were so many systems and they interacted with so many other systems.

Systems administrators often deal with multiple operating systems. In one company where I was a system administrator, I worked with no less than six operating systems from mainframes to PC servers, two of them highly specialized to the hardware.

How the Cloud Changes Things

The systems administrator has the same concerns and impacts of "the cloud" as the DBA and the Systems Architect. They need to educate themselves on the options within this new option (Knowledge), try a few test solutions out (Experience) and of course work with others on various parts of the implementation (Coordination).

I've mentioned the three big buckets of cloud computing, dealing with Virtual Machines (IaaS) writing code (PaaS) and using software that’s already written and being delivered via an Application Programming Interface (API). In my experience, the systems administrator role normally tackles the first "bucket" most often - IaaS, which has at its base the technology of virtualization.

Virtualization

One of the first areas the systems administrator is involved with "the cloud" is in the area of virtualization. This technology isn't new - in fact, I worked on Virtual Machines (VM's) way back in my mainframe days. It's the process of using software to emulate hardware - which has implications far beyond that simple sentence.

Virtualization is normally a standard on-premises process. When you take Virtual Machines and host them in another location, this is called Co-Location, or CoLo. Personally, I don't define either of these activities as "Cloud" computing - it's simply virtualization. Infrastructure as a Service (IaaS) normally involves several more components, at the very least being able to set up the systems (provision) and deploy them in a standard, automated way. It also involves (at a minimum) the ability to monitor, move and alter the systems using a prescribed methodology. There are other parts of IaaS to be sure, but this level above simply scripting installations or virtualizing a machine is where the system administrator becomes involved in this new "cloud computing" paradigm.

There are multiple VM technologies available, from the hypervisor that is built-in to the Windows operating system (Hyper-V) to third-party alternatives such as VMWare. The choice of cloud provider often dictates the selection of hypervisor. Windows Azure uses Hyper-V, and allows you to move systems from the cloud to the desktop and back again. Other providers use VMWare, or a proprietary format. Some allow you to push or pull images from the cloud service, others do not. The systems administrator must educate themselves on the business need and then select the cloud provider that best fits the requirements for a workload. It's also common to use several cloud providers within a single company.

Resources: Windows Azure Virtual Machines: http://www.windowsazure.com/en-us/manage/windows/tutorials/virtual-machine-from-gallery/ and System Center: http://blogs.technet.com/b/server-cloud/archive/2011/12/01/managing-and-monitoring-windows-azure-applications-with-system-center-2012.aspx

Cloud Computing Architecture - Private, Public and Hybrid

It's important to note that IaaS can be on-premises, at another facility, or both. The first is called "private cloud", the second "public cloud", and the third "hybrid cloud". Yes, these are marketing terms, but they are useful in describing where the decisions are for deploying a system. If data security is paramount, then private cloud may be the right choice for a given workload. If agility or cost is an issue, public cloud may be the right answer for another workload. And in many cases - perhaps most - using both architectures is the right way to split the workload.

The key is to understand the workload well. In the past the system administrator needed to know the component requirements, such as how much memory, CPU, network and storage a workload needed. In cloud computing, these are also concerns, but you need to add in the questions of cost, business use, location of users, security and other vectors. These concerns bring the systems administrator closer to the business and its goals.

Resources: Windows Azure Hybrid Systems: http://msdn.microsoft.com/en-us/library/hh871440.aspx?AnnouncementFeed

DevOps

One new term introduced into cloud computing is "DevOps" - short for Developer Operations. Not everyone agrees that this is even a real "thing" - that it's a made-up term by cloud vendors. Regardless, there is a new set of tasks that the cloud brings that may sit within the purview of the system administrator.

Basically it involves the administration needed at the PaaS or SaaS level. The IaaS function of cloud computing holds most of the same characteristics as an on-premises system, defined the in the first list I mentioned above. But when the organization uses Platform as a Service, the operating system, much of the security, scale and other components of infrastructure are abstracted into the platform, and are often even controlled by the developer.

But once the application "goes live", there are a host of billing, controlling, scaling and other security questions that developers aren't equipped to handle. Who takes care of those? As companies are finding out, they need to appoint someone to cover these overlapped areas between developers and administrators.

References: How DevOps brings order: http://searchcloudcomputing.techtarget.com/feature/How-DevOps-brings-order-to-a-cloud-oriented-world and Managing Windows Azure: http://www.windowsazure.com/en-us/manage/overview/

How Does the Cloud Change a Database Administrator’s Job?

BuckWoody — Tue, 29 Jan 2013 15:08:32 GMT

I recently posted a blog entry on how cloud computing would change the Systems Architect’s role in an organization. In a way, the Systems Architect has the easiest transition to a new way of using computing technologies. In fact, that’s actually part of the job description. I mentioned that a Systems Architect has three primary vectors to think about for cloud computing, as it applies to what they should do:

Knowledge - Which options are available to solve problems, and what are their strengths and weaknesses.
Experience - What has the System Architect seen and worked with in the past.
Coordination - A system design is based on multiple factors, and one person can't make all the choices. There will need to be others involved at every level of the solution, and the Systems Architect will need to know who those people are and how to work with them.

The Database Administrator Role

But a Database Administrator (DBA) is probably one of the harder roles to think about when it comes to cloud computing. First, let’s define what a Database Administrator usually thinks about as part of their job:

Planning, Installing and Configuring a Database Platform
Planning, designing and creating databases
Planning, designing and implementing High Availability and Disaster Recovery for each database (HADR) based on requirements for its workload
Maintaining and monitoring the database platform
Implementing performance tuning on the databases based on monitoring
Re-balancing workloads across database servers based on monitoring
Securing databases platforms and individual databases based on requirements and implementation

That’s just a short list, and each of those unpacks into a larger set of tasks.

The issue is that I’ve never actually met a DBA that does all of those things, or just all of those things. Many times they do much more, sometimes the systems are so large they specialize on just a few of them.

And as you can see from the list, some of these areas are shared with other roles. For instance, in some shops, the DBA plans, purchases, sets up and configures the hardware for database servers. In others that’s done
by the Infrastructure Team. In some shops the DBA designs databases from software requirements, and in others the developers do that – or perhaps it’s done as a joint effort. The same holds true for database code – sometimes the
DBA does it, other times the developer, and still others it’s a shared task.

In fact, you could argue that there are few other roles in IT where the roles are so intermixed. Also, the DBA works with software the company develops, and software the company buys. They work with hardware, networking, security and software. There are certain aspects of design and tuning that are outside the purview of some of those things, and inside the others.

With all of these variables, simply telling a DBA that they should “use the cloud” is not the proper approach.

How the Cloud Changes Things

To be sure, the DBA has the same vectors as the Systems Architect. They need to educate themselves on the options within this new option (Knowledge), try a few test solutions out (Experience) and of course work with others on various parts of the implementation (Coordination). But it goes beyond that.

There are three big buckets of cloud computing, dealing with simply using a Virtual Machine (IaaS) to writing code without worrying about the virtualization or even the operating system (PaaS) and using software that’s already written and being delivered via an Application Programming Interface (API). Each of these has so many options and configurations that it’s often better to think about the problem you’re trying to solve rather than all of the technology within a given area - although some of that is certainly necessary anyway.

Database Platform Architecture

I’ll start with when the DBA should even consider cloud computing for a solution. Once again, it’s not an “all or nothing” paradigm, where you either run something on premises or in the cloud – it’s often a matter of selecting the right components to solve a problem. In my design sessions with DBA’s I break these down into three big areas where they might want to consider the cloud –and then we talk about how to implement each one:

Audiences
HADR
Data Services

Audiences

If the users of your database systems all sit in the same facility, you own the servers and networking, and the application servers are separate from the database server, it doesn’t usually make sense to take that database workload and place it on Windows Azure – or any other cloud provider. The latency alone prevents a satisfactory performance profile, and in some cases won’t work at all. It doesn’t matter if the cloud solution is cheaper or easier – if you’re moving a lot of data every second between an on-premises system and the cloud it won’t work well.

However – if your users are in multiple locations, especially globally, or you have a mix of company and external customer users, it might make sense to evaluate a shared data location. You still need to consider the implications of how much data the application server pushes back and forth, but you may be able to locate both the application server and SQL Server in an IaaS role. Assuming the data sent to the final client will work across public Internet channels, there may be a fit. There are security implications, but unless you have point-to-point connections for your current solution you’re faced with the same security questions on both options.

Your audience might also be developers looking for a way to quickly spin up a server and then turn it down when they are done, paying for the time and not the hardware or licenses. This is also a prime case for evaluating IaaS. And there are others that you'll find in your own organization as you work through the requirements you have.

Resources: Windows Azure Virtual Machines: http://www.windowsazure.com/en-us/manage/windows/tutorials/virtual-machine-from-gallery/ and Windows Azure SQL Server Virtual Machines: http://www.windowsazure.com/en-us/manage/windows/common-tasks/install-sql-server/

HADR

The next possible place to consider using cloud computing with SQL Server is as a part of your High Availability and Disaster Recovery plans. In fact, this is the most common use I see for cloud computing and the Database Administrator. The key is the Recovery Point Objective (RPO) and Recovery Time Objective (RTO). Based on each application’s requirements, you may find that using Windows Azure or even supplementing your current plan is
the right place to evaluate options. I’ve covered this use-case in more detail in another article.

References: SQL Server High Availability and Disaster Recovery options with Windows Azure: http://blogs.msdn.com/b/buckwoody/archive/2013/01/08/microsoft-windows-azure-disaster-recovery-options-for-on-premises-sql-server.aspx

Data Services

Windows Azure, along with other cloud providers, offers another way to design, create and consume data. In this use-case, however, the tasks DBA’s normally perform for sizing, ordering and configuring a system don’t apply.

With Windows Azure SQL Databases (the artist formerly known as SQL Azure), you can simply create a database and begin using it. There are places where this fits and others where it doesn’t, and there are differences, limitations and enhancements, so it isn’t meant as replacement for what you could do with “Full-up” SQL Server on a Windows Azure Virtual Machine or an on-premises Instance. If a developer needs an Relational Database Management
(RDBMS) data store for a web-based application, then this might be a perfect fit.

But there is more to data services than Windows Azure SQL Databases. Windows Azure also offers MySQL as a service, RIAK and MongoDB (among others) and even Hadoop for larger distributed data sets. In addition you can use Windows Azure Reporting Services, and also tap into datasets and data functions in the Windows Azure Marketplace.

The key for the DBA with this option is that you will have to do a little investigation this time, and potentially without a specific workload in mind this time. I think that’s acceptable thing to ask – DBA’s constantly keep up with data processing trends, and most will consider different ways to solve a problem.

References:

Windows Azure SQL Databases: http://www.windowsazure.com/en-us/home/features/data-management/

Windows Azure Reporting Services: http://www.windowsazure.com/en-us/manage/services/other/sql-reporting/

HDInsight Service (Hadoop on Azure): https://www.hadooponazure.com/

MongoDB Offerings on Windows Azure: http://www.windowsazure.com/en-us/manage/linux/common-tasks/mongodb-on-a-linux-vm/

Windows Azure Marketplace: http://www.windowsazure.com/en-us/store/overview/

How Does the Cloud Change a Systems Architect’s Job?

BuckWoody — Tue, 22 Jan 2013 15:43:59 GMT

I know - I said I didn't like the "cloud" term, but my better-phrased "Distributed Systems" moniker just never took off like I had hoped. So I'll stick with the "c" word for now, at least until the search engines catch up with my more accurate term.

I thought I might spend a little time on how the cloud affects the way we work - from Systems Architects to Database Administrators and Developers, and Systems Administrators - a group often referred to as "IT Pro's". But each role within these groups have different aspects when using cloud computing. In this post we'll take a look at the role of the Systems Architect, and in the posts that follow I'll talk more about the other roles in the IT Pro area.

The Systems Architect Role

What does a "Systems Architect" do? Like most IT roles, it depends on the company or organization where they work. In fact, the term isn't even specific to technology, but I'll use it in that context here. In general, a Systems Architect takes the requirements for a given system, and assembles the relevant technology areas that best fulfill those requirements. That's a single-sentence explanation, and needs further unpacking.

As an example, a Systems Architect at a medical firm is presented with a set of requirements for tracking a patient through the entire care cycle. The Systems Architect first looks at all of the requirements for the data that needs to be collected based on business, financial, regulations, and other requirements, and then how that data needs to flow from one system to another. They check the security requirements, performance, location and other aspects of the system. They then check to see which options are available for processing that data, and which parts they should "build or buy".

For instance, the requirements might be so specific that only custom code is the proper solution - but even there, choices still exist, such as which language(s) to use, what type of data persistence (a Relational Database Management System or or other data storage and processing) will be used, what talent within the company is available for the system and a myriad of other decision.

All of this boils down to three primary vectors:

Knowledge - Which options are available to solve problems, and what are their strengths and weaknesses.
Experience - What has the System Architect seen and worked with in the past.
Coordination - A system design is based on multiple factors, and one person can't make all the choices. There will need to be others involved at every level of the solution, and the Systems Architect will need to know who those people are and how to work with them.

How the Cloud Changes Things

From the outset, it doesn't seem that using a distributed system would change anything in the Systems Architect role. Isn't the cloud simply another option that the Systems Architect needs to learn and apply? Yes, that is true - but it goes a bit deeper. Let's return to those vectors a moment to see what a Systems Architect needs to take into account.

Knowledge

The first and probably most obvious impact is learning about cloud technologies. But the important part of that knowledge is to learn when and where to use each service. It's a common misconception that the cloud should be an "all or nothing" approach. That's just not true - every Windows Azure project I work on has some element of on-premises interaction, and in some cases only one small part of a solution is placed on the Windows Azure architecture. Since Windows Azure contains IaaS (VM's) PaaS (you write code, we run it) and even SaaS (Such as Hadoop or Media Services), a given architecture can use multiple components even within just one provider. And I've worked on several projects where the customer used not only Windows Azure and On-Premises environments, but also components from other providers. That's not only acceptable, but often the best way to solve a given problem.

As part of the learning experience, it's vital to keep in mind what you need to pick as key decision points. In your organization, cost could be ranked higher than performance, or perhaps security is the highest decision point.

To stay educated, there are various journals, websites and conferences that Systems Architects use to keep current. Almost all of those are talking about "cloud" - but there is no substitute for learning from the vendor about their solution. I'm speaking here of the technical information, not the marketing information. The marketing information is also useful, at least from a familiarity standpoint, but the technical information is what you need.

Resource: For Windows Azure, the Systems Architect can start here: http://blogs.msdn.com/b/buckwoody/archive/2012/06/13/windows-azure-write-run-or-use-software.aspx

Experience

Cloud computing is relatively new - it's only been out a few years, and the main competitors are only now settling in to their respective areas. It might not be common for a Systems Architect to have a lot of hands-on experience with cloud projects.

Even so, there are ways to leverage the experience of others, such as direct contact or even attending conferences where customers present findings from their experiences.

You can also gain hands-on experience by setting up pilots and proof-of-concept projects yourself. Most all vendors - Microsoft included - have free time available on their systems. The key to an experiment like this is choosing some problem you are familiar with that exercises as many features in the platform as possible. There is no substitute for working with a platform when you want to design a solution.

Coordination

Probably one of the largest changes in the Systems Architect role that the cloud brings is in the area of coordination. When a Systems Architect deals with the business and other technical professionals, there is a 20+ year history of technology that we are all familiar with. When you mention "the cloud", those audiences may not have spent the time you have in understanding what that means - and often they think it means the "all or nothing" approach I mentioned earlier.

I've found that a series of "lunch and learns" for the technical staff is useful to explain to each role-group how the cloud is used in their area is useful. In the posts that follow this one, I'll give you some material for those. For managers and business professionals, you'll want to go a different route. I've found that an "Executive Briefing" e-mail, consisting of about a page, with headings that are applicable to your audience.

Resource: Writing Executive Summaries: http://writing.colostate.edu/guides/guide.cfm?guideid=76

High Availability for IaaS, PaaS and SaaS in the Cloud

BuckWoody — Tue, 06 Nov 2012 15:15:28 GMT

Outages, natural disasters and unforeseen events have proved that even in a distributed architecture, you need to plan for High Availability (HA). In this entry I'll explain a few considerations for HA within Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS) and Software-as-a-Service (SaaS). In a separate post I'll talk more about Disaster Recovery (DR), since each paradigm has a different way to handle that.

Planning for HA in IaaS

IaaS involves Virtual Machines - so in effect, an HA strategy here takes on many of the same characteristics as it would on-premises. The primary difference is that the vendor controls the hardware, so you need to verify what they do for things like local redundancy and so on from the hardware perspective.

As far as what you can control and plan for, the primary factors fall into three areas: multiple instances, geographical dispersion and task-switching.

In almost every cloud vendor I've studied, to ensure your application will be protected by any level of HA, you need to have at least two of the Instances (VM's) running. This makes sense, but you might assume that the vendor just takes care of that for you - they don't. If a single VM goes down (for whatever reason) then the access to it is lost. Depending on multiple factors, you might be able to recover the data, but you should assume that you can't. You should keep a sync to another location (perhaps the vendor's storage system in another geographic datacenter or to a local location) to ensure you can continue to serve your clients.

You'll also need to host the same VM's in another geographical location. Everything from a vendor outage to a network path problem could prevent your users from reaching the system, so you need to have multiple locations to handle this.

This means that you'll have to figure out how to manage state between the geo's. If the system goes down in the middle of a transaction, you need to figure out what part of the process the system was in, and then re-create or transfer that state to the second set of systems. If you didn't write the software yourself, this is non-trivial.

You'll also need a manual or automatic process to detect the failure and re-route the traffic to your secondary location. You could flip a DNS entry (if your application can tolerate that) or invoke another process to alias the first system to the second, such as load-balancing and so on. There are many options, but all of them involve coding the state into the application layer. If you've simply moved a state-ful application to VM's, you may not be able to easily implement an HA solution.

Planning for HA in PaaS

Implementing HA in PaaS is a bit simpler, since it's built on the concept of stateless applications deployment. Once again, you need at least two copies of each element in the solution (web roles, worker roles, etc.) to remain available in a single datacenter. Also, you need to deploy the application again in a separate geo, but the advantage here is that you could work out a "shared storage" model such that state is auto-balanced across the world. In fact, you don't have to maintain a "DR" site, the alternate location can be live and serving clients, and only take on extra load if the other site is not available. In Windows Azure, you can use the Traffic Manager service top route the requests as a type of auto balancer.

Even with these benefits, I recommend a second backup of storage in another geographic location. Storage is inexpensive; and that second copy can be used for not only HA but DR.

Planning for HA in SaaS

In Software-as-a-Service (such as Office 365, or Hadoop in Windows Azure) You have far less control over the HA solution, although you still maintain the responsibility to ensure you have it. Since each SaaS is different, check with the vendor on the solution for HA - and make sure you understand what they do and what you are responsible for. They may have no HA for that solution, or pin it to a particular geo, or perhaps they have a massive HA built in with automatic load balancing (which is often the case).

All of these options (with the exception of SaaS) involve higher costs for the design. Do not sacrifice reliability for cost - that will always cost you more in the end. Build in the redundancy and HA at the very outset of the project - if you try to tack it on later in the process the business will push back and potentially not implement HA.

References: http://www.bing.com/search?q=windows+azure+High+Availability (each type of implementation is different, so I'm routing you to a search on the topic - look for the "Patterns and Practices" results for the area in Azure you're interested in)

The Windows Azure Software Development Kit (SDK) and the Windows Azure Training Kit (WATK)

BuckWoody — Wed, 12 Sep 2012 13:40:40 GMT

Windows Azure is a platform that allows you to write software, run software, or use software that we've already written. We provide lots of resources to help you do that - many can be found right here in this blog series. There are two primary resources you can use, and it's important to understand what they are and what they do.

The Windows Azure Software Development Kit (SDK)

Actually, this isn't one resource. We have SDK's for multiple development environments, such as Visual Studio and also Eclipse, along with SDK's for iOS, Android and other environments. Windows Azure is a "back end", so almost any technology or front end system can use it to solve a problem.

The SDK's are primarily for development. In the case of Visual Studio, you'll get a runtime environment for Windows Azure which allows you to develop, test and even run code all locally - you do not have to be connected to Windows Azure at all, until you're ready to deploy.

You'll also get a few samples and codeblocks, along with all of the libraries you need to code with Windows Azure in .NET, PHP, Ruby, Java and more.

The SDK is updated frequently, so check this location to find the latest for your environment and language - just click the bar that corresponds to what you want:

http://www.windowsazure.com/en-us/develop/downloads/

The Windows Azure Training Kit (WATK)

Whether you're writing code, using Windows Azure Virtual Machines (VM's) or working with Hadoop, you can use the WATK to get examples, code, PowerShell scripts, PowerPoint decks, training videos and much more. This should be your second download after the SDK. This is all of the training you need to get started, and even beyond. The WATK is updated frequently - and you can find the latest one here:

http://www.windowsazure.com/en-us/develop/net/other-resources/training-kit/

There are many other resources - again, check the http://windowsazure.com site, the community newsletter (which introduces the latest features), and my blog for more.

How I Work– Working From Home

BuckWoody — Tue, 17 Jul 2012 13:54:24 GMT

I work from home – and I would find it difficult to work any other way. For decades I’ve worked in traditional offices, cubicle-farms, and open-plan work areas. I find them too bright, too loud, too distracting, and in some cases, too depressing to be effective. Working from home offers me a zero-commute, largely stress-free way of working that I really like. I just find it one of the most effective, efficient and enjoyable ways to work.

But there are some caveats.

I’ve read a lot of blogs that talk about how difficult it is to work from home, articles that explain how ineffective home workers are, and how it’s difficult to monitor and manage remote workers. I find those interesting, since I’ve not experienced that. I thought I might take a few moments to explain how I work at home, and the attributes that make this style of working most effective. Every person is different, so perhaps it’s a combination of factors that make working remotely successful for one person and not another.

Personal Work Style

I am a very self-motivated person. I am goal-oriented, which means that when I’m given an assignment, I start a work-back schedule and immediate sort out my time and start working the tasks to get to the goal. And I don’t have to be given a specific goal – if my company gives me the direction they want to go, I’ll make goals that take us both in that direction. That’s a large part of working remotely – you have to be self-motivated.

I also like to focus during certain periods of time. No music, no talking, no e-mail, twitter, or Facebook; no interruptions, just an hour or so of “heads-down” work. I don’t work like that all the time, but when I do, I want that extreme focus available. Working remotely allows me to control the environment enough to do that.

In fact, I don’t like having set tasks to do each day. I can certainly do that kind of work, but I’d much rather work towards a goal, and set up my own tasks.

To work remotely, you’ll need to be self-motivated as well. In many cases, those who work remotely don’t have a daily stand-up meeting or scrum that they are part of. That means no one is watching to see if they are on-task – and for some folks, that allows them not to be as focused as having that pressure of knowing they are being tracked.

You’ll need to be able to stay focused on the work you need to do. It is incredibly easy to be sidetracked when you work remotely. You have to be ruthless about looking forward to the goal, creating a schedule and staying on that schedule. I live and die by my Outlook Tasks. Every e-mail I get is turned into a task if there is anything needed from me. I make those tasks visible to my boss, so he knows at all times what I’m up to without bothering me for status. When I finish the work, I close the task.

You’ll need to be able to be “compartmentalized”. When you’re at work, be at work. When you’re done, shut it off. When you’re working at home, it’s far too easy for those lines to become blurred, and for your home and work life to mix. Some of that is unavoidable, but I minimize it as much as I can. Even my family helps – they know that when I’m in my room that serves as my office with the door closed, I’m not home. I’m at work. Unless it’s an emergency, they treat it as if I’ve left the house.

That doesn’t mean I work 9-5 – far from it. I get up very early, and make coffee, open up the house, and turn on the laptop. I check e-mail (but I don’t answer them yet) social networks, and my RSS feeds (I have several hundred I glance through). When my family wakes up a few minutes later, I stop work, we go for a walk around a pond near our home (about a half-a mile) rain or snow. We come back, I get back to work on the day’s tasks. I stop again, we eat breakfast together, and then my wife is off to work and my daughter is off to school. I’m now ready to start phone calls, client visits, or work on architectures and solutions. When I can I eat lunch, take the dog for another walk, and get back to work. My daughter comes home from school, and I stop and chat with her about her day. She goes off to do her homework, and I get back to work. When my wife gets home in the evenings, I stop work, cook dinner, and then read most nights when I’m not teaching college. On the weekends we go various places around Washington and Oregon, and on Sundays I go to church. I do have to work a weekend here or there, but I don’t work on Sunday – or even open the laptop very often – unless I have to. That’s my schedule, and it suits me not to have to sit in traffic or deal with office politics. I’m most effective this way.

Work

Not every job can be done remotely.

I chose the Role I have at Microsoft carefully, and in the largest part because it allows me to work remotely. My job is to support customers when they are investigating using our technologies to solve a problem. It’s at various parts of the sales cycle, but I’m assigned a territory (the entire Pacific Northwest of the U.S.) to work directly with our clients and whatever team at Microsoft is helping them at the moment. I currently work with Windows Azure, specializing in things like architecting systems, data systems, and security. That means I’m either on a web-conference, at the client’s office, or working architecture designs. That lends itself well to working remotely.

So your work needs to be able to be done remotely. If you’re able to use e-mail and the phone/webcam to communicate, if your work doesn’t require a lot of in-person meetings, and if you’re able to work independently and then submit your part of the work to a larger group using a computer network, then you’re probably able to work remotely. Also, if your work involves visiting clients often you’re not in the office most of the time anyway – so you could probably work remotely then as well.

Environment

Having the location and tools to work remotely is essential to success. First, you need a place where you can work. I have a home office, a dedicated room in the house that I use for work. It has a door, and when I close that I can isolate myself for calls, web broadcasts, collaboration and to focus. For me, a specific, private room or place in the house is very important.

That being said, I very often take my office with me to another location. When the family leaves for the day, I can pick up my laptop (I use a Lenovo X220 with SSD’s and a lot of RAM, with Windows 8 and Microsoft Office) and head outside when the weather is nice. I grab a cup of coffee, head to the porch, and stay heads-down on the work. Sometimes I go to a coffee shop, sometimes I go to the Public Library, a University of Washington location, or just by a lake or stream. When I am remote I take my phone with me.

You’ll need a fast Internet connection – business class if you can get it – if you’re doing web conferences. You want to remain professional, and nothing says “I’m an amateur” faster than a spotty network connection, especially if you use an IP phone. My company doesn’t pay for the connection, I just treat it as my commute-money. It’s worth it. And you’ll need a backup. I do have a MiFi device in case I’m by that pond or mountain or when the Internet goes out – I don’t rely on public Internet when I’m out. And I try to be on my business class Internet when I’m on a call or web conference. If I have to I’ll drive in to a Microsoft office to use a dedicated connection if the one at home is out. I won’t compromise the client’s experience on my convenience.

I have an inexpensive server in the house, where I run Windows 2008 R2 Server. I have several VM’s there that I remote into, and those are my demo machines, development workstations and so on. I back that up to an external drive which I keep at a friend’s house. I also make sure all of my critical work files are kept in Windows Azure storage for a remote backup.

I use my regular laptop for just a few tools:

OneNote – For everything. I use it for typing, a light spreadsheet and math tool, graphics, presenting (yup, that’s what I use), whiteboarding, everything. I use my Windows Live SkyDrive for all my notebooks, and then I open them using the local OneNote tool. That way I have the rich tool interface, backups of all my data, and when needed I can use the web OneNote to do my work on any computer connected to the Internet. I can even share out the NoteBooks (I do this all the time) and work with clients or internal teams real-time. It’s a one-product SharePoint, at least for me.
Outlook – From e-mail to RSS feeds to calendar and tasks, I live and die on Outlook. Once again I use the “fat” client for almost everything local, but connected over the web interface whenever I need to, and of course that’s backed up on corporate.
PowerPoint – don’t be a hater. I love PowerPoint. In fact, every time (every single time) I’ve seen someone whining about how bad PowerPoint is I find out they stink at using PowerPoint. If you take the time to learn it, you can do everything from presentations to simple designs, whiteboarding and even full on training sessions using it. I also store the Presos on my Skydrive, and I can (and have) presented from Office Live Apps.
Visual Studio – I do all my coding, database and data work, and design work in VS. I check all my code into GIT or VSTS, depending on the client.
Lync – One of the best communication tools I’ve used. It’s my IP phone, internal chat, internal groups, webcam presentation, and a really nice collaborative whiteboard tool. I do entire design sessions using only Lync.

You’ll have to decide whether you like a noisy, busy environment, or a quiet, serene one. I like both, depending on what I’m doing. I normally work mostly here at home listing to music, but from time to time I’ll get out where there are more people I can interact with.

Communication

You’ll find that working at home can be very solitary. The whole point of being away from everyone is that you’re away from everyone – but this can be a problem. You need periodic interaction. I have regular staff meetings online with my team, I stay connected and chat often using Lync, and I do visit client and Microsoft offices from time to time.

And I use social media. Twitter I use for general chatter, and I do this throughout the day when I’m waiting for a compile or some other task to finish. Facebook is where I post more family or longer interactions. LinkedIn, which I use a great deal, is for only work-related information, chats, groups and so on. Can these be time-wasters, or not be useful? Of course. That’s where the self-discipline comes in. I put into those networks exactly what I want out of them, and when I’m busy, I shut them off.

I don’t like e-mail. I get tons of it – but I try to use the phone whenever I can. I think that’s especially important for a remote worker. I also make sure my boss knows what I’m up to. I want to make sure come promotion time that he knows I’m effective, efficient, and valuable. That means chatting with him on a weekly basis, and letting him know my successes and challenges. I don’t want our first conversation to be at review time.

And I go to in-person events as often as I can. From PASS to SQL Saturdays, user groups, anything I can make time for. These are not part of my job – I’m not paid to do public speaking, nor is it a requirement of my job – but I’ll volunteer to speak so that I can go to an event.

The key is to develop your lines of peer-to-peer contacts with whatever tools you like so that you’re not an island. I use Lync, internal groups at Microsoft, and LinkedIn to stay up to date with my industry peers. There’s a lot of chaff in all of those, but I put the work in to make them effective for me.

Company

Your company has to buy in to all this.

If they simply allow you (begrudgingly) to work at home, you’re doomed to fail. You’ll be left out of the loop, not invited to meetings, forgotten, and your professional brand will slowly wither and die. You have to get your manager and your company to see the value in your not taking up space in their building. You have to sell it.

Microsoft is very remote-worker friendly. They have an entire set of tools to make us successful. We have training for working remotely, and our managers are trained to have remote workers. Some do that well, others don’t. I don’t work for the managers that aren’t good at managing remote workers.

So it seems that there are a lot of factors that go into making a successful remote worker – it’s not just staying at home in your pajamas, checking e-mails now and again. It’s a combination of how you work, what the work is, how well you can stay connected, and the level your company supports you.

And I love it.

In the Cloud, Everything Costs Money

BuckWoody — Tue, 10 Jul 2012 12:55:50 GMT

I’ve been teaching my daughter about budgeting. I’ve explained that most of the time the money coming in is from only one or two sources – and you can only change that from time to time. The money going out, however, is to many locations, and it changes all the time. She’s made a simple debits and credits spreadsheet, and I’m having her research each part of the budget. Her eyes grow wide when she finds out everything has a cost – the house, gas for the lawnmower, dishes, water for showers, food, electricity to run the fridge, a new fridge when that one breaks, everything has a cost. She asked me “how do you pay for all this?” It’s a sentiment many adults have looking at their own budgets – and one reason that some folks don’t even make a budget. It’s hard to face up to the realities of how much it costs to do what we want to do.

When we design a computing solution, it’s interesting to set up a similar budget, because we don’t always consider all of the costs associated with it. I’ve seen design sessions where the new software or servers are considered, but the “sunk” costs of personnel, networking, maintenance, increased storage, new sizes for backups and offsite storage and so on are not added in. They are already on premises, so they are assumed to be paid for already.

When you move to a distributed architecture, you'll see more costs directly reflected. Store something, pay for that storage. If the system is deployed and no one is using it, you’re still paying for it. As you watch those costs rise, you might be tempted to think that a distributed architecture costs more than an on-premises one.

And you might be right – for some solutions. I’ve worked with a few clients where moving to a distributed architecture doesn’t make financial sense – so we didn’t implement it. I still designed the system in a distributed fashion, however, so that when it does make sense there isn’t much re-architecting to do.

In other cases, however, if you consider all of the on-premises costs and compare those accurately to operating a system in the cloud, the distributed system is much cheaper. Again, I never recommend that you take a “here-or-there-only” mentality – I think a hybrid distributed system is usually best – but each solution is different. There simply is no “one size fits all” to architecting a solution.

As you design your solution, cost out each element. You might find that using a hybrid approach saves you money in one design and not in another. It’s a brave new world indeed.

So yes, in the cloud, everything costs money. But an on-premises solution also costs money – it’s just that “dad” (the company) is paying for it and we don’t always see it. When we go out on our own in the cloud, we need to ensure that we consider all of the costs.

Windows Azure End to End Examples

BuckWoody — Tue, 29 May 2012 13:45:59 GMT

I’m fascinated by the way people learn. I’m told there are several methods people use to understand new information, from reading to watching, from experiencing to exploring.

Personally, I use multiple methods of learning when I encounter a new topic, usually starting with reading a bit about the concepts. I quickly want to put those into practice, however, especially in the technical realm. I immediately look for examples where I can start trying out the concepts. But I often want a “real” example – not just something that represents the concept, but something that is real-world, showing some feature I could actually use.

And it’s no different with the Windows Azure platform – I like finding things I can do now, and actually use. So when I started learning Windows Azure, I of course began with the Windows Azure Training Kit – which has lots of examples and labs, presentations and so on. But from there, I wanted more examples I could learn from, and eventually teach others with. I was asked if I would write a few of those up, so here are the ones I use.

CodePlex

CodePlex is Microsoft’s version of an “Open Source” repository. Anyone can start a project, add code, documentation and more to it and make it available to the world, free of charge, using various licenses as they wish. Microsoft also uses this location for most of the examples we publish, and sample databases for SQL Server.

If you search in CodePlex for “Azure”, you’ll come back with a list of projects that folks have posted, including those of us at Microsoft. The source code and documentation are there, so you can learn using actual examples of code that will do what you need. There’s everything from a simple table query to a full project that is sort of a “Corporate Dropbox” that uses Windows Azure Storage.

The advantage is that this code is immediately usable. It’s searchable, and you can often find a complete solution to meet your needs. The disadvantage is that the code is pretty specific – it may not cover a huge project like you’re looking for. Also, depending on the author(s), you might not find the documentation level you want.

Link: http://azureexamples.codeplex.com/site/search?query=Azure&ac=8

Tailspin

Microsoft Patterns and Practices is a group here that does an amazing job at sharing standard ways of doing IT – from operations to coding. If you’re not familiar with this resource, make sure you read up on it. Long before I joined Microsoft I used their work in my daily job – saved a ton of time. It has resources not only for Windows Azure but other Microsoft software as well.

The Patterns and Practices group also publishes full books – you can buy these, but many are also online for free. There’s an end-to-end example for Windows Azure using a company called “Tailspin”, and the work covers not only the code but the design of the full solution. If you really want to understand the thought that goes into a Platform-as-a-Service solution, this is an excellent resource.

The advantages are that this is a book, it’s complete, and it includes a discussion of design decisions. The disadvantage is that it’s a little over a year old – and in “Cloud” years that’s a lot. So many things have changed, improved, and have been added that you need to treat this as a resource, but not the only one. Still, highly recommended.

Link: http://msdn.microsoft.com/en-us/library/ff728592.aspx

Azure Stock Trader

Sometimes you need a mix of a CodePlex-style application, and a little more detail on how it was put together. And it would be great if you could actually play with the completed application, to see how it really functions on the actual platform.

That’s the Azure Stock Trader application. There’s a place where you can read about the application, and then it’s been published to Windows Azure – the production platform – and you can use it, explore, and see how it performs.

I use this application all the time to demonstrate Windows Azure, or a particular part of Windows Azure.

The advantage is that this is an end-to-end application, and online as well. The disadvantage is that it takes a bit of self-learning to work through.

Links: Learn it: http://msdn.microsoft.com/en-us/netframework/bb499684 Use it: https://azurestocktrader.cloudapp.net/

Book Review (Book 11) - Applied Architecture Patterns on the Microsoft Platform

BuckWoody — Tue, 15 May 2012 16:50:34 GMT

This is a continuation of the books I challenged myself to read to help my career - one a month, for year. You can read my first book review here, and the entire list is here. The book I chose for April 2012 was: Applied Architecture Patterns on the Microsoft Platform. I was traveling at the end of last month so I’m a bit late posting this review here.

Why I chose this book:

I actually know a few of the authors on this book, so when they told me about it I wanted to check it out. The premise of the book is exactly as it states in the title - to learn how to solve a problem using products from Microsoft.

What I learned:

I liked the book - a lot. They've arranged the content in a "Solution Decision Framework", that presents a few elements to help you identify a need and then propose alternate solutions to solve them, and then the rationale for the choice. But the payoff is that the authors then walk through the solution they implement and what they ran into doing it.

I really liked this approach. It's not a huge book, but one I've referred to again since I've read it. It's fairly comprehensive, and includes server-oriented products, not things like Microsoft Office or other client-side tools. In fact, I would LOVE to have a work like this for Open Source and other vendors as well - would make for a great library for a Systems Architect. This one is unashamedly aimed at the Microsoft products, and even if I didn't work here, I'd be fine with that. As I said, it would be interesting to see some books on other platforms like this, but I haven't run across something that presents other systems in quite this way.

And that brings up an interesting point - This book is aimed at folks who create solutions within an organization. It's not aimed at Administrators, DBA's, Developers or the like, although I think all of those audiences could benefit from reading it. The solutions are made up, and not to a huge level of depth - nor should they be. It's a great exercise in thinking these kinds of things through in a structured way.

The information is a bit dated, especially for Windows and SQL Azure. While the general concepts hold, the cloud platform from Microsoft is evolving so quickly that any printed book finds it hard to keep up with the improvements.

I do have one quibble with the text - the chapters are a bit uneven. This is always a danger with multiple authors, but it shows up in a couple of chapters. I winced at one of the chapters that tried to take a more conversational, humorous style. This kind of academic work doesn't lend itself to that style.

I recommend you get the book - and use it. I hope they keep it updated - I'll be a frequent customer. :)