Search results matching tag 'Microsoft'

Geoflow? Is that it, Microsoft?

jamiet — Sun, 14 Apr 2013 15:42:21 GMT

I didn’t travel to the SQL Pass Business Analytics conference this week but I keenly followed what was going on via the #passbac hashtag on Twitter. Seemingly the big announcement was Geoflow Preview for Excel 2013, an add-in for Excel that visualises data over space and time (read more at Public preview of project codename “GeoFlow” for Excel delivers 3D data visualization and storytelling). Geoflow certainly looks compelling at first glance though I must say I found it rather strange that it got top billing given that Microsoft were talking about it at some SharePoint conference five months ago but nonetheless the keynote demo was apparently very impressive indeed. Here’s a screenshot of Geoflow:

I think Geoflow looks great, I really do; the questions I immediately had about it were:

Can I share my Geoflow’d Excel workbooks and have people view them on SharePoint? Answer: No, there’s no SharePoint collaboration story.
Is Geoflow part of Power View? Answer: No, its a separate installation.
OK so I have to install it. I presume then that its available in the Office App Store given that’s the new model for distributing Excel add-ins? Answer: No, you have to download it from Microsoft’s download site.

So the big reveal from Microsoft at this conference was an Excel add-in that does something very very cool but had already been announced, is only a preview, doesn’t fit with Microsoft’s BI collaboration strategy, doesn’t use their modern distribution platform and isn’t part of their Excel-based data visualisation tool. Well thank god I didn’t stump up the cost of travel, accommodation, loss of income and time away from the family for that! Doubtless there were a lot of other good reasons to go to the conference but I would have been going with high expectations of news from Microsoft that is going to be compelling and help me sell Microsoft’s BI offering to my clients – Geoflow doesn’t do that, not by a long chalk.

What I was hoping for, nay expecting, was a concrete announcement regarding Microsoft’s mobile BI strategy. We first saw Power View demonstrated on an iPad at the PASS 2011 conference and I assumed that in the intervening eighteen months they might have built something we could actually install and play around with. Apparently not.

Microsoft are getting killed in this area. At my current client all the management folk walk around with iPads glued to their hands – this is the tool on which they consume information and Microsoft doesn’t have anything for them. I was working for a client two years ago that had just invested in a product called RoamBI because it enabled them to view Reporting Services reports on an iPad. Two years ago for pity’s sake, and Microsoft haven’t released anything mobile-BI-related since!

@Jamiet

OT: Thank You, Microsoft

andyleonard — Sat, 31 Mar 2012 13:36:00 GMT

cross-posted from AndyLeonard.me…

Each April 1st for the past five years, I have been honored to receive an email from Microsoft informing me I have been recognized as a SQL Server MVP. Tomorrow will be different. Back in January – when I wrote this – I requested Microsoft not consider me for renewal.

I have enjoyed serving as a Microsoft MVP. I only got to see what it is like to be a SQL Server MVP, and I think we are part of a special community that makes being an MVP even more special. I have read the thoughts of others departing the MVP program. Many are not happy. I believe the MVP program could be better but I know the SQL Server MVP program is pretty cool.

“If it’s so cool, Andy, why are you leaving?” That is an excellent question.

The simple answer is: priorities.

Warning

The less simple answer may bore or offend you – especially if you prefer to not read about matters of faith. So I am warning you now: a lot of what follows is about faith. If this offends you and yet you find yourself wanting to learn more, skip this section and read Community and Technology below.

Having made that last statement, I feel compelled to state that being an MVP is not an impediment to faith. It is not a sin or diversion any more than anything else can be a sin or diversion, and I know plenty of MVPs who are people of faith; Christian and other faiths.

God spoke to my heart from Luke 9:23: “Whoever wants to be my disciple must deny themselves and take up their cross daily and follow me.” This is me wanting to be Christ’s disciple; denying myself, taking up His cross, and following Him. I am not “blaming God” for this decision. To the contrary: My experience as an MVP has reinforced my belief that a small number of individuals, working together, can make a measurable impact on a community.

If you have read this far you are either curious or bored, or perhaps compelled. So be it. Years ago, I participated in a Bible study entitled Experiencing God based on the book by Henry Blackaby of the same title. In it, Mr. Blackaby asserts “God speaks by the Holy Spirit through the Bible, prayer, circumstances, and the church to reveal Himself, His purposes, and His ways.” I believe God has been speaking to me through these means, to these ends.

For a while now, some friends and I have been meeting semi-regularly to pray and study the Bible. We are geographically distant so we meet online. We don’t broadcast this fact but many folks know about it because we also don’t hide it. Last year we began a study called Masterlife. It is a powerful guide to learning more about the Christ, the Bible, the church, the world, and our role in relating to all of them.

As we continue to study, we are all growing spiritually. As we grow, we are noticing new and different stuff. Although I will not speak for the other guys, I am sure they will agree when I say God is using this study to change stuff in our hearts, minds, and lives. One person recommended a book they had read called Radical: Taking Back Your Faith from the American Dream. I have to admit the title put me off some, but the recommendation was positive so I purchased the e-book.

I didn’t read it.

It sat on my Kindle for months. I read a bunch of other stuff. Some sci-fi series, a couple business books, some awesome Seth Godin works. Finally, I started reading it. This is a hard book to read. It is not anti-America, it is pro-Christ. I believe the book did a good job separating stuff that is part of America’s culture from the Christian faith. I found some of the book painful to read. I identified with parts of it in ways I did not expect; positively and negatively. And this shifted my priorities.

Some of the priority shift has already made it into this electronic journal. Some remains to be seen.

Looking back, I see alignment in some stuff that has happened to me over the past few years. I believe God used circumstances to challenge my priorities. Although I do not have it all figured out at the moment, I now know I should be doing other things with some of my time. You may wonder what those things may be. So do I. As I write this, I have very little idea. I simply know I have to re-organize my current priorities to make way for the new set. I am praying, reading the Bible, paying attention at church, and observing the circumstances to learn more. If you are so inclined, I would appreciate your prayers.

Community and Technology

I have no plans to change my community activities. I plan to continue presenting at conferences and SQL-Community events (if they will have me). I plan to remain engaged with technology as I have the past 37 years. But you will know (if you did not skip over the preceding section) that “my plans” are less and less my own.

An Interesting Side-Effect

One thing shocked me over the past few months of re-prioritizing: I began enjoying technology more. I did not realize some of my joy had been sapped, but it had. It must have happened slowly over time because I did not notice it until it began to reverse.

Thank You, Microsoft

I would like to sincerely thank Microsoft for recognizing me with the MVP award. It was a career- and life-changing experience. I got to hang out with some of the smartest people in our field. I learned a lot and grew as an individual and a technologist. Although I never felt I deserved the award (and secretly suspected they lowered the standard to allow my participation), I was honored Microsoft did.

I am excited about what is next.

:{>

Big Data - A Microsoft Tools Approach

BuckWoody — Mon, 20 Feb 2012 21:16:00 GMT

(As with all of these types of posts, check the date of the latest update I’ve made here. Anything older than 6 months is probably out of date, given the speed with which we release new features into Windows and SQL Azure)

I don’t normally like to discuss things in terms of tools. I find that whenever you start with a given tool (or even a tool stack) it’s too easy to fit the problem to the tool(s), rather than the other way around as it should be.

That being said, it’s often useful to have an example to work through to better understand a concept. But like many ideas in Computer Science, “Big Data” is too broad a term in use to show a single example that brings out the multiple processes, use-cases and patterns you can use it for.

So we turn to a description of the tools you can use to analyze large data sets. “Big Data” is a term used lately to describe data sets that have the “Four V’s” as a characteristic, but I have a simpler definition I like to use:

Big Data involves a data set too large to process in a reasonable period of time

I realize that’s a bit broad, but in my mind it answers the question and is fairly future-proof. The general idea is that you want to analyze some data, and using whatever current methods, storage, compute and so on that you have at hand it doesn’t allow you to finish processing it in a time period that you are comfortable with. I’ll explain some new tools you can use for this processing.

Yes, this post is Microsoft-centric. There are probably posts from other vendors and open-source that cover this process in the way they best see fit. And of course you can always “mix and match”, meaning using Microsoft for one or more parts of the process and other vendors or open-source for another. I never advise that you use any one vendor blindly - educate yourself, examine the facts, perform some tests and choose whatever mix of technologies best solves your problem.

At the risk of being vendor-specific, and probably incomplete, I use the following short list of tools Microsoft has for working with “Big Data”. There is no single package that performs all phases of analysis. These tools are what I use; they should not be taken as a Microsoft authoritative testament to the toolset we’ll finalize for a given problem-space. In fact, that’s the key: find the problem and then fit the tools to that.

Process Types

I break up the analysis of the data into two process types. The first is examining and processing the data in-line, meaning as the data passes through some process. The second is a store-analyze-present process.

Processing Data In-Line

Processing data in-line means that the data doesn’t have a destination - it remains in the source system. But as it moves from an input or is routed to storage within the source system, various methods are available to examine the data as it passes, and either trigger some action or create some analysis.

You might not think of this as “Big Data”, but in fact it can be. Organizations have huge amounts of data stored in multiple systems. Many times the data from these systems do not end up in a database for evaluation. There are options, however, to evaluate that data real-time and either act on the data or perhaps copy or stream it to another process for evaluation.

The advantage of an in-stream data analysis is that you don’t necessarily have to store the data again to work with it. That’s also a disadvantage - depending on how you architect the solution, you might not retain a historical record. One method of dealing with this requirement is to trigger a rollup collection or a more detailed collection based on the event.

StreamInsight - StreamInsight is Microsoft’s “Complex Event Processing” or CEP engine. This product, hooked into SQL Server 2008R2, has multiple ways of interacting with a data flow. You can create adapters to talk with systems, and then examine the data mid-stream and create triggers to do something with it. You can read more about StreamInsight here: http://msdn.microsoft.com/en-us/library/ee391416(v=sql.110).aspx

BizTalk - When there is more latency available between the initiation of the data and its processing, you can use Microsoft BizTalk. This is a message-passing and Service Bus oriented tool, and it can also be used to join system’s data together than normally does not have a direct link, for instance a Mainframe system to SQL Server. You can learn more about BizTalk here: http://www.microsoft.com/biztalk/en/us/overview.aspx

.NET and the Windows Azure Service Bus - Along the same lines as BizTalk but with a more programming-oriented design are the Windows and Windows Azure Service Bus tools. The Service Bus allows you to pass messages as well, and opens up web interactions and even inter-company routing. BizTalk can do this as well, but the Service Bus tools use an API approach for designing the flow and interfaces you want. The Service Bus offerings are also intended as near real-time, not as a streaming interface. You can learn more about the Windows Azure Service Bus here: http://www.windowsazure.com/en-us/home/tour/service-bus/ and more about the Event Processing side here: http://msdn.microsoft.com/en-us/magazine/dd569756.aspx

Store-Analyze-Present

A more traditional approach with an organization’s data is to store the data and analyze it out-of-band. This began with simply running code over a data store, but as locking and blocking became an issue on a file system, Relational Database Management Systems (RDBMs) were created. Over time a distinction was made between data used in an online processing system, meant to be highly available for writing data (OLTP) and systems designed for analytical and reporting purposes (OLAP).

Later the data grew larger than these systems were designed for, primarily due to consistency requirements. In analysis, however, consistency isn’t always a requirement, and so file-based systems for that analysis were re-introduced from the Mainframe concepts, with new technology layered in for speed and size.

I normally break up the process of analyzing large data sets into four phases:

Source and Transfer - Obtaining the data at its source and transferring or loading it into the storage; optionally transforming it along the way
Store and Process - Data is stored on some sort of persistence, and in some cases an engine handles the acquisition and placement on persistent storage, as well as retrieval through an interface.
Analysis - A new layer introduced with “Big Data” is a separate analysis step. This is dependent on the engine or storage methodology, is often programming language or script based, and sometimes re-introduces the analysis back into the data. Some engines and processes combine this function into the previous phase.
Presentation - In most cases, the data wants a graphical representation to comprehend, especially in a series or trend analysis. In other cases a simple symbolic representation, similar to the “dashboard” elements in a Business Intelligence suite. Presentation tools may also have an analysis or refinement capability to allow end-users to work with the data sets. As in the Analysis phase, some methodologies bundle in the Analysis and Presentation phases into one toolset.

Source and Transfer

You’ll notice in this area, along with those that follow, Microsoft is adopting not only its own technologies but those within open-source. This is a positive sign, and means that you will have a best-of-breed, supported set of tools to move the data from one location to another. Traditional file-copy, File Transfer Protocol and more are certainly options, but do not normally deal with moving datasets.

I’ve already mentioned the ability of a streaming tool to push data into a store-analyze-present model, so I’ll follow up that discussion with the tools that can extract data from one source and place it in another.

SQL Server Integration Services (SSIS)/SQL Server Bulk Copy Program (BCP) - SSIS is a SQL Server tool used to move data from one location to another, and optionally perform transform or other processes as it does so. You are not limited to working with SQL Server data - in fact, almost any modern source of data from text to various database platforms is available to move to various systems. It is also extremely fast and has a rich development environment. You can learn more about SSIS here: http://msdn.microsoft.com/en-us/library/ms141026.aspx BCP is a tool that has been used with SQL Server data since the first releases; it has multiple sources and destinations as well. It is a command-line utility,and has some limited transform capabilities. You can learn more about BCP here: http://msdn.microsoft.com/en-us/library/ms162802.aspx

Sqoop - Tied to Microsoft’s latest announcements with Hadoop on Windows and Windows Azure, Sqoop is a tool that is used to move data between SQL Server 2008R2 (and higher) and Hadoop, quickly and efficiently. You can read more about that in the Readme file here: http://www.microsoft.com/download/en/details.aspx?id=27584

Application Programming Interfaces - API’s exist in most every major language that can connect to one data source, access data, optionally transforming it and storing it in another system. Most every dialect of the .NET-based languages contain methods to perform this task.

Store and Process

Data at rest is normally used for historical analysis. In some cases this analysis is performed near real-time, and in others historical data is analyzed periodically. Systems that handle data at rest range from simple storage to active management engines.

SQL Server - Microsoft’s flagship RDBMS can indeed store massive amounts of complex data. I am familiar with a two systems in excess of 300 Terabytes of federated data, and the Pan-Starrs project is designed to handle 1+ Petabyte of data. The theoretical limit of SQL Server DataCenter edition is 540 Petabytes. SQL Server is an engine, so the data access and storage is handled in an abstract layer that also handles concurrency for ACID properties. You can learn more about SQL Server here: http://www.microsoft.com/sqlserver/en/us/product-info/compare.aspx

SQL Azure Federations - SQL Azure is a database service from Microsoft associated with the Windows Azure platform. Database Servers are multi-tenant, but are shared across a “fabric” that moves active databases for redundancy and performance. Copies of all databases are kept triple-redundant with a consistent commitment model. Databases are (at this writing - check http://WindowsAzure.com for the latest) capped at a 150 GB size limit per database. However, Microsoft released a “Federation” technology, allowing you to query a head node and have the data federated out to multiple databases. This improves both size and performance. You can read more about SQL Azure Federations here: http://social.technet.microsoft.com/wiki/contents/articles/2281.federations-building-scalable-elastic-and-multi-tenant-database-solutions-with-sql-azure.aspx

Analysis Services - The Business Intelligence engine within SQL Server, called Analysis Services, can also handle extremely large data systems. In addition to traditional BI data store layouts (ROLAP, MOLAP and HOLAP), the latest version of SQL Server introduces the Vertipaq column-storage technology allowing more direct access to data and a different level of compression. You can read more about Analysis Services here: http://www.microsoft.com/sqlserver/en/us/solutions-technologies/business-intelligence/analysis-services.aspx and more about Vertipaq here: http://msdn.microsoft.com/en-us/library/hh212945(v=SQL.110).aspx

Parallel Data Warehouse - The Parallel Data Warehouse (PDW) offering from Microsoft is largely described by the title. Accessed in multiple ways including using Transact-SQL (the Microsoft dialect of the Structured Query Language), This is an MPP appliance scaling in parallel to extremely large datasets. It is a hardware and software offering - you can learn more about it here: http://www.microsoft.com/sqlserver/en/us/solutions-technologies/data-warehousing/pdw.aspx

HPC Server - Microsoft’s High-Performance Computing version of Windows Server deals not only with large data sets, but with extremely complicated computing requirements. A scale-out architecture and inter-operation with Linux systems, as well as dozens of applications pre-written to work with this server make this a capable “Big Data” system. It is a mature offering, with a long track record of success in scientific, financial and other areas of data processing. It is available both on premises and in Windows Azure, and also in a hybrid of both models, allowing you to “rent” a super-computer when needed. You can read more about it here: http://www.microsoft.com/hpc/en/us/product/cluster-computing.aspx

Hadoop - Pairing up with Hortonworks, Microsoft has released the Hadoop Open-Source system - including HDFS and a Map/Reduce standardized software, Hive and Pig - on Windows and the Windows Azure platform. This is not a customized version; off-the-shelf concepts and queries work well here. You can read more about Hadoop here: http://hadoop.apache.org/common/docs/current/ and you can read more about Microsoft’s offerings here: http://hortonworks.com/partners/microsoft/ and here: http://social.technet.microsoft.com/wiki/contents/articles/6204.hadoop-based-services-for-windows.aspx

Windows and Azure Storage - Although not an engine - other than a triple-redundant, immediately consistent commit - Windows Azure can hold terabytes of information and make it available to everything from the R programming language to the Hadoop offering. Binary storage (Blobs) and Table storage (Key-Value Pair) data can be queried across a distributed environment. You can learn more about Windows Azure storage here: http://msdn.microsoft.com/en-us/library/windowsazure/gg433040.aspx

Analysis

In a “Big Data” environment, it’s not unusual to have a specialized set of tasks for analyzing and even interpreting the data. This is a new field called “data Science”, with a requirement not only for computing, but also a heavy emphasis on math.

Transact-SQL - T-SQL is the dialect of the Structured Query Language used by Microsoft. It includes not only robust selection, updating and manipulating of data, but also analytical and domain-level interrogation as well. It can be used on SQL Server, PDW and ODBC data sources. You can read more about T-SQL here: http://msdn.microsoft.com/en-us/library/bb510741.aspx

Multidimensional Expressions and Data Analysis Expressions - The MDX and DAX languages allow you to query multidimensional data models that do not fit well with typical two-plane query languages. Pivots, aggregations and more are available within these constructs to query and work with data in Analysis Services. You can read more about MDX here: http://msdn.microsoft.com/en-us/library/ms145506(v=sql.110).aspx and more about DAX here: http://www.microsoft.com/download/en/details.aspx?id=28572

HPC Jobs and Tasks - Work submitted to the Windows HPC Server has a particular job - essentially a reservation request for resources. Within a job you can submit tasks, such as parametric sweeps and more. You can learn more about Jobs and Tasks here: http://technet.microsoft.com/en-us/library/cc719020(v=ws.10).aspx

HiveQL - HiveQL is the language used to query a Hive object running on Hadoop. You can see a tutorial on that process here: http://social.technet.microsoft.com/wiki/contents/articles/6628.aspx

Piglatin - Piglatin is the submission language for the Pig implementation on Hadoop. An example of that process is here: http://blogs.msdn.com/b/avkashchauhan/archive/2012/01/10/running-apache-pig-pig-latin-at-apache-hadoop-on-windows-azure.aspx

Application Programming Interfaces - Almost all of the analysis offerings have associated API’s - of special note is Microsoft Research’s Infer.NET, a new language construct for framework for running Bayesian inference in graphical models, as well as probabilistic programming. You can read more about Infer.NET here: http://research.microsoft.com/en-us/um/cambridge/projects/infernet/

Presentation

Lots of tools work in presenting the data once you have done the primary analysis. In fact, there’s a great video of a comparison of various tools here: http://msbiacademy.com/Lesson.aspx?id=73 Primarily focused on Business Intelligence. That term itself is now not as completely defined, but the tools I’ll show below can be used in multiple ways - not just traditional Business Intelligence scenarios. Application Programming Interfaces (API’s) can also be used for presentation; but I’ll focus here on “out of the box” tools.

Excel - Microsoft’s Excel can be used not only for single-desk analysis of data sets, but with larger datasets as well. It has interfaces into SQL Server, Analysis Services, can be connected to the PDW, and is a first-class job submission system for the Windows HPC Server. You can watch a video about Excel and big data here: http://www.microsoft.com/en-us/showcase/details.aspx?uuid=e20b7482-11c9-4965-b8f0-7fb6ac7a769f and you can also connect Excel to Hadoop: http://social.technet.microsoft.com/wiki/contents/articles/how-to-connect-excel-to-hadoop-on-azure-via-hiveodbc.aspx

Reporting Services - Reporting Services is a SQL Server tool that can query and show data from multiple sources, all at once. It can also be used with Analysis Services. You can read more about Reporting Services here: http://www.microsoft.com/sqlserver/en/us/solutions-technologies/business-intelligence/reporting-services.aspx

Power View - Power View is a “Self-Service” Business Intelligence reporting tool, which can work with on-premises data in addition to SQL Azure and other data. You can read more about it and see videos of Power View in action here: http://www.microsoft.com/sqlserver/en/us/future-editions/business-intelligence/SQL-Server-2012-reporting-services.aspx

SharePoint Services - Microsoft has rolled several capable tools in SharePoint as “Services”. This has the advantage of being able to integrate into the working environment of many companies. You can read more about lots of these reporting and analytic presentation tools here: http://technet.microsoft.com/en-us/sharepoint/ee692578

This is by no means an exhaustive list - more capabilities are added all the time to Microsoft’s products, and things will surely shift and merge as time goes on. Expect today’s “Big Data” to be tomorrow’s “Laptop Environment”.

Big Data and the Cloud - More Hype or a Real Workload?

BuckWoody — Tue, 18 Oct 2011 13:57:36 GMT

Last week Microsoft announced several new offerings for “Big Data” - and since I’m a stickler for definitions, I wanted to make sure I understood what that really means. What is “Big Data”? What size hard drive is that? After all, my laptop has 1TB of storage - is my laptop “Big Data”?

There are actually a few definitions for this term, most notably those involving the “Four V’s” Volume, Velocity, Variety and Variability. Others disagree with this definition. I tend to try and get things into their simplest form, so I’m using this definition for myself:

Big data is defined as a large set of computationally expensive data that is worked on simultaneously.

Let me flesh that out a little. To be sure, “Big Data” has a larger size than say a few megabytes. The reason this is important is that it takes special hardware to be able to move large sets of data around, store it, process it and so on. (large set)

If you store a LOT of data, but only use a small portion of it at a time, that really isn’t super-hard to do. It’s mainly a storage issue at that point. But, if you do need to work with a large portion of the data at one time, then the memory, CPU and transfer components of the system have to adapt to be responsive - new ways to work with that data (game theory, knot-algorithms, map-reduce, etc.) need to be brought into play. (computationally expensive)

Once that data is loaded into the processing area (memory or whatever other mechanism is used) it must be worked on in parallel to come back in a reasonable time. You have two options here - you can scale the system up with more internal hardware (CPU’s, memory and so on) or you can scale it out to have multiple systems work on it at the same time using paradigms such as map/reduce and so on. Actually, when you lay this out in an architecture diagram, scale up or out doesn’t actually change the logical structure of the process - in scale out the network becomes the bus, and the nodes become more RAM and computing power. Of course, there are changes in code for how you stitch the workload back together. (worked on simultaneously)

So back to the original question. Is Big Data, as I have defined it here, a workload for Windows and SQL Azure? Absolutely! In fact, it’s probably one of the main workloads, and I believe it represents the latest, and perhaps also the earliest frontier of computing. Jim Gray, a former researcher here at Microsoft and a hero of mine, was working on this very topic. I believe as he did - all computing is simply an interface over data.

Microsoft has multiple offerings on the topic of Big Data. In posts that follow from myself and my co-workers, we’ll explore when and where you use each one. Whether you are a data professional or a developer, this is the new frontier - don’t wait to educate yourself on how to leverage Big Data for your organization.

Hadoop on Windows Azure and SQL Server - Microsoft’s partnership to include Hadoop workloads on Windows Azure and SQL Server/Parallel Data Warehouse (PDW)

LINQ to HPC - Microsoft’s High-Performance Computing SKU of HPC is now in Azure

Windows Azure Table Storage - A key/value pair type storage with full partitioning that is immediately consistent, able to handle huge loads of data and works with any REST-compatible language

Other offerings - Including the new Data Explorer, Project Daytona (with a Big Data Toolkit for Scientists and researchers), Power View and more.

The era of Big Data is here. And you can use Windows and SQL Azure to bring it to your organization.

You Probably Already Have a “Private Cloud”

BuckWoody — Tue, 14 Jun 2011 13:05:05 GMT

I’ve mentioned before that I’m not a fan of the word “Cloud”. It’s too marketing-oriented, gimmicky and non-specific. A better definition (in many cases) is “Distributed Computing”. That means that some or all of the computing functions are handled somewhere other than under your specific control.

But there is a current use of the word “Cloud” that does not necessarily mean that the computing is done somewhere else. In fact, it’s a vector of Cloud Computing that can better be termed “Utility Computing”. This has to do with the provisioning of a computing resource. That means the setup, configuration, management, balancing and so on that is needed so that a user – which might actually be a developer – can do some computing work. To that person, the resource is just “there” and works like they expect, like the phone system or any other utility.

The interesting thing is, you can do this yourself. In fact, you probably already have been, or are now. It’s got a cool new trendy term – “Private Cloud”, but the fact is, if you have your setup automated, the HA and DR handled, balancing and performance tuning done, and a process wrapped around it all, you can call yourself a “Cloud Provider”.

A good example here is your E-Mail system. your users – pretty much your whole company – just logs into e-mail and expects it to work. To them, you are the “Cloud” provider. On your side, the more you automate and provision the system, the more you act like a Cloud Provider. Another example is a database server. In this case, the “end user” is usually the development team, or perhaps your SharePoint group and so on. The data professionals configure, monitor, tune and balance the system all the time. The more this is automated, the more you’re acting like a Cloud Provider.

Lots of companies help you do this in your own data centers, from VMWare to IBM and many others. Microsoft's offering in this is based around System Center – they have a “cloud in a box” provisioning system that’s actually pretty slick.

The most difficult part of operating a Private Cloud is probably the scale factor. In the case of Windows and SQL Azure, we handle this in multiple ways – and we're happy to share how we do it. It’s not magic, and the algorithms for balancing (like the one we started with called Paxos) are well known. The key is the knowledge, infrastructure and people. Sure, you can do this yourself, and in many cases such as top-secret or private systems, you probably should. But there are times where you should evaluate using Azure or other vendors, or even multiple vendors to spread your risk. All of this should be based on client need, not on what you know how to do already.

So congrats on your new role as a “Cloud Provider”. If you have an E-mail system or a database platform, you can just put that right on your resume.

Have SSIS' differing type systems ever caused you problems?

jamiet — Mon, 14 Mar 2011 16:46:00 GMT

One thing that has always infuriated me about SSIS is the fact that every package has three different type systems; to give you an idea of what I am talking about consider the following:

The SSIS dataflow's type system is made up of types called DT_* (e.g. DT_STR, DT_I4)
The SSIS variable type system is based on .Net datatypes (e.g. String, Int32)
The types available for Execute SQL Task's parameters are based on something else - I don't exactly know what (e.g. VARCHAR, LONG)

Speaking euphemistically ... this is not an optimum situation (were I not speaking euphemistically I would be a lot ruder) and hence I have submitted a suggestion to Connect at [SSIS] Consolidate three type systems into one requesting that it be remedied. This accompanying blog post is not however a request for votes (though that would be nice); the reason is actually subtler than that. Let me explain.

I have been submitting bugs and suggestions pertaining to SSIS for years and have, so far, submitted over 200 Connect items. If that experience has taught me anything it is this - Connect items are not generally actioned because they are considered "nice to have". No, SSIS Connect items get actioned because they cause customers grief and if I am perfectly honest I must admit that, other than being a bit gnarly, SSIS' three type system architecture has never knowingly caused me any significant problems.

The reason for this blog post is to ask if any reader out there has ever encountered any problems on account of SSIS' three type systems or have you, like me, never found them to be a problem? Errors or performance degredation caused by implicit type conversions would, I believe, present a strong case for getting this situation remedied in a future version of SSIS so if you HAVE encountered such problems I would encourage you to leave a comment on the Connect submission accordingly. Let me know in the comments too - I would be interested to hear others' opinions on this.

@Jamiet

MVPs and the Community

andyleonard — Fri, 11 Feb 2011 15:51:00 GMT

Introduction

Earlier this month, David Woods decided to drop his MVP award. The move inspired some interesting comments and discussion among MVPs.

David's points are:

MVP Expertise
Microsoft Technology Products
Microsoft "Listens"
Cost-Benefits for an MVP

MVP Expertise

After mentioning he's encountered MVPs who are not experts, David states: "The way you get in is by contributing to the community."

Honestly, I don't know the specifics of how someone becomes a Microsoft MVP. And I'm ok with that. It's Microsoft's program and they can run it however they please. That's not a complaint, compliment, excuse, or endorsement. It's just the way it is. I've watched others become MVPs. I've nominated several people for the award and some of them have been awarded. They weren't awarded because I nominated them - they were awarded because the good folks at the Microsoft MVP Award program checked them out, vetted them, and agreed they should be MVPs.

What exactly does Microsoft consider? I don't know.

I cannot speak to other programs, but I know a lot of SQL Server MVPs. To a person, they're all experts. Do they all know everything about everything in SQL Server? No. But they know an awful lot about their preferred corner of SQL Server. Some rock on the relational engine and T-SQL. Others are masters of storage and SAN administration. Some know business intelligence, some don't. Quite a few SQL Server MVPs are proficient in several areas or disciplines. Very few are laser-focused on one aspect of the technology.

I can hear you thinking "Why?" I'm glad you asked! SQL Server is HUGE! Database professionals refer to the collection of technologies associated with SQL Server as the stack. The stack includes the SQL Server relational database engine, workstation tools for development and administration (SQL Server Management Studio, Busienss Intelligence Development Studio, SQL Server Profiler, Configuration tools, etc.), SQL Server Analysis Services (SSAS), SQL Server Reporting Services (SSRS), and SQL Server Integration Services (SSIS). There are sub-specialties within technologies as well: Notification Services (SQL Server 2005 only), Extended Events, XML, Spacial data, Data Warehousing, Change Data Capture, and more. In other words, there's a lot to keep up with!

I'm not sure who David encountered in his adventures, but it's entirely possible to find a SQL Server MVP who doesn't know much about some portion of the SQL Server stack or sub-specialties.

One of the coolest things about working with SQL Server is the community. The SQL Server community is chock full of expertise! And - news flash - the expertise doesn't lie solely with SQL Server MVPs. What makes this community so cool is: Everyone shares. If I get stuck on some aspect of the technology, I know who to contact for help. If someone gets stuck on a part of the stack that I know, they contact me.

This happens all the time!

Microsoft Technology Products

If you step back, you realize that Microsoft is a software development shop. They work with others to make devices (Zune, XBox, etc.), but David takes aim at a few software platforms and products.

Being a software development shop means sometimes you're ahead, sometimes you're behind. Sometimes your software is the hottest thing on the market; sometimes your software is not so hot. It happens to me, you... and Microsoft. Because Microsoft is larger and more experienced at building software, they're held to a higher standard than you or me. I don't hear them complaining about that. Instead, I see them working very hard to improve processes and the quality of their deliverables. This feeds into David's complaints about...

Microsoft "Listens"

David states "most products you never hear about until it is too late." I understand how this can be frustrating to anyone providing feedback to Microsoft who doesn't understand development lifecycles. I'm not sure I can share what I know about how Microsoft develops software internally.

Aside: MVPs know things that are not common knowledge. This bugs some people. If this bugs you, I'm sorry. Your government is keeping things from you. If you're in business, your competitor is as well. You never know everything, or even everything you want to know. That's part of life.

I know everyone who builds software employs some kind of development life cycle. At some point in this life cyle, there's a feature lock where nothing new is added to the product without major adjustment to the remainder of the life cycle. There are ways to develop software to facilitate for constantly-shifting requirements and these methodologies lend themselves to developing certain types of applications and platforms. But let's take a step back and think about the nature of a constantly-shifting-requirements application: this is messy. Will developing software in this manner lead to improved processes and quality for the deliverables?

Cost-Benefits for an MVP

I honestly don't know how to address this complaint. It smacks of what-have-you-done-for-me-lately. Maybe I'm being harsh - that is not my intention. I will share this: I know lots of people who aren't MVPs that travel to events to share their passion and expertise, and they do so at their own expense. I know plenty of MVPs who do the same, and all I know did so before they were awarded MVP. It may very well be part of the reason they are MVPs today, but (again) I do not know the criteria Microsoft uses to award MVPs.

Personally, I don't look at the expense. I don't (usually) drop in, present, and leave. I also receive training while attending events - especially events like the PASS Summit, Code Camps, and SQL Saturdays. I always learn something!

I also know of a lot of non-MVP community people who work behind the scenes to manage User Group meetings and the events listed above. They're all volunteers. I'm not sure of all their motivations, I can't see their hearts. But I see their tireless actions.

Volunteering isn't for everyone. And even those who enjoy volunteering can burn out. Perhaps that's what happened here. If so, I hope David takes a well-deserved break and returns to his community.

Conclusion

My experiences with the Microsoft MVP program have been positive. That's not to say there hasn't been the occasional bump in the road, but even during rough patches I've been treated with trust and respect by my peers (MVPs or no) and Microsoft.

:{>

Microsoft TechEd Session Preference Survey

andyleonard — Sat, 15 Jan 2011 17:00:00 GMT

There's a Session Preference Survey live at the TechEd North America 2011 website. I like this idea a lot: getting feedback from the Community!

:{>

And The Winner Is...

andyleonard — Fri, 09 Jul 2010 03:00:00 GMT

David Taylor!

There were a lot of really great comments on the post titled A Visual Studio 2010 MSDN Seeding Card Giveaway Contest! The judging was completely subjective - just me reading the comments and making a call.

One suggestion was apparently plagarized... bummer.

All of the projects sound awesome and I hope those who commented will have the opportunity to build them!

Thanks everyone for participating!

:{>

A Visual Studio 2010 MSDN Seeding Card Giveaway Contest

andyleonard — Sat, 03 Jul 2010 11:00:00 GMT

I have a Visual Studio 2010 MSDN Seeding Card to give away. It's good for a year.

Rather than just hand it out, I've decided to hold and judge a contest. So tell me - in the comments of this post - what cool project you would like to use the software you can access free for a year? Tell me what you think you can do - what you'd like to try. I'll announce a winner Friday 9 Jul 2010 on this blog.

Reminder: I moderate comments on this blog. Yours will not appear immediately.

Go!

:{>

Note: As pointed out by friends in the Comments, the subscription is valid for a year but must be activated witin 180 days. I have updated the post to correct this... Apologies for my confusion!