Big Data and the Cloud - More Hype or a Real Workload?

BuckWoody — Tue, 18 Oct 2011 13:57:36 GMT

Last week Microsoft announced several new offerings for “Big Data” - and since I’m a stickler for definitions, I wanted to make sure I understood what that really means. What is “Big Data”? What size hard drive is that? After all, my laptop has 1TB of storage - is my laptop “Big Data”?

There are actually a few definitions for this term, most notably those involving the “Four V’s” Volume, Velocity, Variety and Variability. Others disagree with this definition. I tend to try and get things into their simplest form, so I’m using this definition for myself:

Big data is defined as a large set of computationally expensive data that is worked on simultaneously.

Let me flesh that out a little. To be sure, “Big Data” has a larger size than say a few megabytes. The reason this is important is that it takes special hardware to be able to move large sets of data around, store it, process it and so on. (large set)

If you store a LOT of data, but only use a small portion of it at a time, that really isn’t super-hard to do. It’s mainly a storage issue at that point. But, if you do need to work with a large portion of the data at one time, then the memory, CPU and transfer components of the system have to adapt to be responsive - new ways to work with that data (game theory, knot-algorithms, map-reduce, etc.) need to be brought into play. (computationally expensive)

Once that data is loaded into the processing area (memory or whatever other mechanism is used) it must be worked on in parallel to come back in a reasonable time. You have two options here - you can scale the system up with more internal hardware (CPU’s, memory and so on) or you can scale it out to have multiple systems work on it at the same time using paradigms such as map/reduce and so on. Actually, when you lay this out in an architecture diagram, scale up or out doesn’t actually change the logical structure of the process - in scale out the network becomes the bus, and the nodes become more RAM and computing power. Of course, there are changes in code for how you stitch the workload back together. (worked on simultaneously)

So back to the original question. Is Big Data, as I have defined it here, a workload for Windows and SQL Azure? Absolutely! In fact, it’s probably one of the main workloads, and I believe it represents the latest, and perhaps also the earliest frontier of computing. Jim Gray, a former researcher here at Microsoft and a hero of mine, was working on this very topic. I believe as he did - all computing is simply an interface over data.

Microsoft has multiple offerings on the topic of Big Data. In posts that follow from myself and my co-workers, we’ll explore when and where you use each one. Whether you are a data professional or a developer, this is the new frontier - don’t wait to educate yourself on how to leverage Big Data for your organization.

Hadoop on Windows Azure and SQL Server - Microsoft’s partnership to include Hadoop workloads on Windows Azure and SQL Server/Parallel Data Warehouse (PDW)

LINQ to HPC - Microsoft’s High-Performance Computing SKU of HPC is now in Azure

Windows Azure Table Storage - A key/value pair type storage with full partitioning that is immediately consistent, able to handle huge loads of data and works with any REST-compatible language

Other offerings - Including the new Data Explorer, Project Daytona (with a Big Data Toolkit for Scientists and researchers), Power View and more.

The era of Big Data is here. And you can use Windows and SQL Azure to bring it to your organization.

How to get Microsoft to speak at your event

BuckWoody — Mon, 22 Feb 2010 14:35:36 GMT

If you’re holding a special event, such as a SQL Saturday or a user group, one of the challenges is to get speakers for the event. Now, the best speakers come from the community – people who use the product day-in, day-out. They have a wealth of expertise, and many of them are really great presenters.

But from time to time you might want to get a Microsoft person to speak at your group or event. Microsoft is a big company, and you can get everything from Marketing (yes, there’s a place for that) to deep technical topics in that skillset. But how can your group get Microsoft to speak for you?

It might be easier than you think.

Microsoft has three or four main areas that you might be able to pull from, starting with the folks that are closest to you: Sales.

Don’t make that face, there is sales and then there is sales. Within the sales team are four kinds of folks – your account manager, who owns the relationship between Microsoft and your company, the Sales professional, who owns the revenue and licensing for your account, the Account Technology Specialist (ATS) who knows about multiple Microsoft products, and the Technology Specialist (TSP or TS) that has a deep knowledge in a single technology, like SQL Server. These last two folks are the people who could deliver technical talks, especially around the newest products. And many of them are willing to do it. They are tied to a geographical area, so if your group has people in it that work at a company that is headquartered in that geography, the TSP and ATS might be willing to come out and chat with your group. And you might even want a Salesperson to come. Ever have a licensing briefing? Ever have questions? Ask them!

How to find them: Call your local Microsoft Office (the building, not the software). Find yours here: http://www.microsoft.com/worldwide/

The product group. If you live in the Redmond area, you should be coming to the PASS chapter monthly user group meetings here. It’s held at the Microsoft campus, it’s free, and it has a Microsoft product group member speaking just about every month. When I ran a user group for five years, it was impossible for me to get this kind of attention from the product group. It’s amazing.

Wait – you don’t live next to Microsoft? Not to worry. If you have a projector and a good set of speakers, the product teams can set up a LiveMeeting and present. I’ve done this several times myself.

How to find them: Twitter, Facebook, LinkedIn. Lurk on forums and see what the signature blocks say. Let’s see….oh, how about blogs like this one. You can find lots of blogs from Microsoft folks here: http://www.microsoft.com/sqlserver/2008/en/us/community.aspx

There are other groups at Microsoft that are willing to speak, and all you have to do is ask around. From “evangelists” to marketing, from training to even the documentation group, folks are out there that have really deep knowledge in just about every area of the product.

How to find them: TechEd, PASS and other conventions. Give them your card, get their e-mail addresses.

Here’s something to keep in mind: Microsoft works us pretty hard. In fact, sleep is really a luxury most of us aren’t granted; I think you get to do that when you’re a president or something. Maybe. So the folks you contact will be busy – really busy – just like you. I have traveled out of my district to present from time to time, but I have to take vacation time to do it, and Microsoft doesn’t pay my flight, hotel, food, anything like that. Any time we give you is “out of our own pocket”, so travel and late nights should be taken into account. So don’t abuse them, don’t always expect the invitation to be accepted. Most of us don’t get paid by Microsoft to do the extra presentations any more than you do, so keep that in mind and be polite and understanding. But do ask – most of us are really enthusiastic about this technology, and we love to help others learn, and to learn from the folks we meet.

Search results matching tags 'PASS' and 'Microsoft'

Big Data and the Cloud - More Hype or a Real Workload?

How to get Microsoft to speak at your event