Search results matching tags 'SQL Server', 'Microsoft', and 'Career'

Big Data and the Cloud - More Hype or a Real Workload?

BuckWoody — Tue, 18 Oct 2011 13:57:36 GMT

Last week Microsoft announced several new offerings for “Big Data” - and since I’m a stickler for definitions, I wanted to make sure I understood what that really means. What is “Big Data”? What size hard drive is that? After all, my laptop has 1TB of storage - is my laptop “Big Data”?

There are actually a few definitions for this term, most notably those involving the “Four V’s” Volume, Velocity, Variety and Variability. Others disagree with this definition. I tend to try and get things into their simplest form, so I’m using this definition for myself:

Big data is defined as a large set of computationally expensive data that is worked on simultaneously.

Let me flesh that out a little. To be sure, “Big Data” has a larger size than say a few megabytes. The reason this is important is that it takes special hardware to be able to move large sets of data around, store it, process it and so on. (large set)

If you store a LOT of data, but only use a small portion of it at a time, that really isn’t super-hard to do. It’s mainly a storage issue at that point. But, if you do need to work with a large portion of the data at one time, then the memory, CPU and transfer components of the system have to adapt to be responsive - new ways to work with that data (game theory, knot-algorithms, map-reduce, etc.) need to be brought into play. (computationally expensive)

Once that data is loaded into the processing area (memory or whatever other mechanism is used) it must be worked on in parallel to come back in a reasonable time. You have two options here - you can scale the system up with more internal hardware (CPU’s, memory and so on) or you can scale it out to have multiple systems work on it at the same time using paradigms such as map/reduce and so on. Actually, when you lay this out in an architecture diagram, scale up or out doesn’t actually change the logical structure of the process - in scale out the network becomes the bus, and the nodes become more RAM and computing power. Of course, there are changes in code for how you stitch the workload back together. (worked on simultaneously)

So back to the original question. Is Big Data, as I have defined it here, a workload for Windows and SQL Azure? Absolutely! In fact, it’s probably one of the main workloads, and I believe it represents the latest, and perhaps also the earliest frontier of computing. Jim Gray, a former researcher here at Microsoft and a hero of mine, was working on this very topic. I believe as he did - all computing is simply an interface over data.

Microsoft has multiple offerings on the topic of Big Data. In posts that follow from myself and my co-workers, we’ll explore when and where you use each one. Whether you are a data professional or a developer, this is the new frontier - don’t wait to educate yourself on how to leverage Big Data for your organization.

Hadoop on Windows Azure and SQL Server - Microsoft’s partnership to include Hadoop workloads on Windows Azure and SQL Server/Parallel Data Warehouse (PDW)

LINQ to HPC - Microsoft’s High-Performance Computing SKU of HPC is now in Azure

Windows Azure Table Storage - A key/value pair type storage with full partitioning that is immediately consistent, able to handle huge loads of data and works with any REST-compatible language

Other offerings - Including the new Data Explorer, Project Daytona (with a Big Data Toolkit for Scientists and researchers), Power View and more.

The era of Big Data is here. And you can use Windows and SQL Azure to bring it to your organization.

Do you have a data roadmap?

BuckWoody — Thu, 13 May 2010 12:16:55 GMT

I often visit companies where they asked me “What is SQL Server’s Roadmap?” What they mean is that they want to know where Microsoft is going with our database products. I explain that we’re expanding not only the capacities in SQL Server but the capabilities – we’re trying to make an “information platform”, rather than just a data store.

But it’s interesting when I ask the same question back. “What is your data roadmap?” Most folks are surprised by the question, thinking only about storage and archival. To them, data is data.

Ah, not so. Your data is one of the most valuable, if not the most valuable asset in your organization. And you should be thinking about how you’ll acquire it, how it will be distributed, how you’ll archive it (which includes more than just backing it up) and most importantly, how you’ll leverage it. Because it’s only when data becomes information that it is truly useful. to be sure, the folks on the web that collect lots of data have a strategy for it – do you?

The Three Things that Brought Me Here

BuckWoody — Tue, 19 Jan 2010 14:34:00 GMT

I've been asked to explain three major events that brought me to where I am today. That's a tall order for me, since I view everything as a continuum of multiple small events. But there are some "seminal" events that tend to either show or dictate a further direction, so in the context of working with technology in general and SQL Server in specific, I'll mention a "pivotal" three things that I believe brought me here, working on and with SQL Server.

1. It's Spock's fault.
Like many - perhaps most - technically-inclined folks, I'm a science-fiction geek. I read Jules Verne and the older sci-fi masters, and then on to the newer things. But it was, of course the "old" Star Trek that I really loved as a child. We were quite poor, but we had a black-and-white TV that someone gave us and I didn't miss an episode. I especially liked Mr. Spock, the "Science Officer" on the show. I saved, scrimped and did without to buy a small computer book which explained electronic circuits, and learned to build a computer. I hooked it up to the BW TV, and a cassette recorder. From there I built a program or two in assembly.

2. The wild blue yonder.
Because I grew up poor, I had no money at all when I left home at 18. I had been working wince I was 13, but that was just to basically take care of myself. So, with no immediate prospect of sleeping indoors, I decided that I would try my hand at the military services. I knew I wasn't tough enough to be a marine, didn't think I cold handle being on ship for a long time, and wasn't sure about the army. My uncle (a marine) described the Air Force as "a business that kills people. You wear a suit, stay on something called 'base', and you send the officers off to fight instead of the other way around." That sounded good to me.

While I was in the Air Force (in one of those "business" roles) they began to install the first PC's. I jumped in, and earned the strange title of "Special Assistant to the Base Commander for Small Computer Affairs". Hey, I don't write this stuff, folks, I just report what happened. Anyway, I was able to basically serve as a computer consultant. I came in to an office, evaluated computer needs, ordered them, installed and configured the hardware and software, and then taught the other airmen how to use them. Lots of fun.

3. (the Emporer's theme from Star Wars)
And of course what really brought "here" is Microsoft. I was a SQL Server MVP, had authored five books on SQL Server, presented for Microsoft and ran the Tampa (Florida) SQL Server user group for a few years when they made me an offer. At first I refused, because they wouldn't let me live in Florida. I really like Florida. Anyway, they explained for that role I really needed to be here in Seattle. So relented, moved up, and promptly ran into someone who lived about three miles from my house in Tampa - wait for it - who had EXACTLY the same role I had at Microsoft. (not that I'm bitter. No, not me.)

And so here I am. As I mentioned earlier, I think it's less about "big" events, than it is about the hundreds of decisions we make each day that make us who we are.

So what about you? Tag....you're it.

Know Your Product Specifications

BuckWoody — Wed, 13 Jan 2010 14:57:01 GMT

As the Data Professional in your organization, the rest of the org looks to you to ensure that the system can handle what the business requires. To do that, you need to know two things: what the business requires, and what SQL Server can do.

But of course there’s a bit more to it than that. Knowing the business side of the requirements – well, I teach an entire course on that. But knowing what SQL Server can do is something you can find out on your own.

SQL Server comes in versions, which are released based on date, and editions, which are based on features and capabilities. It’s that last part that I want to focus on today.

As Microsoft SQL Server matures, you’re going to see even more separation between what each edition of SQL Server can do and where it should be used. In the past, most folks have only focused on three editions – Express (the “free” one), Standard, and Enterprise. The rule of thumb was that if Standard was good enough at the moment, put it in. And it is true (and a good thing) that you can upgrade from one edition to another fairly easily.

But as time goes on, we should spend a little more time understanding what each edition does, what it’s features and capabilities are, and where and when we should put them in. As I study this information, I’ll throw in my 2 cents and you can as well based on what you see. One thing I’ve found so far is that once I have the business requirements, there’s a mix of what I can write in code and what might already be included in a different edition. It’s important to look long and hard at that choice – writing a feature on my own is certainly cheaper in the short term than moving to a “higher” edition, but in some cases it makes sense to let Microsoft handle that lifting.

These links are ones that you should bookmark and take a peek at periodically. They are the “header” links for more information on those features and capabilities:

SQL Server 2008: http://msdn.microsoft.com/en-us/library/ms143287.aspx

SQL Server 2008 R2: http://msdn.microsoft.com/en-us/library/ms143287(SQL.105).aspx

In addition, you might start learning a little more about SQL Azure. I’ll talk more about that later.