Search results matching tags 'PASS' and 'SQL Server'

Big Data and the Cloud - More Hype or a Real Workload?

BuckWoody — Tue, 18 Oct 2011 13:57:36 GMT

Last week Microsoft announced several new offerings for “Big Data” - and since I’m a stickler for definitions, I wanted to make sure I understood what that really means. What is “Big Data”? What size hard drive is that? After all, my laptop has 1TB of storage - is my laptop “Big Data”?

There are actually a few definitions for this term, most notably those involving the “Four V’s” Volume, Velocity, Variety and Variability. Others disagree with this definition. I tend to try and get things into their simplest form, so I’m using this definition for myself:

Big data is defined as a large set of computationally expensive data that is worked on simultaneously.

Let me flesh that out a little. To be sure, “Big Data” has a larger size than say a few megabytes. The reason this is important is that it takes special hardware to be able to move large sets of data around, store it, process it and so on. (large set)

If you store a LOT of data, but only use a small portion of it at a time, that really isn’t super-hard to do. It’s mainly a storage issue at that point. But, if you do need to work with a large portion of the data at one time, then the memory, CPU and transfer components of the system have to adapt to be responsive - new ways to work with that data (game theory, knot-algorithms, map-reduce, etc.) need to be brought into play. (computationally expensive)

Once that data is loaded into the processing area (memory or whatever other mechanism is used) it must be worked on in parallel to come back in a reasonable time. You have two options here - you can scale the system up with more internal hardware (CPU’s, memory and so on) or you can scale it out to have multiple systems work on it at the same time using paradigms such as map/reduce and so on. Actually, when you lay this out in an architecture diagram, scale up or out doesn’t actually change the logical structure of the process - in scale out the network becomes the bus, and the nodes become more RAM and computing power. Of course, there are changes in code for how you stitch the workload back together. (worked on simultaneously)

So back to the original question. Is Big Data, as I have defined it here, a workload for Windows and SQL Azure? Absolutely! In fact, it’s probably one of the main workloads, and I believe it represents the latest, and perhaps also the earliest frontier of computing. Jim Gray, a former researcher here at Microsoft and a hero of mine, was working on this very topic. I believe as he did - all computing is simply an interface over data.

Microsoft has multiple offerings on the topic of Big Data. In posts that follow from myself and my co-workers, we’ll explore when and where you use each one. Whether you are a data professional or a developer, this is the new frontier - don’t wait to educate yourself on how to leverage Big Data for your organization.

Hadoop on Windows Azure and SQL Server - Microsoft’s partnership to include Hadoop workloads on Windows Azure and SQL Server/Parallel Data Warehouse (PDW)

LINQ to HPC - Microsoft’s High-Performance Computing SKU of HPC is now in Azure

Windows Azure Table Storage - A key/value pair type storage with full partitioning that is immediately consistent, able to handle huge loads of data and works with any REST-compatible language

Other offerings - Including the new Data Explorer, Project Daytona (with a Big Data Toolkit for Scientists and researchers), Power View and more.

The era of Big Data is here. And you can use Windows and SQL Azure to bring it to your organization.

Speaking at PASS Summit 2011 - Lightning Talk Accepted!

Argenis — Wed, 24 Aug 2011 22:31:00 GMT

Quick post - I got news that I will be speaking at the PASS Summit 2011. What a huge honor. As I understand it, I am the first Venezuelan ever to speak at PASS. I'm super excited!

The Lightning Talk is called "Quick Tips to Enhance tempdb Performance", and I have to practice this talk a lot. I'll have exactly 5 minutes to deliver it, and discussions about tempdb performance can go on for hours. So I need to keep it concise and to the point.

A full lineup of all Lightning Talks can be found here: http://www.sqlpass.org/summit/2011/SummitContent/LightningTalks2011.aspx

Also, I changed my Twitter handle to @DBArgenis, to better reflect who I am, my passion and commitment towards the SQL Server community.

Can't wait for the Summit now.

See you in Columbus Saturday?

drsql — Mon, 06 Jun 2011 04:32:45 GMT

Assuming all goes as planned, I will be in Columbus, OH this Friday night and Saturday for SQL Saturday 75. I really love SQL Saturday events the best of all of the events because they are very intimate in nature. As a fairly antisocial person, I sometimes get overwhelmed by the size of other events, even the SQL Rally was just barely in my comfort range. Here the number of people and size of rooms just feels like home, like you are shooting the breeze with a group of friends.

My session will be at 9:00 AM (full schedule), so don’t be late!

Characteristics of a Great Relational Database
When queried, most database professionals would mention normalized as one of the most important characteristics that tell the difference between a good and bad database design. I won't disagree in the least, but there is so much more to be considered. Even if you did a great job of normalization, poor naming, poorly implemented keys, too many or too few indexes, and so on can derail your design. In this session I will present seven primary characteristics of a design that differentiates between an ugly design that will have your colleagues nitpicking you to death and one that will have them singing your praises. Characteristics such as comprehendible, documented, secure, well performing, and more (including normalized, naturally) will be discussed.

It is the second time I will do this presentation, and the first time where I can see the faces of the recipients, so it will be nice to gauge how people like it. It is a lot of fun actually, though no matter what I am talking about I want to talk more about normalization, which I believe is the key to improving the databases of the future. So many of the session that are given at these things are geared towards systems that are already screwed up and limping along, and I really want to evangelize the merits of doing it right. That is where this presentation fits in, the time period between design and performance tuning, where you determine the future work that is done with the system. Is it well performing, understandable, easy to maintain and use? Or does it take a crew of ten thousand DBAs doing nothing but putting their fingers in the leaks to keep the thing running?

And don’t forget all of those BI sessions too… The better you do with the relational database, the easier the dimensional designer/implementers have it too.

Anyhow, I hope to see you all there (well, not all of you, just the SQL nerds who are reading this. The history buffs who are still wondering why we are going to be inside the founder of our continent this weekend, well, you I feel sorry for you.

PASS Goals and the SQL Server Community

andyleonard — Mon, 22 Nov 2010 12:00:00 GMT

Introduction

I was honored to sit at the blogger's table during the keynote sessions at the PASS Summit 2010! Our official job was to promote the Summit keynotes via social media and our blogs. Our unofficial job was to pick on each other.

Rob's response? "Realised is spelled with an 's', not a 'z'."

Goals Are Good

In the kickoff session for the PASS Summit 2010, Rushabh Mehta announced some lofty goals for the organization to achieve by 2015:

250,000 members
1,000,000 hours of technical learning available to members
PASS Summit 2015 attended by 7,500

These goals are a stretch and I admire them for that reason. And I am confident the goals are achievable and that PASS can achieve them. I think there will need to be significant changes in the PASS organization to accomplish these goals, but I see the potential beginnings of that change in the organization's leadership - the PASS Board - even now.

Votes and NDA

One important change is Individual PASS Board votes will now be disclosed. According to Rushabh's PASS blog:

This is a topic that has been under active discussion for a few months and was also part of our board agenda for today. The board voted unanimously to make individual votes public except for topics which are considered NDA by the Board.

Conclusion

The PASS Board is making changes and moving in the right direction with these goals and changes. I applaud them.

:{>

Blogging from the PASS Keynote : 2010-11-09

AaronBertrand — Tue, 09 Nov 2010 17:15:00 GMT

The keynote starts off with a tine Turner impersonator (or maybe it really was Tina?):

Rushabh Mehta (donning the Tina wig) delivers the first segment of the keynote, telling the SQL Server community that we are "simply the best." He then introduces the Board of Directors and founding partners. He tells us the Board of Directors will be available for chatting at the Meet & Greet on Thursday from 5:30 - 7:30 PM, and that PASS is eager to hear about our ideas for how to improve the organization and its events.

This Summit has 3,807 attendees from 48 countries. And there are over 4,500 registrants from 49 countries for the streaming keynotes and WIT event. There are 191 speakers, 44 of them MVPs, and 111 out of the 168 sessions are 300+ level. And for the first time, the DVDs will include all 168 sessions *AND* the pre- and post-con recordings. (And probably David DeWitt's keynote again, if history is any indication.)

Exhibitors are an important part of PASS, obviously. Visit the Exhibit Hall (which sold out for the first time ever). The hours are Tuesday from 10:45 to 4:00 and Wednesday from 10:45 to 5:00. Also all attendees are welcome to the Exhibit Hall Reception tonight from 6 - 8.

Mark Souza comes on stage and talks about Ask the Experts, SQL Clinic, and several ways you can interact with the MVPs and Microsoft employees that are here at PASS. One of the giveaways each day will be an XBox Kinect. They give away an Alienware laptop after placing an envelope under a random chair in the crowd.

Before Ted Kummert takes the stage, there is an entertaining video about the evolution of SQL Server - from breaking out of the Sybase code ($100 million/yr) to SQL Server 2008 R2 ($3 billion/yr). Ted comes on stage and exclaims that this is the largest PASS ever. He talks about the focus of SQL Server over the years, and more importantly, the future. He explains that SQL Server 2008 R2 Parallel Data Warehouse (formerly "Project Madison") is available now, and will be shipping on appliance servers in December.

Jesse Fountain (Principal Program Manager, CAT) comes on stage and shows off two nice racks loaded up with PDW, and then explains a new CREATE TABLE statement that incorporates the CLUSTERED INDEX, HASHING KEY and PARTITION boundaries all in the WITH clause - looks like very simplified syntax compared to setting up partitioning today. Then he demonstrates an 800 billion row query that runs in 19 seconds.

Paulo Resende, Chief Architect, Bank of America comes on stage and talks about he implemented SQL Server solutions to reduce costs, improve performance, and be more agile.

Dave Mariani, VP Development at Yahoo, talks about ad distribution for 600 million users (3.5 billion events, amounting to 1.2 TB of data per day). He discusses using a 12 TB cube in Analysis Services for data mining to deliver relevant ads and optimize the revenue of their advertising customers.

Ted announces updates to support & service offerings such as Premier Mission Critical, and the new Microsoft Critical Advantage Program - end-to-end support and service starting with Parallel Data Warehouse appliances.

Then Ted talks about the beta for Codename "Atlanta" - an agent that runs on your server and uploads various performance and health metrics (not actual data) about your SQL Server 2008 and SQL Server 2008 R2 to a secure service in the cloud. You can log in to the portal to view various metrics about your servers, but it also assesses best practices and configurations to provides guidance on preventing incidents, and better enables support to analyze your systems in case you do have an incident.

Bob Ward comes on stage and demonstrates troubleshooting the dreaded "Cannot generate SSPI Context" error message. He says that Atlanta is all about preventing problems, being proactive, and minimizing downtime. He shows that the Atlanta user interface has pointed out an issue in Kerberos configuration and actually guides him to a KB article that solves the problem.

You can sign up for Microsoft Atlanta at http://www.microsoftatlanta.com/ (requires a Windows Live ID, and Silverlight once signed in).

If you're at PASS and you want to learn more about Atlanta, you can attend the following spots:

Breakout Session (DBA226M) Tue 10:15 AM in room 612
SQL Clinic

o   Come talk to the product team and the guys behind the rules
o   Product team (Paul Mestemaker, Tim Ng, Lorenzo Rizzi)
o   Knowledge v-team (Bob Ward, Suresh Kandoth, Adam Saxton)

2 Product Pavilion Showcase Theater Demos

o Tue 6:00pm – 6:30pm
o Wed 1:30pm – 2:00pm

2 Focus Groups (please attend to influence our Wave 3 planning!)

o   FG#1 – Room 305 – Wed 3:00pm – 4:30pm
o   FG#2 – Room 305 – Thu 4:30pm – 6:00pm
o   If you know you’d like to come, RSVP to TimNg@microsoft.com. If you have some free time and want to stop by, we’ll allow you in if there is room. We are targeting about ~8-10 people per focus group.

Next, Ted talks about the cloud, and Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS). He says the key benefits to SQL Azure in terms of PaaS are that it is self-managed, has elastic scale, and is both agile and familiar. He talks about the new CTPs that were announced at PDC : Web Admin, Reporting, and Data Sync. He talks about Windows Azure Marketplace DataMarket - where you can get both public domain and commercial data sets to pull into your own applications. Adam Wilson (SQL Azure PM) demonstrates this feature for a fictional bike store, which pulls sales data and correlates it to weather data pulled from the Marketplace.

Finally, Ted talks about what we're all waiting for: SQL Server Codename "Denali" - the next version of SQL Server. The press release is available here:

http://www.microsoft.com/Presspass/press/2010/nov10/11-09PASS10PR.mspx

Quentin Clark et. al. will be providing much more detailed information about Denali in tomorrow's keynote, but we are getting some high-level information from Ted today. He talks about Mission Critical, Productivity, and Pervasive Insight.

Amir Netz comes on stage to discuss enhancements in PowerPivot for Denali, and shows off Excel browsing 100 million rows of data. It's very fast and impressive, though I'm curious what type of machine that is really running on. I think a lot of business users are stuck on decrepit hardware that may not exhibit such smooth performance.

Then Amir shows how Visual Studio, Management Studio and BIDS have been combined into a single, unified interface, and how when looking at data on a server, the interface can sort and filter 2 billion rows "in a flash." Again, quite impressive, and garnered a separate round of applause. Then he shows updating reports in real time, representing scanning a trillion rows per minute. Then he talks about a new columnar storage mechanism in Denali. Yes, you heard that right, we're getting vertipaq technologies in the next version of SQL Server.

Next, they talk about Codename "Crescent" - data visualization feature that allows you to represent data in a report in many different ways, including animated charts that can show, for example, the evolution of movie sales over time. Another round of applause here for the animated chart, and what he called "instant replay" - which allows him to slow-motion the chart animation. Then he shows live data being updated inside a PowerPoint presentation - I don't think I'll be able to pull this off using Keynote. I can't really describe this for you, I'm just going to have to figure it out and build out a demo.

The big tech sites already had prepared articles waiting for the press release announcement, where you can read about some of the key features:

ZDNet
The Register

You can download the CTP here, or wait for DVDs available after the keynote tomorrow:

http://www.microsoft.com/sqlserver/en/us/product-info/future-editions.aspx

PASS Workshop “Creating a BI solution from A to Z” - Interview

manowar — Thu, 16 Sep 2010 22:16:27 GMT

If you’re still unsure if the workshop I’ll deliver at PASS Summit the next November - “Creating a BI solution from A to Z” - is a good choice for you or not, you can get some more details reading the brief interview here:

PASS Summit Post-Con Preview - Davide Mauri

Where you’ll find answers to the following questions:

Is there an audience that would benefit especially from this session?
After having attended your seminar, what are two or three things that an attendee will be able to take back to the office and put to use right away?
What background should attendees ideally have to be fully prepared for your seminar?
What experience are you, as a speaker, bringing to this session?

That may help you to take your final decision. More than 20 people already decided to come: you’ll be in good company! :)

SQL Saturday 27 (Portland, Oregon)

BuckWoody — Mon, 24 May 2010 13:49:00 GMT

I’m sitting in the Seattle airport, waiting for my flight to Silicon Valley California for the SQL Server 2008 R2 Launch Event. By some quirk of nature, they are asking me to Emcee the event – but that’s another post entirely.

I’m reflecting on the SQL Saturday 27 event that was just held in Portland, Oregon this last Saturday. These are not Microsoft-sponsored events – it’s truly the community at work. Think of a big user-group meeting – I mean REALLY big – held in a central location, like at a college (as ours was) or some larger, inexpensive venue like that. Everyone there is volunteering – it’s my own money and time to drive several hours to a hotel for the night, feed myself and present. It’s their own time and money for the folks that organize the event – unless a vendor or two steps in to help. It’s their own time and money for the attendees to drive a long way, spend the night and their Saturday to listen to the speakers. Why do all this?

Because everybody benefits. Every speaker learns something new, meets new people, and reaches a new audience. Every volunteer does the same. And the attendees? Well, it’s pretty obvious what they get. A 7Am to 10PM extravaganza of knowledge from every corner of the product. In fact, this year the Portland group hooked up with the CodeCamp folks and held a combined event. We had over 850 people, and I had everyone from data professionals to developers in my sessions.

So I’ll take this opportunity to do two things: to say “thank you” to all of the folks who attended, from those who spoke to those who worked and those who came to listen, and to challenge you to attend the next SQL Saturday anywhere near you. You can find the list here: http://www.sqlsaturday.com/. Don’t see anything in your area? Start one! The PASS folks have a package that will show you how. Sure, it’s a big job, but the key is to get as many people helping you as possible. Even if you have only a few dozen folks show up the first time, no worries. The first events I presented at had about 20 in the room. But not this week.

See you at the Launch Event if you’re near the San Francisco area tomorrow, and see you at the Redmond SQL Saturday and TechEd if not.

Back from PASS Europe 2010

manowar — Sun, 25 Apr 2010 11:24:04 GMT

PASS Europe 2010 is finished and I’m now finally back at home and will stay here for a while.

I would like to thanks all the people who has come to my sessions for all their feedback, especially for the “Adaptive BI” session!

Slides and demos should be available for download from the PASS European Conference website in a couple of days.

Meanwhile if you want to rate my session online, you can do it here:

Adaptive BI
http://speakerrate.com/talks/3136-adaptive-bi-best-practices

Blazing Fast Queries
http://speakerrate.com/talks/3135-blazing-fast-queries-when-indexes-are-not-enough

How to get Microsoft to speak at your event

BuckWoody — Mon, 22 Feb 2010 14:35:36 GMT

If you’re holding a special event, such as a SQL Saturday or a user group, one of the challenges is to get speakers for the event. Now, the best speakers come from the community – people who use the product day-in, day-out. They have a wealth of expertise, and many of them are really great presenters.

But from time to time you might want to get a Microsoft person to speak at your group or event. Microsoft is a big company, and you can get everything from Marketing (yes, there’s a place for that) to deep technical topics in that skillset. But how can your group get Microsoft to speak for you?

It might be easier than you think.

Microsoft has three or four main areas that you might be able to pull from, starting with the folks that are closest to you: Sales.

Don’t make that face, there is sales and then there is sales. Within the sales team are four kinds of folks – your account manager, who owns the relationship between Microsoft and your company, the Sales professional, who owns the revenue and licensing for your account, the Account Technology Specialist (ATS) who knows about multiple Microsoft products, and the Technology Specialist (TSP or TS) that has a deep knowledge in a single technology, like SQL Server. These last two folks are the people who could deliver technical talks, especially around the newest products. And many of them are willing to do it. They are tied to a geographical area, so if your group has people in it that work at a company that is headquartered in that geography, the TSP and ATS might be willing to come out and chat with your group. And you might even want a Salesperson to come. Ever have a licensing briefing? Ever have questions? Ask them!

How to find them: Call your local Microsoft Office (the building, not the software). Find yours here: http://www.microsoft.com/worldwide/

The product group. If you live in the Redmond area, you should be coming to the PASS chapter monthly user group meetings here. It’s held at the Microsoft campus, it’s free, and it has a Microsoft product group member speaking just about every month. When I ran a user group for five years, it was impossible for me to get this kind of attention from the product group. It’s amazing.

Wait – you don’t live next to Microsoft? Not to worry. If you have a projector and a good set of speakers, the product teams can set up a LiveMeeting and present. I’ve done this several times myself.

How to find them: Twitter, Facebook, LinkedIn. Lurk on forums and see what the signature blocks say. Let’s see….oh, how about blogs like this one. You can find lots of blogs from Microsoft folks here: http://www.microsoft.com/sqlserver/2008/en/us/community.aspx

There are other groups at Microsoft that are willing to speak, and all you have to do is ask around. From “evangelists” to marketing, from training to even the documentation group, folks are out there that have really deep knowledge in just about every area of the product.

How to find them: TechEd, PASS and other conventions. Give them your card, get their e-mail addresses.

Here’s something to keep in mind: Microsoft works us pretty hard. In fact, sleep is really a luxury most of us aren’t granted; I think you get to do that when you’re a president or something. Maybe. So the folks you contact will be busy – really busy – just like you. I have traveled out of my district to present from time to time, but I have to take vacation time to do it, and Microsoft doesn’t pay my flight, hotel, food, anything like that. Any time we give you is “out of our own pocket”, so travel and late nights should be taken into account. So don’t abuse them, don’t always expect the invitation to be accepted. Most of us don’t get paid by Microsoft to do the extra presentations any more than you do, so keep that in mind and be polite and understanding. But do ask – most of us are really enthusiastic about this technology, and we love to help others learn, and to learn from the folks we meet.

Quest DMV Breakfast Code – Disk IO

drsql — Wed, 04 Nov 2009 17:52:43 GMT

For any of you who attended the Quest breakfast at PASS (and anyone else really) I wanted to go ahead and post the code and give a little wrap up/explanation. (I will post the stuff on perf counters later. They are cool, but this is really what I wanted to make sure people saw.)

I had a plan for my presentation, but time was less than I expected, and the good stuff kinda fell off. So here is the stuff around the Disk IO DMV section that I really messed up and didn’t show enough. I read a few of the comments that my presentation wasn’t that useful to you, and that is clearly on me. If the index ones are useful, these are too, and can be used to help spread load around your disks.

So, here is the slide I presented:

By knowing the amount of data written to a database file, you can see how busy the file is, the filegroup is, the database, the disk drive, etc, by aggregating the data in the following query in various way. The query is pretty simple and will return all files in all databases (there are parms on the function to change that, but check BOL for more info):

--since reboot
SELECT db_name(mf.database_id) as databaseName,
            mf.physical_name, divfs.num_of_reads,
            divfs.num_of_bytes_read, divfs.io_stall_read_ms,
            divfs.num_of_writes, divfs.num_of_bytes_written,
            divfs.io_stall_write_ms,
            divfs.io_stall,size_on_disk_bytes
FROM sys.dm_io_virtual_file_stats(null,null) as divfs
            JOIN sys.master_files as mf
                ON mf.database_id = divfs.database_id
                     and mf.file_id = divfs.file_id

The “stall” columns give you how long any processes have waited on your disk drives to serve up data for this file. This information plus the information in _bytes_written and _bytes_read columns can give you how much data was moved, and how long your CPU had to wait for the data before it could get working on it. Very powerful stuff, and can really give you a look at opportunities for performance tuning.

That query gives you data from the last reboot of the server, but usually you want to see up to date values. So you have to get a baseline of values, then compare them at a later point. You can build tools that do this, and you can use the performance data warehouse in 2008, and I will demonstrate the former. Using a temp table and a WAITFOR DELAY command, I build a quick tool to see waits and stats over 20 seconds. Note that I use a #temp table to allow you to run the last query over and over to see the comparison to the baseline until you want to drop the #temp table.

--over the following delay
declare @delay varchar(10) = '00:00:20'

SELECT db_name(mf.database_id) as databaseName,
       mf.physical_name, divfs.num_of_reads,
       divfs.num_of_bytes_read, divfs.io_stall_read_ms,
       divfs.num_of_writes, divfs.num_of_bytes_written,
       divfs.io_stall_write_ms,
       divfs.io_stall,size_on_disk_bytes,
       getdate() as baselineDate
INTO #baseline
FROM sys.dm_io_virtual_file_stats(null,null) as divfs
            JOIN sys.master_files as mf
                ON mf.database_id = divfs.database_id
                     and mf.file_id = divfs.file_id

WAITFOR DELAY @delay

;WITH currentLine as (
    SELECT db_name(mf.database_id) as databaseName,
            mf.physical_name,num_of_reads, num_of_bytes_read,
            io_stall_read_ms, num_of_writes,
            num_of_bytes_written, io_stall_write_ms,
            io_stall,size_on_disk_bytes,
                getdate() as currentlineDate
     FROM sys.dm_io_virtual_file_stats(null,null) as divfs
                JOIN sys.master_files as mf
                        ON mf.database_id = divfs.database_id
                            and mf.file_id = divfs.file_id)

SELECT DATEDIFF(s,#baseline.baselineDate,currentlineDate),
        currentLine.databaseName,
      left(currentLine.physical_name,1) as drive,
      currentLine.physical_name
     ,currentLine.io_stall - #baseline.io_stall as io_stall_ms
     ,currentLine.io_stall_read_ms -
               #baseline.io_stall_read_ms as io_stall_read_ms
     ,currentLine.io_stall_write_ms -
              #baseline.io_stall_write_ms as io_stall_write_ms
     ,currentLine.num_of_reads - #baseline.num_of_reads as
                                               num_of_reads

    ,currentLine.num_of_writes - #baseline.num_of_writes
                                            as num_of_writes
   ,currentLine.num_of_bytes_written -
        #baseline.num_of_bytes_written as num_of_bytes_written
FROM currentLine
      join #baseline
          on #baseLine.databaseName = currentLine.databaseName
             and #baseLine.physical_name =
                              currentLine.physical_name
go
--drop table #baseline

The most interesting part of this query is often the stall columns. When I see really bad IO, the number of seconds waited can end up being way more than the actual number of seconds. Disk IO is more or less serial (particularly on my mini HP with Win7 Starter Edition), so if you have many queries running concurrently, every one of them needs data from the same IO source, and if one query used the disk for 1 second, the other processes might wait for 1 second each. When wait stats are really low, that means the disk were ready, willing, and able to serve all processes.

On your test server, you can use a query like:

--create database DMV
--go
use DMV
go
create table testDMV
(
testDMVid int identity,
bigun char(8000)
)
go
set nocount on
insert into testDMV(bigun)
select REPLICATE('a',8000)
go 5000

--drop table testDmv

select *
from testDMV

To create some data. Note that data will be written to the disk in various counts. I set all of the rows to take a full data page, and each of the rows is an individual transaction. But it won’t be a 1-1 match with the number of rows written or read. SQL Server flushes data to the disk in the manner it wants to, and some manner of caching will change the number over time. Run the select * from testDMV query over and over and you will not see any change in the disk IO, because it is in cache now.

I hope this is better. Sometimes as a presenter I am a good writer, and frankly the lack of bacon really threw me off this morning. Feel free to email me at louis@drsql.org or tweet me at drsql and I will be happy to share more. All of this will be in our RedGate book that will be coming out before the end of the millennium, at the very minimum.