Search results matching tags 'performance' and 'DBA'

Read the New TPC Database Benchmarking Series

KKline — Mon, 22 Apr 2013 18:17:00 GMT

Let's talk about database application benchmarking.

This is a skill set which, in my opinion, is one of the major differentiators between a journeyman-level DBA and a true master of the trade. In this article published in my monthly column at Database Trends & Applications magazine, I'll give you a brief introduction to TPC benchmarks and, in future articles, I'll be telling you how to extract specific pieces of valuable information from the published benchmark results.

But let's get started with an overview … read more.

Many thanks,

-Kevin

- Follow me on Twitter!
- Google Author

Learn More About SQL Server IO and Query Tuning in These Webcasts

KKline — Fri, 14 Dec 2012 18:50:00 GMT

I'm doing two new webcasts next week on Wednesday, December 19th, one in the morning and the other after lunch.

SSDs are a Game Changer for SQL Server Storage

No, session is not exclusively about SSDs. But this is my first session on IO and storage tuning that emphasizes SSDs over hard disks. As Bob Dylan said "Times, they are a'changin'". This session on Wednesday, December 19th at 11:30 AM EST, sponsored by Astute Networks, takes you through all of the basics of storage and IO tuning, regardless of the underlying storage technology. I'll show you how SQL Server handles storage structures, how to identify IO activity on Windows and SQL Server, and best practices for minimizing IO bottlenecks. Register now for: Storage IO Best Practices for SQL Server and a New Approach to Solving Application Performance Issues.

Write Better SQL Queries

The next webcast on Wednesday, December 19th at 2 PM EST, is with me, Aaron Bertrand (Twitter | Blog) and SQLCruise Impresario & Microsoft MVP Tim Ford (Twitter | Blog) as we take you through the query tuning process, discussing important DMVs to use during query tuning, as well as demonstrating several essential query tuning techniques that every SQL developer should know. Not only are we presenting an hour of top quality technical content, we’ll also be giving away some cool prizes, including the grand prize of a paid registration for the upcoming SQLCruise Miami, a $1,395 value! Register now for: SQL Server Query Tuning Best Practices, Hosted by Kevin Kline and Aaron Bertrand with special guest Tim Ford

I hope to see you at both of these sessions next week! Best regards,

-Kev

-Follow me on Twitter!

Quick Tip - Speed a Slow Restore from the Transaction Log

KKline — Wed, 14 Nov 2012 15:59:00 GMT

Here's a quick tip for you:

During some restore operations on Microsoft SQL Server, the transaction log redo step might be taking an unusually long time. Depending somewhat on the version and edition of SQL Server you've installed, you may be able to increase performance by tinkering with the readahead performance for the redo operations. To do this, you should use the MAXTRANSFERSIZE parameter of the RESTORE statement. For example, if you set MAXTRANSFERSIZE=1048576, it'll use 1MB buffers.

If you change the MAXTRANSFERSIZE, keep an eye on the PerfMon objects for Buffer Manager and Readahead IO. You may also wish to keep an eye on LOGBUFFER wait stats.

I'd love to hear your feedback. Have you tried this technique? Did it work as advertised? Did it require some changes to work on a specific version or edition?

Many thanks,

-Kev

-Follow me on Twitter!

The Zombie PerfMon Counter That Never Dies! Quick Tip

KKline — Mon, 08 Oct 2012 11:55:00 GMT

The PerfMon Counters That Just Won't Die

One of the things that's simultaneously great and horrible about the Internet is that once something gets posted out in the ether, it basically never goes away. (Some day, politicians will realize this. We can easily fact check their consistency). Because of longevity of content posted to the Internet, a lot of performance tuning topics become "zombies". We shoot 'em in dead, but they keep coming back!

In other words, those old recommendations were a suggested best practices for long ago, for a specific version of SQL Server, but are now inappropriately for the newer version. It's not uncommon for me, when speaking at a conference, to encounter someone who's still clinging to settings and techniques which haven't been good practice since the days of SQL Server 2000. Here's an example of Microsoft SQL Server 2000 Best Practices that are very version-specific.

So here's an example. The %Disk Time counter and the Disk Queue Length were heavily recommended as a key performance indicator for IO performance. SQL Server throws a lot of IO at the disks using scatter/gather to maximize the utilization of the disk-based IO subsystem. This approach leads to short bursts of long queue depths during checkpoints and readaheads for an instance of SQL Server. Sometimes the server workload is such that your disk can't keep up with the IO shoved at it and when that happens, you'll see long queue lengths too. The short burst scenario isn't a problem. The lengthening queue length scenario usually is a problem. So is that a good practice?

In a word, not-so-much.

Those counters can still be of some use on an instance of SQL Server which only has one hard disk drive. But that's exceedingly rare these days. Why?

The PerfMon counter %Disk time is a bogus performance metric for several reasons. It does not take into account asynchronous I/O requests. It can't tell what the real performance profile is for an underlying RAID set may be, since they contain multiple disk drives. The PerfMon counter Disk Queue Length is also mostly useless, except on SQL Server's with a single physical disk, because the hard disk controller cache obfuscates how many IO operations are actually pending on the queue or not. In fact, some hard disks even have tiny write caches as well, which further muddies the water was to whether the IO is truly queued, in a cache somewhere between the OS and the disk, or has finally made it all the way to the CMOS on the disk.

Better IO PerfMon Counters

Instead of using those PerfMon counters, use the Ave Disk Reads /sec, Avg Disk Write /sec, and Avg Disk Transfers/sec to track the performance of disk subsystems. These counters track the average number of read IOs, write IOs, and combined read and write IOs to occured in the last second. Occassionally, I like to track the same metrics by volume of data rather than the rate of IO operations. So, to get that data, you may wish to give these volume-specific PerfMon counters a try: Avg Disk Transfer Bytes/sec, Ave Disk Read Bytes /sec, and Avg Disk Write Bytes/sec

For SQL Server IO Performance, Use Dynamic Management Views (DMV)

And unless you've been living in a cave, you should make sure to use SQL Server's Dynamic Management Views (DMVs) to check on IO performance for recent versions of SQL Server. Some of my favorite DMV's for IO include:

Sys.dm_os_wait_stats
Sys.dm_os_waiting_tasks
Sys.dm_os_performance_counters
Sys.dm_io_virtual_file_stats
Sys.dm_io_pending_io_requests
Sys.dm_db_index_operational_stats
Sys.dm_db_index_usage_stats

Many of these DMVs are fully document in this Books Online article here at Microsoft SQL Server 2012 Index Related Dynamic Management Views and Functions.

So how are you tracking IO performance metrics? Which ones are you using?

I look forward to hearing back from you!

Enjoy,

-Kev

-Follow me on Twitter!

Two New Slide Decks. Plus, the Week in Colorado.

KKline — Mon, 20 Aug 2012 15:03:00 GMT

I had the honor of traveling the great state of Colorado last week, speaking at the PASS chapters in Boulder, Colorado Springs, and Denver. At all three events, we had a stellar attendance and, at least in Denver, broke all the records in recent memory both in terms of overall attendance and in first-timers. Denver, in fact, was standing room only and had nearly 30 first time attendees. Great news! I also want to give a special shout-out of thanks and appreciation to Chris Shaw (Twitter | Blog) whose hard work and tenacity ensured that all of Colorado got to see me speak. From left to right, Gabriel Villa (Twitter), me, Chris Shaw, and Rebecca Mitchell (Twitter). If it weren't for Chris, I wouldn't have been there. Thanks for putting in the time, amigo!

New Slide Decks!

During the 3-day jaunt, I presented two of my more popular sessions. These are updated slide decks, in case you want to download them here:

UG - End-to-End Troubleshooting

and

UG - Top 10 SQL Server Administration Mistakes

Be sure to check in the Slides area of the website, if you want to see the links for SpeakerRate, and in the case of several of my presentations, white papers, video recordings, etc. It's the People that Matter

A Blast from the SQLPASS Past!

I've always tried to maintain the relationships I built with the founding members of the PASS board of directors. After their time on the PASS board, almost all of them have moved on from SQL Server to other adventures. Pam Smith, the first president of the organization, is now a professor. Guy Brown, the second president, is now the director of IT at his same employer, rather than just SQL Server as when he was on the PASS board. A few, such as Kurt Windisch, a former VP of PASS, and my good friend Joe Webb (Twitter | Blog), are still active in the SQL Server space. One relationship that I've enjoyed over the years is with PASS' original Microsoft liaison and now a Denver-area software entrepreneur Steve Murchie (at right) running his own healthcare IT outfit. Steve has been a source of inspiration to me and also of great advice for all things startup-related. It was great to connect with Steve and catch up on his latest doings.

I also got to enjoy an evening out with the local attendees after the Denver user group meeting. It was great to hang out with folks there. I got to meet Kevin Cox (twitter), a member of Microsoft's incredibly talented SQLCAT group, and for whom I was a technical editor on a SQL Server v6.5 book back in the Neanderthal era. That shows just how old both Kevin and I actually are. Other cool folks that I got to meet included Steve Wake, Mike Fal, Marc Beacom, Jason Kassay, Jason Horner and my ol' buddy, Kelly the Greeley Geek.

On top of that, long-time SQL Server MVP and all-around awesome guy Steve Jones (Twitter | Blog) visited. I kept him out way too late that night which, to be honest, isn't usually my style. But the good conversation propelled us on past midnight.

The Good Folks at SQL Server Professional and Windows IT Professional Magazines

I've written for SQL Server Professional (formerly the artist known as "SQLMag") in some form or another starting from my first cover article for them way back in the mid 1990's. My Tool Time column has been going strong there since, oh, around 2006 iirc. For most of the time I've known the folks at SQLMag, they were located in Loveland, CO but they were able to move to some incredibly nice digs just up the road in Fort Collins. In all the many years, I've written for them, I'd never been to their offices - until now. It was great to visit and break bread with Megan (to my right), Blair (across), and Jaylee (across and to my right)!

I've always supported SQLMag and encourage you to subscribe. On top of the goodness already in the digital magazine, there are some neat developments coming down the pipeline with SQLMag which I think we'll all enjoy. Be sure to subscribe today! (You can click the badge on the left or simply go to http://www.sqlmag.com).

What's Next?

My current travel schedule is pinned up on this blog post HERE. However, I also know of a couple on-line appearances and probably a trip between the long gap between now and my next in-person appearance at the Orlando SQL Saturday at the end of September, where I'll also be teaching a pre-conference seminar (register HERE for the seminar).

SSWUG

The first on-line event to note is my a presentation by SSWUG of my Leadership Skills for IT Professionals video series, starting on August 24th. Sign up using the hyperlink (note that a video plays immediately upon loading the webpage, in case you want to be ready to pause or stop it). You can also buy a DVD set of the 14 hours of leadership training content.

24HOP - The 24 Hours of PASS Event

I'll also be speaking on the topic of influence in the next 24 Hours of PASS coming up on September 20th and 21st. Registration for the twenty-four hours of around the clock presentations is completely free and well worth your time. Check the schedule for the event and register! Even if you can only watch one or two sessions (or even zero sessions), be sure to register so that you'll automatically be notified when the sessions become available as streaming media.

Enjoy!

-Kev

-Follow me on Twitter, LinkedIn, and Facebook

New White Paper: SQL Server Extended Events and Notifications

KKline — Wed, 25 Apr 2012 13:13:00 GMT

SQL Server comes with a wide array of tools for monitoring your environment. There are logs and traces that provide information when errors occur, but these are often used passively to react to events that have already occurred. There's PerfMon, and Profiler, and loads of Dynamic Management Views to check. But where to look?

As database administrators (DBA), we need to monitor our environments proactively and create solutions as issues arise. In this white paper, we will look at a couple technologies – event notifications and extended events – that can help you achieve these goals. With these two features, we’ll look at the error log and deadlocks, and demonstrate how you can get relevant information delivered as it occurs. We’ll also look at ways that run-time errors can be captured and used to help reduce the amount of time required to investigate issues.

This white paper, written by SQL Server MVP Jason Strate (blog | twitter), is a free download but requires a registration. Download the Extended Events white paper here.

And, as always, I enjoy your feedback. Thanks!

-Kev

Follow me on Twitter!

Accelerate OLTP with HP and Microsoft's New High Performance Reference Architecture

KKline — Tue, 06 Mar 2012 15:25:00 GMT

If you haven't started to read Shashank Pawar (blog), you're missing out. Shashank is part of Microsoft Australia and has been writing some very good content lately. Here's an example from the Reference Architecture for High Performance SQL Server:

HP and Microsoft engineering teams have worked together to create a reference architecture to Accelerate Online Transaction Processing (OLTP) database workloads with a fully-flash based HP/Microsoft architecture and achieve significant performance increases, simplified database manageability, and industry leading TCO.

The details come in a torrent after that leading paragraph with lots of pretty pictures and charts to help explain. This is great stuff, especially for competitive platforms such as Oracle Exadata.

Read more about the new HP High Performance Reference Architecture for SQL Server 2012 here.

And just out of curiousity, are any of you using high performance architectures such as Oracle Exadata, IBM Netezza, or Teradata? I'd love to hear your feedback, questions, and comments.

Enjoy,

-Kev

-Follow me on Twitter

-More on my KevinEKline.com

Everybody Needs a Test Harness

KKline — Mon, 31 Oct 2011 14:17:00 GMT

When you're developing new Transact-SQL code or modifying some existing code, do you just launch directly into programming?

I know that I did just that, for years. It wasn't until I was trying to performance tune some existing code that I realized I hadn't actually taken caching of data and execution plans into account. So all those modified stored procedures that I was so proud of might not actually be faster than the first generation of procedures because I hadn't checked to ensure that I was testing cached programs against uncached programs (and, by extension, the data used by those programs). That's easy enough to fix with a test harness. Test harness were originally an actual, physical harness used by engineers to clamp down parts of an electrical or mechanical device they were prototyping. Ours is no different. It locks down all of the assumptions about our code (like my early, false assumption that I didn't need to clear the caches) and adds a metric or two for good measure - literally - so we can better measure what's happening in that code.

Here's what my test harness looks like:

/* Transact-SQL test harness by Kevin Kline, http://KevinEKline.com, Twitter at kekline */

/* Flush dirty pages from the buffer to the database files. */

CHECKPOINT;

/* Flush the data cache and procedure cache, respectively. For DEV environments only! */

DBCC DROPCLEANBUFFERS;

DBCC FREEPROCCACHE;

/* Enable statistics tracking for IO and timings. Remember, SET commands remain enabled during a session until disabled. */

SET STATISTICS IO ON;

SET STATISTICS TIME ON;

-- Whatever SQL code you'd like to process goes below.

SELECT SalesOrderID

FROM Sales.SalesOrderHeader H

WHERE CustomerID = 344

GO

SET STATISTICS IO OFF;

SET STATISTICS TIME OFF;

/* Textual Execution Plans, if desired.

SET SHOWPLAN_TEXT ON;

SET SHOWPLAN_TEXT OFF;

*/

I also like to include the execution plans a lot of the time. You might wonder why I don't save the execution plans for the GUI in SSMS? Well, I'm a big advocate of scripting in general because I like to automate activities. By pulling the execution plans using scripts, I can use SQLCMD to schedule a large number of query executions during the evening and have the results ready for analysis when I come back into the office in the morning. Workin' smarter, not harder, Baby!

So how does this test harness work for you? Do you use other elements in yours? If so, share your experiences here!

Thanks,

-Kevin

-Follow me on Twitter

Sometimes you just have to break it up

Andrew Kelly — Tue, 14 Apr 2009 23:19:02 GMT

I will be the 1st to admit that I am no Itzik Ben-Gan when it comes to writing complex SQL statements and I wonder at times how people come up with some of the Selects that I see. Sometimes they are so complex or convoluted that I simply have a hard time figuring out the intent or the logic behind them. But as a performance consultant I pretty much find myself looking at these on a regular basis when they end up scanning very large tables or indexes. I find that many developers have a hard time splitting up the query into what I think are more manageable pieces, especially if there is a temp table involved. Lets face it temp tables are evil aren’t they :). Well that’s a whole another discussion which we won’t get into now but sometimes they can be be your friend. For instance today I took a rather unwieldy query that was being used in a stored procedure in which the optimizer simply couldn’t come up with an efficient query the way it was written. And one of the issues was that they needed a total count for paging purposes so they were running the same query twice, once for the count and once to get the TOP xx rows. In a nutshell I changed the query to use a UNION which allowed the optimizer to focus better on the SARG’s for each part of the query and placed the resultant rows into a temp table. This gave me the ability to get a count and retrieve the rows using TOP xx without running it twice. As you can see from the picture the costs were dramatic. The reads went from over 3 Million to just under 5 thousand and the duration went from over 23 seconds to 150 milliseconds. So bottom line is don’t be afraid to make the query more maintainable and break it up at times. You might be surprised what you can accomplish.