Search results matching tags 'Productivity' and 'backup'

March Update to Rules-Driven Maintenance

merrillaldrich — Tue, 12 Mar 2013 02:56:00 GMT

This month I have a minor update to the Rules-Driven Maintenance code I originally posted back in August 2012. This update has just two enhancements, but they are nice ones, I think:

Much improved handling for instances that use database snapshots.
Ability to do intra-day differential backups purely by setting maintenance times and limits in the policy table.

The code posted here is cumulative, and replaces entirely the original code, but please refer back to that original blog post for description, instructions and details. This update can be installed right over an existing deployment, or be installed all by itself as a complete solution but – as always – please test and be cautious.

Enjoy!

Update to Rules-Driven Maintenance

merrillaldrich — Wed, 16 Jan 2013 18:34:00 GMT

Back in August I posted a first version of a rules-driven solution for backups, index and statistics maintenance and integrity checks. The system in general has been working well, and has saved my team a huge amount of time and effort. We are coming to the anniversary of its use in production soon.

Today I offer an update that contains a few enhancements, performance improvements and a bug fix.

To recap, this is a system or framework to manage many small databases across many instances on many servers in a policy-based/automated way. The details about how to deploy and how to use the system are in the original post, but at a high level:

First deploy the code across a collection of servers using the included PowerShell script DeployMaintenance.ps1. That will create an administrative database on every instance (or use one that you provide) and populate it with the required tables, procedures, etc. The deployment script will also make a handful of SQL Agent jobs that handle all the maintenance on all the databases.
Visit the instances. If there is maintenance already in place, disable or remove it so as not to perform this work twice. If necessary, change or override the default preferences for this solution in the admin database for each type of maintenance. This includes the time of maintenance windows, whether to use features like differential backups or Litespeed, whether there are databases that should be treated as exceptions, with different settings, etc.
Enable the new policy jobs in SQL Agent.
Monitor to make sure things are running smoothly.

The theory behind this project is to save work configuring servers (efficiency), to ensure that the maintenance is in fact deployed and working for ALL databases in the environment (quality), to ensure that it works in a truly consistent way everywhere (consistency), and to make it so that the servers don’t have to be touched by a DBA when changes happen, such as adding or dropping databases (resiliency).

Changes

The updates in this 1.1 version include

Better reporting of errors. The jobs now write log files to the default error log location for SQL Server.
Better recovery from errors. The jobs in some cases continue on error – for example, one failed backup will not stop the backup job and leave other databases without backups.
Performance enhancements for examining backup history and system counters (mainly % log used).
One bug fix for the log backup logic on a server that uses both log shipping and Litespeed.

Disclaimers

The code for the system is posted here, and you are welcome to download it and try it out. Please note that this is not a commercial solution, and that while you may test or deploy this, you do so at your sole risk. Edits to the code almost certainly will be required for your environment. Please read, understand and test the code thoroughly before you even imagine using this in production. I do not want to cause you or your employer any pain. License for using this solution is GPL 3.0: http://opensource.org/licenses/GPL-3.0. You may use and modify this code, but not sell it.

This has been run on SQL Server versions from 2005 to 2008 R2. It probably works on 2012, but I have not tested it thoroughly.

I run only case-insensitive collation; if you are on a case-sensitive system, then you have my sympathy, and you might have to edit a few things to make this work.

There is coverage for Quest Litespeed backups in the solution; other third-party backup products could be incorporated easily, but have not been. It should be possible to follow the pattern used to handle Litespeed and extend it to any other third-party stored procedures.

If you haven’t done index maintenance for a while, and you flip this solution on you might blow up your transaction log. Be sure that you have plenty of log space and plenty of log backup space, and watch the process the first few runs. This is especially true if the combination of Standard Edition of SQL Server and the preference for Online index maintenance applies in your environment, because the maintenance process will perform index maintenance using reorganize (defrag), which produces a lot of log.

Lastly, this is a solution for the type of environment that has many small databases, not the type with a few big, performance intensive databases. Databases that are large and require detailed attention may not be suitable for a one-size-fits-all solution like this.

Must-Have Resources - SQL Server Backup & Recovery

KKline — Thu, 08 Dec 2011 20:36:00 GMT

One of the things that drives me crazy as I'm getting older is that my brain is losing the capacity to differentiate version numbers. As I speak travel around speaking with customers and at conferences, I find my self saying things like "I can't recall if this problem was fixed in SQL Server 2000 or 2005. But you don't have to worry about that any more." Or things like "That feature was added in SQL Server 2008 R2, eh, or was that version 6.5. DOH!" followed by a loud slapping sound as I whap my palm into my forehead. The Internet doesn't forget either. Recommendations that were once helpful, if not outright essential, now are neutral or even downright bad. So now, whenever I put together new presentations, I always spend a lot of time in research, reassessing my knowledge on the topic. (That doesn't mean that I'll extemporaneously say it wrong, because I speak in an off-the-cuff and rapid style. But at least my notes are usually correct).

Case In Point

Take backup and recovery (B&R), for example. I've been writing about and performing B&R for years. I'm even part of the team that builds the most popular B&R tool in the SQL Server space. It's an extremely important part of what I do. And yet, even after spending a couple days re-researching topics (yet again), I still manage to get a few of the more specific details wrong because of changes over the years in the feature set. How so? During an Expert's Perspective webcast last week (and available on-demand) covering the top backup and recovery mistakes on Microsoft SQL Server, I mis-explained the details concerning how SQL Server performs a differential backup. A very clever SC on our team who was also attending the webcast pointed out to me after the session that I'd described differentials working at an 8k page-level when, in fact, they work at the extend-level (that's a block of eight 8k pages). #FACEPALM! When describing the differential backup I said that a bit was flipped on each page header and SQL Server would only back up those pages. Instead, whenever a page is changed a bit is recorded on the differential change map (1bit per extent), the backup process then queries these map pages and backs up those extents that have been marked as changed.

Watch the on-demand video of Performance Tuning SQL Server for Backups and Restores, with me, Iain Kick, and Brent Ozar (blog | twitter). Free, but registration required.
Watch the on-demand video of Ten Incredibly Painful SQL Server Backup and Recovery Mistakes, with me and David Gugick.

The Research Reveals

The cool revelation here is not that I can admit my mistakes. The take-away from this blog post are all the great articles I read writing my latest slide deck. These blogs and articles were so good that I needed to spread the word. Add these to your must read list:

Virtual Log Files must be tamed!

Performance impact: a large number of virtual log files Part I and Part II by one of my favorite not-so-prolific bloggers, Linchi Shea (blog). The thing I love about Linchi's content is its rich benchmarking information. Few SQL Server experts out there routinely test the behavior and performance of specific SQL Server features like Linchi does.
Slow recovery times and slow performance due to large numbers of Virtual Log Files, by Graham Kent (blog) of Microsoft Sweden, shows that VLFs impact not only the processing time of transactions but also B&R processes.
Lots of VLFs are Bad, by Tony Rogerson (blog | twitter), was the article that first got me interested in this behavior.

Backup and Recovery Myth Busting

Breaking the Backup Chain - Redux (Or 'Eating Crow'), by Michael K. Campbell (blog), shows that out-of-band database backups break differentials not transaction log backups.
Backup with COPY_ONLY, How to Avoid Breaking the Backup Chain, by the eminent Paul Randal (blog | twitter), uses clear writing and first-class demos to illustrate the principle of breaking backup chains in Microsoft SQL Server. Other great blogs of a related nature from Paul include:

Other Good Sources

And just in case you need a refreshing in the basics of SQL Server backup and recovery, check these out: