THE SQL Server Blog Spot on the Web

Welcome to SQLblog.com - The SQL Server blog spot on the web Sign in | |
in Search

Allen White

Emergency Procedures

In a followup to Aaron Bertrand's recent blog post on Disaster Recovery, I'd like to mention something I always mention when I teach SQL Server classes.

One of the best sources of training I received for being a DBA was, in fact, the training I received to become a pilot. The two things I took from pilot training are the use of checklists, which I blogged about here, and practicing Emergency Procedures.

When you learn to fly you spend the majority of your time in the air learning how to handle the things that can go wrong. Anyone can fly an airplane when things go smoothly, but the real pilots are the ones who keep their cool and handle things when things go wrong. They are able to do this because they practice.

Likewise, a good DBA not only has a solid backup strategy, but has actually TESTED those backups to ensure they're usable should the need arise. High Availability failover strategies are not only configured, but tested, both at the database and server level and at the application level, to ensure that the applications continue to support the business when a failover occurs.

It would be useful to make a list of all the ways in which things can go wrong, and develop a checklist on procedures to solve those problems. Use that checklist to practice recovering from those problems so that when they do occur you can calmly and professionally handle the problem.

The important thing to remember is to practice your solutions.

Allen

Published Monday, June 22, 2009 4:34 PM by AllenMWhite

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

 

Joe said:

I would start by making a list of the people with the knowledge to rebuild the environment,

this would include the database, the web or app server, the OS, and the networking components

if the main site were to be lost, along with the people inside,

lets assume a remote site has all the hardware, and you have been sending backups there.

are there people who could get it up? do they have admin accounts and passwords?

June 22, 2009 3:47 PM
 

Mike Walsh said:

Great point and a pet peeve of mine. Have never been a pilot (4 hours of lessons is it... when the kids are older I'll get my ticket) but I am a big fan of checklists and repeatable processes.

Actually missed your original blog post when I blogged about lessons DBAs could learn from Pilots, Doctors and Chefs (http://www.straightpathsql.com/blog/2009/3/16/checklists-recipes-and-algorithms.html) but this and the previous entry from you are great and full of good advice. Thanks!

June 22, 2009 9:23 PM
 

TiborKaraszi said:

I agree. At the more entry level, I use to tell my students (when discussing backup and restore) that they should plan their restore strategy so they can be awaken in the middle of the night and 2 minutes later sit in fron of the screen and feel comfortable doing the restore.

June 23, 2009 3:56 AM

Leave a Comment

(required) 
(required) 
Submit

About AllenMWhite

Allen White is a consultant and mentor for Upsearch Technology Services in Northeast Ohio. He has worked as a Database Administrator, Architect and Developer for over 30 years, supporting both the Sybase and Microsoft SQL Server platforms over that period.

This Blog

Syndication

Powered by Community Server (Commercial Edition), by Telligent Systems
  Privacy Statement