I was tagged by David Stein, a.k.a. Made2Mentor [ blog | twitter ], to talk about "MacGyver Moments" in my career - times when I've turned nothing into something. One clearly comes to mind, though it was more like keeping a dying server on life support than building something fantastic with .
Several years ago we built out a few QA servers to roughly mimic our production environment for one of our ASPs. At the time, this was a control DB server and a peer DB server - the control server held the databases that ran the system (mostly metadata and config about our customers), and the peer servers held the actual data and metadata that our customers in turn used. Both servers were virtualized with 1 core, had very little disk space, and I believe 512 MB of RAM. It was a constant juggling act to ensure that there was enough space for typical QA activities that would grow a peer database or TempDB.
But one of the servers, the control server, behaved a little bit differently than any other server I've ever managed. We never figured out what was specifically wrong with the server, but for some reason, it would take 45 minutes to establish an RDP session to the box, and clicking the mouse and waiting for the visual response was at least a 15 minute affair. To avoid having to rebuild from scratch, I just dealt with it. SQL Server itself seemed to run okay, and everything I needed to do at the OS level (such as create folders) I
would do via xp_cmdshell in a remote SSMS session from my own desktop or another server in the data center. Every once in a while I would forget how useless this server was in terms of direct interaction, and curse my memory after waiting 15 minutes and still not being able to enter my credentials to log in. I was never able to install SP1 or any Windows Updates, so this machine was at RTM for both Windows 2003 and SQL Server 2005 for ages... it's amazing the server fulfilled its duties for as long as it did, and I always tried very carefully to not disrupt the box.
A few weeks ago, this server was finally retired (some kind of issue with the I/O for the VM that was unrecoverable according to IT). It was kind of sad to see it go, and it was quite a bit of work to migrate its functionality to another server, but when all has been said and done: good riddance. Now our QA environment is all SQL Server 2008 (albeit 32-bit) and we can actually RDP into all of the servers and get stuff done locally.
Tagging: Jonathan Kehayias, Tom LaRock, and Buck Woody.