SQL Server with Mr. Denny

Oct 29 2012   10:15PM GMT

Now that it’s too late, what DR prep can we do today?

Denny Cherry Denny Cherry Profile: Denny Cherry

While preparing for a major disaster is what should be done ahead of time, that’s a little late for some people on the east coast of the US today.  So if you find yourself (or your company or your servers) on the east coast, what are some things you can do today to help keep things running if things go very, very bad.

On the SQL Server side of things…

  • Verify your most recent full backup by restoring it to another server (both user and system databases)
  • Make sure that your backups are copied to another servers or USB Drive
  • Get a copy of the backups out of the facility (a USB drive is a great way to do this, the network will work if needed)
  • Generate some scripts to ensure that you can restore the databases to another machine when ready
  • A way to get the transaction log backups off site, maybe FTP’ed to a personal website (after encrypting the files)

On the Hyper-V / VMware side of things…

  • Do you have backups of critical VMs?  If not take them now.
  • Get a copy of the backups out of the facility (a USB drive is a great way to do this, the network will work if needed)
  • Don’t forget to do a system state backup of Active Directory

If there is a disaster, the things that you’ll need at the new data center will include…

  • Servers (get your order in as soon as you know that your data center doesn’t exist any more, you won’t be the only one ordering servers)
  • Switches, Routers, etc.
  • Lots of USB Hard Drives (The SAN won’t be available, and it’ll take weeks or months to get a new one)
  • Zip ties/cable ties
  • Network cables
  • All the backups that you took from above
  • Installation media for all your OSes and databases servers
  • As lots of things

As you are getting the installations all setup, some things to keep in mind…

  • You’ll want to enable Instant File Initialization
  • You’ll want to restore system databases, then user databases
  • Restore AD from the system state backup, otherwise all the Windows authentication SIDs will be useless within SQL Server

If your data center looses power and it’ll be out for a while, and you don’t have a big expensive generator to run the data center know where you can get a generator.  A few small ones from Lowes, Home Depot, etc. will work as well for keeping the most critical systems online.  Get some large box fans to keep the air in the data center moving.  If you are using small generators, use a different one for the fans as they will have very dirty power, which could damage the power supplies of the servers.  Plastic sheeting can be used to direct air in or out of the data center so that you get cool air coming in from outside and so you can direct the hot air back out (ask me some time about turning a 6 story building into a giant chimney for a data center).

Once we are in a disaster situation we are no longer worried about peak performance, we are worried about keeping the business running for our customers.  Don’t worry about redundant power supplies, or passive nodes, reporting servers, etc.  Just worry about getting the core systems online so that websites work, and so that customers can login to your applications.  If your application supports it be sure to put up a message telling your customers that performance won’t be very good because of the weather.  Just because you know that the weather is a mess doesn’t mean that your customers are aware that there’s a problem.  Most people are pretty understanding and if you tell them that the weather is causing performance problems they will be fine with that.  This will save you a lot of phone calls and a lot of headache.

No matter what, when getting things setup, rebuilt, brought back online, etc. do it safely.  Don’t get hurt and risk hurting yourself or others to get things back up and running.  It just isn’t worth it.

Denny

 Comment on this Post

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: