To solve this problem, I added a DDL trigger to the server which will cause the new database to be backed up as soon as the database is created.  The trigger itself is rather simple.  Most of the trigger is setting variables.  Then I make sure that the database isn’t a database snapshot, as database snapshots can’t be backed up.  If it isn’t a snapshot we continue with everything else.

Then I create a subfolder for the backups to be put into (the backups for each database go into their own folder, so as this is a new database the folder needs to be created).  Then I commit the transaction, as database backups can’t be taken within a transaction.  Then we do the actual database backup.  I then throw a message to the end user using the RAISERROR statement telling them that they can ignore the other error that they are going to get about the transaction being closed before the trigger was complete.  This is just an annoyance of my needing to commit the transaction before taking the backup.  Sure I could have setup a job which takes the backup and emails if there was a failure, but that just seems to complex for something so simple.  The code for my trigger is below.

CREATE TRIGGER BackupNewDatabase
ON ALL SERVER
FOR CREATE_DATABASE
AS
declare @database sysname, @event_data XML = EVENTDATA(), @folder nvarchar(4000), @file nvarchar(4000)

SET @database = @event_data.value(‘(/EVENT_INSTANCE/DatabaseName)[1]‘, ‘sysname’)

set @folder = ‘X:\Backups\’ + @database

set @file = @folder + ‘\’ + @database + ‘.bak’

if exists (select * from sys.databases where name = @database and source_database_id is null)
BEGIN
EXEC master.dbo.xp_create_subdir @folder

COMMIT

BACKUP DATABASE @database to disk=@file

raiserror( ‘You can ignore the error message which says that the transaction ended within the trigger.’, 16,1)
END
GO

Hopefully you find this solution helpful if you get into a situation like this,
Denny

I’ve moved lots of companies into data centers before, and each time I’ve done so it has included a site visit during to RFP process to check on things like cooling, generator, site configuration, etc.  During these site visits I stick my head through every door that the sales guy and the engineer who are doing the tour will open.  If they won’t open a door for me they better have a damn good reason to not show me what’s in there.

If I went to do a site visit at a facility that was located just at sea level just a few blocks from the ocean, they’d better have some pretty impressive plans for how they are going to handle flooding.  If the answer was “we’ve never had a problem with flooding” or something similar they’d be off my list as they haven’t done their due diligence to insure that the site will be online during the worst emergencies possible.

Now before you start telling me that I’ve got no idea what I’m talking about, and that data centers in the North East have different problems from data centers in the South West.  I actually do as I’ve moved companies into data centers on both coasts.  Most recently I moved the data center for Phreesia from a managed service facility in Texas to a colo in New Jersey.  As a part of this move we looked at a number of facilities in the New York / New Jersey area.  Many of the New York City data centers were eliminated due to cost, or being just to close to the water as we didn’t want to deal with situations like this exact one.

The data center we settled on is in New Jersey about a 30-40 minute drive from the Phreesia office (once you get out of New York City).  While the data center is near a river, the river is a little over a mile away.  The data center itself is on a slope with a  cinder block wall on the outer edge which will divert water away in the event of a river overflow (it also protects the facility from someone driving a car or truck into the facility).  The generators and fuel pumps are all mounted on raised feet (not very tall, but tall enough) so that they keep running in the event of a flood.  The cables from the generators to the building have all been buried under the ground so that tree branches which are torn loose during the hurricane can’t cut those cables.

Our diligence in selecting a data center paid off.  While the folks mentioned in that article were dragging buckets of fuel up 17 floors worth of stairs the team at Phreesia just sat back and rode out the storm with their families.  The sites didn’t go down and the team didn’t have to rush into a very hazardous situation. The team was able to focus on their families and neighbors without having to worry about the systems.  Those of us that aren’t in the New York area monitored the systems remotely and founds that everything was running perfectly normally just on generator power instead of city power.

This entire event just shows that when doing data center evaluations the worst possible case situation needs to be planned for and expected.  If they aren’t they are going to come back and bite you.  Especially in todays world of storms which are ever increasing in destructive power.

If you are in a facility which has risks such as fuel pumps which are below sea level (they are in the basement and the road is at sea level) then a second data center becomes very important very quickly.  This became very clear during this hurricane when some very large important websites went offline because they didn’t have that DR facility that they could fail over to, unlike sites like StackOverflow (and ServerFault and the Stack Exchange network).

If you are at risk now is an excellent time to sit down with management and go over all the “what ifs” of your current hosting solution and think about the cost of a DR site along with the cost of not having one.

Denny

Bull Shit!

While it is true that a lot of companies are moving SOME services to the cloud, most companies are not looking at moving every service to the cloud.  Even if they wanted to move everything to the cloud so that Amazon, Rackspace, Microsoft (Azure), etc. could handle the managing of the servers there are some things which will simply never sit in the public cloud.  Here’s a short list to start with…

• Medical Data (there’s this thing called HIPPA that requires you know who has accessed the data)
• Confidential data
• Any customer data from companies in Europe
• Most legacy applications which contain personally identifiable information

There are also plenty of services that IT provides that simply can’t be replaced by the cloud.  This includes things like:

• Network infrastructure at the office
• DHCP, DNS, Authentication, etc. services at the office
• Telecom Services
• File Servers
• Account provisioning in what ever applications are running in the cloud

Even when services are moved to the oh so magical cloud there are still plenty of non-code things that need to be done by someone who works for the company who’s application will be hosted by the cloud.

• Virtual Machine Architecture (remember when Amazon’s EC2 had that little problem with some of the groups going offline?)
• Application deployments (separation of duties is still a SOX requirement when applications are in the cloud)
• Disaster Recovery planning (realistically how would the business have reacted during that EC2 outage?)
• Disaster Recovery testing (if you plan for a disaster and don’t test the plan, you’ve got nothing useful.)
• Scaling applications (not everything automatically scales like they marketing material says)
• Moving applications from one cloud provider to another (if I can save $10 a month by moving from EC2 to Rackspace for example, why shouldn’t I)
• Database Index Tuning
• Server Patching

On top of all of that, there’s other risks with moving everything that runs a company into the cloud.  Now that you no longer own the servers you can’t control what the other servers are running on the physical hardware.  So if that tier 1 application that requires millisecond response time isn’t getting it, there’s basically nothing that you can do.  And what happens where there’s an internet outage at the office and your employees can’t access the software they need to do their jobs?  Suddenly the savings of moving that tier 1 application to the cloud wasn’t the greatest plan.

Now there are definitely benefits to moving SOME application and services to the cloud, but the thought that everything will move to the cloud and if you want to stay employed you better learn how to write application code, is total crap.

The reason for this is pretty basic.  How is the SQL Server supposed to know what settings to use, or how often to do those things?  Every database needs to have the indexes rebuild at a different interval.  There are a variety of recovery options which would change the backup policy for a database.  Not to mention that to properly build backup settings the SQL Server would need to know things like your RPO (Recovery Point Objective aka. how much data are you willing to loose in the event of a major failure).  Without a way to tell the SQL Server these sorts of things the SQL Server wouldn’t be able to properly maintain your environment.

So there’s the why.

Denny

The big thing that you loose when doing this is that you can no longer restore the SQL Server database to a specific point in time if you have a system failure.  If you don’t need the ability to restore so a specific point in time, and restoring to your latest full or differential backup that you have (you do know that you need to backup the vCenter databases right?) then changing the recovery mode from FULL to SIMPLE is just fine.  However if you do need that ability (and I would recommend that you have that ability so that if the vCenter server fails or someone deletes data you can get back as much trending data as possible) then you will want to leave the recovery mode as FULL, but instead setup transaction log backups (hopefully you have a DBA or a database consultant that can help you with this) so that they run every few minutes (I typically recommend every 10-15 minutes).  Just keep in mind that when selecting how often they should run in the event of a server failure you may loose up to that amount of data from the database.  So if you aren’t aloud to loose more than 10 minutes worth of data, don’t backup the log every 15 minutes, back it up every 10, or maybe every 8 just to be safe.  Make sure that you get in writing from your management that this is the config that they want so if you get yelled at you are covered.

If you haven’t ever backed up the transaction log your disk is probably pretty full (or totally full).  In this case you will want to shrink the database just once.  But before you do you need to know what size you should shrink it to.  This can be much easier if a DBA does it, but I’ve got faith in you to not screw it up.

First connect to the SQL Server using Management Studio.  Take a backup of the transaction log to disk by right clicking on the database, selecting Tasks from the context menu then selecting Backup from the pop out menu.  Select transaction log backup, set a file to backup the log to (you’ll need a lot of disk space for this) and make sure that there’s only one file listed in the file list then start the backup.  This will probably take a while.

If you don’t care about loosing the log for now change the database from full to simple recovery and back.  Do this by running this script.

ALTER DATABASE vcdb
SET RECOVERY SIMPLE
GO
ALTER DATABASE vcdb
SET RECOVERY FULL
GO

Now do a full backup as shown above (just do a full database backup instead of a transaction log backup).

Now with everything running as normal wait for how ever long it’ll be between transaction log backups.  Then connect to the SQL Server in SQL Server Management Studio.  Right click on the vCenter database (vcdb) and select Reports from the context menu, then “Disk Usage” from the list of reports.  At the top it’ll give you the size of the transaction log in MB, and the graph on the right will show you a percentage breakdown of how full the transaction log file is.  Take the Used percentage and double it.  Now take that percentage and multiple it by the size of the transaction log.  That’s the minimum size you should set your transaction log for.  Personally I would round up to the 8 gig size above that size.  If you need 150 Megs of log space, set the log for 8 Gigs in size.  If you need 10 Gigs of log space, let the log for 16 Gigs of space.  A little extra space used is fine, it is better for performance of the database if we don’t need to grow this transaction log.  You can shrink the transaction log by using the DBCC SHRINKFILE statement as shown.

DBCC SHRINKFILE ('vbdc_log', 8388608)

Keep in mind that the number, in this case 8 Gigs, is passed in in megs.  Do NOT for any reason create any sort of scheduled task to regularly shrink any database file.  This will do nothing good and will cause performance problems of its own.  If anyone has told you to setup a job which shrinks the database file regularly they are WRONG.

Denny

While you can normally attach a database without the log file, it doesn’t always work.  Sometimes SQL Server won’t be able to recover from this and create a new log file and you’l have lost the database.

If your database is growing and has gotten very large, then you’ll want to change the recorvery mode from FULL to SIMPLE.  This will help keep the transaction log file small, and is perfectly safe to do.  The only thing you will loose is the ability to do point in time restores of the transaction log.  However point in time restores are only available if you backup the transaction log (which would have prevented the log from growing out of control to begin with).

The reason that many people do this, is that the name of the file is misleading.  Normally logs are very safe to delete, however the transaction log isn’t your normal log file.  A transaction log keeps a record of every change to the database, along with the before and after values.  All changes are written to the transaction log before they are commited to the database itself so that if the server fails the database is left in a consistant state.

If your log file has simply gotten huge, and you need to make it smaller, you can use the DBCC SHRINKFILE command to shrink the log file to a smaller size.  This shouldn’t be done on a regular basis.  If you are doing this on a regular basis leave the file large it’s at the size it needs to be or change the database recovery to SIMPLE.

Denny

1. RAM – RAM is cheep, and easy to upgrade.  For a couple of thousand bucks you could easily add another 32-64 Gigs of RAM (more if you buy off brand).

2. Drive Space – Disks are also cheep (not as cheep as RAM, but close).  You could drop some extra spindles in there and more tempdb off to its own drive, or even break the database into a couple of physical files and double the available IO.

3. Faster CPUs – CPUs usually aren’t all that cheep.  If you have dual core chips, and you want to get some core chips you are probably looking at ~$1500 per chip, so that probably isn’t an option.  Now if you have a single quad core CPU in there, you could drop a second one in double your CPU power.  This will throw off your licensing if this is an internet facing SQL Server.  Those 6 core CPUs may look might sexy, but sadly they are probably way out of your budget at ~$3k each.

Those are pretty much your options.  Now if your IO is being slammed that might be a good place to throw that cash, but if your IO is being slammed is it because of writes or reads?  If it’s because of writes then by all means throw the cash at the storage.  If it is because of reads, then check your hit cache ratio, and page life expediency.  If they are low, then some more RAM is in order.  This will increase the amount of data in cache, increasing the amount of time the data stays in cache, and reducing the IO requirements on to the disk to a crawl.

Everyone wants more CPU power in there servers.  But sadly CPU power is still very expensive, so it isn’t really going to be an upgrade option unless you’ve got a pretty small server.  Not to mention the licensing issues that can quickly lead to if you add more physical CPUs than you are licensed for.

All things being equal I’d probably go for the RAM upgrade.  Long term I think that’ll get you the most bang for the buck, however that’s just me.  And depending on the server I might change my mind.  Before jumping into any system upgrade think carefully about what you want to purchase, what you want to achieve, and make sure that those things cross.

Denny

P.S. My hardware prices were from Dell’s website.  Your prices will vary depending on where you do your shopping.

The problem with this is that you have bought a server with no idea how to fit the database on it and still get good performance from the server.  If you ask me this is just ass backward.  When buying a new SQL Server you should figure out what the server needs to run the database, then buy the server to match that configuration.  You may be spending a little bit more money up front, but your system will perform much better because you’ve actually designed the server to support the database rather than trying to cram the database into the server.

Denny

Denny

 The backups are simple, use the SQL Maintenance plan to backup the logs, and then copy them to the remote machine.  But how do you restore the logs to the reporting server nightly.

 Well I’ve got a two step SQL job which should help you out.

Step 1 kills all current sessions in the database, and step 2 does the actual restores.

The code for step 1 is: 

declare @spid varchar(20)
declare cur CURSOR FOR
select spid
from sys.sysprocesses
where dbid = db_id('Your Database Name Here') /*<---Put your database name here*/
and spid > 50
open cur
fetch next from cur into @spid
while @@FETCH_STATUS = 0
BEGIN
exec ('kill ' + @spid)
fetch next from cur into @spid
END
close cur
deallocate cur

The code for step 2 is:

create table #Files
(FileName nvarchar(4000),
Depth int,
IsFile bit)insert into #Files
exec xp_dirtree 'd:\', 1, 1

I hope this makes your process easier. Now this code only works on SQL Server 2005 and up as the system stored procedures which I use were not included until SQL Server 2005.

Denny