EMC

#EMCWorld 2009 is starting tomorrow, and I’m on my way

Well tomorrow begins my annual trek to EMC World.  This year I’m headed back to Orlando as EMC World is being held at the Orange County Convention Center.  As I’ve done the last couple of years I’ll post as often as I can during the conference both on here on my blog, as well as on Twitter.

This years EMC World event should be a blast and very educational.  They’ve got tons of sessions on VMware, and one that I’m really looking forward to on setting up Exchange under VMware using a CLARiiON for the storage.  This is something that I was hoping to get done before EMC World, but when I saw that session on the schedule I decided to hold off on our Exchange Migration until afterwords so that I could get some additional best practices first.

I’m also looking forward to the sessions that I’ve found about SQL Server on the CLARiiON.  I haven’t found all that many of these up there, something that I’ll be sure to mention in my eval this year as I would assume tha the bulk of data stored on SANs is database data, and contrary to popular believe database servers are not file servers and should not be treated as such.

If you will be at EMC World come on over and say hi.  I’ll be on twitter so shoot me a message or a DM or find me in the Web 2 lounge or the EMC returning attendees lounge, or the exhibit hall somewhere.

Don’t forget to check back here for photo’s of the event.  I can’t upload in real time as my phone doesn’t have a camera, so I have to wait until I get back to the hotel or the convention center to upload them.

Denny

Datacenter Migration In Progress

For those of you who know me, or have heard me talk at a Code Camp in the last year, you’ve heard me talk about a data center migration that I want to do from Rackspace in Texas to our own equipment in the LA area.  Well that day has finely come.

Our current environment has served us well, but we have outgrown the services that Rackspace can offer us, and we have purchased our own production environment.  This isn’t any rinky dink environment either.  We are starting out with a fully redundant, highly available environment which can be scaled by simply deploying more VMs, and in the event that the VMware hardware is over tasked by simply plugging another VMware server into the mix, and shifting the VMs from one node of the cluster to another.

We are very proud of our new environment, so I figured that I’d give you some of the tech specs of it (yeah, I’m totally bragging here).

On the storage side of things we’ve got an EMC CX4-240 with ~35TB of storage laid out in three tiers.  This is connected via multiple 4 Gig fibre cables to a pair of Cisco fibre switches.  Each fibre switch is connected to each of the SAN attached servers.

We went with Dell servers (I would have preferred HP servers, but I was overruled).

The SQL Servers and the VMware servers are identical.  Quad chip, quad core servers each with 64 Gigs of RAM.  Each pair will be clustered for High Availability.  The VMware servers will look a little like they puked cables out of the back.  Because of all the various subnets and to ensure that each subnet is connected to each of the redundant network switches each of the VMware ESX servers will have 11 Ethernet cables, and 2 fibre cables coming out of the back.

The VMware vCenter services are running on a little single chip quad core server.  This is the only part of the system which isn’t redundant, but ESX can run fine for up to 14 days without the License server running, and since this machine has a 4 hour turn around on parts we’ll be fine if the machine dies.

The file servers which host the screenshots, emails, etc which have been captured by our application and will be served to the website upon request are a pair of dual chip, quad core servers also clustered for high availability.

All the servers are SAN attached via the fibre and all data will be saved on the SAN.

Our current environment is much smaller.  A single SQL Server, three web servers, and two file servers.  The only redundant pieces are the fibre cables from the SQL Server to the SAN, and the fact that we have three web servers.  However if the newer web server goes out in the middle of the day, the other two will choke at this point.

Rackspace has been pretty good to us over the years.  It just wasn’t cost effective for us to purchase this level of hardware before now, and Rackspace was able to provide us with a good service level for a reasonable price.  But at this point, because of the amount of hardware we were looking to move into, and the amount of bandwidth we are going to be using it simply became more cost effective for us to host the systems at a local CoLo.

The main reason that I’m telling everyone this is that if you have been trying to find me for the last two weeks or so this is why I can’t be found.  I’ve been spending pretty much every waking moment this together and getting it all setup so that we can migrate over to it.

Needless to say its an awesome project.  How many people get the chance to build a new data center and design it the way they want to from scratch.  Pretty much no one.  Data centers usually grow from a small config of a server or two in a sporadic way, and they are inherited from one person to the next.  But this time I get to design everything they way I want to from the grown up.  It’s going to be a blast.

Denny

EMC World 2008 is over :(

EMC World was a blast this year.  I’ve posted some highlights of the event through out the week.  I’ll continue to post as much of the info as I can through out the next few weeks.

 Denny

EMC World 2008 Day 3

Wednesday at EMC World was a session packed day to be sure.

I started my morning with CLARiiON Rebuild Settings and Data which was an in depth look at how exactly the CLARiiON systems handle rebuilds, and how long various rebuilds take.  In addition we went into detail as to how the CLARiiON will not see that a disk is going to fail, and instead of waiting for it to fail and then having to rebuild it will actually copy the data from the disk ahead of time to a hot-spare then mark the disk as failed so that it can be replaced before the disk actually fails there by providing you with no point in time where your data is unprotected.

 Another session went through the changes to the EMC PowerPath product and all the new features they are building into PowerPath such as the encryption of data when it leaves the server on its way to the disk.

Another session went through some tuning tips an tricks for getting the best performance from the EMC CLARiiON product.  These include the strip size, cache settings at both the LUN and SP level. 

The night ended with the Billy Crystal performance.  I figured that the show would be funny.  I was wrong, it was hilarious.  Like everyone else I’ve seen Billy Crystal on TV and in Movies and thought he was funny, but in person he was probably the best comedy performance that I had ever seen.

Denny

EMC World 2008 Day 2

Yesterday was an very busy day.  I didn’t have time to think, much less put together a post about it.  I hit every session which I was looking for including the always hard to get into Navisphere Manager Hands-on workshop.

The session I probably got the most out of was the session on what’s new in the FLARE version 26 which was released a few months ago.

FLARE 26 now supports Active/Active presentation of the LUNs.  What this means is that in the event of a fibre cut on either the front end or the back end the host machine (Server) will no longer need to trespass the LUN to the other SP.  The LUN can simply send the IO request to the other SP.  The non-preferred SP will then forward the request to the preferred SP automatically for completion.  Upon the preferred SPs connectivity coming back online the requests will then be sent to the preferred SP.  The newest version of PowerPath is required for this to work, or the native multipathing driver such as the Windows 2008 driver must support ALVA.

The support for supporting a broken connection between the host and the storage is from ALVA.  The support for handling the request when the connection is broken between the SP and the DAE is an EMC only extension of ALVA.

FLARE 26 also includes RAID 6 support.  When comparing RAID 6 with RAID 5 on the same system read performance will typically be better as the data is spread across all the drives in the RAID 6 array.  Unlike a lot of other systems the EMC CLARiiON array spreads the parity sectors of (RAID 5 and) RAID 6 across all the drives in the RAID Group.  So because there is an extra drive in the array a 4+2 RAID 6 RAID Group will give better read performance that a 4+1 RAID 5 RAID Group.  When doing a full strip write the write speed between a RAID 5 and RAID 6 array will be basically the same.  When doing smaller writes a RAID 5 array will have a faster write time than a RAID 6 array because RAID 6 has the extra parity to account for.  The rebuild times for rebuilding after a failed drive will be about the same between a RAID 5 and RAID 6 array which have suffered a single drive failure.  If the RAID 6 array has to recover from a dual drive failure it will take longer to recover than the single drive failure as the data must be recalculated from the two parity bits rather than from a single parity bit.  However the odds of a dual disk failure are slim. 

Just like with RAID 5 within the CLARiiON the RAID 6 supports the proactive hot spare.  This is where when the system sees that a drive is going to fail it will automatically copy the data from the failing disk to a hot spare and mark the disk as bad.  As the data does not have to be rebuilt this is a very quick operation.

FLARE 26 now supports a Security Administrator role.  Members of this role have no access to the storage settings it self.  They can only create accounts within the Array.

A very important change is that the SPs can now be setup to sync thier system time to a networked NTP time server.  This will force the time on the SPs to be the same.  Until now the times could end up getting a little off which could make tracking down event information very hard to do as the log entries would have different times on each SPs log file.

FLARE 26 now supports replication over the built-in iSCSI ports on the new CX3 line of systems.  This is a great change as before you had to use the iSCSI ports on a FC-IP switch to do this replication.  This includes SAN Copy, MirrorView, etc.

MirrorView /S should only be used for connections within ~100 miles as beyond that you start to get to much latency between the arrays.

Starting later this year (Q3 or so) there will be an extension to MirrorView /S called MirrorView /SE (Cluster Enabler) for Microsoft Cluster Service.  This will give you the ability to use CLARiiON to setup a geographically disbursed cluster.  In other words you can have servers in two different cities setup in a Windows Cluster.

EMC World 2008 Day 1 (Keynotes)

Here are some of the key points from the keynotes from EMC world.

Gartner says that by 2011 there will be 1337 Exabytes.

 179 Exabytes of information has been created so far this year.

In the next two years we will see a trend moving further away from using Tape for backup and recovery.  It will still be used for long term archiving, but day to day restores will come from disk.

During the raw data into useful information is becoming more and more of a challenge for IT departments and business units. 

What I’m gathering from the keynotes is that the amount of data is exploding.  I see this as a great thing for us DBAs as, the bulk of data that companies have is going to be stored within the database.  I know that the company which I work for (Awareness Technologies) creates about 110 Gigs of information per day.  Now with our products policy we keep this data for 14 days (unless the customer pays us to keep the data longer).  This means that we have about 1.5 Terabytes of information within our data center that our customers are looking at on a daily basis.  This much data, with such a high data change rate is just a crazy amount of data change.

Sites with high data change rates have specific challenges some of which I will be talking about (specifically how they relate to Microsoft SQL Server) on 6/11/2008 at 12:00 PM PST when I present a geekspeak session “geekSpeak: Spatial Data in SQL Server 2008 with Denny Cherry”.  I’ll post the link to this webcast when I get it.

Since my laptop battery is dieing I’ll have to wrap this up for now.  Check back later for more.

Denny

EMC World 2008 Day 0

Today is the check-in and welcome party for EMC World.  EMC and Brocade had the Goo Goo Dolls perform at the welcome party.  The party and show were fantastic.

Funniest comment of the concert would have to be “This is the first all you can eat buffet rock show I’ve ever been to.”

The video was taken by my wife (who also had a great time, thanks for coming with me).  What you don’t see (but you can here) is John Rzeznik talking to one of the people serving the food about 2 feet from the front of the stage.  If anyone has better shots or video please post them and links to them.

Here are some stills which my wife took. She takes all the photos. I can’t take a picture to save my life.  (Click the image to see the bigger version.)
  

I’ll continue to post from the conference as best I can.

Denny

****UPDATE****

I’ve added another picture so the group.