I know, I know … whenever I see “killer app” I roll my eyes too, but I’ve been shown the light, or indoctrinated, whichever way you want to look at it. So here is my take: I’ve found storage’s killer app, and it’s hiding in plain sight.
There … I said it … EMAIL.
At work we’ve been deeply involved in identifying the platform for our next generation SAN. We’ve been busy identifying performance metrics (the benchmarks I’ve taken off some of these machines are incredible–I’ll blog about it soon!!) and precisely what we use our storage for. I’m a big VMware fan and throughout this discovery process I’ve had in the back of my head that VMware is the biggest thing we need to plan for. I was looking at VMware/host virtualization as the killer app for storage and I was wrong.
While we don’t and wouldn’t actually look at what is in end users’ mailbox, we do see the size of the mailboxes. We have quotas for most but some folks warrant an exception. It dawned on me during this process that we’ve been building out our Vmware infrastructure to mainly provide the filler for messaging and collaboration. I’m a cynic by nature so I decided to take a deeper look into my own email habits to see if this theory held water.
If you ever really wanted to surprise yourself, take a look into the dusty corners of your home machine’s email program.
So my basic conclusion is my dinky laptop hard drive is no place for my archive.pst. I need RAID 10 striped across 14 drives … for just the imbroglio (this was a great SAT word submission!!) I call my inbox. How many attachments I have, how much email I actually get, how much of it I keep, how it gets indexed by my desktop search, and finally how it gets archived all lead to a surprising portion of my work laptop hard drive dedicated to email. I also started looking at where I pull the information from to put in my email and found that almost all the work I produce gets emailed to someone, then they store it or email it to someone else.
At work I have about 800MB of active email data and about 4GB archived for the last 6 months. At home it’s triple that–I actually had to build a virtual machine to handle my third-tier email archives (I like to have my email indexed and available) which leads me to why I believe email is the killer app for storage.
My personal email is always online, indexed and searchable. If I need a piece of information and I can remember one or two unique words I stand a great chance of retrieving it from just about anywhere (I use IMAPS) that I can install Thunderbird or Evolution. It is very convenient and once I figure out how to search from my BlackBerry I won’t ever have to remember anything but keywords.
This convenience of course requires storage, and not only that, but storage that can chew through tons and tons of 2K and 4K files (I use Postfix and Cyrus IMAP on virtual machines at home) to find the bit of information I’m looking for.
Scale me to an enterprise level, think Ultraman. Today there are people in the work force who don’t know what POP is (remember PINE? Ahh the good old days…), have been using their Yahoo.com email account since they could type, and email things to themselves when they want to save it. There is a new generation coming to the workforce after them that EXPECT to be able to have their entire lives searchable and indexed, Google-style.
At work we are moving towards the Sharepoint 2007 /Exchange 2007 /Office 2007 hegemony in the next year or two, and I am concerned that we as an industry don’t really and truly understand what collaboration does to storage requirements.
If I can now collaborate completely on my computer how long will my organization have to keep my Onenote stuff around? Will there be some sort of e-discovery for the group whiteboard? Where will we store all this stuff?