Contests

Contest winner for Tales from the Data Center

Grand Prize Winner

An anonymous entry takes the Nintendo Wii grand prize.
Click to see why this member wishes to remain anonymous.

Data Center Contest: Now that’s what I call multitasking

The Tales from the Data Center contest is now over and we had a lot of fun with it. The lucky winner has been selected and will be receiving a Nintendo Wii.

Visit the ITKE Community Blog or subscribe to our newsletter to learn more about our other contests. If you’d like to subscribe to our newsletter (currently twice a week) just login to IT Knowledge Exchange and click the Edit My Account link. Then check the box to receive our IT Knowledge Exchange Community Newsletter.

I listed some other contest entries below for your amusement. Thanks for stopping by to see who won.

Community Manager

Enjoy these other contest entries…

Data Center Contest: Disaster recovery using non-Microsoft “Windows”
Data Center Contest: Successful treatment of Mayo Clinic kill switch
Data Center Contest: Ooh, ooh that smell …can’t you smell that smell?
Data Center Contest: Oh what a tangled web we weave
Data Center Contest: Bathroom or Server room?
Data Center Contest: Saran Wrap to the rescue!
Data Center Contest: The 2:00 AM Wind Tunnel
Data Center Contest: Nuns bring network to its knees
Data Center Contest: The great Chicago flood of ‘91
Data Center Contest: Zut alors! How many cables?
Data Center Contest: Hose down those hard drives!
Data Center Contest: The Orville Redenbacher backup technique
Data Center Contest: Last one out, turn off the lights

Data Center Contest: Oh what a tangled web we weave

We’re gathering stories and photos for a cool contest sponsored by American Power Conversion (APC). First place in our Data Center Contest wins a Nintendo Wii game system. And we’re giving away twenty (20) copies of System Specifications and Project Manual for Data Centers (a $250 value!) to members just for entering the contest with a valid entry. Enjoy the contest entry below — and feel free to comment.

We also received a couple of photos with this contest entry. These ought to curl your toes…

Community Manager

From a Member who wishes to remain anonymous:

Evolution is supposed to be an systematic increase in order, complexity and efficiency. Well, along the way, Darwin’s concept was forgotten as this “wiring closet” was evolving.

The original panel and wiring was installed in the late 70’s and as extensions and additional outside lines were added, and changed, this arrangement became more …um…interesting.

About half of this company’s PBX telephone gear still runs through this panel, along with all six of their fax lines. The eight incoming telco lines are also present here (somewhere…).

The most recent addition is the network switches and their APC battery backup (barely visible above the switches) that handle about 20 of the company’s 50 workstations.

There is no documentation, no labels, no tags. What the image does not show is the fact that there is about 8 feet of space between the bottom edge of the photo and the floor. So any work done on this panel must be done from a ladder.

Long term plans are to replace the current Executone PBX system (circa 1992) with a new VOIP system, until then, nobody wants to even breathe on this panel for fear of knocking something loose.

Data Center Contest: Ooh, ooh that smell …can’t you smell that smell?

We’re gathering stories and photos for a cool contest sponsored by American Power Conversion (APC). First place in our Data Center Contest wins a Nintendo Wii game system. And we’re giving away twenty (20) copies of System Specifications and Project Manual for Data Centers (a $250 value!) to members just for entering the contest with a valid entry. Enjoy the contest entry below — and feel free to comment.

Community Manager

Cynthia submitted this entry:

This probably isn’t the Craziest Story you’ve had, but for us, it’s pretty funny. We have a policy that won’t allow us to take photos inside the facilities, so I have to share the story instead.

One of our IT employees smelled an “electrical burning” smell. After sniffing around, we found the iSeries console had been fried. On the table where it sat was quite a bit of water and more was dripping from the ceiling. We had just had a particularly hard downpour and come to find out, the roof had sprung a leak.

Maintenance acted immediately bringing us a 55-gallon trash can and a huge roll of plastic! They temporarily fixed the roof by putting some plastic down and holding it with a few bricks!

We pushed the iSeries and it’s attached devices a few inches away from the dripping, draped plastic over it’s front and sides, and dug our last dumb terminal out of storage to replace the console unit. It took almost 2 weeks to get the roof repaired. Thankfully, we were blessed with mostly dry weather.

It never ceases to amaze me that our maintenance department doesn’t seem concerned about the state of our computer room which houses hundreds of thousands of dollars worth of equipment.

I wish I had a picture of the iSeries & tape drive draped in plastic, with the huge blue trash can, ceiling tiles and floor tiles removed, and a ladder in the middle of it all. It was a mess!

Data Center Contest: Successful treatment of Mayo Clinic kill switch

We’re gathering stories and photos for a cool contest sponsored by American Power Conversion (APC). First place in our Data Center Contest wins a Nintendo Wii game system. And we’re giving away twenty (20) copies of System Specifications and Project Manual for Data Centers (a $250 value!) to members just for entering the contest with a valid entry. Enjoy the contest entry below — and feel free to comment.

Community Manager

Gary submitted this entry:

In 1994, I joined the Mayo Clinic Rochester, MN campus as a senior Systems Administrator. My first day was spent getting the guided tour of the data center facilities at the Clinic. We were touring the main data center. This facility housed the Clinic’s mainframe, and a host of other distributed enterprise systems. The day was getting rather late as we were approaching the end of the tour. The lateness of the day also marked the end of the shift for the Data Center operations staff.

Our tour group was being escorted through the command center on our way out to the main data center operations area. Located along the wall next to the data center entrance was a large red kill switch.

Looking back now, this reminds me of the big red “Easy Button” depicted in popular Staples’ television media adds. This kill switch when pressed engaged the emergency power shutdown for the entire data center. The kill switch was within easy reach from the entrance door. There was no protective encasement or other security feature over the kill switch. Somebody could accidentally trip and hit the switch, which would engage the emergency power shutdown.

This fact must have been on the mind of our guide as well. Our guide strategically positioned himself between the kill switch and the door entrance to prevent such an accident from occurring. After everyone safely assembled together on the production floor, our guide resumed the tour. Our group moved over to opposite end of the production floor to view new StorageTek tape silos. From this vantage point, we had a clear unobstructed line of sight to witness the events that followed.

Observing a data center power shutdown is really something to see. One minute, we were gazing at the swinging robotics arm contained within the StorageTek silo fetching tape cartridges. The next minute the only viewable lighting was coming from the emergency lights. In retrospect, the outage lasted only a few minutes. But during this time there were teams of programmers and other IT support specialists that flooded out onto the production floor. During the power outage, our tour guide remained calm and collected. During these events he kept the group together and out of everyone’s way.

We later learned at the end of our tour of the events that led up to the data center power shutdown. A developer had called the command center to request a hard reboot on one of the servers in the command center. Confused, the operator asked the developer where the server was located. The developer instructed the unfortunate operator to look for a red power switch on the server located next to the wall. The operator saw one red switch. The rest was history.

The next day when I was out on the production floor doing server maintenance I noticed there was a plastic security encasement covering that one particular red switch!!!

Data Center Contest: Disaster recovery using non-Microsoft “Windows”

We’re gathering stories and photos for a cool contest sponsored by American Power Conversion (APC). First place in our Data Center Contest wins a Nintendo Wii game system. And we’re giving away twenty (20) copies of System Specifications and Project Manual for Data Centers (a $250 value!) to members just for entering the contest with a valid entry. Enjoy the contest entry below — and feel free to comment.

Community Manager

From a Member who wishes to remain anonymous:

My first job in IT (at the time it was just called Data Processing) was in the early 80’s at a community college that I had just graduated from with an Associates in computer programming. I was hired as an operator for an IBM system 38 that was housed in a large room on the first floor of the school with a cubicle in the corner, which served as my office and of the programmer who had been there for 2 years before me.

Anyway, our back up system consisted of a weekly ritual of saving the most important files to diskettes housed in magazines that held 10 diskettes each. The normal weekly back up used about 10 magazines. The monthly backup was of the whole system, which used about twice as many magazines. It was cumbersome, but that was my main job since I worked 2nd shift. After the back up finished, I had to place each magazine on a bookshelf that sat in the corner.

The bad part was our disaster recovery system, for which we did drills for once a month. When the alarm went off, the programmer had to run outside and stand outside the window to our office. I then had to power down the computer and take each magazine of diskettes and toss it out the window to her until all of them were out. Only then could I “save” myself and join everyone else outside. The programmer then had to take the magazines and put them in her car and drive off site. Needless to say, all of the students and faculty got a big kick out of our little ritual.

Well, wouldn’t you know it, an actual fire broke out at the school one day and it just so happened to be the day the programmer wasn’t there. My boss, the data processing manager, had to run down the hall and take her place. Everything went without a hitch since the fire was in a part of the building far away from the computer room. After I threw all of the magazines out the window, I proceeded outside where I noticed that my boss had already put them in his car and was driving away from the building.

After all of the hoopla died down and the minor fire was put out, we were allowed back inside. I asked my boss when he returned where the magazines were and he informed me not to worry about them, he would just keep them in his car and for me to initialize and use new diskettes for that night’s backup, which I did.

Well, a week later while preparing for the monthly back up, I had to go to my boss and ask for the magazines since I didn’t have enough to do the whole back up. It was then that he informed me in a small voice that he had forgotten to take them out of the back of his trunk, and the diskettes had all warped and melted from the heat.

Needless to say, he then authorized me to go and buy new diskettes and magazines and made me promise not to tell anyone what happened. Also, soon afterward, he hired a company to house all of our back ups off site and we adopted a rotation system for our backups as well as purchasing a tape drive.

Data Center Contest: Last one out, turn off the lights

We’re gathering stories and photos for a cool contest sponsored by American Power Conversion (APC). First place in our Data Center Contest wins a Nintendo Wii game system. And we’re giving away twenty (20) copies of System Specifications and Project Manual for Data Centers (a $250 value!) to members just for entering the contest with a valid entry. Enjoy the contest entry below — and feel free to comment.

Community Manager

Adrian submitted this entry:

We’d just moved into a new data centre and had a Lights Out operation in this one area where we had a Tech replacing some part. We left him there for a while and when he left, he reached back to turn the lights off and instead hit the Emergency Power Off (EPO) button, dropping power to the entire area.

We quickly put covers over all the EPOs (like you see on a fire alarm) and labeled them more clearly.

Are the Uptime Institute’s Data Center Rating Tiers Out of Date?

“Share your ‘Tales from the Data Center’ and you could win a Nintendo Wii”

Let me start by saying I have the utmost respect for the Uptime Institute’s Pitt Turner, P.E., John Seader, P.E., and Ken Brill and the work they have done furthering the cause of providing some standards to an otherwise standard-less subject like data center design. However, as a data center designer I feel their definitive work, Tier Classifications Define Site Infrastructure Performance, has passed its prime.
The Institute’s systems have been in use since 1995, which is positively ancient in the world of IT.
In its latest revision, the Uptime Institute’s Tier Performance Standards morphed from a tool for IT and corporate decision makers to consider the differences between different data center investments into a case study for consulting services pushing for certification against their standard.
While the data their standard is based upon has been culled from real client experiences, the analysis of the data has been interpreted by only one expert company, ComputerSite Engineering which works in close collaboration with the Uptime Institute. Surely, the standard could be vastly improved with the outside opinion and influence of many of the, just as expert, data center design firms that exist.
Case in point, the Uptime Institute has repeatedly defended the notion that there is no such thing as a partial tier conforming site (Tier 1+, almost Tier III, etc.). They argue that the rating is definitive and to say such things is a misuse of the rating guide. While I understand the argument that a site is only as good as its weakest link, to say that a site incorporating most, but not all of the elements of the tier definition is mathematically and experientially wrong.
PTS’ actual experiences bear this out. Our clients that have all the elements of a Tier II site, except for the second generator, are clearly better than those with no UPS and/or air conditioning redundancy (Tier I). Therefore, if not for Tier I+, how do they suggest to account for the vast realization between the real availability of the two sites?
It is interesting that most data center consulting, design, and engineering companies nationwide utilize elements of the white paper as a communications bridge to the non-facility engineering community, but not as part of their design process. In fact, most have developed and utilize their own internal rating guides.
While I will continue to utilize their indisputable expertise as a part of my own interpretation in directing PTS’ clients with their data center investment decisions, I suggest that clients would be wise not put all of their eggs in the Institute’s basket at this point in time.

What is your outlook on the Uptime Institute’s Tier Performance Standards? Is the four-tier perspective outdated or is it still a meaningful industry standard?

Pete

Data Center Contest: Now that’s what I call multitasking

We’re gathering stories and photos for a cool contest sponsored by American Power Conversion (APC). First place in our Data Center Contest wins a Nintendo Wii game system. And we’re giving away twenty (20) copies of System Specifications and Project Manual for Data Centers (a $250 value!) to members just for entering the contest with a valid entry. Enjoy the contest entry below — and feel free to comment.

Community Manager

From a Member who wishes to remain anonymous:

Like the proverb says, “A picture is worth a thousand words.” This photo was sent in by one of our members, stating that an IT coworker had taken this picture while visiting a newly acquired company. As you can see — this data center has all the comforts of home.

Let’s Talk About: The Role of Sprinklers in Computer Room Fire Protection

A number of clients have asked us about the viability of replacing their ‘wet’ sprinkler systems with a dry-type fire suppressions system, such as FM-200. Not many IT personnel understand the role of water-based fire suppression systems, but all realize the potential for water in the data processing environment to be a “bad thing.”

The short answer is that sprinkler systems protect the building and dry-type systems protect the equipment. In most cases a dry-type system cannot take the place of a sprinkler system, it can only be installed in addition to it. At the end of the day, the local fire inspection is the authority and has jurisdiction over what is permissible. This is the reason that pre-action sprinkler systems are primarily used for computer room fire protection. That being said, fire prevention provides more protection against damage than any type of detection or suppression equipment available. For Tier I and Tier II computer rooms, PTS often recommends installing only a pre-action sprinkler system activated by a photo-electric smoke detection system and forgo a dry-type system and VESDA system. We find the most effective strategy is to emphasize prevention and early detection. This allows the client to maximize availability by investing in solutions for areas of higher risk, such as fully redundant power and cooling systems.

For more information on fire protection, read our vendor white paper “Mitigating Fire Risks in Mission Critical Facilities,” which provides a clear understanding of the creation, detection, suppression and prevention of fire within mission critical facilities. Fire codes for Information Technology environments are discussed. Best practices for increasing availability are provided.

Pete

Data Center Contest: The Orville Redenbacher backup technique

We’re gathering stories and photos for a cool contest sponsored by American Power Conversion (APC). First place in our Data Center Contest wins a Nintendo Wii game system. And we’re giving away twenty (20) copies of System Specifications and Project Manual for Data Centers (a $250 value!) to members just for entering the contest with a valid entry. Enjoy the contest entry below — and feel free to comment.

Community Manager

Shava submitted this entry:

I was the manager of tech support for a Data General OEM back in the very early 80’s. One day, our only customer in Montana opened his call with, “I want you to get [undisclosed] from Data General tech support FIRED!”

He had a one Eclipse computer shop. The Eclipse was, essentially, a
DEC PDP-11 knock off. It was entirely held in one 19″ rack mount, and still had a switch bank to manually boot the system in case of boot sector corruption. We’re talking ancient of days, here.

The CPU unit was mounted above a 5MB fixed/removable disk. This is to say, the whole computer ran off a 5MB fixed platter (pre-Winchester sealed drives) and to back it up, you opened the drawer and put in another 5MB removable cartridge. This wasn’t a floppy – it was a 24″ round, 4-5″ deep, maybe 12# assembly, with a handle in the center to heft it out when you had to remove it.

The gentleman with the complaint had set his accounting data to back up and gone to the movies. Coming back to the office from the movies, he placed the remnant of his extra large tub of buttered, salted popcorn atop the 19″ rack. Removing the backup cartridge, in preparation for inserting the transaction logging removable cartridge, he swung the backup cartridge to set it on top of the rack.

His popcorn dumped into the disk drive, salt and butter and all. Remember, these drives are not sealed.

So, knowing the drive enclosure was supposed to be kept very clean and free of dust, he went to the men’s room and grabbed a wad of coarse (linty) paper towels and tried to wipe out the salt, butter, and popcorn as well as he could – probably shoving a good deal of it into the chamber where the ceramic read heads were recessed during cartridge change.

By this point, I had the guy on speakerphone, and my crew of about a dozen was just about DYING trying to suppress raucous laughter.

Having done “what he could” to clean up the mess, he put in the transaction disk, and spun up the drive. At this point, the drive started to scream and wail – no doubt, a massive head crash. We could all imagine the heads shaving oxide off of those big brown platters like a Dremel tool.

He tried spinning the drive down, but the noise didn’t stop right away, so he tried spinning it back up – and then in panic, he cut the power to the machine. He could smell smoke

He then opened the drawer to look inside.

Now, these disks weren’t *sealed* but they tried to keep air out when they were running, so there was a *pop* as the removable cartridge came out. But that was a minor effect as, heated probably over 500 degrees F, some bits of popcorn and the atomized butter ignited, causing a Flash Paper-like burst of flame, singeing the guy’s eyebrows a bit.

Everyone in my office listening to this guy was completely losing it by this point.

“Thank God this didn’t happen in a couple weeks when we’re putting the sprinklers into the computer room!” he puffed.

“Yes, sir,” I said calmly, “and this is when you called Data General?”

“I hate calling field service,” the guy admitted, “And they never get here fast – we’re a couple hours from the nearest tech. But I called him up and you know what the jerk told me?”

“Uh, no sir.”

“He told me my contract didn’t cover me for errors of gross stupidity! I want that man *fired*!”

Well, weeks later, the man did get his computer and drive replaced (the CPU had fried from the heat or some effect of atomized grease on the insulation, who knows?). Although it’s not clear how much of the damage his contract should have covered, DG covered it all, including (I think) three site visits in all. We always speculated that they thought they got value back in the hilariousness of this man’s story.