A week ago, after coming across some rather graphic server room pictures (SFW unless your office believes in the sanctity of server equipment), I asked the community for its favorite server room DIY or horror stories, promising the winner fame and fortune (or at least some free swag). Well, we have a winner: IT Knowledge Exchange member Batye’s tale of a Russian server room that might’ve stumped the Hardy Boys. As thanks, we shipped Batye free copies of:
This went out without any apparent cause: Total breakdown in communication, only to return again later. It wasn’t a virus, and all the hardware seemed to be fully functional, until it completely disappeared.
The facility was quite modern: Secure parking, video security, armed guards with AKM’s and Saigas. The office even required passing a facial recognition scan to access the facilities. After having a specialized name badge made up, complete with his photo and a unique barcode, he was given access to the server room, the epicenter of the irregular crashes.
The scene made our leaked server room pics look like a networking Taj Mahal: The server room was a refinished washroom/utility closet. Nobody had bothered to put in server racks, leaving the high-end equipment and air conditioner to sit directly on the floor amid moldy walls and leaky pipes.
The problem quickly became apparent: Anytime someone flushed the toilet, water (one prays) from the leaking pipes got on the UPS. The UPS’ surge protection would kick in, automatically powering itself down along with the servers connected to it, hence the disappearing pings.
A few hours later, the air conditioner would dry up the flushed water and the UPS would reset itself, and all would be more or less right with the world again – until someone else needed to go.
Blame the setup on new fangled IT theories: The server room was managed by a 17-year-old kid, the son of one of the company’s executives.