Data Center

New open source IT management tool: Lighter-weight than Nagios, more granular than Cacti

Theo Schlossnagle, CEO and founder of managed services and hosting provider OmniTI, hopes to solve some of the common complaints with open source systems management tools with his company’s new tool Reconnoiter.

OmniTI manages 15 data centers with heterogeneous architectures for multiple clients, and Schlossnagle said he’s used every tool under the sun: Zenoss, Tivoli, OpenView, Nagios, Cacti and more.

The recurring problems Schlossnagle found with open source management tools — scaling issues, repeated effort for configuration management, and requirements for powerful server infrastructure – frustrated his team to the point where OmniTI built its own toolset for monitoring metrics, graphing data for capacity planning, and post-mortem analysis of problems.

The tool uses an agent-based system. Users would install a Noit Daemon in each important portion of infrastructure and configure it to monitor different services. The software is written in C, plug-ins are written in C or Lua. Reconnoiter uses SNMP, ITMP, HTTP among other protocols.

The company is offering it under BSD license on its Website for free.

According to Schlossnagle, challenges using the open source management software Nagios were a major driver for developing the Reconnoiter tool.

“Nagios is quite inefficient in the way it collects data,” Schlossnagle said. “It follows the age-old Unix philosophy that you use the right tool for each job. This means that Nagios ends up launching thousands of small applications to test things. While the lots of little tools philosophy is often convenient, it heavily conflicts with high performance, low latency requirements. Often purpose built tools need to take over in that role — that is what Reconnoiter is.

“I have to buy a big, expensive box to run Nagios — I don’t with the Reconnoiter agents,” Schlossnagle continued. “Nagios does fault detection, but not trending — which means I have to double my efforts by configuring both Nagios and another tool.”

Schlossnagle also said Nagios’ monitoring was centralized, so it was difficult to adding checks in the field. Managing configurations was hard to track as you deployed new services and machines.

The Reconnoiter tool polls systems to see if they’re healthy in a similar way that Nagios does, but of the open source-commercial hybrid products that are out there, Schlossnagle said the product is most similar to Hyperic.

“Hyperic takes a more holistic view of monitoring in that it includes both trending and fault detection. Reconnoiter takes this approach as well.”

The Reconnoiter tool is also designed to help IT managers analyze Web traffic events in a very granular way, even ones that happened in the distant past. “RRDTool is specifically designed to retain data within size constraints. You define how long you wish to retain data on various granularities,” Schlossnagle said. “In most systems that use rrdtool (like Cacti) recent data (like one week) is retained on five minute granularity, while data older than a week is reduced to a granularity of one hour. So, if you want to compare a spike today to one from six months ago, it is very likely that you have a defeating skew: 288 five-minute intervals for “today” and four six-hour intervals for the day in question six months back.”

Reconnoiter approaches this by taking the stance that storage is cheap. “There is not excuse for throwing any of that data away. I’ll go buy a terabyte of disk. I’m not going to search back 12 months very often, so it doesn’t need to be fast, but I need to be able to do it.”

According to Schlossnagle, watching the spike happen gives you a better understanding how traffic patterns shift during a major event, for example a Web site being picked up by a large social media site like Digg.

“If I’m looking at that spike on my systems at thirty second granularity, I can tell you how fast that spike happened. If I use the RRD tool with Nagios and Cacti, I can only see that day at that level of granularity for about six hours.”

This tool can help IT managers plan for capacity during spike scenarios and compare to events in the past.

“Our primary goal was to make our lives easier. This tool replaces an enormous amount of headache at OmniTI,” Schlossnagle said. “Making it a successful open source tool makes it even easier. One of the short term goals it to have it adopted other places and get the tool deployed in large environments.”

Today, OmniTI is slowly introducing Reconnoiter to its managed services clients. The company is currently monitoring tens of thousands of metrics across five data centers, approaching a terabyte of metric data.

OmniTI does not plan to develop a commercial version at this time, like Hyperic or Zenoss. “An open source approach with a strong community is better,” Schlossnagle said. “I don’t want to be in the tools business. If a company wants to give us money for support and indemnify them with IP rights, we won’t turn away that money.

“The key difference being the product we deliver, support and indemnify, would be the same product, not the one that has special neat features that paying customers get.”

You can give Reconnoiter a test run at labs.omniti.com.

Server sales tank as folks stretch hardware life cycles

According to IDC’s recent server report, server sales are plummeting due to the slumping global economy. According to IDC, this is the lowest quarterly server revenue since the analyst firm began tracking the server market on a quarterly basis 12 years ago.

The commodity server market took a big hit in general, while Dell specifically had a bad start to 2009. Dell’s server revenue declined 31.2% year over year. Even blade servers slumped, for the first time since IDC had tracked blades.

According to our 2009 data center economy survey, IT pros are stretching stretching server life cycles, putting off buying new hardware.

Servers are typically replaced every three years. Two-thirds of IT shops have extended the production life of server deployments in 2009. More than 35% say they’ll keep servers in production for six months to a year longer, 34% say they’ll extend server life cycles by two years.

Forty-foot Unix history poster will dominate your office

For a mere $340, you can send a major message to your office-mates about how much you love Unix. A new poster from Leighton Jones at Calgary-based Floating Point Digital Images depicts Eric Levenez’s diagram of the Unix operating system with fractal art by Alan Tenant. You can download the Unix history chart for free here.

Photo by Floating Point Digital Images.

Why buy this tear and weather-resistant 10lb poster? According to the purveyors, “it could be as simple as the desire to wrap yourself several times in its informational goodness that documents the history of 1000+ versions of more than 150 different Unixes.”

Found this link at The Register.

Is open source affiliation keeping upstart systems management tools out of the enterprise?

According to IT management blogger John M. Willis, upstart systems management vendors like Zenoss and Hyperic need to tone down their open source rhetoric and take a page from their competitors like Solar Winds and Nimsoft.

“Zenoss and Hyperic beachfront with open source too much and it keeps them out of the enterprise,” Willis said. “Stop it with the open source stuff. Stop even mentioning it. Solar Winds is kicking your butt all over the place and all they’re talking about is price and performance.”

Willis isn’t advocating that Zenoss and Hyperic drop open source altogether, rather make it a line item instead of a headline.

“Almost everybody I talk to in enterprise IT management isn’t keen on open source,” Willis said. “If there’s a team that wants to run Nagios, then they usually can get a checkmark on it if they don’t have anything already in place. But if you want to rip and replace Tivoli with Zenoss, management will say ‘Eh…. I’m not sure about that.’”

If a company is going the Nagios, Zenoss or Hyperic route, they’re going hook line and sinker, according to Solar Winds senior VP Kenny Van Zant. “They’ll suffer the manual cost of configuration and maintenance that open source will bring, because they don’t even have $5000 to spend,” Van Zant said. “Or people are open source fans who want to use open source wherever they can. When there is a gap on Zenoss, the fill it with nTop, Cacti, or some other open source product of the month and integrate them all together.

“We bump up against those free tools when the open source person leaves the company,” Van Zant said. “We replace Nagios deployments. It just takes too much to keep it up and running.”

Are you willing to bring open source systems management tools into your shop? Did your management team object? Email me or leave feedback in the comments.

Intel: Mega data centers sucking up chips

Jason Waxman, Intel’s general manager of high-density servers, told The Register that in the next few years, about one-quarter of all chip sales will go to so-called “mega data centers.”

Currently, that number is at about 10%, but Waxman predicts it growing as “the world continues to embrace distributed grid cloud architectures from the net’s biggest names,” according to the story.

It’s important to note what Waxman considers a “mega data center:” Google, Amazon, Microsoft, “but also telcos doing hosting like AT&T and Verizon.” He further defined it as companies purchasing thousands of machines a month and putting them into megawatt data centers.

As the story points out, this growth only occurs if cloud computing takes hold and grows. Currently Google and Microsoft have put the brakes on data center construction, for data centers largely thought to be cloud computing-based.

Will Facebook-style features increase value and accuracy in CMDBs?

The value of a configuration management database (CMDB) is proportionally dependent on the level of involvement from the IT staff using the tool. Two of the biggest challenges of a successful CMDB implementation are propagating the configuration items and keeping the thing up to date.

If data center managers could get employees to spend as much time updating server configurations as they do updating their Facebook status, the accuracy and immediacy of the tool would be a huge boon.

This is the concept behind Novell’s myCMDB, a software layer interfacing between the CMDB and the users that purports to bring social networking aspects to CMDB data use. Today Novell announced the rebranded myCMDB, from Novell’s acquisition of Managed Objects in October 2008.

The tool is designed for companies that have homegrown CMDBs, built by users on MySQL or Sybase, but will also work on CMDB offerings from HP, BMC and IBM. According to Peter O’Neill, Research Vice President at Forrester, around half of the existing CMDBs in production are homegrown, and “limited in their reporting and visibility outside of the team that created it.”

Web 2.0, you know: Wikipedia, Facebook, del.icio.us and us!
During my conversation with Siki Giunta, former CEO of managed objects and Richard Whitehead, director of marketing for data center solutions with Novell, they spent a lot of our briefing comparing the product to Facebook and Wikipedia.

“Incorporating Web 2.0 in the myCMDB design allows a CMDB to propagate faster, drives more adoption and improves the quality of the data,” Giunta said. “Wikipedia is a huge database, contributed by the end user, federated by news sources. A CMDB is created by people, federated by HelpDesk. Why do people go to Wikipedia? They feel that they can contribute.”

Giunta said myCMDB uses inboxes, RSS feeds, and the atmosphere and look and feel of Facebook. It also features “Google-like” search, and for social bookmarking, myCMDB took a page from del.icio.us. “When you’re navigating this data, it’s easy to lose your place.”

Are these just marketing buzzwords, or are there real “Web 2.0” attributes to this product?

“Sure, there are functional comparisons to be made,” said Michael Coté, an analyst with Redmonk. “The emphasis on including people’s profiles and activity streams is the most relevant. They’re also trying to pull the community and sharing aspects you’d expect to see in consumer apps. These collaborative IT management features, like being able to share different reports or views in myCMDB, are pulled from the Web 2.0 world.”

O’Neill agreed. “The release does provide CMDB insight and reporting in a very modern mode (“Web 2.0” being the metaphor for that) – much more than any other provider,” he said. “This new style is being increasingly adopted, and preferred in businesses. One of the reasons for the adoption of software-as-a-service solutions is their modern user interface.”

But the question remains, is an updated Facebook-like user interface (UI) enough to encourage employees to spend more time on the CMDB, thereby utilizing it more and also keeping it up to date and useful?

“Definitely. The IT management space has a chronic case of terrible UI syndrome. I often consider a fresh, well done UI that matches current trends in UI and usability a self standing feature on its own,” Coté said. “While myCMDB has a nice looking UI, the thing that will make the difference with it is getting users to interact with the system and build up the ‘content’ in it.”

Giunta said this may be the only way to get the next generation of IT administrators to interact with systems management tools in the future. “In the IT operations side, if you keep maintaining old consoles, all the kids will go to work on the application side of the house and we’ll end up with only old people in the data centers,” she said.