Open Source Insider

Jun 2 2014   9:25AM GMT

How to make Hadoop logfile analysis fun & interesting

Adrian Bridgwater Adrian Bridgwater Profile: Adrian Bridgwater

Tags:
DevOps
Hadoop
logfile
logs
Testing

21e12devwev.jpgThe heady world of server log analysis kicks up another gear this week with new products proffered forth from XpoLog Ltd, the company that says it “invented” augmented search for IT log analysis.

What’s the augmentation and why is it interesting?

We’re glad you asked — this is a process by which DevOps professionals can discover hidden:

• errors,

• anomalies,

• problems and,

• patterns…

… that may be hiding inside IT log data.

What is IT log data or a logfile?

This is a file that records “events” occurring throughout an operating system or software application (in the case of XpoLog this software is focused on open source Hadoop) and it can also incorporate message information between users in a network — a list of other log types can be found here if you need bedtime reading.

Augmented auto-detected intelligence

6y5tgrefvv.jpg

XpoLog is aiming to bring more to the party than plain old log searches and now offers what it calls search with “augmented auto-detected intelligence” based on user search context.

We digress, back to the news… XpoLog Augmented Search 5.0 brings XpoLog’s troubleshooting capabilities to the Hadoop platform.

The company states the obvious but still pertinent factor at play here i.e. testing applications on Hadoop (a large-scale, distributed data processing platform) isn’t a trivial task.

Pattern and anomaly detection

“XpoLog adds intelligence to log file search context with semantic analysis, and pattern and anomaly detection (to uncover insights and trends into application problems, systems, and user behaviour),” said XpoLog VP Solutions Omry Koschitzky.

“This helps users analyze problems within the Hadoop infrastructure and applications that run on the platform. It offers visibility into the distributed architecture, automatically triaging issues and errors for severity, and presenting results in a dashboard interface,” he added.

The company has made the XpoLog platform free for processing up to 1 gigabyte of log data per day.

More fun and interesting than you thought?

1  Comment on this Post

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.
  • Ashish Mohindroo
    Check out a new Cloud Log Monitoring & Management Platform: OohLaLog (http://www.oohlalog.com). It offer broad support for apps, DBs, and systems including Hadoop. With it's non-intrusive architecture along with features like custom alerts and counters, devOPs and Developers can now discover & resolve issues 75% faster with zero impact on their production applications. Try it for Free! http://www.oohlalog.com
    0 pointsBadges:
    report

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: