Enterprise IT Watch Blog

Jan 5 2015   4:33PM GMT

Shining analytical light deeply into dark data

Michael Tidmarsh Michael Tidmarsh Profile: Michael Tidmarsh

Dark data

Dark data image via Shutterstock

By James Kobielus (@jameskobielus)

The typical organization has more information than they know what to do with, a phenomenon that many refer to as “dark data.”

This recent article is typical of how this topic is often discussed. Author Ed Tittel alludes to the potential of “dark data” as a business resource. However, he comes down primarily on the “dark threats” side of the equation. “[E]ven untapped and neglected,” he states, “dark data can pose security risks should it fall into the wrong hands, or range outside its owner’s control.”

To Tittel’s credit, he provides a comprehensive discussion of the risks of failing to protect this “dark” resource, examining the legal, regulatory, competitive, and reputational downsides. But he seems to be compelled by this risk-centric approach to spin a negative from what would otherwise be a positive: the data’s potential business utility. If the data were left exposed, he states, “given that the organization has decided not to invest in analysis and mining of dark data by definition, concerted efforts by third parties to exploit its value represent potential losses of intelligence and value based upon its contents.”

That’s a convoluted way of expressing a no-brainer: you’ve chosen to leave your data “dark” because you lack a business case strong enough to justify shining the proverbial analytical light on it. Lacking a viable business-value narrative for that data, you’ve forced yourself into a defensive posture: keeping it out of others’ hands lest they leverage it for useful purposes the likes of which you’ve failed to identify.

This represents a vicious circle of cost and risk. The data-protection burden becomes the perennial “dark” cloud of cost overhead enshrouding your “dark” data resource. The way to end the vicious circle is by ridding yourself of the notion that any data is “dark.” Whenever it sticks its shadowy head into the picture, that term puts a risk spin on what might otherwise be a business-value discussion. You should regard this data as simply an undiscovered resource. It can yield value if you make the requisite investments in tools, platforms, personnel, and business applications.

Of course, nothing can stop smart people from using any analytical tool and correlating any data at their disposal to make brilliant inferences. If the core data they covet remains “dark,” smart people (e.g., data scientists) will use all their ingenuity and tooling to infer what they’re lacking, perhaps with a high degree of confidence.

This is the essence of sleuthing, which is a skill that many people are using advanced analytics to accelerate. Dark patterns have a way of revealing themselves to the determined mind.

 Comment on this Post

There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: