In this end-of-the-year edition of the Talking Data podcast, Senior Executive Editor Ed Scannell joined me to speak with Mike Matchett, founder and principal analyst of the Small World Big Data consultancy, as we rambled through some of the signal events of big data in 2018.
Mergers and acquisitions, naturally, tend to be the stepping stones when you look back at the path just traveled. Cloudera and Hortonworks, IBM and Red Hat – these deals set the tone for our end-of-year big data ruminations.
But what rises in importance in our podcasters’ ponderings, are not the mergers in their dollar terms but instead the mergers as they reveal the underlying currents and eddies of telling trends. What surfaces?
*Hadoop-centric big data analytics is morphing into machine learning and deep learning analytics.
*It is not that the shortcomings of Hadoop data processing have been solved, however.
*Rather, the vendors have declared victory, and moved on to the next world to conquer – the more mysterious one of AI, machine learning and statistics safely beyond the layperson’s ken. It’s happened before.
*AI is what you do with big data. The Web and cloud have become irresistible honeypots for said data, and the result is that the balance of power – for data, IT and business — is moving to the cloud.
*A long view would say that it’s taken more than 10 years for cloud computing to become an overnight success, and that assorted after effects will play out for some time to come.
In the podcast we talk about the pendulum effect, of which 1990s client/server computing is a ready example. In that case there was a swing away from central IT, which was called “the glass house.” The era saw independent departments within businesses beginning to set their own technology courses.
We see that with cloud today. A pendulum swing has put more technology decision making in the hands of developers within lines of business. They can use credit cards to start projects, and they can get very high-end systems via top cloud providers.
We may look back one day and see things shifting back toward central IT. If so, it will no more resemble today’s central IT than today’s resembles the IT shop of the glass house days.
The recent years have been topsy-turvy – and not just on the big data front. Thanks for hanging with us on the Talking Data podcast. – Jack Vaughan