Big data image via Shutterstock
By James Kobielus (@jameskobielus)
Big data thrives on data scientists who use their expertise in statistical modeling and business analytics to find meaningful patterns in data sets that can grow extraordinarily complex.
Organizations everywhere are stepping up their hiring, recruitment, and training of data scientists. But there seems to be a growing alarm that we won’t have enough data scientists to go around. Will the big data revolution screech to a halt due to a shortage of data scientists?
Don’t worry. Any talent gaps that may exist are likely to be short term and ephemeral. Data science is not an elite priesthood of quants, geeks, and Ph.Ds, but rather, a set of skills and practices that are being nurtured and cultivated everywhere in the business world, not just in academia. The core data-scientist aptitudes–curiosity, intellectual agility, statistical fluency, research stamina, scientific rigor, skeptical nature–are widely distributed throughout the population.
The following trends explain why the supply of data science talent in the business world will continue to expand and the productivity of data scientists will continue to improve over the foreseeable future:
- Data scientists are automating more of their tedious data discovery, acquisition, and preparation tasks through sophisticated tools (e.g., as IBM InfoSphere Server, IBM SPSS Modeler).
- Data scientists are developing fewer models from scratch, because more big data projects run on application-embedded analytic models integrated into commercial solutions, such as IBM Unica.
- Data scientists are being trained within enterprise centers of excellence, which provide forums and resources for long-time business analytics professionals to enhance their skills in hot new areas such as text mining, graph modeling, and behavioral analytics.
- Data scientists are offering their services to the business market through professional services organizations such as IBM’s Business Analytics and Optimization group; through specialized, boutique consulting firms; and through the open-source communities.
- Data scientists are taking advantage of increasingly low-cost, self-service analytics power tools to teach themselves the core skills and become productive developers of sophisticated models
Data science is a highly skilled profession with a significant learning curve. But throngs of smart people are flocking to it in ever greater numbers to advance their careers in the age of big data. Rather than lose sleep over some overblown threat of a chronic “talent gap,” you should simply cultivate the right data scientists for your specific needs.