The concept of Big Data — the practice of acquiring, analysing and interpreting ridiculously huge data sets — is something much of the technology and business world is extremely excited about. But excited is about as far as it goes because, currently, there just aren’t enough practitioners to make it work.
While there’s much fuss made about the endless possibilities of what can be done with the streams of data garnered from the web, the hard truth is that few people know what to do with it. The Wall Street Journal points to a report by McKinsey, which explains:
“A significant constraint on realising value from Big Data will be a shortage of talent, particularly of people with deep expertise in statistics and machine learning, and the managers and analysts who know how to operate companies by using insights from Big Data. We project a need for 1.5 million additional managers and analysts in the United States who can ask the right questions and consume the results of the analysis of Big Data effectively.”
But finding those people might not be very easy. Hilary Mason — inspirational chief scientists for bit.ly, one of Fortune’s 40 Under 40 to Watch and one of the leading proponents of Big Data — explains to the Wall Street Journal that people who work successfully in the sector have a rare combination of skills:
“They can take a data set and model it mathematically and understand the maths required to build those models; they can actually do that, which means they have the engineering skills…and finally they are someone who can find insights and tell stories from their data. That means asking the right questions, and that is usually the hardest piece.”
The way to fix that? As with any emerging field, there needs to be a change in the way we think about the sector and the way we train people to enter it. Pat Gelsinger, president and chief operating officer of EMC Corphe, a giant US data company, explained to the Journal:
“IBM started a generation of Cobol programmers. 30 years ago we didn’t have computer-science departments; now every quality school on the planet has a CS department. Now nobody has a data-science department; in 30 years every school on the planet will have one.”
That may not be all that’s required, of course, but with the concept of Big Data refusing to go away, we’ll have to work the rest out as we go. [Wall Street Journal]
Image: ARTSILENSEcom/Shutterstock