By Decade's End, Big Data Will Be Bigger Than Ever

Specialized tech knowledge will be necessary in our data-driven society.

By + More

But the data industry is well into the build-it-and-they-will-come phase. The data is being built and the users are coming. The tools to use it, whether designed for the non-tech user or IT engineers, are likely to follow.

"It's all about data," says Adam Honoré, research director at the Aite Group, a Boston-based consultancy specializing in information technology research in the financial services industry. "We need people who can move it, transform it, and correlate it. There will be large volumes of global market data, coupled with a myriad of additional data sources like news, social media, and by 2020, audio and video. We also need people who can apply semantics to it and make it all machine-readable for both risk management and alpha generation," a solution used by algorithmic trading to develop quantitative trading models.

Honoré also cites the need for visualization specialists. They typically work as graphic designers, but their job functions often bleed into analytics and research as they look to improve the way data is represented. "The securities industry, as one example, can't keep adding screens to a traders' desktop in response to the data torrent they get," says Honoré, "although tablets are changing the way people react to data."

And integration specialists have an important role. "Managing complex integrations between global venues and market participants is going to require a higher skill set than currently supported," he says. "Further, as more unstructured information becomes part of the trading mainstream, multilingual skills will be important for semantic analysis."

The demand for hard computer skills is trickling down to non-technical job descriptions that cross industries—including financial services, pharmaceuticals, and manufacturing.

[See The New Concerns of an Evolving Workforce.]

"I'm constantly asked to recruit people who understand both marketing and IT," says Kathleen Shelby, director of recruiting at Montvale, N.J.-based FlexTime Solutions, a management search firm specializing in marketing, communications, and design.

"We deal a lot with the pharmaceutical industry now and training materials are primarily interactive," she says. "The relationship between content developers—people writing the words and producing graphics—and Web design, even coding up a website is becoming tighter." She adds, "Everybody wants something in the way of an app for the iPad. We can't find enough developers."

In general, there have been gaps in the move to automation. Insurance carriers, including industry leader Allstate, have been able to estimate which of their customers are likely to make a claim using predictive analytics, a variety of statistical techniques from computer modeling, data mining, and game theory that analyze current events and historical facts to make predictions about the future. Claims adjusters may bring handheld devices to the scene of an accident, but that doesn't prevent information from being transferred to a paper claim form.

But one thing is clear: The hybrid of half-automated, half-manual processing will go the way of the Pontiac and eight-track tapes. Data is the medium of exchange and for it to flow most efficiently, it is being digitized.