“Big data and how best to utilise it has become a perennial subject,” asserts David Senior, CEO of Lowdownapp Ltd, “and the endless debates around it rarely come to a satisfactory conclusion.” You still read a lot of hype about big data as well as how big data has failed to live up to that hype. Senior believes one reason for this discontent is that big data arrived ahead of the technology that was needed to properly analyze it. “It is technology that will unlock the benefits,” he writes, “and narrow AI (artificial intelligence) in particular that will bring together big data and other sources of information to create large and informative data pictures.” Kevin Kelly (@kevin2kelly), the founding executive editor of Wired magazine, agrees with Senior that artificial intelligence is the key that will unlock the future. He explains:
“Like all utilities, AI will be supremely boring, even as it transforms the Internet, the global economy, and civilization. It will enliven inert objects, much as electricity did more than a century ago. Everything that we formerly electrified we will now cognitize. This new utilitarian AI will also augment us individually as people (deepening our memory, speeding our recognition) and collectively as a species. There is almost nothing we can think of that cannot be made new, different, or interesting by infusing it with some extra IQ.”
David Schatsky (@dschatsky), an analyst with Deloitte, writes, “If ‘artificial intelligence’ sounds like science fiction or hype to you, it’s time to get better informed.” He adds, “Artificial intelligence is the field that has given us cognitive technologies such as computer vision, speech recognition, machine learning, and automated reasoning. These technologies seem to get more powerful every day.” As President/CEO of a cognitive computing company, I can attest to how quickly the field is maturing. New algorithms are constantly being developed to help academics, governments, and businesses unlock the potential of big data. That’s why Peter Sondergaard (@PeterSonderg), Senior Vice President & Global Head of Research at Gartner, insists we are entering into an Algorithm Economy. He explains:
“Big data is the oil of the 21st century. But for all of its value, data is inherently dumb. It doesn’t actually do anything unless you know how to use it. Oil is useless thick goop until it’s refined into fuel. Big data’s version of refined fuel – proprietary algorithms that solve specific problems that translate into actions – will be the secret sauce of successful organizations in the future. The next digital gold rush will be focused on how you do something with data, not just what you do with it. This is the promise of the algorithm economy. … Where does this ultimately lead? Software that thinks and does. Cognitive software that drives autonomous machine-to-machine interactions. Artificial intelligence.”
Humans are good at pattern recognition. Pattern recognition helps us make many of the quick decisions needed to get through each day. MIT’s Larry Hardesty observes, however, that artificial intelligence is generally as good as humans when it comes to pattern recognition; but, AI systems are much, much faster. He elaborates:
“Big-data analysis consists of searching for buried patterns that have some kind of predictive power. But choosing which ‘features’ of the data to analyze usually requires some human intuition. In a database containing, say, the beginning and end dates of various sales promotions and weekly profits, the crucial data may not be the dates themselves but the spans between them, or not the total profits but the averages across those spans. MIT researchers aim to take the human element out of big-data analysis, with a new system that not only searches for patterns but designs the feature set, too. To test the first prototype of their system, they enrolled it in three data science competitions, in which it competed against human teams to find predictive patterns in unfamiliar data sets. Of the 906 teams participating in the three competitions, the researchers’ ‘Data Science Machine’ finished ahead of 615. In two of the three competitions, the predictions made by the Data Science Machine were 94 percent and 96 percent as accurate as the winning submissions. In the third, the figure was a more modest 87 percent. But where the teams of humans typically labored over their prediction algorithms for months, the Data Science Machine took somewhere between two and 12 hours to produce each of its entries.”
As Senior noted, these results are being achieved by narrow AI systems (i.e., systems designed to perform specific functions); they are not designed for artificial general intelligence (i.e., systems that become self-aware). That’s why Christopher Mims (@mims) asserts, “The age of intelligent machines has arrived, only they don’t look at all like we expected.” He adds, “Forget what you’ve seen in movies; this is no HAL from ‘2001: A Space Odyssey. … It’s more akin to what happens when insects, or even fungi, do when they ‘think.’ (What, you didn’t know that slime molds can solve mazes?) Artificial intelligence has lately been transformed from an academic curiosity to something that has measurable impact on our lives.” Former IBMer Irving Wladawsky-Berger observes, “Our increasingly smart machines are now being applied to activities that not long ago were viewed as the exclusive domain of humans. Cognitive computing, robotics, self-driving cars and AI in general are some of the hottest topics in academia and business these days.” Wladawsky-Berger agrees with the other pundits cited above that AI is going to have a measurable and positive impact on our lives in the years ahead. “Given that in the foreseeable future we should expect even more powerful parallel computing, lots more data, and deeper algorithms,” he writes, “AI will keep improving and become an increasingly important part of everyday life.”
 David Senior, “How artificial intelligence is the key to unlocking big data,” TechRadar, 22 September 2014.
 Kevin Kelly, “The Three Breakthroughs That Have Finally Unleashed AI on the World,” Wired, 27 October 2014.
 David Schatsky, “It’s Time to Get Informed About Artificial Intelligence,” Huffington Post The Blog, 9 October 2015.
 Peter Sondergaard, “Big Data Fades to the Algorithm Economy,” Forbes, 14 August 2015.
 Larry Hardesty, “Automating big-data analysis,” MIT News, 16 October 2015.
 Christopher Mims, “Artificial Intelligence Comes to Your Inbox,” The Wall Street Journal, 25 August 2014.
 Irving Wladawsky-Berger, “The Future of AI: An Ubiquitous, Invisible, Smart Utility,” The Wall Street Journal, 21 November 2014.