“Big data, the tech story of a few years ago, is now beginning to show big results,” writes Michael S. Malone. “The science of using powerful computers, ubiquitous sensors and the Web to crunch mountains of raw data to uncover previously invisible insights is increasingly used in businesses, universities and government agencies. It is transforming our understanding of everything from fetal development to cosmology.”[1] On the other hand, Alex Woodie (@alex_woodie) writes, “Over the past five years, the amount of hoopla and hysteria over big data has reached epic proportions at times. That’s why some in the industry are calling for a slowdown in the analytic platitudes, a return to sanity about what big data can really do.”[2] Who’s correct? Are we entering or leaving the Age of Big Data? Those are not easy questions to answer because, as Woodie observes, there has been a lot of hype surrounding big data. In spite of his article’s headline (“Is 2016 the Beginning of End for Big Data?”), Woodie doesn’t believe we are seeing the beginning of the end for big data — maybe just the beginning of the end of labeling advanced analytics with the “big data” modifier. He explains:
“Here’s the thing: … The Internet of Things, is just getting started. We’re just beginning to explore and exploit the potential to squeeze more automation from our stuff, which ostensibly will give us more time to pursue more value-added activities. … When you consider the advances being made in the fields of machine learning, cognitive computing, and artificial intelligence — and how those rapidly evolving technologies will help us to build even smarter devices over the next 10 years — it’s tough not to be excited. That’s not to say there won’t be changes and there won’t be spectacular failures of big data analytics. While the technologies and techniques behind advanced analytics have proven they can deliver a competitive advantage, it is not easy to make use of them. There is still a giant skills gap in the field, and while big data analytic software vendors are adding more automation to their wares, the most innovative applications still require the oversight of a trained and competent data scientist. It’s no surprise that leaders in the field are equally optimistic about the future of big data.”
The Internet of Things (IoT) is going to generate so much data that calling it “big” will sound both quaint and anachronistic. Malone notes, “One of the most extraordinary features of big data is that it signals the end of the reign of statistics. For 400 years, we’ve been forced to sample complex systems and extrapolate. Now, with big data, it is possible to measure everything, from the movement of billions of stars to every beat of the human heart.” Unfortunately, like many buzzwords, big data is likely to linger in our lexicon long after it has any real meaning — much like the term “horseless carriage” lingered well into the age of automobiles. Ben Rossi (@BenRossi89) asserts, “Instead of defining big data as a number or a size, it is more interesting and relevant to define it in terms of history, growth, compute and value.”[3] The reason for that, he notes, is, “As a term, ‘big data’ still has no clear definition. For some, a dataset over a terabyte is big data — for others, it might be a million rows, and others still may have smaller datasets that is changing many times a second.” Rossi explains that the first big data project started, in 1937, at the height of the Great Depression. “At that time,” he reports, “the [Roosevelt] administration in America [was] looking to keep track of social security contributions from some 26 million Americans, and 3 million employers.” IBM’s punch card technology got the job. If data in the millions sounds quaint, terrabytes of data is following suit. “For the first time since its conception,” Rossi explains, “global internet traffic will surpass 1 zettabyte (1 billion terabytes) in 2016, according to a Cisco research paper, having risen fivefold in the past five years.” And the IoT will make that number seem outdated in the years ahead. Kenneth Cukier (@kncukier), Data Editor of The Economist, gave an interesting TED talk about big data that is worth watching. In the video Cukier explains why big data is better data and why there are reasons to be both cautious and optimistic.
Jonathan Vanian (@JonathanVanian) believes we are entering a period during which there will be less hype and more anecdotal evidence about how big data analytics benefits organizations. “Big data is no longer the hot buzzword it was a few years ago that people strained their brains to understand,” he writes. “It’s now entered the mainstream and can be viewed as an extension of traditional data crunching.”[4] If you watched Cukier’s video, you’ll probably agree that describing big data analytics as “an extension of traditional data crunching” is underselling big data’s potential. Rossi notes, “In Gartner’s hype cycle, the term ‘big data’ was once a staple of the yearly report. It moved swiftly into the peak of inflated expectations, weathered its way through the trough of disillusionment, and is now prevalent — somewhere between the slope of enlightenment and the plateau of productivity. Expectations of the technology are high, a Gartner survey in September 2015 showed more that 75% of companies are investing or planning to invest in big data in the next two years, and 37% of those projects are being driven from board level.” Malone touts the value of advanced analytics for society as a whole as well as for business. “Big data,” he writes, “is … making its way, almost silently, into everyday life. … Farmers in the Midwest will soon have measurements taken in their fields nearly down to individual plants — and that will combine data on soil, water and weather to come up with trillions of scenarios for insurance coverage. Victims of natural disasters are now being pinpointed and given aid through real-time analysis of tweets, instant messages and phone calls. Even one of the oldest of arts, geography, is being revolutionized with maps that will change by the second. But it is in the commercial world where big data has seen its greatest inroads to date.”
Tom Phelan (@tapbluedata), co-Founder and Chief Architect of BlueData, agrees with Malone that advanced analytics are having their greatest impact in the business arena. “Over the next year,” he writes, “a growing number of customers will realize the vast business benefits of Big Data and will deploy Big Data solutions across their organization.”[5] Vanian asserts that advanced analytics embedded in cognitive computing systems are likely to replace discussions about big data. He explains:
“The new hot topic is cognitive computing, which generally refers to computer systems built to simulate human thinking and reasoning using artificial intelligence techniques like machine-learning algorithms. Cognitive computing systems can recognize speech, identify objects in pictures, and even learn to adapt to dangerous road conditions if embedded within a self-driving car, for example. If companies are able to use these advanced data crunching techniques to solve traditional business problems, … cognitive computing might eventually be lumped in with how people view traditional data analytics.”
According to Rossi, “It’s clear big data is growing — it’s here to stay and, right up to board level, companies have woken up to its potential. Companies see that with big data analytics they can find insights that enable them to outperform their competitors and reduce costs. With the right tools, these insights are easier to obtain than ever.” I’m not sure I answered the question I asked in this article’s headline — Are we entering or leaving the age of big data? — but that’s because it really has no answer. I agree with Rossi that, whatever you call it, big data analytics are here is to stay and they will be hallmark of the Cognitive Computing Era.[6]
Footnotes
[1] Michael S. Malone, “The Big-Data Future Has Arrived,” The Wall Street Journal, 22 February 2016.
[2] Alex Woodie, “Is 2016 the Beginning of the End for Big Data?,” Datanami, 5 January 2016.
[3] Ben Rossi, “What are the numbers, facts and figures behind big data?” Information Age, 4 March 2016.
[4] Jonathan Vanian, “Has Big Data Gone Mainstream?” Fortune, 13 January 2016.
[5] Tom Phelan, “Eight Things to Watch for in Big Data,” Inside Big Data, 27 February 2016.
[6] Stephen DeAngelis, “The Era of Cognitive Computing,” Enterra Insights, 10 February 2016.
