“The world around us is changing so rapidly,” writes Datafloq founder Mark van Rijmenam (@VanRijmenam), “that even the hype big data is already outdated. Big data is nothing new anymore and by now we all know that more data is coming our way, rapidly.”[1] Daniel Fallman, CEO of Mindbreeze agrees. “Big data is currently on the fast track,” he writes. “The rapid ongoing developments are not showing any signs of slowing down.”[2] You can sense an urgency in tone from both van Rijmenam and Fallman. Terms like “rapidly” and “fast track” signal there is little time to get on board the big data train before your company finds itself sidetracked or run over. Ashish Thusoo (@ashishthusoo), CEO and co-founder of Qubole, bluntly states, “Companies that are not capitalizing on data analytics will start to go out of business, and the enterprise will realize that the key to staying in business and growing the business is data refinement and predictive analytics.”[3] If big data is really such a big deal, then knowing where big data is heading in the years to come is important for every company. Following are some of the trends that analysts have identified in the big data arena.

Internet of Things (IoT)

Although the Internet of Things has been referred to by various names (e.g., the Industrial Internet, the Internet of Everything, and the Internet of Anything), most pundits agree that a new era of connectivity is about to emerge as more and more “things” get connected. Hortonworks Chief Technology Officer Scott Gnau (@Scott_Gnau) predicts, “Next year businesses will look at deriving value from ALL data. It’s not just the Internet of Things but rather Internet of Anything that can provide insights. Getting value from data extends beyond devices, sensors and machines and includes ALL data — including that produced by server logs, geo location and data from the Internet.”[4] Thusoo adds, “With the ever increasing amount of data produced through connected environments and apps, the need for big data and analytics is just getting more and more pronounced. As this market transitions to the mainstream, there is a need to simplify the process of unlocking big data insights. … Cloud-based big data services are driving a lot of this, and increasingly making big data accessible in a simplified manner to the mainstream market. This trend will keep accelerating. We will also see an increased focus on vertical analytical applications in industries, such as the healthcare sector, that will further simplify the usage of big data and its overall adoption.” One prediction I know will come true is that, as the IoT grows, the term “big data” will be inadequate to describe the oceans of data being generated. Mark Clayman, President and COO of TriCore Solutions, notes the IoT still has a long way to go before it reaches its full potential. “It is clear that organizations understand the value of IoT data,” he writes, “but it is unclear how to actually get from collecting the data to deriving value from the data. Until that’s resolved, IoT projects may prove more frustrating than useful. There are still some major challenges to make the Internet of Things a reality.”[5]

Artificial Intelligence and Advanced Analytics

Roxanne Alexander predicts, “Artificial intelligence and advanced analytics will shape the future of business.”[6] Many analysts are even more specific and point to cognitive computing as the AI field that will have the greatest impact. For example, Accenture analysts state that cognitive computing will provide the “ultimate long-term solution” for many business challenges.[7] And Jorge Garcia (@jgptec), Senior BI and Data Management Analyst at Technology Evaluation Centers, writes, “Machine learning, along with many other disciplines within the field of artificial intelligence and cognitive systems, is gaining popularity, and it may in the not so distant future have a colossal impact on the software industry.”[8]

Data Scientist Scarcity

Fallman notes, “Big data systems need to be planned and implemented by specialists who have a deep understanding of the business processes and the core business. These experts, ‘data scientists’, are a pleasantly exotic mixture of mathematicians, statisticians and computer scientists.” This need for data specialists could be a problem. Carlton Doty, an analyst at Forrester Research, along with his colleagues Brian Hopkins, Jennifer Belissent, predict, “Firms will try to come to terms with data science scarcity. Two-thirds of firms will have built predictive systems capability by mid-2016, but will struggle to find data science talent.”[9] This is a challenge we’ve anticipated at Enterra Solutions®, and the Enterra Enterprise Cognitive System™ (ECS), a system that can Sense, Think, Act, and Learn®, has embedded expertise in it. Enterra’s approach empowers the business expert by automating the statistical expert’s and data expert’s knowledge and functions, so the ideation cycle can be dramatically shortened and more insights can be auto-generated. Gnau agrees that democratizing data analysis is important. “There is a market need to simplify big data technologies,” he writes, “and opportunities for this exist at all levels: technical, consumption, etc.”

Real-time Analytics

Doty and his colleagues also predict, “Demand for real-time customer engagement will test streaming technology. Traditionally, marketers use analytics retrospectively to analyze highly structured, at-rest customer and campaign data from customer databases. But as marketing technologists try to build contextual marketing engines, they will need to apply analytics as customer interactions occur in order to drive deeper engagement. To succeed, CI pros must prioritize in-motion data and time-sequenced analysis.”

Moving from Business Intelligence to Insights

According to Doty and his colleagues, “Only a few elite teams will take the leap from BI to systems of insight. Only a few BI teams are taking baby steps toward agile BI, and Forrester expects that in 2016, less than a third of these will be ready to make the leap to systems of insight. Why? Because BI and big data are firmly rooted in the technology while business executives impatiently await results. We see better business outcomes in organizations that merge BI and customer insights teams.” Cognitive computing systems are well-suited to help companies derive actionable insights from big data analytics. Katie Kennedy (@MissKatherineLK), a research analyst at Cognitio Corp, predicts, “As Big Data use cases continue to evolve, there will be an increasing number of business missions that will be served by new approaches to data.”[10] Forrester Vice President and principal analyst Brian Hopkins (@practicingEA), writes, “Closing the gap between insight and action is the big unfilled hole we found in 2015. In 2016, that gap will be hard to close for all but the most advanced leaders. However, by the end of 2016, energy around big data will be substantially redirected towards insights execution.”[11]

Conclusions

Big data analytics is finally poised to deliver on its hype. Kennedy writes, “By mid-2016 we expect to see corporations using enterprise data hub approaches to enhance agility and decision-making that leads to market dominating strategies. Across multiple sectors, we can expect a new corporate differentiator to be how the use of data leads to agility. By the end of 2016, there will be two types of companies: those using big data to dominate – and those being dominated.”

Footnotes
[1] Mark van Rijmenam, “7 Reasons why the Algorithmic Business will Change Society,” Datafloq, 16 November 2015.
[2] Daniel Fallman, “Big Data: Where are we headed?” ITProPortal, 1 November 2015.
[3] David Weldon, “Do Data Big, Or Die in 2016, Warns Big Data Veteran,” Information Management, 5 November 2015.
[4] Ibid.
[5] Mark Clayman, “Big Data Becomes Fast and Accelerated with IoT,” Big Data Quarterly, 22 December 2015.
[6] Roxanne Alexander, “Predictions for Big Data in 2016,” Vivit, 21 October 2015.
[7] “From Digitally Disrupted to Digital Disrupter,” Accenture, 2014.
[8] Jorge Garcia, “Machine Learning and Cognitive Systems: The Next Evolution of Enterprise Intelligence (Part I),” Wired, July 2014.
[9] Carlton Doty, “2016 Predictions: All That Data Will Finally Drive Business Action,” Information Management, 17 November 2015.
[10] Katie Kennedy, “The 2016 Big Data Prediction: Which of these two types of organization will you be in?” CTOvision.com, 1 September 2015.
[11] Brian Hopkins, “Forrester’s 2016 Predictions: Turn Data Into Insight And Action,” Brian Hopkin’s Blog, 9 November 2015.