“There is such a preponderance of business and personal data whirling around the globe every millisecond,” writes Chris Preimesberger (@editingwhiz), Editor of Features & Analysis at eWEEK, “a zillion more enterprises are paying attention to the potential value all this data can bring to balance sheets around the world.”[1] Given Preimesberger’s assertion, there is little wonder that a number of predictions have been made about how the big data sector will unfold during this coming year. Neil Mendelson (@neilmendelson), Oracle’s vice president of Big Data and Advanced Analytics, was the Oracle executive brave enough to offer his predictions for Preimesberger. In a subsequent article by Preimesberger, he reports on several more big data predictions offered by James Richardson, a business analytics strategist at Qlik.[2] A couple of other pundits who have offered up their thoughts are Neil Biehn (@neilbiehn), vice president of science & research at big data software company PROS, and Dr. Rado Kotorov (@rado_kotorov), Chief Innovation Officer at Information Builders. I’ll use the Mendelson’s predictions as the focus of this article and will supplement them with predictions from the other pundits as appropriate.
Prediction 1: Corporate boardrooms will talk about data capital, not big data. “Data is as necessary for creating new products, services and ways of working as financial capital,” Mendelson stated. “For CEOs, this means securing access to, and increasing use of, data capital by digitizing and datafying key activities with customers, suppliers and partners before rivals do. For CIOs, this means providing data liquidity: the ability to get data the firm wants into the shape it needs with minimal time, cost and risk.” Accenture refers to this as the data supply chain. Biehn agrees. “‘Big data, becomes a term of the past,” he writes.[3] He adds:
“In 2015, we’ll see the buzz term ‘big data’ significantly erode and begin to vanish. In contrast, we’ll see increased focus on the hidden assets found in data using predictive and prescriptive analytics. It’s these analytics that provide actual business value and help companies make easier, faster and smarter decisions about how to better engage with customers and drive top-line revenue. CEOs are looking not for more data, but how they can connect data with predictive and prescriptive insights to capture strategic business value resident in their systems. … If you’re not using big data analytics you’re already behind the competitive curve. The amount of data in corporate infrastructures grows every year both in volume and complexity. In 2015, if companies aren’t using prescriptive and predictive analytics to take advantage of the strategic assets resident in their vast array of systems, they’re losing their ability to compete and win. The art of doing business grows in complexity every year, and prescriptive and predictive analytics are now table stakes.”
Frankly, I’m not sure how quickly the term big data will disappear. Next year seems a bit quick. Admittedly, however, big data will only get bigger and at some point it becomes so gargantuan that “big” simply doesn’t do it justice and the term will disappear as the type of analytics being applied takes it place in most discussions.
Prediction 2: Big data management will grow up. “Hadoop and NoSQL will graduate from mostly experimental pilots to standard components of enterprise data management, taking their place alongside relational databases,” asserts Mendelson. “Over the course of the year, early majority firms will settle on the best roles for each of these foundational components. The demand for data liquidity will compel architects to find new ways to make the full big data environment — Hadoop, NoSQL, and relational technologies — act as a mature enterprise-grade system.” As big data management grows up the demand for data scientists increases. Kotorov adds, “Despite the benefits of driving BI and analytics use across an organisation, the role of the analyst and data scientist is still of great importance. There is high demand for these roles but unfortunately not the supply to meet it.”[4] He continues:
“In the next year, I believe we’re going to see the role of the Chief Data Officer (CDO) become essential as data is recognized as the most important asset in the enterprise today. We’ll also see the Chief Analytics Officer (CAO) role grow in response to the need to analyze trends. The challenge here is to find people with business acumen, not just the technical skills.”
Prediction 3: Companies will demand a SQL for all seasons. “SQL is not just a technology standard,” writes Mendelson. “It’s a language based on 100 years of hard thinking about how to think straight about data. Applications, analysts, and algorithms rely on it daily to run everything from fraud analyses to freight forwarding. Companies will demand that SQL works with all big data, not just data in a Hadoop, NoSQL (Oh, the irony!), or relational silo. They’ll also demand that this big data SQL works just like full-fledged modern SQL that their applications and developers already use. This will put pressure on nascent Hadoop-only SQL to mature overnight.”
Prediction 4: Just-in-time transformation will transform ETL. “New in-memory streaming technologies change the rate at which we can act on data,” Mendelson writes, “causing a re-examination of extract, transform, and load (ETL) activities. Data scientists will increasingly opt for real-time data replication tools instead of batch-oriented ones to get data into Hadoop, which has been the norm. They’ll also take advantage of distributed in-memory processing to make data transformation fast enough to support interactive exploration, creating new data combinations on the fly.” Speed continues to grow in importance for many business functions. As the Internet of Things (IoT) matures, the need for real-time (or near-real-time) analysis will only increase. Richardson agrees. He predicts, “Real-time interaction with BI will become a requirement. … The speed of business has accelerated, and IT systems must keep pace. As analytics becomes part of the standard operating procedure, users rely more and more on speed to drive fast and agile business decisions. For example, retailers who once had two major fashion seasons a year are now being pushed to design and distribute new lines each week to keep up with new trends.”
Prediction 5: Self-service discovery and visualization tools will come to big data. “New data discovery and visualization tools will help people with expertise in the business, but not in technology use big data in daily decisions,” Mendelson writes. “Much of this data will come from outside the firm and, therefore, beyond the careful curation of enterprise data policies. To simplify this complexity, these new technologies will combine consumer-grade user experience with sophisticated algorithmic classification, analysis and enrichment under the hood. The result for business users will be easy exploration on big data, such as knowing where the oil is before digging the well.” I agree completely with this prediction. More and more vendor solutions will come with built-in analytic capabilities that permit non-technical business users to get the most from business applications. This will be essential because, as Kotorov noted, there won’t be enough data scientists to go around. Richardson adds, “Inside each organization, users want to be actively engaged with their data; however, they haven’t had the technology to do so. By providing users BI solutions that allow true self-service, they move from passively consuming the data to actively using it to glean important information. We live in a world of data — both at a personal and professional level — and people express themselves through the work they do with it.”
Prediction 6: Security and governance will increase big data innovation. “Many large firms have found their big data pilots shut down by compliance officers concerned about legal or regulatory violations,” Mendelson reports. “This is particularly an issue when creating new data combinations that include customer data. In a twist, firms will find big data experimentation easier to pen up when the data involved is more locked down. This means extending modern security practices such as data masking and redaction to the full big data environment, in addition to the must-haves of access, authorization and auditing.” Privacy is not an issue that is going to disappear even though more consumers appear willing to share data with companies with which they do business. Consumers want perks, but they also want to trust the companies that want them to share their personal information.
Prediction 7: Production workloads blend cloud and on-premises capabilities. “Once companies see enterprise security and governance extended to high-performance cloud environments,” writes Mendelson, “they’ll start to shift workloads around as needed. For example, an auto manufacturer that wants to combine dealer data borne in the cloud with vehicle manufacturing data in an on-premises warehouse may ship the warehouse data to the cloud for transformation and analysis, only to send the results back to the warehouse for real-time querying.” The cloud is changing the virtual business landscape and the movement to the cloud will become an imperative as the IoT matures.
Kotorov predicts that for many of the reasons noted above machine learning will become a necessity. “As with any labor shortage problem,” he writes, “the solution lies in technology and in this case machine learning is the solution. Looking back, Deloitte made some interesting points here in its ‘Analytics Trends 2014’ presentation, that managers had previously steered clear of machine learning for decision-making as there was ‘no hypothesis or human explanation behind them.’ However, now ‘data projects are often moving too quickly for traditional hypothesis-driven analytics.’ This explains why businesses are embracing machine learning to help them deal with the large volumes of multi-dimensional data that they have access to.” Accenture believes that the ultimate solution is cognitive computing, which is a step beyond machine learning. Biehn offers a prediction with which all of the above pundits would probably agree, “If you’re not using big data analytics you’re already behind the competitive curve.”
Footnotes
[1] Chris Preimesberger, “Why 2015 Will Be Year of Big Data: Oracle’s Seven Predictions,” eWeek, 16 December 2014.
[2] Chris Preimesberger, “Trends to Expect in Business Intelligence, Big Data in 2015,” eWeek, 30 December 2014.
[3] Neil Biehn, [“Big Data predictions for 2015,” BetaNews, 22 December 2014]
[4] Rado Kotorov, “10 things that will happen in analytics in 2015,” posted by Chloe Green, InformationAge, 22 December 2014.
