An overview of a report by Markets and Markets states, “Big Data has grown in significance over the last few years because of the pervasiveness of its application, across areas ranging from weather forecasting to analyzing business trends, fighting crime, and preventing epidemics, etc.” Big Data’s significance is also translating into big bucks. The overview reports, “The Hadoop market in 2012 is worth $1.5 billion and is expected to grow to about $13.9 billion by 2017, at a CAGR of 54.9% from 2012 to 2017.” Those are big numbers and there has been a lot of discussion about whether Big Data analytics have been over-hyped. The numbers indicate that business executives think not and are demonstrating that belief by opening their corporate wallets. Nevertheless, there are still naysayers. For example, Gary Drenik believes that there is a “pending big data disillusionment which may result from the hype and failure of big data to deliver meaningful, strategic solutions for senior level executives.” [“Going Beyond Big Data To Knowledge,” Forbes, 11 March 2014]
Drenik’s real concern (as the headline of his article reveals) is that the analytical processes applied to Big Data sets won’t really provide the kind of value that vendors are promising. He doesn’t argue that Big Data analytics aren’t worth pursuing if done correctly. He explains, “It’s clear that most of the big data commotion gets down to media applications based upon questionable models. Maybe that is why a recent Gartner survey found that through 2015, 85% of Fortune 500’s will fail to exploit big data for competitive advantage.” He continues:
“Data is the starting point and basic building block in a knowledge-based organization. … Strategy requires a broader view of data. Strategy requires data that serves as fuel, but logic and experience still need to be applied to generate knowledge-based systems. Knowing not only what happened, but why it happened (diagnostic), what will happen (predictive) and how we can make it happen (prescriptive) is important for moving beyond big data to knowledge.”
Drenik is CEO of a company called Prosper Insights & Analytics and he goes on to tout the Big Data products provided by his firm. I can’t fault him for that. As President & CEO of a company that also develops Big Data solutions, Enterra Solutions®, I often use this blog to explain how our products can benefit businesses. As CEOs of Big Data firms, we understand that at the end of the day what executives need is actionable insights that can improve their company’s bottom line. As Drenik notes, good Big Data analysis looks back, looks forward, and provides valuable insights. David Smith, a self-proclaimed data scientist, asserts that, before the advent of Big Data analytics, companies relied on business intelligence. The problem with business intelligence, Smith notes, is that it is really backward looking. On the other hand, data science can look forward as well as backward. [“Statistics vs. Data Science vs. BI,” SmartData Collective, 17 May 2013] He provided the following chart to make his point.
Maria Deutscher writes, “Data holds the answers for many of the critical questions faced by decision makers today, with the quality of the query determining the accuracy the result.” [“An Insider Take on Using Analytics to Ask Big Questions,” Silicon Angle, 22 October 2013] The most important thing that determines whether actionable insights are provided is the quality of the questions that are asked. The best data scientists and analysts ask the best questions (for more on this topic, read my post entitled “Big Data Analytics: Good Questions Result in Better Answers“). With the advent of cognitive computing systems, like Enterra’s Cognitive Reasoning Platform™ (CRP), artificial intelligence processes can help frame questions and even develop questions on their own. That’s important because, as Deutscher notes, “Many CIOs are recognizing that the ability to ask the right questions can be transformative for their organizations, but the shortage of skilled data scientists is proving to be a barrier to insights for all but the biggest enterprises.”
One of the reasons that some critics believe Big Data is overhyped is because it is too often discussed on the macro-level (i.e., divorced from any real world situation or circumstance). McKinsey & Company analysts, Brad Brown, David Court, and Tim McGuire, note, “The reality of where and how data analytics can improve performance varies dramatically by company and industry.” [“Views from the front lines of the data-analytics revolution,” Telecom, Media, & High Tech Extranet, 12 March 2014 (registration required)] They go on to explain some the various kinds of Big Data applications that are useful.
“Customer-facing activities. In some industries, such as telecommunications, this is where the greatest opportunities lie. Here, companies benefit most when they focus on analytics models that optimize pricing of services across consumer life cycles, maximize marketing spending by predicting areas where product promotions will be most effective, and identify tactics for customer retention.
“Internal applications. In other industries, such as transportation services, models will focus on process efficiencies — optimizing routes, for example, or scheduling crews given variations in worker availability and demand.
“Hybrid applications. Other industries need a balance of both. Retailers, for example, can harness data to influence next-product-to-buy decisions and to optimize location choices for new stores or to map product flows through supply chains. Insurers, similarly, want to predict features that will help them extend product lines and assess emerging areas of portfolio risk. Establishing priorities wisely and with a realistic sense of the associated challenges lies at the heart of a successful data-analytics strategy. Companies need to operate along two horizons: capturing quick wins to build momentum while keeping sight of longer-term, ground-breaking applications. …
“New opportunities will continue to open up. For example, there was a growing awareness, among participants, of the potential of tapping swelling reservoirs of external data — sometimes known as open data — and combining them with existing proprietary data to improve models and business outcomes.”
Curt Monash asserts, “It’s hard to make data easy to analyze.” [DBMS2, 13 February 2013] He continues, “There are many ways to help with preparing data for analysis. Some of them are well-served by the industry. Some, however, are not.” Since good analysis begins with data, you need to ensure that it is prepared correctly for analysis. Greg Satell adds, “We live in a disruptive age, in which the average lifespan of a Fortune 500 company is plummeting while new businesses seemingly spring out of nowhere and become multibillion dollar businesses almost overnight. … As the big data revolution gets underway, it’s becoming clear that the gap between firms will be not only of skills and investment, but of mindset.” His conclusion, “Companies That Can’t Figure Out Data Are Getting Left Behind.” [Business Insider, 25 August 2013] According to an IDC Retail Insights study, there could be a lot of companies left behind. The study concludes, “Big data and analytics have become top priorities for a growing number of retail executives. Yet many retailers don’t yet have mature big data and analytics competencies across five critical dimensions — intent, people, process, technology and data.” [“Retailers Lack Mature Big Data and Analytics Competencies,” Information Management, 14 February 2014]
The mountains of data being collected are only going to continue to grow. There are valuable insights that can be gleaned from that data and can improve everything from business operations to marketing. As the above discussion notes, getting to those insights is not always easy but obtaining them can make your business more competitive. If Greg Satell is correct, the very survival of your business may depend on mastering Big Data analytics.
