Many cultures retell the story of a flood that changed the world — the most famous story, of course, is that of Noah and his ark. Brian Deagon writes about a new flood that is changing the world — the flood of data that mounts every day. “Each minute Google gets more than 2 million queries, while about 47,000 people download an Apple app,” he writes. “Some 100,000 tweets hit Twitter, and almost 300,000 people log on to Facebook. But that’s just a sliver of the data produced daily.” [“Big Data Floods Opens Opportunity To Mine Information For Better Results,” Investor’s Business Daily, 3 April 2012] Deagon notes that “last year data produced 1.8 trillion gigabytes, double the amount from 2009. And that will swell 50% in 2012, says EMC.” As Deagon states, “Welcome to the age of Big Data.”
Like other kinds of floods, the flood of data, if uncontrolled, is not a good thing. Deagon reports that “the information pipeline has gotten so fat that commonly used business software chokes on it.” Obviously, that is not a good thing. Another thing worth noting is that unanalyzed “data” is not the same thing as “information” or “knowledge.” Unanalyzed Big Data is about as useful as unrefined crude oil (i.e., it has potential but is really messy). Although the World Economic Forum has declared data a new asset class, the value in that data lies in the insights that can be obtained from it. Businesses want actionable insights that make them more efficient, increase their market share, and maximize profits. Scott Gnau, head of research and development at Teradata, told Deagon, “Now businesses want to know who bought the blouse and what drew them to that store in the first place.”
Deagon points out that there is nothing new about data analysis. Businesses have being using various forms of analysis since the first business lifted the curtain on its stall. Deagon doesn’t go back that far. He points out that “Big Data is a grand extension of earlier tools used to mine data, such as enterprise resource planning and customer relationship management software.” As he points out, ERP and CRM use relational databases. He continues:
“With relational database technology, data are generally organized in rows and columns for fast access to a single point of information. But it’s not adept at developing insights from unstructured data — massive volumes of disparate information from many sources. ‘Newer forms of data are wild and crazy, and the old tools don’t work,’ said Gnau. ‘An entirely new class of technology is being developed to leverage Big Data.’ Consumers now provide tons of information on what they eat, where they visit, what they like and don’t. Much of that comes via social networking sites like Facebook, Twitter and the boom in smartphone Web surfing.”
Obviously the kind of data generated through social media channels is not neatly slotted in boxes defined by rows and columns. That’s why old tools don’t work when businesses want to mine it for insights. Deagon reports, “Businesses spent more than $4 trillion in the past six years to handle the data explosion, says research firm IDC.” By any measure, that’s a lot of money and IT-related companies are scrambling to get into the money stream. Deagon continues:
“The Big Data trend has led to a new surge in spending on technology and services to understand what to make of it. Revenue specifically targeting Big Data is projected to reach $17 billion in 2015 from $3 billion in 2010, says IDC. Much will focus on analytics technology to manipulate, analyze and model complex data to derive intelligent solutions.”
As president/CEO of a company whose business does exactly that, you’d be correct in assuming that I love to hear predictions like that being made. With so much money being thrown at Big Data, Deagon points out that the field will soon be crowded with companies looking to get a piece of the action. IDC analyst Dan Vesset told Deagon that “Big Data startups have attracted about $500 million of venture capital funding.” And Deagon points out that big firms, not just startups, are very active in the field. Deagon concludes:
“Big Data taps into two other big tech trends — cloud computing and virtualization. With cloud computing, data can be dispersed on storage systems worldwide, accessible via the Internet. Virtualization lets a single server function as multiple computers, raising performance at a lower cost.”
Trevor Miles states, “Not too long ago companies suffered from having too little data with which to manage the company’s operations. The ERP age has brought in a different problem of too much data, but too little information.” [“Connecting changes to consequences: The missing piece for getting business value from business intelligence,” The 21st Century Supply Chain, 14 February 2012] As a side note, Miles’ statement also highlights one of the challenges of Big Data — words such as “information” aren’t used consistently. Miles uses word “information” to represent the results of analyzed data, while earlier Deagon used the word “information” to mean unanalyzed data. That’s why using an ontology as part of a Sense, Think, Act, and Learn® system, like we do at Enterra Solutions®, is important. Such a system can consider context and know which meaning is the correct one from the content being analyzed. Since a Sense, Think, Act, and Learn system can, as its name suggests, learn from what it analyzes it has other benefits as well. Miles details the problem with some analytic tools. He writes:
“BI tools suffer from two major drawbacks that prevent them from providing greater value and therefore obtaining greater adoption: They cannot identify causality and, as a consequence, they cannot provide a prediction of future performance and risk. In SCM, without the deep supply chain analytics required to identify causality, BI tools cannot identify future risk based upon the current state of the supply chain. For example, while the information that a receipt of a shipment indicates that 20% of the shipment is damaged is important; the real value comes from being able to identify the orders that are impacted and therefore, being able to evaluate the potential financial consequence. In other words, BI tools do not provide actionable information and therefore fail to address a critical aspect of the day-to-day lives of operational people: knowing what levers to pull to affect change. What has been missing is the predictive analytics to connect changes to consequences, both operational and financial. There is little value in knowing about a problem after it has occurred (or just before it occurs when you do not have time to react.) And knowing that something has changed is also of little value. But knowing the operational and financial consequences or root causes of these changes, and therefore what to work on, has immediate value.”
One of the objectives that Enterra Solutions has pursued in its solutions is doing exactly what Miles suggests (i.e., discovering relationships so that downstream perturbative consequences can be determined and either be prevented or mitigated). I agree with him that good analytics provide actionable intelligence (i.e., knowledge that permits a decision maker to respond within the necessary decision cycle to make a difference). Miles continues:
“Knowing what to do to overcome the risks that these changes present is of greatest value. For example, there may be several orders that will be delayed because of a shortage of one component. This component could be a fairly small part of a company’s overall purchase costs so the shortage may be overlooked using traditional BI tools. But its impact on future finished goods availability may be quite large. Predictive analytics identifies future risks associated with current changes. And once you know what future risks are faced by your company, the natural next step is to find ways of testing the effect or impact of choosing one course of action over another to mitigate these risks.”
That is exactly the kind of solutions we develop at Enterra Solutions. Paul Teague, leading thinker in the procurement world, reports, “Gartner’s recent CIO survey concluded that analytics and business intelligence are the top-ranked technologies for 2012; Consultancy A.T. Kearney has long pushed for increased use of analytics in supply chain management; and the American Management Association has just added a new course on how to improve analytical skills to its list of educational seminars.” [“Got data? Learn how to use it,” Procurement Leaders Blog, 14 February 2012] He continues:
“It’s hard to argue against the importance of analytics, and, in fact, only latter-day technology Luddites do. Virtually everyone agrees that the ability to pull information out of data-collection systems, to slice it and dice it to extract just what you need, and to use it for timely decision making and to predict future events is essential for effective management of any business function. … Analytics, of course is critical in determining total cost of ownership, an exercise that’s essential in outsourcing decisions. But, it also helps in determining which pieces of information you’ve gathered are really relevant, and to reconcile conflicting information.”
Jacques Bughin, John Livingston, and Sam Marwaha conclude, “Large-scale data gathering and analytics are quickly becoming a new frontier of competitive differentiation.” [“Seizing the potential of ‘big data’,” McKinsey Quarterly, October 2011] They conclude, “Too few leaders fully understand big data’s potential in their businesses, the data assets and liabilities of those businesses, or the strategic choices they must make to start exploiting big data. By focusing on these issues, senior executives can help their organizations build a data-driven competitive edge.” Needless to say, I agree with them completely. Big Data analysis is the ark that is going to keep many businesses afloat in the future.
