Jessica E. Vascellaro writes, “The big hype around Big Data is getting a reality check.” [“Hadoop Has Promise but Also Problems,” Wall Street Journal, 23 February 2012] Vascellaro doesn’t really question the fact that new insights can be gained by analyzing Big Data, she simply points out that obtaining those insights might not be as easy as people are making it seem. She writes:

“For years, technologists have predicted that software that makes it easier and cheaper to analyze vast amounts of data would revolutionize business. One of the hottest contenders is Hadoop, a much-discussed variety of software that sprang from developments at companies that include Google Inc. and Yahoo Inc. Hadoop, which comes in free versions under what the industry calls an open-source license, aims to lower costs by storing data in chunks across many inexpensive servers and storage systems. The software can help speed up certain types of simple calculations by sending many queries to multiple machines at the same time. The technology has spawned a set of new start-ups, such as Hortonworks Inc. and Cloudera Inc, which help companies implement it. But some early adopters of Hadoop now say using the technology is challenging and rolling it out will take time.”

To be fair, technologists have rarely stated that analyzing Big Data would be easy just worth the effort. Few things worthwhile are easy to obtain. Roger Liew, chief technology officer of travel-search company Orbitz LLC, told Vascellaro, “The software isn’t easy to work with. … I believe in the power of it, but you have to adopt it eyes wide open.” Vladimir Boroditsky, a director of software engineering at Motorola Mobility Holdings Inc., told Vascellaro, that Hadoop is “immature” and he reminded her that free software is never really free. Vascellaro continues:

“Still, demand for companies that help businesses implement Hadoop appears to be rising. Mike Olson, chief executive of Palo Alto-based Cloudera, says customers and revenue have doubled in the past year, though he declined to give figures. Mr. Olson says Hadoop is being held back by the lack of software applications that companies can use in conjunction with it to make the data they are storing useful. “We need to see a proliferation of big data-analysis applications,” he says.”

Almost everyone agrees that the era of Big Data remains in its infancy. Like most new things, Big Data analysis will have some growing pains before it becomes fully mature. Stephen Messer, co-founder of Collectivei, told Vascellaro, “Hadoop is like taking an aspirin for a broken leg.” She explains:

“[This is] because it helps companies store large amounts of data but doesn’t help them with certain types of processing. His new start-up sells cloud-based software that helps companies determine what data are relevant for analyzing their customers, eliminating the need to store lots of data that may turn out to be worthless. Rob Bearden, CEO of Hortonworks in Sunnyvale, says the impulse to store lots of data because it can be cheap can lead to storing too much and make answering simple questions harder. ‘You want to have some sort of control over what data you push into an application,’ he says. ‘Otherwise, your juice isn’t really worth the squeeze.'”

Stephen Few agrees that “getting the juice” is important but that sometimes the problem is with how analytical results are presented rather than with how they are obtained. A year ago, he wrote:

“Big, old, traditional BI companies are good at producing technologies that enhance the infrastructure of business intelligence—more and faster—but not the actual use of data in ways that lead to greater intelligence. Being big, focused primarily on technology from an engineering perspective, and devoutly sales driven makes it difficult for companies like SAP to develop useful tools for activities that support decision making: data exploration, sensemaking, and communication. To meet this challenge, they must shift their focus from technology to the humans who use it—our needs and abilities—and expand their perspective to embrace design.” [“Old BI and the Challenge of Analytics,” Visual Business Intelligence, 7 March 2011]

If big companies are unable change, Few asserted that “analytics [will] become the exclusive realm of smaller and more agile vendors, leaving traditional BI companies in the back room to maintain the infrastructure (data collection, transformation, cleansing, warehousing, and production reporting).” Over the past year, the IT sector has seen larger companies go on a buying binge to acquire some of the more agile vendors about which Few wrote. The reason that big companies will adapt (either internally or through mergers and acquisitions) is because they fully intend to stay in front of the money.

While mostly large businesses use enterprise resource planning (ERP) software as well as Big Data analytics, medium- and small-sized businesses (SMBs) are beginning to understand how they can use Big Data analytics and intelligence as well. Mark Cox reports, for example, “A new study by the SMB-focused analyst firm Techaisle study on Business Intelligence Adoption and Trends among SMBs shows that the minority of SMBs who use a BI solution will more than double this year, with the cloud driving adoption.” [“SMB Business Intelligence adoption shifting from spreadsheets to Cloud,” eChannel Line, 26 February 2012] Cox concludes, “Big data analytics is not only for enterprises but at least 18% of mid-market businesses are seriously considering big data analytics.”

The reason that companies are using or considering Big Data analytics is to gain an edge over their competition. Back in 2007, Tom Davenport and Jeanne Harris wrote a best-selling book entitled “Competing on Analytics.” As Steve Banker points out, “The premise of the book was that companies could create a competitive advantage by using Business Intelligence (BI) tools in smart ways.” [“Competing on Supply Chain Analytics,” Logistics Viewpoints, 5 March 2012] Banker concludes, “Most companies know they should be competing on analytics; too few do.” A year ago, Ann Grackin, chief executive officer of ChainLink Research, was making a similar point that Big Data analytics needs to play a larger role in most companies’ strategies. Grackin, however, stressed the importance of demand management. [“Supply Chain and Marketing Executives Can Leverage Information, Expertise,” SupplyChainBrain, 18 February 2011] She wrote:

“In 2010, we did extensive research in several demand management categories that often are thought of separately but actually form the foundation of a new holistic model:

• Selling channels – Our model includes partner channels, distributors and retailers and their behaviors and information.

• Shopping channels – In addition, we looked at multichannel shopping, often called multichannel retail – web, catalog, store, wholesaler, as well as the delivery and information models that support these shopping options.

• Information platform channels – web/cloud, social networks, direct and mobile. Mobile has apps, bar-coding/QR, RFID, Near Field Communication, etc., all coming across a unified platform in a big way. Users are opting in and providing multi-dimensional views into consumer tastes, trends and tribes.

“So far, even the leaders have not fully figured out how to create a holistic vision out of all of this. However, they have implemented components and are beginning to utilize and leverage them. Glimpses of the vision are appearing in the mist, though.”

A 2011 report by the Aberdeen Group noted, “Today’s business decision maker is challenged by a multi-dimensional and continually evolving competitive environment. In one dimension, the evolution of technology and the subsequent acceleration of information flow have created an environment that handsomely rewards timely generation of business insight. In another dimension, the volume, disparity, and overall complexity of that information is expanding at an alarming rate.” [“Business Answers at Your Fingertips: The Real-Time Value of BI,” February 2011] In other words, collecting the right information, analyzing it intelligently, and presenting it to decision makers in a timely and user-friendly way are critical for today’s decision makers. The Aberdeen Group study recommends taking four actions to make this happen:

• Start measuring “time-to-information.”
• Develop programs to coach/train/develop analytical talent in-house.
• Investigate technologies to improve data quality.
• Combine real-time data with predictive modeling applications.

The study fully explains why each of these actions is recommended. Bob Ferrari is a fan of the second recommendation (upgrading analytical talent) because analytics and business intelligence must provide “supply-chain wide information and intelligence.” To overcome a lack of in-house talent in companies, some vendors are providing what Ferrari calls “narrow scope analytics applications.” The problem with these applications, he asserts, is that they place “analytics and business intelligence data back into functional stovepipes.” [“How is Your Organization Improving its Supply Chain Advanced Analytical Skills?” Supply Chain Matters, 8 February 2012] He concludes:

“These narrow scope analytics applications may also be the means of some software providers to hang on to customers while not necessarily supporting the broader and more extensive need. … Regardless of the systems and technology approach, the reality remains that both supply chain management and planning teams need to consider realistic methods for deepening individual skills and literacy into leveraging advanced analytics within supply chain business decisions.”

The one thing that almost everyone agrees on is that business analytics and intelligence are going to be critical for companies and decision makers in the years ahead. Getting it right won’t be easy, but it will be worth it.