Supply chains come in all shapes and sizes. Supply chain complexity increases as it becomes larger or more geographically extended or more data intensive. Lora Cecere, a partner at Altimeter Group, recently wrote a post focused on “the big data supply chain.” [“User in the Era: Big Data Supply Chains,” Supply Chain Shaman, 1 June 2011]. Since “big data” may be a new term for some readers, Cecere begins her post by explaining what she means by “big data.” She writes:

“The concept is simple. The answer is complex. Big data is a term used to describe data sets that grow so large, and so fast that conventional reporting and analytics are insufficient. … Let’s examine some trends:

“1) You see it in new tagging systems for safe and secure supply chains.

“2) It is ever-present in demand sensing and the design of listening posts from social networks. These technologies [up] the ante on the use of unstructured text and building supply chain systems that can sense [not just] respond. It is one that starts from the outside-in to define the enterprise response.

“3) It takes the form of mobile devices that are redefining the workplace. Mobile data has grown 8 fold in five years.

“4) It is a new world of convergence of visualization, geolocation, and digital media.

“5) Partner data is growing exponentially. What we once thought was just a simple downstream data repository is now being used as the data translator and harmonizer at both ends of the supply chain. It is redefining the world of business-to-business relationships. Trading partners are starting to share daily data daily.”

A recent study by McKinsey & Company asserts, “The scale and scope of the changes that such ‘big data’ are bringing about have reached an inflection point.” [“The challenge—and opportunity—of ‘big data’,” McKinsey Quarterly, June 2011] Although the study concedes that data collection makes some people suspicious, it concludes “that collecting, storing, and mining big data for insights can create significant value for the world economy, enhancing the productivity and competitiveness of companies and the public sector and creating a substantial economic surplus for consumers.” It’s no secret that staggering amounts of consumer information is now being gathered and analyzed. But rather than wanting less information, manufacturers and retailers are asking for more. Supply chain transparency and visibility, two descriptors that most supply chain analysts believe will define future supply chains, require the best and most current information available. That’s why Cecere insists that supply chain professionals need to be ready to manage big data supply chains. She continues:

“This is far different than the world of five years ago when data was shared less often; and when it was … usually monthly data monthly or weekly data weekly. … As we enter the world where data is more available from trading partners, we can navigate across the supply chain into customer’s customers and supplier’s suppliers. In this world of big data, relational databases and desktop applications – spreadsheets, statistical packages and reporting—are insufficient. Instead, it requires the use of parallel software running on tens, hundreds or even thousands of servers. It is the world of terabytes, exabytes and zettabytes of data.”

The age of big data began a number of years ago. The late Jim Gray, a database software pioneer and a Microsoft researcher, gave a speech in January 2007 shortly before he was lost at sea off California’s coast. In that speech, Gray argued that we are entering a “fourth paradigm” in computer science. [“A Deluge of Data Shapes a New Era in Computing,” by John Markoff, The New York Times, 14 December 2009] Markoff explains that “the first three paradigms were experimental, theoretical and, more recently, computational science.” Gray claimed we were entering a new paradigm that was brought about by “an ‘exaflood’ of observational data.” Gray insisted that a new generation of computing tools was needed “to manage, visualize and analyze the data flood.” Gray anticipated the big data era discussed by Cecere and the kinds of computational tools that would be required to make sense of it all. Although Gray didn’t use the term “cloud computing,” he clearly had something like it in mind. The point being made by Gray and Cecere is business as usual doesn’t work in the big data era. Turning to how big data affects the supply chain, Cecere writes:

“Today’s era of big data supply chains is an even bigger step change opportunity, but to take advantage of the opportunity, we must re-wire our thoughts to see new possibilities. It is not just about supply, it is about making tradeoffs to improve value. It is not just about linear relationships or a chain reaction, it is about sensing networks. It is not just about right product, right place, at the right time. Instead, it is about the redesign of value networks that use information to reduce latency, streamline cash flow and drive profitability. Today’s supply chain systems are not designed for the world of big data. It is coming. The data will be colossal. The use of data in the supply chain will differentiate.”

Rajiv Mehrotra agrees with Cecere that “today’s decision managers must be able to analyze data along any number of business dimensions, at any level of aggregation, with the capability of viewing results in a variety of ways. They must have the ability to drill down and roll up along the hierarchies of multiple business dimensions.” [“Memory Centric Business Analysis and Supply Chain Visibility: One Complementing the Other,” Supply Chain Management, 30 March 2011] This kind of analysis can only be achieved when mountains of data can be adequately analyzed. Mehrotra continues:

“Memory-centric data management and OLAP (online analytical processing) technologies and tools are the answer. With the cost of memory dropping and speed increasing, the in-memory computing and analytics platform is an efficient way which has helped users carry out complex analysis on enterprise wide data by consolidating, and editing enormous volumes of data across different operational and financial measures. Memory-centric technologies permit real-time and multidimensional OLAP analysis, with much more speed and computational flexibility as opposed to disk-centric technologies, whose performance is more like ‘come back after lunch’.”

Cecere offers five predictions about the future of big data supply chains.

“#1 A One Vendor World is not the Answer. The big data supply chain will not be a one vendor world (.9 probability). … Gaining competitive advantage from the big data supply chain will not be a “one throat to choke” scenario. You cannot afford to tie your apron strings to the innovation of ERP vendors. If you do, you will move too slowly. … We have defined supply chain execution too narrowly. It is more than order to cash. There are new opportunities in S&OP execution, demand and supply visibility and demand orchestration. I am also excited by the focus and energy of Manhattan and Red Prairie to tackle this opportunity more holistic[ally].”

There is a lot of buzz about sales and operations planning nowadays. To learn more, read my posts entitled Some Thoughts Concerning Sales & Operations Planning, Part 1 and Part 2, S&OP: Supply Chain’s Foot in the Boardroom Door, and Supply Chain S&OP Technology. Many analysts believe that the term S&OP is too limiting and prefer the term integrated business processes (IBP). Whether you prefer S&OP or IBP, Cecere is correct that getting the process right opens up new possibilities. Cecere’s second prediction:

“#2. Line of Business (LOB) Meet Data. In companies where the line of business leader steps up to own the big data supply chain, there will be a 3X increase in the ROI of IT investments (.8 probability). I have done research studies over the past five years on IT investments of BI in the supply chain. One factor is clear to me. When projects are owned by the Line of Business Leaders, and those LOBs are knowledgeable and capable team players, there is a dramatic difference in the impact and ROI on the project. In the face of the great recession, companies that were better at demand sensing changed their supply chains five times faster. The issue is finding leaders that are both knowledgeable and capable.”

Cecere’s prediction reinforces the old adage “knowledge is power.” There is a difference between data and knowledge. You can gather mountains of data; but, if you can’t make sense of it, it does you little good. Master the data and enjoy the rewards. The McKinsey report cited earlier believes this may be the long pole in the tent for many companies. It predicts that by 2018 there will be between 140,000 to 180,000 unfilled positions that require workers with data analysis expertise. Cecere’s third prediction:

“#3. Not a Project. It cannot be solved one project at a time. Companies that approach this evolution as a program, not a project will increase speed to value by 70%. As I study the evolution of Business Intelligence (BI) in supply chain, it is clear to me. Project-based evolution absent a program and a strategy is problematic. Companies that have multiple projects that do not build on a consistent data model, with clear data governance, and definition of the meta-data structures, have built a bridge to nowhere.”

Cecere makes an excellent point. The difference between a project and a company-wide program is that projects strengthen business silos and programs help break silos down. In the information age, business silos are more likely to hinder than help a company thrive. Cecere’s fourth prediction:

“#4. We must part with Tradition. It will require taking a leap of faith. … It will require a RETHINKING of supply chains to abstract the supply chain into sensing attributes that can sense market changes quickly, easily translate these changes into the world of supply and transmit them in a meaningful way to the supplier. The design is outside-in, not inside out. It is [no] longer the world of the language of SKU (item at a location.) This language gets co-opted by the language of attributes. We will have to remap supply chains, rebuild demand and supply hierarchies, and redefine BI –portals, scorecards, dashboards, and predictive analytics—to think in the world of attributes. … This evolution will make current Enterprise Resource Planning (ERP), Customer Relationship Management (CRM) and Advanced Planning Solutions (APS) solutions obsolete over the next five years (.9 probability). Because current analyst models are funded by these late stage technology solutions, you will find little from analysts on the rise of big data supply chains. The reason? The shift is discontinuous.”

In a post entitled Supply Chain Evolution and Transformation, I argue that supply chains probably need transform rather than evolve. I believe that is what Cecere is arguing as well. Her final prediction:

“#5 MDM will not take us to the Pearly Gates. As companies enter into the world of big data, Master Data Management (MDM) concepts that we know– and never loved– will fall by the wayside. They will be scrapped. New technologies will evolve to better handle MDM. There are three technologies that I am watching closely that I think offer promise, both singularly, and together.

• “Search Engine Optimization (SEO): Endeca is using their SEO tools to improve flexibility in parts management in the automotive industry. The use of tagging and attributes improves flexibility.
• “Artificial Intelligence: Enterra Solutions redefinition of security data for the Iraq war was applied successfully to Conair and Newell Rubbermaid supply chains in 2011 to sense supply issues and redefine the response.
• “Intelligent Workflow for Governance: Kalido has introduced intelligent governance workflow for line of business users.

“Within five years, the landscape of master data solutions will be redefined (.9 probability).”

I’m always grateful when my company gets a plug, especially when it comes from an analyst as well-respected as Ms. Cecere. She concludes her post with a five recommendations for companies that want to be prepared for the big data era. She writes:

1. “Stabilize. Take a look at your product portfolio and stabilize traditional approaches, especially ERP projects. Focus on the use of ERP for seamless movement of transactions. Throw away the enterprise application lexicon that you have learned and get ready for a new world.
2. “Define. Map the supply chain from the outside-in focused on how customer attributes translate to service and product attributes. Think about how and why you sense and what a decrease in information latency can mean for your supply chain.
3. “Build. Focus on building an inter-enterprise data model. Focus on the ends of the supply chain…. Realize that there was never ‘R’– or relationship– in CRM or SRM applications. Think about what you could accomplish through the building of business-to-business relationships through a combination of social, sensing/listening technologies and predictive analytics to transform B2B.
4. “Will require a Team. Invest in a BI team of excellence to look at how companies can drive insights from data. Staff it cross-functionally, but align the reporting relationships to a line-of-business thought leader that has cross-functional responsibilities. Experiment with new master data management systems. Develop a holistic BI strategy for your value networks.
5. “Get good at data. Train teams on the evolving world of business intelligence and the use of trading partner data in data-driven decisions. Reward innovation through the use of predictive analytics. Focus on data reuse, meta-data definitions, and data enrichment strategies. Overlay the BI team of excellence on top of sales and supplier relationships to build data-driven sensing to drive supply chain requirements.”

The McKinsey report identifies five additional “ways to leverage big data.” They are:

1. “Make big data more accessible and timely. Transparency, enabled by big data, can unlock a great deal of value. In the public sector, increasing access to data across separate departments can sharply reduce search and processing times. In manufacturing, integrating data from R&D, engineering, and manufacturing units to facilitate concurrent engineering can cut time to market.
2. “Use data and experiments to expose variability and raise performance. As organizations create and store more transactional data in digital form, they can collect more accurate and detailed performance information on everything from product inventories to sick days.
3. “Segment populations to customize. Big data allow organizations to create ever-narrowing segmentations and to tailor services precisely to meet customer needs. This approach is well known in marketing and risk management but can be revolutionary in areas such as the public sector.
4. “Use automated algorithms to replace and support human decision making. Sophisticated analytics can substantially improve decision making, minimize risks, and unearth valuable insights that would otherwise remain hidden. Such analytics have applications from tax agencies to retailers.
5. “Innovate with new business models, products, and services. To improve the development of next-generation offerings and to create innovative after-sales services, manufacturers are leveraging data obtained from the use of products. The emergence of real-time location data has created a new set of location-based mobile services from navigation to people tracking.”

I agree with Cecere and the analysts at McKinsey that the big data era is rising like the tide. In fact, Cecere believes it’s approaching more like a tsunami. Some companies are just getting their toes wet, while others are already waist deep in data and analytics. An incredible amount of work is being done in this area — including at my company. Cecere and the folks at McKinsey have sounded the warning call and others will undoubtedly join them. Companies that respond will be that much further ahead when the big data tsunami makes landfall because it’s a wave of data that never ends.