Supply chain analysts Lora Cecere and Bob Ferrari have engaged in an online dialogue about big data in the supply chain. In this post, I’ll look at Cecere’s original post on the subject. Tomorrow, I’ll look at Cecere’s follow-on post and Ferrari’s response. In the third post of this series, I’ll look at what else is happening in the world of big data.

Cecere begins, “Data volumes are exploding, data velocity is increasing and data types are proliferating. Most companies that I am working with are struggling. The question is how to develop a road map to best use the many types of data the right way.” [“Minding our P’s and Qs: It is more than R’s and P’s,” Supply Chain Shaman, 16 August 2011] She continues:

“The list is long and includes: transaction data, sensor transmission, social text proliferation, downstream channel data, distributor network sales, warranty information, customer contracts, product IDs for serialization, geo-location and map data. Yes, data is exploding both in type and volume. It will continue to grow, but more importantly, it allows us to define new capabilities. This growth will be exponential. Today, it is on the doorstep of our supply chain, early adopters are experimenting and will use it to power supply chain innovation; and within five years, I believe that the holistic use of this data will be mainstream.”

If Cecere is correct, then companies don’t have any time to waste in getting their IT infrastructures in place. Five years comes very fast. Cecere believes that there are a number of drivers propelling this high-speed trend down the tracks. They include: convergence, supply chain transformation, and global infrastructure. She explains:

“A major force is convergence: unleashing the power of mobile, geolocation, digital and social data together. Innovators … are busy marrying structured and un-structured data to harness new opportunities in their big-data supply chains. … The second opportunity is building TRUE customer-centric supply chains. Demand-driven is not sufficient. We must do more than sense and shape demand. … The third driver is global infrastructure. The average manufacturing client has three ERP instances (one for each geography), and each has over a terabyte of data. The good news is that hardware is cheaper, the bad news is that these systems have become so integral to the business environments, that they can no longer afford the outages associated with maintenance upgrades and system upgrades.”

Cecere goes to describe what she calls a “Big Data Supply Chain.” She writes:

“Let’s start with a definition. What do I mean by Big Data Supply Chains? They are value networks that extend from the customer’s customer to the supplier’s supplier that sense, shape and respond by listening, testing and learning with minimal latency.

• “What it looks like: It combines structured and unstructured data to sense, listen, test and learn to shape the intelligent response horizontally and cross-functionally. The processes are outside-in not inside out. They are bi-directional from buy to sell-side markets connecting the customers’s customer to the supplier’s suppliers. Because the data volumes are so immense –with high velocities and variabilities– the creation of big data supply chains will require new techniques for the capture, storage, search, visualization and sharing of data. It is the world of terabytes, exabytes and petabytes.

• “The building blocks: A pre-requisite is excellence in today’s business analytics: reporting, scorecards/dashboards, optimization, and intelligent rules. It will challenge today’s world of supply chain applications. I believe that it will transform the Advanced Planning and Scheduling (APS) market and will redefine Customer Relationship Management (CRM) and Supplier Relationship Management (SRM) applications. Hence the title of this blog post. It will transform today’s R&P applications–APS, CRM, SRM, ERP–but, only for those that mind their Ps and Qs of supply chain understanding to harness the power of big data supply chains.”

Cecere doesn’t pretend that the transition to a Big Data Supply Chain is going to be easy or quick. That’s why her 5-year horizon for big data ubiquity looms so large. Among the challenges are: a lack of visionary business leaders; the continued presence of siloed business operations; a lack of understanding that the supply “is the business”; and the fact that most business road maps are leading their companies towards the big data promised land. The reason for the last challenge, she writes, is that the road that must be taken is “unproven … with dead-ends, uncertainty, and lots of opportunity.” She laments the fact that “supply chain investments historically have focused on improving efficiency. Supply chains respond. It is seldom an intelligent response.” She continues:

“These new approaches, allow the supply to learn and predict. This machine to machine learning is a radical shift for supply chain leaders. We can learn the impacts in the financial and insurance industries where technology enabled a continuous learning environment, allowing the organization, to listen, learn and then drive an intelligent response. Based on many-to-many rules mapping (versus traditional one to one fixed mapping), the new approach allows the learning to be around the clock and across geographies. The larger challenge will then become change management. Organizationally, we don’t know how to listen and learn. And, within organizations, not all people want to be measured or even share their data.”

Cecere insists that in spite of the challenges that are going to be faced during any transformation to a Big Data Supply Chain, the end results are going to be worth the effort. She also knows that change management is never easy. In order to foster change management, she asserts that a shift of focus is required. She writes:

“As supply chain teams mature, the focus shifts from vertical process excellence, to cross-functional and horizontal processes to deliver a supply chain strategy. These horizontal processes gain more value from external data sources. Examples include Sales and Operation Planning (S&OP), Revenue Management, Supplier Development, Demand Sensing and shaping, etc. The combination of structured and unstructured data gives a more holistic view to make these horizontal processes more effective.”

She believes that convincing people that the promised land exists is going to be difficult because many of the past promises proffered by those peddling integration have largely been unfulfilled. She writes:

“Enormous integration challenges still abound in companies with structured data, and it is even greater with unstructured data. This is true even in companies that have standardized on a common ERP system. The proliferation of data sources from outside the enterprise external to ERP will make this issue worse before it gets better. The emerging technologies associated with Big Data Supply Chains offers hope. … While the goal was to standardize on ERP and eliminate disparate systems, this was largely a pipe dream. It has not happen[ed].”

She believes that things will be different this time around because computing power has increased 20 times since the early client server days a decade ago. Despite this increase in computing power, Cecere writes, “The evolution of ‘R and P’ technologies has largely ignored the possibility of what can be done with parallel processing and in-memory capabilities.” She issues a clarion call for visionary leaders to step up and capture the potential. She insists, “The pace of adoption and gaining competitive advantage will be gated by the lack of business visionaries and their ability to pull their noses off the grindstone to envision how this can benefit the business.”

Another reason Cecere believes that things are going to be different this time around is the fact that costs are coming down. She writes:

“The beauty of the new techniques is that the cost of computing is going down, and the techniques are no longer just for the VERY large corporations. Cloud, mobile, in-memory, and vertical or functionally targeted applications and services are making the barrier to entry much lower than ever before.”

She concludes her post with a few guiding Principles:

“I want to be sure that we are grounded in guiding principles to take this work on [a] deliberate, systemic path versus a [course of] splintered projects that lead to nowhere.

• “Build cross-functional teams to unleash the power. All too often, the groups that know the most about disruptive technologies are adjunct to eCommerce teams. Make this mainstream and challenge the group to think about what new processes could be if they build horizontal processes with minimal latency from the outside-in.
• “There is no data mart cheap enough. This was a quote that I heard recently from eBay. I think that it is very true. Our foray into data marts is largely driven by a project approach versus a deliberate, and conscious choice on building effective value networks.
• “Focus on meta-data design and master data will be easier. Try new master data techniques. While I hear many business users fret about master data issues, many times the issue is lack of attention to metadata design (especially customer, product and supplier data). I also see innovators attempting new techniques to solve the master data issues: bypassing traditional techniques by indexing their data for rapid assembly. This gives flexibility to embrace the differences between master data registry and master data reference.
• “Never let anything come between the user and their data. Empower the business user by focusing on self-service. To maximize the use of data for insights, empower the business users to directly use the data and even manage it (mapping, extracting etc.) themselves. Who knows the data better than the user, right? Focus on training and design to enable self-service by the line of business user.
• “Write once and read many times. In big data supply chains, focus on one system of record. Everyone has the moments when they show up at a business meeting only to argue about ‘whose report has the right data’. Solve this problem by writing once and using many times.
• “Success. While many companies feel that success happens when a project is installed on time and on budget, I feel that success happens when line of business managers USE the systems. For most companies today, that use Excel and Access, after pouring millions of dollars into ‘R & P’ systems, this should be a lesson learned and a mistake that should be avoided. I feel that adoption, usage, and user satisfaction are better success criteria.”

In her first post, Cecere promised to keep writing about this subject in future. In her next post, Cecere provides us with some “definitions for the supply chain leader’s big data supply chain vocabulary. Words to know to understand what is happening.” I’ve only touched the surface of Cecere’s blog (which is about twice as long as this post). If the topic is of interest to you, I recommend you read her post in its entirety.