In Part 2 of this series, I discussed some thoughts on the future of big data offered by marketing technologist Scott Brinker. He wrote that the “big leap” for marketing companies will be when they add “big testing” to “big data” to create a “big experience” for clients. He believes this will be so revolutionary that “it’s worth expanding a bit on what’s meant by ‘big testing’ and ‘big experience’ — and why they’re such a big leap for most marketers.” [“The big data bubble in marketing — but a bigger future,” Chief Marketing Technologist, 21 January 2013] Brinker first discusses big testing. He writes:
“A few months ago, I wrote a post, “We want to test bold, new ideas that always work.” It highlighted recent research by the Corporate Executive Board showing that the far majority of Fortune 1000 marketers think their organizations are not effective at test-and-learn experiments. One reason why: at least half didn’t think an experiment should ever fail. I believe this is the single biggest obstacle most organizations face: their culture and politics dissuade people from trying experiments because failure of an experiment implies failure of the tester. Who wants to stick their neck out in that environment? So either people don’t test anything, or they test in a non-controlled manner so that outcomes are comfortably subject to caveats and interpretations.”
In my posts about innovations and innovative companies, I’ve repeatedly pointed out that a fear of failure is a sure sign that a company doesn’t have a innovative culture. In a TEDx Kyoto talk, Catherine Courage, a leader for a Silicon Valley product design group, noted that Thomas Edison failed to find the right filament for his light bulb a thousand times. Edison didn’t see this as 999 failures, but 999 steps in a 1,000-step process to success. That’s a very different mindset than most companies embrace. Brinker claims that even companies claiming to embrace testing often conduct only superficial tests that involve little risk. “Perhaps the greatest damage they do,” he writes, “is that they give people the illusion of engaging in real experimentation. … Big testing is qualitatively different.” He then offers “three ways in which big testing earns its ‘big’ label”(as depicted in the following graphic):
Brinker writes this about big ideas:
“First, it’s about testing big ideas. Headlines and button colors are fine, but they barely scratch the surface. The real power of testing is unleashed when you use it to learn ways of engaging entirely new segments of customers or to pioneer innovative new ways of selling or delivering your product or service. It’s about taking the surprising insights that big data may reveal and proving (or disproving) their value and efficacy. … A good way to know if a test is truly meaningful or not: state out loud what the hypothesis is. If there is no hypothesis, or it’s something patently banal — like ‘chartreuse green buttons will have at least a 0.01% increase in clicks over forest green buttons’ — then you’re just rearranging deck chairs on the Titanic.”
Brinker next talks about “big teams.” Long-time readers of this blog know that I’m a supporter of cross-functional teams. Team members from different disciplines bring with them valuable perspectives that are not shared by others outside their field. Brinker writes:
“Big testing is not restricted to a tiny priesthood of test-masters who jealously guard the data, tools, or governance rights for running tests. Big testing empowers and encourages an open, big team of testers across the organization. Many different people are given the capability to run tests in the context of their particular work. … Of course, this is usually best when there’s a strong shared vision and some basic training — enough to keep people loosely coordinated and out of trouble. But harnessing a (relatively) large and distributed force is a source of great power — both in sheer manpower and in rich diversity of ideas.”
When Brinker talks about “big teams,” he emphasizes the “big.” He calls it “massively parallel marketing.” He claims, “Massively parallel marketing covers a much broader set of possibilities much faster, but it is a fundamentally different kind of organization than the traditional command-and-control hierarchy.” He next discusses why big testing should be a big deal. He writes:
“Big testing is … championed by executives from the top-down. Experimentation isn’t a risk to your job. On the contrary, not experimenting — in particular, not experimenting with big ideas — will put you out of favor. I was at a big data conference … where Gary Loveman, CEO of the Caesars casino empire, gave a presentation about how important meaningful testing is in his organization. ‘There are three ways to get fired from Caesars,’ he states matter-of-factly. ‘You can steal from the company. You can harass a co-worker. Or you can fail to have a control group for an experiment.’ That’s making a big deal of testing. At that same conference, Hal Varian, the chief economist at Google shared that Google runs around 10,000 experiments per year, with about 500 experiments going on at any one time. Testing is an integral part of their company culture. It’s worth pointing out that both of these companies are superstars when it comes to big data, but in both cases, they strongly emphasized the criticality of running real-world tests to capture the value buried in that data. As Hal Varian said simply, ‘Experimentation is the gold standard of causality.’ It tells you which actions generate the best response.”
Brinker’s final discussion is about providing a big experience for customers. Big experiences are customized, personalized experiences. They speak directly to the consumer rather than merely interrupt whatever he or she is otherwise doing. Brinker explains:
“The incredible potential of big data — and big testing, for that matter — is near worthless if an organization can’t employ it in the service of delivering remarkable customer experiences. Big experience should be the big tent under which big data and big testing sing and dance.”
Brinker believes that “there are three important connections between big experience and big data/big testing.” He writes:
“First, data and testing give direction to the burgeoning customer experience movement. Customer experience is becoming the banner under which modern marketing marches to victory. It’s why everything is now marketing. The tectonic shift of marketing’s responsibilities, from mostly customer communications to increasingly lifecycle customer experiences, is one of the 5 meta-trends of modern marketing. It’s an opportunity for marketing to shine at the highest level of the organization.”
The dilemma concerning the creation of a big experience is the fact that “building amazing customer experiences is a lot of work, time-consuming, and expensive.” Brinker obviously believes the effort is worth the result or he wouldn’t have written his post. He continues:
“Using big data and big testing provide a structured and systematic way to hone in on the right experiences to build big. Designers and customer experience professionals can leverage these capabilities to identify rich, new pockets of opportunity for exploration, as well as to refine their productions along the way. Done properly — where big data and big testing are employed in the service of remarkable customer experiences — the relationship between these parts can be harmonious, not contentious. Data and testing minimize the downside and maximize the upside of pursuing big, new customer experience innovations.”
Even if you are headed in the right direction, if you execute poorly you will lose your competitive advantage. Big testing, writes Brinker, is “only valid if the customer experiences in which it’s executed are good.” He continues:
“If you run a split-test of two concepts, say offer A (a price emphasis) and offer B (a quality emphasis), testing a hypothesis of which will motivate a particular customer segment more — but both experiences are kind of crappy — then the results of your test are useless. Probably both will perform poorly, but maybe offer A gets slightly more traction. But it’s not unequivocally because the market prefers price over quality. The audience exposed to offer B may have found the offer of high quality, juxtaposed with an experience that stunk of low quality, to be incongruous and have no credibility. Superficially claiming ‘quality’ is not the same as truly exuding quality.”
He concludes, “It’s far better if big testing is a comparison of two very good customer experiences to see which is best.” The final connection discussed by Brinker involves a customer focus rather than a product focus. He writes:
“Big data and big testing give us a way to construct amazing customer experiences around customers rather than products. … Even though we know we’re largely past the age of mass marketing — at least in terms of channels and touchpoints — the mindset and organizational structure of most businesses still revolves around fixed products and services. Even as they try to embrace a more customer-centric way of selling those products and services, they’re still anchored to segmenting their market by product at their core. ‘Most companies count their profits in terms of products, not customers,’ remarked Gary Loveman at that conference in answer to the question of why most companies are so bad at using analytics to drive better results. The real vision of big experience is using data and testing to reconfigure our organizations around our different groups of customers, finding and favoring the most profitable and crafting ever more tailored experiences to each. I’m not saying that lightly — I appreciate how big of a challenge that is. But ultimately, that is how businesses will unlock the massive value that the big data movement is promising.”
I’m a big believer in the kind of targeted marketing Brinker describes. In conclusion, Brinker offers a few afterthoughts about big data. His first observation is, “Data is fuel — invest more in the engine.” He explains:
“In the face of all this data mania in marketing, the real revolution is shaping a data-driven organization. Wrangling big data is an important part of that, but big testing and big experience are the ways in which that data lead is turned into customer gold. While there is certainly cool technology available to power that entire ‘big stack,’ the biggest part of embracing this revolution will be shifting your organizational structure, behavior, and culture to truly leverage it. … Data is like fuel. It’s certainly valuable, and there’s a big industry emerging around the extraction, refinement, and distribution of that fuel. Hey, Exxon Mobil is a $400 billion mega-corporation. But the big oil companies would be nearly worthless without the transportation industry. Because it’s the engines and the cars and the jet airplanes that use that fuel to move the world. An engine without fuel is fairly useless; but a barrel of fuel without an engine is equally pointless. It’s only the two together — the fuel and the engine — that unleash the potential of both. If you want to really harness the power of big data, build the organizational engine to use it.”
All of the analysts cited in this 3-part series about the future of big data agree on one thing: It’s not the data, but what you do with the data, that matters. I like Brinker’s fuel and engine analogy. That’s why at Enterra Solutions we refer to many of our technology services as data analysis engines created to obtain specific kinds of results. Over-hyped or not, the future of big data still looks bright.
