In an interview with one of the author’s of the book Switch: How to Change Things When Change Is Hard by Chip and Dan Heath, Renee Hopkins writes, “Change is at the heart of innovation—no innovation happens without change, and innovation changes the status quo” [“How Change Happens,” Bloomberg BusinessWeek, 19 April 2010]. She correct, of course, and she is also correct when she writes, “Understanding how to make change happen, then, is of critical interest to innovators.” She then describes the three ways the Heaths claim that change happens. They are:

1. Direct the Rider (the conscious mind), eliminating what looks like resistance but is more often a lack of clarity by providing crystal-clear direction.

2. Motivate the Elephant (the subconscious), eliminating what looks like laziness but is more often exhaustion by engaging emotions to get people on the same path as you.

3. Shape the Path (the situation), eliminating what looks like a people problem but is more often a situation problem, by making the environment more conducive to the change you seek.

I’m all for clever packaging, but when you have to parenthetically explain what you mean by your selection of a name (e.g., Direct the Rider, Motivate the Elephant, and Shape the Path), you’ve probably been too cute. The interviewee in this case was co-author Dan Heath, who answered questions sent to him by Hopkins. Heath asserts that “Switch discusses a simple framework for changing behavior. Innovators will need this skill more than most people, since they need to convince their colleagues to adopt new practices and their customers to embrace new products.” Heath believes that you need to get others “to feel” what you are trying to accomplish because “change comes from feeling, and the feeling provides us the motivation we need to overcome the nuisance of making changes.” Although I agree that motivating people to get on board with change is important, sometimes getting passionate about a bad idea can hurt rather than help a company. Isabelle Royer asserts, “The failures I’ve examined resulted, ironically, from a fervent and widespread belief among managers in the inevitably of their projects’ ultimate success. … The result is what I call collective belief, and it can lead an otherwise rational organization into some very irrational behavior” [“Why Bad Projects are So Hard to Kill,” Harvard Business Review, February 2003]. For that reason, I would add at least one more bullet to the Heaths’ framework: Don’t Spare the Chopping Block. I’ll return to this subject later.

Sometimes change and innovation are thwarted by corporate internecine warfare. One former Microsoft vice president believes that is exactly what is happening to Microsoft [“Microsoft’s Creative Destruction,” by Dick Brass, New York Times, 4 February 2010]. Brass, a vice president at Microsoft from 1997 to 2004, writes:

“Microsoft has become a clumsy, uncompetitive innovator. … While Apple continues to gain market share in many products, Microsoft has lost share in Web browsers, high-end laptops and smartphones. Despite billions in investment, its Xbox line is still at best an equal contender in the game console business. It first ignored and then stumbled in personal music players until that business was locked up by Apple. Microsoft’s huge profits — $6.7 billion for the past quarter — come almost entirely from Windows and Office programs first developed decades ago. Like G.M. with its trucks and S.U.V.’s, Microsoft can’t count on these venerable products to sustain it forever. Perhaps worst of all, Microsoft is no longer considered the cool or cutting-edge place to work. There has been a steady exit of its best and brightest. What happened?”

Brass contends that the problem with Microsoft is that its search for innovative new products is not guided by any type of formalized system. As a result, different divisions of the company can effectively sabotage efforts of other divisions. He continues:

“Unlike other companies, Microsoft never developed a true system for innovation. Some of my former colleagues argue that it actually developed a system to thwart innovation. Despite having one of the largest and best corporate laboratories in the world, and the luxury of not one but three chief technology officers, the company routinely manages to frustrate the efforts of its visionary thinkers. For example, early in my tenure, our group of very clever graphics experts invented a way to display text on screen called ClearType. It worked by using the color dots of liquid crystal displays to make type much more readable on the screen. Although we built it to help sell e-books, it gave Microsoft a huge potential advantage for every device with a screen. But it also annoyed other Microsoft groups that felt threatened by our success. Engineers in the Windows group falsely claimed it made the display go haywire when certain colors were used. The head of Office products said it was fuzzy and gave him headaches. The vice president for pocket devices was blunter: he’d support ClearType and use it, but only if I transferred the program and the programmers to his control. As a result, even though it received much public praise, internal promotion and patents, a decade passed before a fully operational version of ClearType finally made it into Windows.”

Brass also discusses how Microsoft’s 2001 tablet computer was doomed from the start because the vice president in Charge of Microsoft Office “refused to modify … applications to work properly with the tablet.” Brass concludes:

“Internal competition is common at great companies. It can be wisely encouraged to force ideas to compete. The problem comes when the competition becomes uncontrolled and destructive. At Microsoft, it has created a dysfunctional corporate culture in which the big established groups are allowed to prey upon emerging teams, belittle their efforts, compete unfairly against them for resources, and over time hector them out of existence. It’s not an accident that almost all the executives in charge of Microsoft’s music, e-books, phone, online, search and tablet efforts over the past decade have left. As a result, while the company has had a truly amazing past and an enviably prosperous present, unless it regains its creative spark, it’s an open question whether it has much of a future.”

Brass is correct when he writes that it is good to have ideas compete. Competition can result in the strongest ideas surviving and weaker ideas being shelved. He is also correct, however, when he notes that internal competition can become uncontrolled and destructive. So how does a company find the proper balance? How does it remain innovative and still manage to kill off bad ideas or halt failing projects? Zia Khan and Jon Katzenbach assert that it is essential that companies get “their formal and informal organizations in sync” [“Are You Killing Enough Ideas?” strategy + business, 27 August 2009]. Brass would probably agree that it is a lack of corporate synchronization that is hurting Microsoft. Khan and Katzenbach write:

“‘How can we tell which idea is going to be the next big thing? Why is it so hard to get an idea from the drawing board into the market?’ Most telling of all is the question: ‘Why do we still waste resources on ideas that people don’t believe in?’ In other words, even though an idea has been effectively ‘killed,’ it still remains on the agenda, with nobody fully willing to learn from the mistake, put it to rest, and move on to other endeavors. … For a company to excel at innovation, it must have high levels of creative capabilities, such as R&D, and production capabilities, such as operational execution. Moreover, the people with these capabilities need to work together seamlessly. … In most companies, the creative and production capabilities are difficult to integrate. To understand why, it’s helpful to think about organizations as having two parts — a formal organization and an informal organization. Most organizations tend to emphasize one over the other, but both are always present. Production is largely governed through the formal side, which includes decision rights, reporting structures, established incentives, codified processes, and corporate metrics. These elements tend to enable an organization to be disciplined and efficient. … Creative work, however, is typically generated by the informal side — the part that doesn’t exist on paper but that influences behaviors through mutually understood rituals, shared values, social networks, commonly held ideas, and emotional connections to the work itself. These factors tend to motivate an organization to be flexible and responsive. Creative people sit in formal groups and have bosses, but they are more likely to draw on their informal connections with people at multiple levels, be passionate about searching for the best idea, and take risks that often don’t work out.”

Khan and Katzenbach cover a broad range of challenges including everything discussed by Chip and Dan Heath and Dick Brass. As I have written before, innovation is tough to pull off; and Khan and Katzenbach highlight why that is the case. The balance and synchronization they discuss are not easy to achieve. They continue:

“Because real innovation requires that good creative ideas be efficiently produced and launched, a balance between the formal and informal structures is critical. If ‘creativity is king’ and teams are encouraged to chase every idea, there will be a great deal of waste in the system. Winners for the final rounds of funding will be chosen on the basis of informal connections and relationships rather than merit — which often means that the political strength of sponsors, rather than customer interest or analysis of potential impact, will most influence decisions. In the aftermath of such a decision, the level of commitment tends to deflate during execution, and the products are less likely to win in the marketplace. … Conversely, if the organization is weighted too heavily on the formal side, then nascent ideas aren’t allowed enough time to develop. They are subjected to processes and decision criteria similar to those used for running the existing business. This results in a bias toward incremental innovations that resemble the products and services that the existing business currently manages, rather than breakthrough innovations that might require a new way of operating and evaluating business success.”

The authors next touch on a corporate cultural trait that is essential for any truly innovative company: the ability to tolerate (or even embrace) failure. They write:

“Whether in transforming a company’s innovation practices or in maintaining them over time, one of the most revealing indicators of effectiveness is the number of losing ideas. This may at first seem counterintuitive, if the goal is to take ideas to market. However, a high number of losing ideas indicates that the informal and formal aspects of innovation are working well together. It shows that the enterprise is creatively generating enough ideas, evaluating them to predict which will be successful, then applying internal discipline to drop support for those that won’t work while shifting time, money, and attention to driving the best into the market. By contrast, when there is an ineffective balance between formal and informal structures, it often shows up as an inability to manage bad ideas effectively. After a formal decision has been made to advance some ideas but not to pursue others, the company expends considerable effort to plan the next steps for the winners. But no one thinks actively of planning next steps for the losing ideas, to put them to rest, free up their supporting resources, and (ideally) identify and share any lessons or insights gleaned from the experience. … When the balance is ineffective, the informal parts of the organization quietly kick into high gear to ‘defend the abandoned children.’ Advocates of the failed project continue to spend time and resources on it, out of either inertia or false hope.”

Khan and Katzenbach assert that putting projects on the chopping block is especially difficult in companies that have a “hero culture” in which the most recognition is given “to the ‘creative genius’ who pushed forward despite resistance.” They continue:

“The formal parts of the organization tacitly reinforce this pattern. Metrics, and thus career advancement and recognition, are tied to expected results. People know that an innovator’s role in a successful product launch could be part of the justification for a promotion, raise, or merit bonus. And they know that no one is rewarded for failure. Thus, in very formal organizations, no one may explicitly shun a failed product, but employees have little or no incentive to celebrate or learn from failures. People try as long as possible to turn them into successes, and dispose of them only when it is clear no hope remains.”

To put a company in synch, Khan and Katzenbach make four recommendations:

1. Weigh the balance of formal and informal mechanisms to meet your company’s current needs. … It is typically only after several years that leaders recognize that they need formal protocols, decision rules, practices, and metrics, applied across the board, to manage complexity more efficiently and ensure that people are treated fairly. After that, they may overcompensate with formal rules and processes borrowed from other companies. These tend to conflict with the informal rituals — practices that, because of their responsiveness and adaptability, were once core to their success — thereby diminishing the effectiveness of both. … To avoid these patterns, look at your organization as a whole, particularly in relation to R&D. Figure out ways to shape the formal and informal structures together, integrating them from the beginning. …

2. Capture budding ideas from the widest possible net and collect them centrally. In most organizations, a subset of people in marketing or product development are formally tasked with developing and testing new ideas. They are typically chosen not because they are the most creative and insightful individuals, but because they know how to navigate the development process once an idea has been formed: They know how to get it approved or funded. Truly innovative companies like Apple or Google, by contrast, make generating ideas an informal part of everyone’s job and motivate employees largely with nonmonetary recognition. …

3. Involve multiple perspectives in “go/no-go” choices from the outset, and thus make them stick. Most organizations have discrete formal groups and processes that use different lenses for evaluating ideas: Marketing represents the customers, finance evaluates the economics, and engineering determines feasibility for launch. They answer the questions in series, and then ‘throw the problem over the wall’ to the next team. They may not even be aware of one another’s findings. The principles of focused accountability or clear decision rights provide a purported rationale for this fragmented approach. Breaking up the innovation process often seems like the easiest way to make progress. However, it ignores the fact that truly effective innovation needs to integrate choices about customers, finance, and technology; without buy-in at the outset from all these groups, choices made upstream may be undone downstream. …

4. Motivate the right behaviors. Build incentives and practices, both formal and informal, that will support the right kinds of implementation when no one is watching. … Truly innovative companies deliberately celebrate failure as well as success. … These [companies] also carefully plan for reallocating resources after killing an idea. The ‘funeral’ for any failed project resembles a celebration rather than a somber mass. And the lessons from failed projects — including both technical breakthroughs and insights about the market — are applied in other projects.

Isabelle Royer believes that organizations “need to recognize the role of ‘exit champions’: [individuals] with the temperament and credibility to question the prevailing belief, demand hard data on the viability of the project, and, if necessary, forcefully make the case that it should be killed.” She also recommends that companies be on the lookout for cheerleading squads that continue to support bad ideas and that companies establish an early warning system that alerts them that a program is likely to fail. The further down the road a program goes the more difficult it is to kill it off. The best practice is to use the chopping block early in the process, but not so early that ideas get beheaded before they have had a chance to be fully vetted.