Crime shows on American television that highlight modern technologies or techniques almost always exaggerate what is actually available in real life — on the small screen, DNA is isolated and evaluated in minutes, grainy security camera images are made crystal clear, facial micro-expressions instantly reveal liars, and so forth. In the television show “Numbers,” university professors aid the FBI by applying various mathematical formulas and algorithms to hunt down bad guys and solve crimes. In a recent column, Washington Post columnist Shankar Vedantam discusses how such mathematical models really can assist decision makers [“The Computer as a Road Map to Unknowable Territory,” 16 February 2009]. Vedantam discusses the work of scientist named Yaneer Bar-Yam, who developed an economic computer model that uses virtual actors. The nice thing about virtual worlds and virtual actors is that you can do things in them that are forbidden in the real world. In Bar-Yam’s case, he was able to ignore and/or manipulate regulations and “change the way actors behaved and then study how those changes rippled through a complex ecosystem.”
Bar-Yam’s aim was to examine how various regulations affected individual behaviors. Models permit researchers to observe and analyze complex systems in ways that are unavailable using real world observations. As Vedantam notes, “In a system as complex as the economy, where feedback loops of rumor, fear and misinformation regularly trigger panic and herd behavior, the ability of individuals to forecast outcomes can diminish rapidly. The normal rules of human intuition break down: A positive intervention — the federal government announcing it is going to pump trillions of dollars into the economy — can be greeted by a plunge in the stock market. Trivial things can get amplified and assume gigantic proportions.” Because model variables can be manipulated, researchers can discover unintended consequences of change.
During early attempts at modeling complex behaviors, scientists believed that massive numbers of complicated algorithms would have to be used. It turns out that in many cases, complex behavior can be duplicated by introducing just a few simple rules. For example, as I wrote in a previous post: “Ants have never been accused having massive brains that work out complicated survival strategies. Yet, when necessary, they build bridges, construct columns, and dig amazingly complex nests — all by obeying a few simple rules.” By manipulating the rules, changes in behavior can be studied. That is basically what Bar-Yam did. Like most researchers, his curiosity had been peaked by a real-world observation — in his case, why the economy was unstable. Bar-Yam was not sure that the sub-prime loan crisis that caused the real estate bubble to burst was the cause of the crisis or a symptom of some other factor. Vedantam provided an analogy. He wrote: “If you take away one of the supports of a house built on stilts and a storm knocks the house down, the problem is not the storm but the missing support. If you rebuild the house on its shaky base — but put in expensive new storm windows — you are unlikely to fare better when the next storm rolls around.”
Bar-Yam’s model revealed that, in fact, one of the supports of the economic system had been removed and that the resulting consequences contributed to system’s instability.
“In July 2007, the Bush administration eliminated a 69-year-old regulation known as the uptick rule. It had been put in place by Joseph Kennedy, the first commissioner of the Securities and Exchange Commission, who had himself profited from the wild economic gyrations of the previous decade. (Kennedy’s appointment as SEC chief was tantamount to installing a fox to guard the henhouse, but President Franklin D. Roosevelt explained that it took a crook to catch a crook.) The uptick rule was designed to prevent bear raids: If a powerful investor suddenly sells a large number of shares in a company, he can temporarily create a situation in which the supply of shares far outstrips the demand. The fall in the share price can be greatly amplified by feedback loops of rumor and misinformation. Once the stock’s value is in the toilet, the crooked investor can swoop in and buy the shares back at an artificially discounted price.”
Using Vedantam’s analogy, Bar-Yam’s model identified the housing crisis as the storm and the uptick rule as one of the supports that secured the house on its base. Eliminating the uptick rule placed the economic system in jeopardy regardless of what storm came along. Vedantam reports that “the incoming head of the SEC in the Obama administration, Mary Schapiro, has promised to revisit the uptick rule.” Vedantam’s column is not about assessing blame for the financial crisis; rather, it focuses on the utility of models in helping decision makers establish policy or manage crises. Vedantam is quick to point out that models can be useful, but that they are not fortunetellers.
“Several caveats are in order when it comes to the computational analysis, which was conducted at Bar-Yam’s New England Complex Systems Institute. Computer models are not oracles — they cannot tell you with certainty that a change in a complex system causes a particular outcome. To put it another way, Bar-Yam and his co-author Dion Harmon might be wrong about the uptick rule.”
Vedantam points out that old Wall Street hands, such as Jim Cramer of the TV show “Mad Money” called for the elimination of the uptick rule months before the computer analysis came out. But it was also old Wall Street hands that guided the U.S. economy into the morass it now finds itself. The point that Vedantam makes about computational models not being oracles was also made during one episode of the television show “Numbers.” In that episode, the good Professor Eppes, the mathematical genius who helps solve crimes, convinces his brother, the FBI agent, that he has developed a computational model that will help destroy the market for the next big drug before it becomes the next big drug. The problem was that “real life” actors behaved differently than the model’s virtual actors and the plan backfired. So what is the value of computational models if they aren’t predictive? Vedantam writes:
“The virtue of computational models is that when you are confronted by a dizzying array of potential problems, they can tell you where to focus your attention. If market regulators have dozens of options, a model can tell them which ones are more likely to work. … In dealing with complex systems, it is not always clear which problems precede others, because everything in a complex system is interconnected.”
When looking to solve any problem, if you ask the right question you have a better chance of getting the right answer. Ask the wrong question (or fail to identify the right rule) and you are bound to head out on a wild goose chase. Setting up the model correctly, with the right rules, is essential to extracting valuable insights. Vedantam reports that computational models have been in a number of social contexts — everything from air traffic control to public health. The common thread, he notes, “is that leaders in every case are asked to make decisions in complex situations with uncertain outcomes.” The other value of computational models is that they can reveal obscure relationships. Vedantam writes about a computational model developed by V.S. Subrahmanian and Jonathan Wilkenfeld at the University of Maryland. The model was developed to examine the complex security situation between Israel and the Palestinians. “One conclusion of their model is that the militant group Hezbollah is more likely to lob rockets into Israel when elections are being held in Lebanon — some proportion of the attacks are meant to impress a domestic audience.” Another model developed at the University of Maryland, Vedantam reports, demonstrated that infant mortality rates can help predict the likelihood that a particular country will be stable or unstable. He notes that such observations could be made in the field, but that computational models can identify those kinds of relationships faster and more cheaply. They also allow some experimentation to see what kinds of activities might be useful to solve the challenge at hand.
Correctly designed computational models could help those in the development community prioritize projects. For example, is it more important to have the transportation infrastructure in place to move commodities to market or to have the commodities available first? Such “chicken and egg” discussions can be helped through computational analysis. Vedantam asserts, for example, that the model that showed a relationship between infant mortality and instability could “tell us to pay preferential attention to infant mortality over, say, hunger or poverty or religious strife.” Identifying an important variable, like infant mortality, allows decisions makers to ask important “why” and “what if” questions. Infant mortality may be an important predictor of instability, but what is causing a high infant mortality rate (poor nutrition, disease, etc.)? Answering those questions will provide the key to reducing infant mortality and increasing stability. Infant mortality is a symptom of the problem, not the cause. Anything that helps focus attention on critical areas is useful. Models are tools that should be in any kit. Vedantam concludes:
“Our culture celebrates intuitive leaders who make brilliant calls — even when we suspect their success was largely luck. Computational models, which speak the language of scientific doubt, are less sexy, but they can tell a president who takes empirical evidence seriously where public health dollars, battlefield troops and financial interventions can have the greatest impact.”
The problem, of course, is that computational models don’t vote. Too many decisions about how best to claw our way out of the current financial crisis are going to be made using emotional arguments rather than calculated analysis. That is why protectionism is likely to increase along with international tensions. Jobs in industries that will not survive in the long-run are likely to be put on life support using money that could have been better used to create new jobs in sustainable industries. Let’s hope that some of the billions of dollars being splashed about go to scientists creating models that could help policymakers spend the stimulus money more productively.
