Dennis K. Berman writes, “There is [an] important theme gathering around us: How analytics harvested from massive databases will begin to inform our day-to-day business decisions. Call it Big Data, analytics, or decision science. Over time, this will change your world more than the iPad 3.” [“So, What’s Your Algorithm?” Wall Street Journal, 4 January 2012] Berman believes that Big Data analytics are going to be important because they will objectively make sense of the mountains of data being generated each day. Without this objectivity he believes that “we are ruined by our own biases.” He continues:
“When making decisions, we see what we want, ignore probabilities, and minimize risks that uproot our hopes. What’s worse, ‘we are often confident even when we are wrong,’ writes Daniel Kahneman, in his masterful new book on psychology and economics called ‘Thinking, Fast and Slow.’ An objective observer, he writes, ‘is more likely to detect our errors than we are.’ … Computer systems are now becoming powerful enough, and subtle enough, to help us reduce human biases from our decision-making. And this is a key: They can do it in real-time. Inevitably, that ‘objective observer’ will be a kind of organic, evolving database.”
Before we simply accept that Big Data analytics are, by nature, objective, we need to remember that algorithms can be created to feed biases as well as ignore them. Simon Dell, director of TwoCents Group, an Australian marketing, advertising and branding company, believes that as a result of increased use of algorithms, “We’re in danger of losing the spontaneity in our lives. Well, at least our digital lives.” [“Algorithms want to rule the world,” posted by Peter Roper, Marketing Magazine, 10 October 2011] Dell explains:
“Google was built on the back of ‘I’m feeling lucky’, but now we’re slaves to what our social networks and our search engines want to tell us we should be looking at and who we should be connected to. We’re gradually collapsing in on ourselves on the premise that someone somewhere is trying to save us time. … Facebook decided to roll out some significant changes, including a removal of our option to switch between all stories and top news. Instead, our news feed has been decided for us, based on a complex algorithm and delivered to us as ‘top stories’. The tool that originally allowed us to filter meaningless news out of lives, and allow us to choose who and what we followed, has now come full circle and is choosing for us. And there’s no off button.”
Dell is not alone in his concern about how algorithms are trying to “rule the world.” In a speech delivered by Kevin Slavin delivered at TEDGlobal last year, he “argues that we’re living in a world designed for — and increasingly controlled by — algorithms. He shows how these complex computer programs determine: espionage tactics, stock prices, movie scripts, and architecture. And he warns that we are writing code we can’t understand, with implications we can’t control.” [TED site] Returning to Simon Dell’s article, he argues that there is “a shadowy figure that now stands over us: the once-innocent algorithm. “He continues:
“Google have been developing algorithms for years – it’s the basis for their entire business model – but as Eli Pariser revealed in his TED talk, those algorithms now deliver different search results based on who is doing the search. No longer did we all get the same search feed, but our location, age, sex and previous searching and browsing habits combine to deliver a result tailored just for us.”
Pariser argues that algorithms can be used to filter, isolate, and present information in ways we may not like but have no control over. Dell, a marketer, is concerned that algorithms can be used in a very intrusive way to manipulate our lives. Some people seem happy about all this tailoring and even willingly offer up their habits to share with others. Others, of course, are concerned over privacy issues, ethical issues, and control issues. For more on this topic, read my post entitled The Big Data Dialogues, Part 5: Algorithms.
Berman, unlike the commentators cited above, takes a much more positive view of algorithms. He understands that companies providing business intelligence (BI) or analytics to other companies will not survive if the analysis they perform is tainted or biased. He continues:
“These systems can now chew through billions of bits of data, analyze them via self-learning algorithms, and package the insights for immediate use. Neither we nor the computers are perfect, but in tandem, we might neutralize our biased, intuitive failings when we price a car, prescribe a medicine, or deploy a sales force. This is playing ‘Moneyball’ at life. It means fewer hunches and more facts. … Crunching millions of data points about traffic flows, an analytics system might find that on Fridays a delivery fleet should stick to the highways— despite your devout belief in surface-road shortcuts. You probably hate the idea that human judgment can be improved or even replaced by machines, but you probably hate hurricanes and earthquakes too. The rise of machines is just as inevitable and just as indifferent to your hatred.”
Berman recognizes that Big Data analysis is only now becoming “the next big thing” because it took a combination of factors to come together before it could be accomplished at a price that made sense. He continues:
“Until the last few years, [Big Data analysis has] been stymied by the cost of storage, slower processing speeds and the flood of data itself, spread sloppily across scores of different databases inside one company. These problems are now being solved. ‘We’ve just got to the point where the technology really starts to work,’ says Michael Lynch, chief executive of Autonomy Corp. Hewlett-Packard Co. just spent $11 billion to buy Autonomy, which vacuums up ‘unstructured data’ then applies it to these analytic approaches. Of course, the hype is growing fast, too. Company valuations in this space have pushed higher, and surely some will falter along the way.”
The lesson that we learned during dot.com bubble and burst was that sound business practices and numbers still matter. The same will hold true for companies offering Big Data analysis. Those that can provide a good return on investment for their clients will thrive. Those that can’t will fail. Berman continues:
“That won’t matter much in the long run. The story of 2012 is how these technologies are inching closer to each one of us. … ‘There is a whole class of things that couldn’t be done five years ago,’ says Arnab Gupta [CEO of Opera Solutions Inc. of New York, an eight-year-old analytics firm], who just landed an $84 million venture investment from investors including Accel-KKR and Silver Lake Sumeru. His company is now valued at around $500 million. ‘A few years ago it might take a month to run a project involving 30 billion separate calculations. Today it can be done in two to three hours.’ The big goal is to push all the heavy back-end work forward to front-line workers, often as a ‘dashboard’ on a handheld device.”
Obviously, gathering mountains of data is a futile pursuit if it just sits there piling up like garbage in a hoarders home. That’s what makes analysis and visualization so important. Berman explains:
“Soon, a drug saleswoman will have real-time analytics that tell her to focus on the doctors who spent time on social networks that morning, and who are thus more apt to influence colleagues, says Dhiraj C. Rajaram, founder of analytics company Mu Sigma, of Northbrook, Ill. Last week Mu Sigma raised $108 million in venture funding from General Atlantic and Sequoia Capital. A warning awaits, of course. As Mr. Rajaram explains, analytics will eventually become the norm, which will push adaptation and business cycles even faster than they are today. ‘As computers become better and better, our lives are becoming more and more complex. They create new problems as much as they solve old ones.'”
More and more analysts are talking about the clock speed of businesses. Not all businesses have to have real-time analysis; but, they closer they come to that ideal the better sales and operational planning processes they will be able to implement. Berman concludes:
“Until then, we should take some comfort—however difficult it may feel—that machines will help us eliminate our worst human tendencies. Mr. Kahneman reminds us best: ‘We often fail to allow for the possibility that evidence that should be critical to our judgment is missing. What we see is all there is.'”
Some critics believe that the dark side of Big Data involves privacy concerns. One has to admit that there is a touch of Big Brother embedded in the Internet, which tracks our every move adds to what is known about us. To overcome some of this stigma, companies have begun to collect personal data more overtly using “gamification.” [“You’ve Won a Badge (and Now We Know All About You),” by Natasha Singer, New York Times, 4 February 2012] Singer reports, for example, that Samsung uses a game-like promotion called Samsung Nation. She continues:
“If Samsung Nation sounds a little like the social network game FarmVille, minus the farm, it’s no accident. Samsung is embracing a business trend called gamification, which takes elements from games and applies them to other settings. Companies like Recyclebank, for example, use game incentives, like points and rewards, to prompt consumers to perform eco-friendly activities. Other businesses offer awards like virtual badges to induce their employees to embrace corporate goals and increase productivity. Meanwhile, a number of well-known retailers and brands, including Samsung and Warner Brothers, are employing point reward systems as a way to engage customers more deeply. … For companies, the premise of gamification is that it engages people in the kind of reward-seeking behaviors that lead to increased brand loyalty, not to mention increased profits. By tracking the online activities of people who sign up for such programs, companies can also amass more detailed metrics about each user — the better to identify the most active customers.”
Gamification, like most things in life, has its critics. According to Singer, “Critics say the risk of gamification is that it omits the deepest elements of games — like skill, mastery and risk-taking — even as it promotes the most superficial trappings, like points, in an effort to manipulate people.” Ian Bogost, a professor of digital media at the Georgia Institute of Technology, told Singer that he considers gamification software “exploitationware.” Singer explains:
“Consumers might be less eager to sign up, he argues, if they understood that some programs have less in common with real games than with, say, spyware. ‘Why not call it a new kind of analytics?’ says Professor Bogost, a founding partner at Persuasive Games, a firm that designs video games for education and activism. ‘Companies could say, “Well, we are offering you a new program in which we watch your every move and make decisions about our advertising based on the things we see you do.”‘ … Professor Bogost cautions that virtual brownie points have the potential to exploit people without offering them much in return.”
Margaret Robertson, development director at Hide & Seek, a game design studio, told Singer that consumers may start turning off to online gamification because of oversaturation. “There is probably a backlash coming,” she told Singer. In fact, she predicts that some companies may soon trumpet their anti-gamification credentials with promotions touting, “No points. No annoying missions. Just clean services.” You notice, however, that “No gathering data” was missing from that promotion. Big Data collection and analysis is here to stay. Used responsibly, that’s a good thing.
