“We’re not asking enough of Big Data, and we’re still getting in its way,” writes Stephen Collins, Chief Executive Officer of Bazaarvoice. “We should be using Big Data to do what we can’t, not just help us do what we’re already doing, better.” [“Big Data needs big problems,” Bazaarvoice: Blog, 29 July 2013] I like Collins’ headline. It’s optimistic about the potential of big data analytics. When it comes to big challenges, what could be bigger than saving the world? You might be asking, “Save it from what?” Good question. As you will read below and in the final segment of this series, proponents of Big Data have a number of things in mind. Collins believes that we need to establish a new paradigm for approaching problem solving. The old paradigm, he writes, involves defining the problem, then training the machine (i.e., the computer) to solve it. The new paradigm, he insists, should involve training the machine to define the problem. He explains:
“Right now, humans are still better at at least one thing: defining the problem. But that’s not to say Big Data can’t eventually outdo us in this arena, too. If we can train machines to solve problems, we can train them to define them as well. Once this happens, we close the circuit—the true power of Big Data is realized. The system starts working on its own, and better than ever before. Here are the conditions which must be met for Big Data to flourish:
- It must be ubiquitous; accessing data from everywhere we let it.
- It must be always on, constantly gathering, processing, comparing, and acting.
- It must be empowered to act in the moment.”
I certainly agree with the first two conditions; but, I would have to add a few modifiers to the third one. Empowering action implies that whoever is in charge of the computer has the authority to act. When it comes to solving the world’s problems, there simply is no single organization or individual with that kind of authority. Does that mean that Big Data can’t be useful in solving the world’s challenges? Certainly not. Mike Wheatley writes, “Big Data’s reputation has taken a bit of a battering lately thanks to allegations that the NSA is collecting and storing people’s web and phone records, leading to a wider debate about the appropriateness of such extensive data-gathering operations. But this negative publicity detracts from the reality of Big Data today, which for the most part will only benefit society as a whole. There’s more to these massive data sets than simply catching terrorists (or spying on law abiding citizens).” [“Big Data’s Still On Track To Save The World,” SiliconANGLE, 9 July 2013] Analysts involved in the Global Pulse initiative, an innovation initiative launched by the Executive Office of the United Nations Secretary-General, agree with Wheatley that Big Data has the potential of addressing significant world challenges (as the following video demonstrates).
Mark van Rijmenam, founder of BigData-startups.com, also believes that Big Data is going to make the world a better place. He notes that “the Engineering Social Systems department (ESS) of Harvard has collected several inspiring use cases.” [“How Big Data Can Help the Developing World Beat Poverty,” SmartData Collective, 2 August 2013] He continues:
“Big data offers for example the possibility to predict food shortages by combining variables such as drought, weather conditions, migrations, market prices, seasonal variation and previous productions. Or what about the possibility to better understand the dynamics of slum residents using mobile data to develop predictive models to better serve the poorest? For example using [Call Detail Records (CDR)] information to map changes in the slum population and direct latrine and water pipe building efforts for the benefit of the slums residents. Time-series analyses performed on CDR combined with random surveys can lead to better insights about the dynamics of rural economies and provide insights on how governments should respond to economic shocks in rural and poor environments. The World Bank shows an example where big data is used to ensure the right distribution of the right medicines to the right location at the right moment in time. A pilot programme called SMS for Life improved the distribution of malaria drugs at a health facility level in rural Tanzania, reducing facilities without stock from 78% to 26%.”
One thread that weaves its way through the Global Pulse initiative, as well as van Rijmenam’s observations, is ubiquity of mobile devices. Van Rijmenam writes:
“For the vast amount of the poor, a simple or basic mobile phone is the only interactive interaction with the World Wide Web. Although in the developed world smartphones may seem to be the common device, they still only account for 10.44% of the global mobile website traffic. On the other hand, traditional mobiles take up 78.98% of mobile worldwide website traffic (with tablets taking 10.58% of the traffic). Luckily there are vast opportunities for the developing world to use data created by basic mobile devices to identify needs, provide services, and predict and prevent crises for the benefit of the poor.”
Gillian Tett adds:
“These days, there are about 2.5 billion people in emerging markets countries who own a mobile phone. In places such as the Philippines, Mexico and South Africa, mobile phone coverage is nearly 100 per cent of the population, while in Uganda it is 85 per cent. That has not only left people better connected than before – which has big political and commercial implications – it has also made their movements, habits and ideas far more transparent. And that is significant, given that it has often been extremely hard to monitor poor societies in the past, particularly when they are scattered over large regions.” [“Big data is watching you,” Financial Times, 10 August 2012]
Mike Wheatley asserts that one source of global data stands out above the rest when it comes to capturing real-time information – Twitter. “Few are as superior as Twitter,” he writes, “and not just because of its widespread user-base that’s spread across the globe. More important, tools such as TwitterHose facilitate this data calection, allowing anyone to download 1% of tweets made during a specified hour at random, giving researchers a nice cross-section of the Twitterverse.”
Although mobile data provides a wealth of useful information, van Rijmenam understands that mobile data alone is insufficient to address all potential problem areas. He concludes:
“Big data can be as a catalyst for long lasting improvements, but we will have to look further ahead to see that. Mobile data alone is not sufficient to really create opportunities that could impact developing countries on the long term. Therefore, more data sources are required, ranging from data from NGO’s, to public data and social data. … Big data offers many opportunities for the developing world to beat poverty, but it will require different organisations to work together in order to achieve lasting results. In addition, the joining organisations should ensure transparency and availability of the data. Transparency will stimulate awareness of the possibilities, ensure data accountability and reduce bureaucracy and corruption. Availability of the data will ensure that multiple data sources can be fused, such as CDRs, open data, social data, government data, NGO data and corporate data, to create valuable and relevant new insights that will truly have a long term impact.”
Van Rijmenam’s anecdotal use cases touch on many of the areas that have been identified as areas in which Big Data can be used to address seemingly intractable problems: agriculture, finance, healthcare, poverty reduction, and disaster response. In the final part of this discussion, I’ll look a little closer at how Big Data can help solve problems in each of these areas.
