The phrase “size matters” occasionally enters casual conversations — often accompanied by a wink and a smirky smile. When discussing the size of datasets, however, winks and smiles are often difficult to find. How to handle big data is a serious challenge. The term “big” data seems almost quaint when talking about the amount of data currently being generated every second of every day. The World Economic Forum has declared big data a valuable resource like oil. Some pundits have even called big data the new oil. The staff at Analytics Insight notes, “It has been a widely acknowledged fact that big data has become a big game changer in most of the modern industries over the last few years. As big data continues to permeate our day-to-day lives the number of different industries that are adopting big data continues to increase. It is well said that when new technologies become cheaper and easier to use, they have the potential to transform industries. That is exactly what is happening with big data right now.”[1]

Big data analytics are changing industries

Data lying fallow in datasets is no more useful than oil pooled underground. What makes data valuable is advanced analytics. The Analytics Insight staff acknowledges this truism and lists ten industries “redefined the most by big data analytics.” Those industries are:

1. Sports. Anyone familiar with sports has heard about Michael Lewis’ book Moneyball. In the book, Lewis notes, “Data Science is about identifying those variables and metrics that might be better predictors of performance.” The Analytics Insight staff notes, “Most elite sports have now embraced data analytics.”

2. Hospitality. According the Analytics Insight staff, “Hotel and the luxury industry have turned to advanced analytics solutions to understand the secret behind customer satisfaction initiatives.”

3. Government and Public Sector Services. Many cities are hoping big data analytics can transform them into smart cities. The Analytics Insight staff reports, “Data science, and big data have helped a number of cities to pilot the smart cities initiative where data collection, analytics and the IoT combine to create joined-up public services and utilities spanning the entire city.”

4. Energy. The power industry was one of the first to recognize the importance of data analytics and the term “smart grid” soon followed. The Analytics Insight staff observes, “Data and the Internet of Things (IoT) is disrupting the way we use energy in our homes.”

5. Agriculture and Farming. No area is more critical to sustain a growing population than agriculture. Analytics is now being used to help farmers reduce the use of pesticides, fertilizers, and water while simultaneously increasing crop yields.

6. Education. The Analytics Insight staff notes, “Important insights can identify better teaching strategies, highlight areas where students may not be learning efficiently, and transform how the education is delivered.”

7. Banking and Securities. According to the Analytics Insight staff, “The Securities Exchange Commission (SEC) has deployed big data to track and monitor the movements in the financial market. Big data and analytics with network analytics and natural language processors is used by the stock exchanges to catch illegal trade practices in the stock market. … This industry also heavily relies on big data for risk analytics including; anti-money laundering, demand enterprise risk management, ‘Know Your Customer’, and fraud mitigation.”

8. Entertainment, Communications and the Media. The best known use of analytics in the entertainment industry is understanding consumer preferences and then using recommendation engines to suggest content the consumer may enjoy.

9. Retail and Wholesale Trade. The Analytics Insight staff asserts, “Big data is applicable to the retail and wholesale industry to mitigate fraud, offer customized products to the end user thereby improving the user experience.”

10. Transportation. “Big data analytics finds huge application to the transportation industry,” according to the Analytics Insight staff. “Governments of different countries use big data to control the traffic, optimize route planning and intelligent transport systems and congestion management.”

Lex Boost (@aeboost), CEO of Leaseweb USA, reports, “According to an Accenture study, 79% of enterprise executives agree that companies not embracing big data will lose their competitive edge, with a further 83% affirming that they have pursued big data projects at some point to stay ahead of the curve.”[2]

Quality insights are the real goal

Although it seems some companies are gathering data just to tout the size of their datasets, what really matters is the quality of the insights that can be gleaned from collected data. Kalev Leetaru (@kalevleetaru), a Senior Fellow at the George Washington University Center for Cyber & Homeland Security, observes, “It has become an unfortunate trend of the ‘big data’ world that we focus on the size of the datasets we analyze, rather than the number of useful datapoints they contain or the insights we gain from them.”[3] He continues, “In our data-driven arms race, we’ve lost track of the simple fact that ‘big data’ came into being to yield insights not possible at smaller scale, not as a marketing term to tout how much data we have obtained or how inefficiently we can store and work with it.” One of the greatest values of big data analytics is to help companies make better decisions based on the insights provided. Bain analysts, Michael C. Mankins and Lori Sherer (@lorisherer), assert if you can improve a company’s decision making you can dramatically improve its bottom line. They explain, “The best way to understand any company’s operations is to view them as a series of decisions.”[4] They add, “We know from extensive research that decisions matter — a lot. Companies that make better decisions, make them faster and execute them more effectively than rivals nearly always turn in better financial performance. Not surprisingly, companies that employ advanced analytics to improve decision making and execution have the results to show for it.”

Boost adds, “Effectively implementing fast data processing will ensure your company is more up-to-date and relevant, and this is extremely important given how diverse data is becoming — a factor that gives us all the ability to analyze more innovatively.” With so many subject matter experts touting the importance of big data analytics, why aren’t more companies taking advantage of advanced analytics? Helena Schwenk (@hmschwenk), a Global Analyst and Market Insights Manager at Exasol, suggests three things are holding companies back: data fragmentation; legacy systems; and data complexity.[5]

Data Fragmentation: “Data flows from a growing number of sources,” Schwenk writes, “including customer transactions, marketing automation systems and the Internet of Things. This has often resulted in ‘data islands’ across databases and legacy archival systems, with inefficient data duplication and multiple, disconnected repositories of data with inconsistent structures.”

Legacy Systems: According to Schwenk, “Legacy infrastructure has also resulted in inefficiencies and latency in performance as these systems were not built to handle the extreme demands of today’s compute and data-intensive workloads.” Often the shortcomings of legacy systems can be overcome by leveraging cognitive platforms, like the Enterra Enterprise Cognitive System™ (AILA®) — a system that can Sense, Think, Act and Learn® — since they enhance rather than replace those systems.

Data Complexity: “To tackle the huge data volumes within the limited constraints of the legacy IT, and the perhaps even more limited resource of the user’s patience,” Schwenk writes, “data professionals have often been forced to simplify the data to speed up results. … But they don’t give a true understanding of the business or the markets it operates in. Significant anomalies may be averaged out of sight, and it’s difficult to respond meaningfully to a headline figure that isn’t backed with granular detail.” A good cognitive platform can deal with data complexity.

Schwenk concludes, “Insight-driven businesses don’t just collect data for the sake of it. They use data in a meaningful, insightful way, that creates a very real competitive advantage.” Leetaru adds, “Rather than treating ‘big data’ as a marketing gimmick that rewards inefficiency and changes in how we store and utilize data, perhaps we should start talking about the insights we gain not the data we processed.”

Footnotes
[1] Staff, “10 Industries Redefined by Big Data Analytics,” Analytics Insight, 2 June 2019.
[2] Lex Boost, “The Power of Crunching Big Data Effectively,” Inside Big Data, 31 March 2019.
[3] Kalev Leetaru, “Big Insights Not Big Data: Why We Should Stop Talking About File Size,” Forbes, 9 January 2019.
[4] Michael C. Mankins and Lori Sherer, “Creating value through advanced analytics,” Bain Brief, 11 February 2015.
[5] Helena Schwenk, “Three barriers to effective data analytics,” Gigabit Magazine, 28 April 2019.