Home » Big Data » Mining Mountains of Data for Insights

Mining Mountains of Data for Insights

May 6, 2021


Most of us try to improve our understanding of the world around us by comparing things that may be difficult to comprehend with things we do recognize. That’s why metaphors and analogies are so popular in literature. In the area of big data, geological features are often used to depict the enormity of the data involved. For example, Boian Alexandrov and Velimir ‘Monty’ Vesselinov, scientists at the Los Alamos National Laboratory, call big data a “bottomless ocean” and marketing expert Angela Hausman (@MarketingLetter), compares big data to “mountains.” Alexandrov and Vesselinov write, “A seemingly bottomless ocean of ‘big data’ has flooded our world. Bits and bytes are pouring in from sources ranging from satellites and MRI scans to massive computer simulations and seismic-sensor networks, from security cameras to smartphones, from genome sequencing of SARS-Cov-2 to COVID-19 test results, from social networks to texts zipping from phone to phone.”[1] Hausman adds, “Everybody knows that big data is a driving force in business and it’s more important than ever to collect and understand information about your customers, competitors, and business process. The problem with big data is that drawing valuable insights from the mountains of data produced by your business, as well as external data, defies efforts to collect, organize, analyze data leading to valuable insights that optimize your business for success.”[2]


Mining the Mountain


As Hausman notes, collecting, organizing, and analyzing data are difficult challenges. Alexandrov and Vesselinov are trying to make those challenges easier. They are working on an artificial intelligence tool called SmartTensors. They explain, “Without any human guidance, this technology sifts through millions of millions of bytes of diverse data to find the hidden patterns and features that make the data understandable, revealing its underlying processes or causes. SmartTensors also can identify just how many features are needed to make sense of enormous, multidimensional datasets. In analyzing data, finding that optimal number reduces a massive set of data to a scale that’s manageable for computers to process and subject matter experts to analyze. The features that SmartTensors extracts are explainable and understandable chunks of data.”


All cognitive technologies that leverage machine learning mine data to find hidden patterns. They perform best, however, when trained on the types of data they will be routinely analyzing. The businesses that best leverage cognitive technologies know the types of data they need to mine for insights. Hausman reports that many companies haven’t learned that lesson. She explains, “The tools for collecting data are better than ever, leading businesses to believe they should capture every bit of data available. But what can you do with all of that data? The answer is that many businesses … do nothing of any value with the data. So many businesses, especially small companies, find themselves buried underneath a huge mountain of data and they have no idea how to manage it. They’re not drawing any useful insights from the data they collect, which means that collecting and storing data, with the associated costs, is a waste of time.”


In answer to Hausman’s question about what you can do with big data, you have to know what kind of answers or insights for which you are looking. George Karapalidis (@gkarapalidis), head of data science at Vertical Leap, explains the purpose of each type of analytics.[3] They are:


1. Descriptive analytics. Descriptive analytics can help discover what happened in the past. As Karapalidis puts it, “Before we learn where to go, we need to know where we came from. That’s the key question descriptive analytics solutions tackle.”


2. Diagnostic analytics. Diagnostic analytics can help explain why something happened. Karapalidis writes, “Diagnostic analytics tools help you uncover the root cause of some problems.”


3. Predictive Analytics. “Predictive analytics,” writes Karapalidis, “‘joins the dots’ between the accumulated and analyzed data points, conveying what and why something happened, into models suggesting what can happen next. It indicates the probability of certain outcomes with high accuracy and takes the guesswork out of your decision-making process.”


4. Prescriptive analytics. Prescriptive analytics informs you what you should do to achieve a particular outcome. It’s a type of analytics made possible by the emergence of cognitive computing technology. Karapalidis notes, “Prescriptive analytics is yet to move from the margins to the mainstream. It’s an emerging area of analysis attempting to answer the complex question of ‘what actions to take if I want to get outcome A?’ Prescriptive tools come up with multiple future outcomes based on your current/past actions; match those futures with your goal and advise you on the action you need to apply.”


Knowing what you want to know, finding the right data in which the answers are locked, and using the right analytics models will reward your company with the best answers and insights.


Beyond Big Data Analytics


Vivek Kumar, Content Lead at Analytics Insight, observes, “The proliferation of big data analytics solutions has significantly redefined businesses’ data processing over the years. It has already proven a key solution for identifying and deriving meaningful insights from vast datasets. With emerging technologies like artificial intelligence, machine learning and the cloud, data professionals are now leveraging cognitive analytics to drive real-time decision making. It presents much greater potential than big data analytics, unlocking the value of big data by making a system more self-reliant, and information contained more accessible.”[4] Rather than discuss cognitive analytics in general terms, Terence Mills (@terence_mills), CEO of AI.io, discusses five specific ways cognitive analytics can benefit businesses.[5] They are:


1. Customer Acquisition And Retention. Every business needs customers. Mills asserts, “To stand out, organizations must have a unique approach to market their products. By using big data, companies can pinpoint exactly what customers are looking for. They establish a solid customer base right out of the gate. New big data processes observe the patterns of consumers. They then use those patterns to trigger brand loyalty by collecting more data to identify more trends and ways to make customers happy.”


2. Focused and Targeted Campaigns. John Wanamaker, the late department store magnate, once stated, “Half the money I spend on advertising is wasted. The trouble is I don’t know which half.” Cognitive analytics can help. Mills explains, “Businesses can use big data to deliver tailored products to their targeted market. Forget spending money on advertising campaigns that don’t work. Big data helps companies make a sophisticated analysis of customer trends. This analysis usually includes monitoring online purchases and observing point-of-sale transactions. These insights then allow companies to create successful, focused and targeted campaigns, thus allowing companies to match and exceed customer expectations and build greater brand loyalty.”


3. Identification of Potential Risks. “These days businesses are thriving in high-risk environments,” Mills writes, “but these environments require risk management processes — and big data has been instrumental in developing new risk management solutions. Big data can improve the effectiveness of risk management models and create smarter strategies.”


4. Innovative Products. A common saying heard in business circles is, “Innovative or die.” Mills writes, “Big data continues to help companies update existing products while innovating new ones. By collecting large amounts of data, companies are able to distinguish what fits their customer base. If a company wants to remain competitive in today’s market, it can no longer rely on instinct. With so much data to work off of, organizations can now implement processes to track their customer feedback, product success and what their competitors are doing.”


5. Complex Supplier Networks. Supply chains are complex and cognitive analytics can help deal with that complexity. Mills explains, “By using big data, companies offer supplier networks, otherwise known as B2B communities, with greater precision and insights. Suppliers are able to escape constraints they typically face by applying big data analytics. Through the application of big data, suppliers use higher levels of contextual intelligence, which is necessary for their success.”


Concluding Thoughts


Mills concludes, “Advanced software systems greatly reduce analytics time, giving companies the ability to make speedy decisions that help increase revenue, reduce costs and stimulate growth.” As Bain analysts, Michael C. Mankins and Lori Sherer (@lorisherer), assert if you can improve a company’s decision making you can dramatically improve its bottom line. They explain, “We know from extensive research that decisions matter — a lot. Companies that make better decisions, make them faster and execute them more effectively than rivals nearly always turn in better financial performance. Not surprisingly, companies that employ advanced analytics to improve decision making and execution have the results to show for it.”[6]


[1] Boian Alexandrov and Velimir ‘Monty’ Vesselinov, “Finding the needles in ‘big data’ haystacks,” Albuquerque Journal, 7 March 2021.
[2] Angela Hausman, “Drawing Valuable Insights from a Mountain of Data,” Business2Community, 6 February 2021.
[3] George Karapalidis, “Examining the four types of big data analytics,” The Drum, 18 December 2018.
[4] Vivek Kumar, “How Can Cognitive Analytics Go Beyond Big Data Analytics?” Analytics Insight, 8 December 2020.
[5] Terence Mills, “Five Benefits Of Big Data Analytics And How Companies Can Get Started,” Forbes, 6 November 2019.

[6] Michael C. Mankins and Lori Sherer, “Creating value through advanced analytics,” Bain Brief, 11 February 2015.

Related Posts:

Full Logo


One of our team members will reach out shortly and we will help make your business brilliant!