Faced with a constant stream of headlines describing corporate data breaches, consumers are feeling a mixture of anger and fear. As a result, politicians have begun regulating how organizations can gather, store, and use personalized consumer data (e.g., the EU General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) of 2018). These regulations threaten businesses with enormous fines and costly lawsuits if companies fail to protect consumers’ personal data adequately. There is more bad news. Gartner analyst Nick Heudecker (@nheudecker) asserts, for a number of reasons, 85% of big data projects are failures.[1] Alex Woodie (@alex_woodie) notes, “Other tallies of big data, data science, and advanced analytics projects have turned up similar statistics about the rather narrow odds of success in big data.”[2] Given these long odds and potential threats, one might wonder why companies don’t swear off using big data. The answer is simple: Big data is simply too valuable not to be leveraged. Brian Johnson predicts, “Over the next 5 years every mid-large US enterprise will adopt some set of modern technologies such as Artificial Intelligence, Machine Learning, Internet of Things, Big Data, and Advanced Analytics to remain competitive.”[3]

Big data in business

Every business executive looks for ways to keep his or her organization competitive (or, better yet, give the organization a competitive edge). Big data analysis is where most companies are looking for that edge. David Walcott, Managing Partner and COO of Infiniti Partnership Inc., explains, “The world is changing at an unprecedented pace, spurring an entirely new business and social era. … The 21st century brings with it a hyper-connected world, with a flood of data created everyday through the interactions of billions of people using various devices. This brings significant opportunity to derive actionable insights from the data sets being generated, but also brings with it the challenge of being able to adjust our capacity to process ever-growing quantities of data. The promise of data-driven decision making is now being recognized broadly, and there is growing enthusiasm for the notion of ‘big data’, with the hope that organizations will be able to harvest and harness the right data and make the right decisions at the right time.”[4] As Walcott notes, the secret to leveraging big data is its analysis. Data lying fallow in a database is not very useful. Fortunately, Woodie notes, “The rapid evolution of analytics has put a wonderful array of cutting-edge technologies at [business’] fingertips.”

Woodie explains that big data and advanced analytics are great — IF you know what you want to obtain from them. That’s a big “if”. By focusing on technology rather than business problems can result in an enormous waste of time and money. That’s a message Bill Schmarzo, CTO of analytics and IoT at Hitachi Vantara, has been making for years. He states, “This conversation needs to be about business outcome.”[5] He adds, “[Data managers] hold the organization’s single most important asset — data — and it’s out of that data [an organization] can glean customer, product, service and operational insight that [it] can use to improve [its] business and operational process[es] to mitigate risk and compliance issues, to uncover new revenue streams, to deliver a compelling and differentiated customer experience. It’s not an individual technology transformation. It’s a digital business model transformation. It’s transforming the business model with the data and analytic insights you glean from that.” Randy Bean (@RandyBeanNVP), Founder and CEO of NewVantage Partners, indicates businesses have finally caught vision and are leveraging big data to address specific business problems.[6] As a result, they are also seeing the long-awaited benefits of big data projects. He writes:

“After years of hope and promise, 2018 may be the year when artificial intelligence (AI) gains meaningful traction within Fortune 1000 corporations. This is a key finding of NewVantage Partners’ annual executive survey, first published in 2012. … The main finding of the 2018 survey is that an overwhelming 97.2% of executives report that their companies are investing in building or launching big data and AI initiatives. Among surveyed executives, a growing consensus is emerging that AI and big data initiatives are becoming closely intertwined, with 76.5% of executives indicating that the proliferation and greater availability of data is empowering AI and cognitive initiatives within their organizations.”

Cognitive technologies, like the Enterra Enterprise Cognitive System™ (AILA®) — a system that can Sense, Think, Act, and Learn®, usually include a full suite of advanced analytics allowing companies to mine the informational gold contained in big databases.

Leveraging big data

Before the gold can be mined from big data, the right kinds of data must be identified, gathered, stored, and protected. Get any of those steps wrong and your data mining efforts collapse. The first step (i.e., identifying the right kind of data) involves identifying business challenges needing attention. Different challenges require different kinds of data. Once the right data sets are available, selecting the right analytics platforms is the next step. Get it right and big data analytics can help your company “understand patterns, tastes, preferences, and trends created when persons interact with each other and with systems that have been put in place by business enterprises.”[7] Mary E. Shacklett (@MaryShacklett), CEO of Transworld Data, notes, “Big data exploration usually starts at a high level of data abstraction, and then gradually plumbs into the depths of the data as companies learn more from it.”[8] She notes, however, “There is also another ground up approach that has the ability to unlock hidden values of big data. This approach actually starts at the lowest level of the data and then works its way up to more sophisticated data structures to deliver data insights that are helpful to management and staff.” She further explains, “When I was running a marketing department for a bank, we used demographics for one of our checking campaigns by identifying persons in certain locations by age group, and then linking checking products to the various life cycle stages that customers were in. Later, we wanted to improve results, and we added occupation as well as age for targeting our checking products. This is a common scenario for companies. They want to go back to the data to see if they can add more information so they can improve results. By assessing and cataloging the potential information yield of big data at the lowest level of the data, data analysts can be poised to open up the data to more comprehensive analytics that can unlock the answers to questions that the company will want to ask next.” A good data scientist or analytics vendor can help a company determine the best approach for the problem at hand.

Concluding thoughts

Woodie concludes, “If we want to succeed with big data analytics and the array of AI technologies coming down the pike, we’re going to need to think more intelligently about how we apply them, and be very deliberate in adapting specific business processes with them, because the alternative is just more spending on failed big data projects.” Big data and cognitive technologies have a symbiotic relationship offering companies a way to attack their most challenging problems.

Footnotes
[1] Alex Woodie, “Focus on Business Processes, Not Big Data Technology,” Datanami, 1 November 2018.
[2] Ibid.
[3] Brian Johnson, “Building an Intelligent Enterprise with Artificial Intelligence (AI),” Medium, 29 August 2018.
[4] David Walcott, “Big data in big business,” Jamaica Observer, 2 October 2018.
[5] Woodie, op. cit.
[6] Randy Bean, “How Big Data and AI Are Driving Business Innovation in 2018,” MIT Sloan Management Review, 5 February 2018.
[7] Mark Palmer, “How Big Data will Influence the Future of Business,” Datafloq, 13 January 2018.
[8] Mary E. Shacklett, “How to optimize your company’s big data for future use,” TechRepublic, 7 September 2017.