According the IO staff, “Big Data is a confusing and much-misunderstood concept. It refers to the processing of vast amounts of information, far too large for traditional systems and techniques to handle. It comes from a huge variety of sources, including our Internet habits, social media, business transactions, smart devices, and many more.” Other than its size, big data is not easily defined. Some analysts like to explain it using seven “Vs”, namely: Volume (it’s humongous); Velocity (it’s being generated at breakneck speeds); Variety (it comes in structured, semi-structured, and unstructured ways); Veracity (not all data is good data); Value (data must be analyzed); Vulnerability (databases come under constant hacker attack); and, Virtue (the ethical use of big data). Data, of course, has been around since humans started writing things down. What makes the digital era different is how many ways data can be generated and exciting breakthroughs in analytic capabilities using computers.

Tech writer Meenu EG insists, “The advent of technology has brought visible changes in the process of doing business. Big data and predictive analytics has had a great impact on industries.”[3] He also notes that big data has challenges that must be addressed. He highlights three specific data challenges: accuracy, consistency, and bias.

Data Accuracy. Meenu points out that some people believe the greater the volume of data the more accurate it becomes. Their reasoning is that outliers become less impactful in a huge database. He argues, however, that is not always true. He also notes that trying to analyze vast amounts of data in real-time is difficult, leading many companies to rely on sampling. This, too, he notes, can introduce errors. Taking the time to get the data accurate from the beginning really pays off in the end.

Data Consistency. Meenu explains, “The data need to be consistent in order to get the right insights. Data is never static; it keeps changing. Since the collection of data hails from multiple sources, it isn’t easy to maintain consistency. Users might get misinformed if the data is inconsistent. Getting different answers for the same query can lead to such inconsistency.” Continual testing and comparing is the only way to ensure the data and the analytics are accurate.

Data bias. Bias can enter the picture either as a result of the data or through the algorithms used to analyze the data. Meenu writes, “A strain of data might contain the biases and faulty values inherited from its source. Data processing using algorithms can also lead to biases.” Meenu notes that algorithms are often “black boxes” and finding out how biases are introduced through them is a difficult process.

In addition to data challenges, staff members at FutureLearn provide a few more challenges businesses must confront.[4] For example, they note there are definite costs associated with big data projects, including: 1) Hardware and software costs. As well as systems to manage large volumes of data (such as Hadoop and Apache Spark), you also have to think about storing the data (with systems like Cassandra or HBase). 2) Personnel costs . Data science is a growing field, and those with an understanding of it are in high demand. Salaries for new hires or freelancers can be expensive. 3) Development costs. Developing a big data solution specific to your business can take a fair amount of time and resources. They also highlight a particular concern, especially when dealing with personal data: security. They write, “By collecting such a vast amount of data, there arises the issue of keeping that information safe.” Failing to do so can cost a company both money and reputation.