In Part 3 of this series of articles about machine learning, I emphasized the growing belief that insights obtained using big data analytics (in the words of Narendra Mulani [@npmulani]), are the “currency of the new economy” and I indicated that I would discuss how machine learning lies at the heart of cognitive computing and also discuss some of the popular machine leaning algorithms that are currently being used to generate insights.

Machine Learning and Cognitive Computing

Peter Fingar (@PeterFingar) asserts, “Cognitive computing, or self-learning AI, combines the best of human and machine learning and essentially augments us.”[1] Not only can cognitive computing systems provide new insights, they can make many of the routine decisions currently being made by humans — thus freeing them to concentrate on more critical matters. In addition, cognitive computing systems can contain embedded analytic capabilities that can complement a company’s workforce. As Jason Hannula (@jason_hannula) points out, many midsize businesses don’t have access to analytics because they can’t afford to compete for data scientists.[2] The Enterra Enterprise Cognitive System™ is a perfect example of the kind of technology involved. When we talk to clients, we note that when an organization has an analytic problem, it typically has to assemble a team of three experts:

• A business domain expert – the customer of the analysis who can help explain the drivers behind data anomalies and outliers.
• A statistical expert – to help formulate the correct statistical studies, the business expert knows what they want to study, and what terms to use to help formulate the data in a way that will detect the desired phenomena.
• A data expert – The data expert understands where and how to pull the data from across multiple databases or data feeds.

Having three experts involved dramatically lengthens the time required to analyze, tune, re-analyze, and interpret the results. The Enterra® approach empowers the business expert by automating the statistical expert’s and data expert’s knowledge and functions, so the ideation cycle can be dramatically shortened and more insights can be auto-generated. Even some of the business expert’s logic is automated to help tune and re-analyze the data.

Machine Learning Algorithms

Below, Dr. Jason Brownlee (@TeachTheMachine) lists many of the popular machine leaning algorithms and he has grouped them the way he thinks is the most intuitive.[3]

Regression
Regression is concerned with modelling the relationship between variables that is iteratively refined using a measure of error in the predictions made by the model. Regression methods are a work horse of statistics and have been cooped into statistical machine learning. This may be confusing because we can use regression to refer to the class of problem and the class of algorithm. Really, regression is a process. Some example algorithms are:

• Ordinary Least Squares
• Logistic Regression
• Stepwise Regression
• Multivariate Adaptive Regression Splines (MARS)
• Locally Estimated Scatterplot Smoothing (LOESS)

Instance-based Methods
Instance based learning model a decision problem with instances or examples of training data that are deemed important or required to the model. Such methods typically build up a database of example data and compare new data to the database using a similarity measure in order to find the best match and make a prediction. For this reason, instance-based methods are also called winner-take all methods and memory-based learning. Focus is put on representation of the stored instances and similarity measures used between instances.

• k-Nearest Neighbor (kNN)
• Learning Vector Quantization (LVQ)
• Self-Organizing Map (SOM)

Regularization Methods
An extension made to another method (typically regression methods) that penalizes models based on their complexity, favoring simpler models that are also better at generalizing. I have listed Regularization methods here because they are popular, powerful and generally simple modifications made to other methods.

• Ridge Regression
• Least Absolute Shrinkage and Selection Operator (LASSO)
• Elastic Net

Decision Tree Learning
Decision tree methods construct a model of decisions made based on actual values of attributes in the data. Decisions fork in tree structures until a prediction decision is made for a given record. Decision trees are trained on data for classification and regression problems.

• Classification and Regression Tree (CART)
• Iterative Dichotomizer 3 (ID3)
• C4.5
• Chi-squared Automatic Interaction Detection (CHAID)
• Decision Stump
• Random Forest
• Multivariate Adaptive Regression Splines (MARS)
• Gradient Boosting Machines (GBM)

Bayesian
Bayesian methods are those that are explicitly apply Bayes’ Theorem for problems such as classification and regression.

• Naive Bayes
• Averaged One-Dependence Estimators (AODE)
• Bayesian Belief Network (BBN)

Kernel Methods
Kernel Methods are best known for the popular method Support Vector Machines which is really a constellation of methods in and of itself. Kernel Methods are concerned with mapping input data into a higher dimensional vector space where some classification or regression problems are easier to model.

• Support Vector Machines (SVM)
• Radial Basis Function (RBF)
• Linear Discriminant Analysis (LDA)

Clustering Methods
Clustering, like regression, describes the class of problem and the class of methods. Clustering methods are typically organized by the modelling approaches such as centroid-based and hierarchal. All methods are concerned with using the inherent structures in the data to best organize the data into groups of maximum commonality.

• k-Means
• Expectation Maximization (EM)

Association Rule Learning
Association rule learning are methods that extract rules that best explain observed relationships between variables in data. These rules can discover important and commercially useful associations in large multidimensional datasets that can be exploited by an organization.

• Apriori algorithm
• Eclat algorithm

Artificial Neural Networks
Artificial Neural Networks are models that are inspired by the structure and/or function of biological neural networks. They are a class of pattern matching that are commonly used for regression and classification problems but are really an enormous subfield comprised of hundreds of algorithms and variations for all manner of problem types. Some of the classically popular methods include (I have separated Deep Learning from this category):

• Perceptron
• Back-Propagation
• Hopfield Network
• Self-Organizing Map (SOM)
• Learning Vector Quantization (LVQ)

Deep Learning
Deep Learning methods are a modern update to Artificial Neural Networks that exploit abundant cheap computation. They are concerned with building much larger and more complex neural networks, and as commented above, many methods are concerned with semi-supervised learning problems where large datasets contain very little labelled data.

• Restricted Boltzmann Machine (RBM)
• Deep Belief Networks (DBN)
• Convolutional Network
• Stacked Auto-encoders

Dimensionality Reduction
Like clustering methods, Dimensionality Reduction seeks and exploits the inherent structure in the data, but in this case in an unsupervised manner or order to summarize or describe data using less information. This can be useful to visualize dimensional data or to simplify data which can then be used in a supervised learning method.

• Principal Component Analysis (PCA)
• Partial Least Squares Regression (PLS)
• Sammon Mapping
• Multidimensional Scaling (MDS)
• Projection Pursuit

Ensemble Methods
Ensemble methods are models composed of multiple weaker models that are independently trained and whose predictions are combined in some way to make the overall prediction. Much effort is put into what types of weak learners to combine and the ways in which to combine them. This is a very powerful class of techniques and as such is very popular.

• Boosting
• Bootstrapped Aggregation (Bagging)
• AdaBoost
• Stacked Generalization (blending)
• Gradient Boosting Machines (GBM)
• Random Forest

Summary

Peter Fingar writes, “Cognitive Computing uses hundreds of analytics that provide it with capabilities such as natural language processing, text analysis, and knowledge representation and reasoning to make sense of huge amounts of complex information in split seconds [and to] rank answers (hypotheses) based on evidence and confidence, and learn from its mistakes.” He continues:

“Cognitive computing systems get better over time as they build knowledge and learn a domain — its language and terminology, its processes and its preferred methods of interacting. Unlike expert systems of the past that required rules to be hard coded into a system by a human expert, cognitive computing systems can process natural language and unstructured data and learn by experience, much in the same way humans do.”

Machine learning has become so ubiquitous in our lives that some pundits are calling this the age of algorithms. That’s why Lars Hård (@larshard) concludes, “It is becoming ever more crucial for enterprise leaders to understand machine learning, particularly the benefits that it can provide for companies today.”[4]

Footnotes

[1] Peter Fingar, “Peter Fingar: The Cognitive Computing Era is Upon Us,” PSFK, 20 May 2015.
[2] Jason Hannula, “Machine Learning: Faster Than a Data Scientist,” PivotPoint, 14 August 2014.
[3] Jason Brownlee, “A Tour of Machine Learning Algorithms,” Machine Learning Mastery, 25 November 2013.
[4] Lars Hård, “Artificial intelligence in the enterprise — what you need to know,” BetaNews, September 2014.