Victor Medina explains model-based clustering, which treats data as coming from a mixture of probability distributions and assigns membership probabilities instead of hard assignments. Using a 10,000-observation gender dataset, he shows K-means produces a linear split while mixture models reveal two distinct peaks in the data.
Chapters
Hi, I'm Victor Medina. I'm a researcher at Spiff and I really enjoy using R to extract valuable insights from data. This course will introduce you to Mixture models, one of the most interesting and useful statistical frameworks to cluster and extract patterns from data. First, let's clarify what do we mean when we talk about clustering. Simply, clustering is the procedure of partitioning a collection of observations into a set of minimum subclasses or clusters. By meaningful, we suggest that all the observations belonging to a cluster share some similarities but are essentially distinct from the other clusters observations. This procedure let us explore the natural structure in a data set. Cluster analysis is used among several disciplines. In medicine, for example is used to analyze different types of tissues in a medical scan as an aid to the diagnosis of disease in business is used to discover different groups of customers in order to develop targeted marketing programs and in social sciences it can be used to identify zones in a city by the type of crimes to manage law enforcement resources more effectively. As we will see later, there are many approaches to clustering and the choice will depend on the aim of the analysis. Widely used are the partitioning techniques, the hierarchical techniques and the model based methods. The first tries to find the centers of the clusters and assign each observation exclusively to the closest cluster. An example of this approach is K means. The second connects the observations based on their similarity to start forming the clusters, which means the number of clusters is related to the number of connections we have made in the lowest level. For example, when we have no connections between the observations, each of them is a cluster in the highest, however, we have all the observations connected to form a single cluster. The third models the observations as they are generated from a combination of probabilistic distributions. This approach is what concerns us and is suitable when we are interested not only in the clusters but but also in measuring the probability of belonging to a cluster. To clarify the differences between these approaches, let's see an example using the gender dataset. This data comprises 10,000 observations including men and women. Here you can see the first six rows for the variable height, weight and body mass index. Using ggplot2 and the function geompoints, we can visualize a scattered plot of the variables BMI versus weight. The idea is to identify two subclasses that can be interpreted as the gender. You may have an idea that females tend to be in the left bottom of the plot and males in the upper right. We could still improve our beliefs using cluster analysis with K means. For example, we find two subpopulations, but it is not realistic that what seems to be a straight line is an appropriate separation between genders. More suitable is to have a measure of the certainty of belonging to a cluster, like a probability. Adding the density distributions to the original scatterplot, we can observe that in both axes appear two peaks, telling us how the points are concentrated or distributed. In fact, we might get the impression that the two subclasses come from a probabilistic distribution. This is exactly what you have when you fit a mixture model, because the way these models work is based on probability distributions. Instead of optimizing each observation to the cluster centers like k means we optimize a probabilistic model. Now it's time for you to practice.
