Kailashawati uses sugar content from two soft drinks, Choke (11 g/100 mL) and Choke R (8 g/100 mL), to teach Support Vector Machines in R. She plots 25 samples, finds the maximal-margin separator at 9.4 g, and explains why this decision boundary is most robust to noise.
Chapters
Hi, I'm Kailashawati. In this course, I'm going to give you a visually oriented introduction to Support Vector Machines. I'll use the abbreviation SVM for Support Vector Machines. Henceforth, the objective of the course is to develop an intuitive understanding of how SVMs work, the different options available in the SVM algorithm, and the situations in which they work best. I'll assume you have an intermediate knowledge of R and some experience with visualization using ggplot. We'll start with a simple one dimensional example and build our understanding through data sets of increasing complexity. To keep things simple, we'll stick with binary classification problems, that is Problems that have two classes okay, let's get started. A soft drink manufacturer has two versions of their flagship brand. A regular version Choke, which with sugar content 11 grams per 100 milliliters and a reduced sugar offering Choke R with sugar content 8 grams per 100 milliliters. In practice though, the sugar content varies quite a bit. Given 25 random samples of Choke and Choke R, our task is to determine a decision rule to distinguish between the two. Let's see if we can identify such a rule visually. The sugar content data has been loaded into the Drink samples data frame which contains sugar content measurements on a set of samples. We first specify the data frame and tell ggplot to plot the sugar content on the x axis. The Y coordinate is set to zero as there is only one variable in the subsequent lines. We tell ggplot to create a scatterplot and do some labeling. The plot shows two distinct clusters separated by data points at 8.8 grams per 100 milliliters and 10 grams per hundred milliliters. These clusters correspond to the two brands. Now, any point lying between these two points would be an acceptable separating boundary between the classes. A separating boundary between classes is called a separator or decision boundary. Lets pick two points in the interval, say 9.1 and 9.7 grams per 100 ML as candidate decision boundaries. The decision rules for these are shown on the slide. Let's visualize these. We create a data frame with the decision boundaries and add them to the plot using GeoPoint, distinguishing them from the sample points by making them bigger and coloring them red. Here's the plot. An important concept is that of the margin, which is the distance between the decision boundary and the closest data point. For example, for the decision boundary at 9.1 grams per 100 milliliters, the closest point is 8.8 grams per 100 milliliters, so the margin is 9.1 minus 8.8, which is 0.3. You can figure out the margin for the other decision boundary. Now, the best decision boundary is one that maximizes the margin. This is called the maximal margin boundary or separator. It should be clear that the maximum margin separator lies halfway between the two clusters, that is at the midpoint of the line joining the sample Data points at 8.8 and 10 grams per 100 milliliters. Let's add this to our plot. To do this, we create a dataframe containing the separator and add it to the plot using John point to distinguish the maximum margin separator from the sample points and the previous decision boundaries. We'll make it a bit bigger and color it blue. The plot makes it clear that the blue point is the best decision boundary because it is furthest away from both clusters and therefore most robust to noise. This simple example serves to illustrate a key feature of SVM algorithms, which is that they find decision boundaries that maximize the margin. Keep this in mind as we work through examples of increasing complexity in this course. Well, that's it for this lesson. Let's try some examples.
