Scales & Guides:How to Visualize Anything

Hello, my name is Mingshan Peng and I'm one of the members in the VIZ team here. Today I'm going to talk about scales and guides and how to visualize anything. I understand this is a very big ambition which so it is probably not really what it is. So I'm not going to visualize anything anything. But in a sense I'm going to talk about how to visualize scenes that are not traditionally numerical data. So in the end of this talk we are going to see how we can visualize this not so pretty, pretty picture about the bound rating for the states and the reason that it's not so beautiful. I hope that it's self explanatory, but we will see how we can do this without specifying a lot of options. I'm sure that most people know how to visualize this with a lot of options, but the goal is to make it simple. Before I start, I will talk about different data types and we all have been exposed to data types or data. But this is probably for some of the people at least. Before I tackle this problem, this was the first time I had this systematical view to see what kind of data that's on my hand. So I hope that this will give you a different view to look at your own data and then we'll talk about visualizations for anything and in the end, if we have time. So yes, I'm not going to talk about for two hours, so I will still stick with my schedule. But if I have time, then we will look at examples. That's using relational database. Okay. So data affects and statistics collected together and most often they are categorical data collection. I almost wanted to say it is always categorical data, but it is always wrong to say always. So I didn't put that, but you get my point. So when we look at the data, there are usually two variables for each data point. The first one would be the explanatory variable that is mostly the label of the axis, for example the height, the blood type, being a vegetarian or not. And there's a response variable that is responding to the question of the explanatory variable. So this is our data information and as you can see, I made some examples here. The results or the response variables values can vary from different type of the data. It can be numerical data, it can be strings, it can be Boolean or anything actually. And there's something called categorical data analysis that is the field to analyze the response variables. We might learn this kind of statistics analysis when we study statistics, including different kinds of measurements, different tests, model fitting, regression and something more. But this talk is not about that. We have a statistic team, so I'm not going to cross my area there. This talk is going to talk about. How to visualize the data itself. Well, it's still a lot of merit. Just talk about how to visualize data because it depends on different visual elements of the visualizations. We still can get some insights, but with that being said, let's get cut off. Okay, the last sentence was about that There are still some intermediate area about visualizing raw data, between visualizing raw data and doing categorical data analysis. Because a lot of time people still need to come through data. And this combin process is almost essential for a lot of data scientists before they have to do this before they can do any analysis. And we will look at the real raw data in the end and then you will understand why I say that. You know, data combin is so important and I'm not sure why this cut through, but okay, categorical data, this categorization of the data is not the only way to do this. This is probably the most popular way. Popular way to do it. The way to categorizing data is here that I presented is first something called nominal data. And we have ordinal data and then we have interval data and the racial data, the nominal data, you know, the examples here I had cat, dog, bunny. They are different kinds of categories, but in between they don't really have that much relation. They might have some kind of relations, but it all depends on the narratives. Here I am saying that I'm just having different kinds of animals. So in between they don't have any orders. And ordinal data are type of data that the name suggesting has odor embedded. And the interval data are like the dates, the ph values, the time bounding rates, temperatures, specifically in Celsius or Fahrenheit. So what it means is that interval data does have numerical meanings and also has these order relations, but it doesn't have a true zero. And the ratio data, or the name ratio data was. It has its reason that it's named. The ratio data we will see later. But the ratio data has the true zero. You know, you could have zero weight, but you cannot have the zero. I mean, you could have also zero dates, but that zero, your zero date doesn't. It may not mean the same zero date from mine. So it's different. It can be different definition. So here we can see the properties of this data. So nominal data throughout the ratio data, you can say that they all have this categories characteristic. But only ordinal data and the interval ratio have these older relations in the data and such and such. Like I mentioned in arithmetic, nominal data can be compared. Are they equal or not equal? You know, cat is not equal to dog, you know, but you cannot compare Cat is less than the dog or greater than the dog. But someone argues but in ordinal data this has a comparison that you can make in the interval data you can have this add entities that you can do. But ratio data you can do multiply on it because there is a true zero and that will give you a meaning in the statistics. Corresponding measurements are here and you couldn't find a median on nominal data but you could do it on the rest of three. So such and such. I'm going to speed up a little bit now. I'm going to talk about examples for different kinds of data. Interval data and racial data are very similar like I said, except that racial data has this true zero meaning. So we were just going to look at this example which is racial data. And I'm using this, our example data Iris Fisher iris. This is a famous data set. And the length of this Data set has 150 and the labels of them which like I said before, the labels of axis is the explanatory variable. These are telling you that what these measurements are about. The first one is the sepal length and the width and the petal length, width and the species. So I'm taking five examples so you can see that what the data look like. And I'm showing these images of these three different species of irises. This is using web image search so it's probably not the most accurate. These are not like I couldn't approve them because some of them looks kind of similar to me. But just some idea about three different kinds of irises we are trying to measure. And so I can use this function groupby to group the data. And this is how it looks like Setosa. And I have the four measurements and such and the keys are the species and again the axis label I listed here in the picture of what is sepal, what is pedal. So you can see what we are talking about. Okay. The visualization I'm choosing to use is parallel axis plot. I think we introduced this plot in post 12. I think 12 3. Yes. This is a function that is excellent to visualize one dimensional data but with many variables. So as you can see I have four variables here. Each variable is one dimensional data. The line connecting them has no Meanings it just give you, give you a visual cue how the data in the data set is connecting. And so this is how they look like. We can do a little bit the data analysis, just finding some measurements. I'm not going. You know, this is pretty simple. It's just finding the median and finding a quartile. And then quickly you can have this para axis plot that gives you some insight. Here's a little bonus example that how it looks like. Well, if you download notebook you can evaluate this. The point was like if you change the plot range, the whole look of this parallel axis plot will be very different and the line might be something you want instead of list view. Okay. Okay. Next. Is ordinal scale data. I thought I have nominal scale data first we will see. Yes, I think this should be nominal scale, right? So nominal scale data, the characteristic of it is that it can only be categories like banana, orange. You know, this is a simple data set, but how can we visualize this in all of these functions? So we have been thinking about this because there are so many data sets that send to us that contains these strings. And how can we visualize this in a plot? There's no magic that has to be mapped into values before we can plot them. So yes, you could just do some replace banana, go to 1, orange, go to 2, apple, go to 3, replace them to values. You can use this plot to plot it. But then we saw that then people would need to modify the axis and how does it interact with flood range and all these things. Right. So we decided that we can take on this and we are using these scaling functions as the vehicle to inject this nominal scale. This scale basically is telling the LISP plot that my data is nominal data. And I'm using this nominal scale to convert this nominal data into numerical data. And since nominal scale does nominal data does not have any relations in between, so the order is not a problem. So we can use nominal scale with automatic. What it does is that it's going to assign values and to categories as it comes. So this example, let me try to. Right, so the first it comes is banana. So banana is going to be assigned to surprise not one, but three here. And then orange is the second one that comes two and the apple is one. We flipped the order of the mapping because we saw that the visual queue is on the top. So when it comes first we put it on top. So this is how you can visualize stream data. The syntax of this, besides automatic, which is very convenient, sometimes you do want to specify categories because sometimes your data comes with missing data and you don't really want to visualize missing data. So you could have just limited type of categories and then the missing data will be dropped. Otherwise missing data will be shown there as one of the categories and times you, you would want to specify the labels of the categories. For example, you can specify Apple as some emoji pictures here instead of just the category itself. Here is a more realistic data which is a car sale. But here I'm choosing a little bit, I'm choosing an example that has some twist. So the data it comes is not strings, they are actually coded data. Because a lot of times when data curator they are typing in the data, they would just type in integers. So it's much easier to type instead of the brand of the vehicles. So you can also visualize integers, you can treat integers as categories. Here I'm using bubble chart to visualize this type of data. And Here I said 1 is going to be Illinois, 2 is Indiana, 3 is Wisconsin as the state variable. And for the car variable which is on the Y axis, I'm going to say that one is Ford, etc. And I'm doing a tally on this. So this is the first variable, the state and the car. And this is the count. So bubble chart is a three dimensional data visualization function. Flatten it. I have XYZ data, so I can visualize this. And you just need to use scaling functions to convert these categories 1, 2, 3 into states and 1, 2, 3, 4, 5 on the y axis into the brand names. Okay. Okay, now we come to the ordinal scale data. Besides categorical data they can be, they also have meaningful distance. And here is a type of typical ordinal data that is how satisfied you are the customer for certain products. So usually they have these orders like satisfied is better than not satisfied. But the distance in between these categories may not be equal distance. So you can specify what they should be mapped to. These values is not something you can specify in the nominal data and then the rest is the same. And for the other syntax you don't need to specify the mapping itself. If you do not, it will be just one to N. And you can this is the example we already saw above. Or you can use association to specify the values. If you want to change the labels like we saw before, you have to put on the third options. Here is an example. I'm not specifying the mapping, but I'm specifying the labels and then you get this result. Okay, here is example that is the Excise COMPARISON There are three explanatory variables. The first one is the region. Europe, usa, Canada, Australia. Second is the size that in each country or each region, how they define the size of x. And the third variable is the measurements. I believe this is in millimeter. So plotting this, I'm treating the X axis, the region as a nominal data because there's no order on this, but on the Y axis there is the order of it. So I have to specify here. Then you can see different excises have different meaning, different measurements in different region. Now we can see the original plot that I showed in the beginning. So this is the moody bound rating. So the mapping is not something that standard. There is a cut line saying that below which rating is a trash bound. But the distance above them, the distance between each ratings is not strictly defined. And then this is the data. If I can like this right. So I have state as the first variable and the bound rating as the second variable. So this is all you need. Okay, there is more. If you look at the documentation for ordinal scale and nominal scale, you will figure out that there are properties for this scaling objects and that's kind of expected and consistent with the design of scaling functions. The option, because we have advertised that scaling functions take options, sorry, take functions. You can specify your scaling function and inverse scaling functions to compute the scale. So when we design this, we saw that we also need to make this ordinal scale, sorry, ordinal scale and the nominal scale as operators that can work with scaling functions. So here the value list. Is the one that will give you the mapping result. There are a few properties that gives you the mappings. We can see the example here. This is an example from documentation as a wood hardness scale. So we define some boulders or sorry, we define some categories that has different wood hardness, you know, from softer to harder ones. So see, this is all about narratives. Those species itself, there's no distance in between. But because this is a wood harness scale, suddenly they have orders. And if you look at the property value association, it gives you the mapping which is exactly like this. And category associations give you the reverse. Because I said before, in scaling functions we always need a scaling function and an inverse scaling function. So this is basically the inverse scaling function. And if I apply these function values as an operator into a list of sample words, then I get different harness. And we can use this to construct a very simple classification function using nearest. So this value list is going to give me just a bunch of values. Basically the values of these associations of list associations using nearest and whatever the wood that you put in. Sorry, whatever the harness values you put in. For example, I just got a piece of wood here, right? I don't know what species of this, but I have some measurements I can test the harness of the wood. So I I wonder that what kind of this harness, what kind of this wood is closest to to the list? I already know. So I can put the values here and the values is going to fetch this nearest function and will give me the nearest value. And then I use this reverse the inverse scaling functions, the category associations and then will give me the closest word. Okay, I have a couple minutes left. If I don't get to talk much about this, I want to point out that Carlo has a workshop on relational database and Leonard has a talk which is recorded using the data set query language to query relational database. The reason I want to mention this is that this relational database is. Well, obviously it's a very popular thing. That's why we want to use that. But the cool thing is that you can import this relational database and using the framework that warframe language provides and turn into something that we can use entity framework to query. So the data is about some cells and if you are familiar with relational database, it is just very complicated or multiple spreadsheets. And each spreadsheet has some key that connects to another spreadsheet. I mean, yes, it can be all condensed into one spreadsheet, but it's going to be super difficult to navigate. That is why. And there will be a lot of empty spaces. That is why it is very common to spread it into different spreadsheets. And in between there are keys to connect. For example, we have customer here and we have orders there. So customer number and right customer number Here is a connection in between this customer style, not style sheet spreadsheet and the order spreadsheet. The cool thing about this framework, I'm going to point out some keys in here that is the joint across. You can query list two different entity values and join them together using a key. And then you suddenly two different entities can give you a result because of this common key. So there will be some examples here that explanations. It will give you some test about how join across and entity framework can make your relational database completely workable inside the Wolfram language. So I'm going to stop here. That will conclude my talk. Thank you.