The Labeling Framework in Wolfram Language: Live with the R&D Team

Hello everyone, my name is Ming Shenpen from warframe. Today I'm going to talk about labeling everywhere. It's about the labeling framework in Wolfram language. So the goal of this talk is so that you will know what kind of tools woven language provide when you come to labeling, when you add annotations to your plot and hope in the end of this talk in the future when you are visualizing your data, you will know what kind of functions to look for. So let's get started. So I'm going to talk about the basic 5Ws of why we are using automatic labeling. So what I mean by the automatic labeling is that warframe language provides this labeling framework that you do not need to manually put labels. And when I say manually, I meant like specifying the coordinates and using low label text instead. There are many other ways that you can just add labels into your visualizations based on the key values in the associations or the metadata. Or even better, when you are visualizing some entity data, the entity itself contains sufficient information. So we will look through that. So where is labeling used? The applications of automatic labeling has been used in many areas. So here I'm just finding some very random examples on the Internet to show you where labelings are used. And you know, like you can definitely see that how people have make an effort. So the labels in different kinds of graphics has been carefully chosen so that they look nice. For example, in the maps. You know, like in the maps you have to put the labels for the country, for the cities, for the ocean, the different seas. And all these labels are very important in the graphics and they cannot overlap. So this is a very, very typical examples of how labeling is used. And you know, even in the whole world map it's very complicated. And I believe that to the date there are still a lot of applications labeling is required to be manually put. More even complicated cases that you have to label coordinates and all these labels and you can see they kind of very harmonic because they don't run into each other. But how do we do this automatically? That's today's question. Another example is when we visualize in pie chart, how do we put the labels automatically? So you don't need to specify them or in scientific charts or data plots, how do you put the labels under each curve to indicate what they are? So the label positions is important sometimes, you know, they can get very close to each other, I mean to the other curves. So it's a little bit ambiguous where on which curve it is labeling. So we will see that how Warframe language provides the solution to this kind of problem. Another very popular labeling situation is to label in a diagram. And so we will see that how possible that Wolfram language can also help in this situation. So I kind of said why we need automations, but more specifically, why in Warframe language, you know, like why the automation is important? Because if you don't use automations, if there is no algorithm that's helping you put the labels, basically you have to use the low label text to put the labels and you know, you have to specify the coordinates and the relative positions to the coordinates you want to put the label. So when it comes to many points, this can quickly becomes a very difficult problem. And traditionally illustrators would have to generate a plot, something like this and they will have to move these labels one by one by the hand. You know, maybe this is efficient for 10 points or 20 points. But what if today that you know, your client is asking you to add another point? Then what are you going to do? Are you going to redo all the process by hand again? So maybe it would be much helpful if there is a process that can be reproduced over and over without manual labor. And when you are plotting a scientific plot, adding labels by hand, meaning that you need to do post processing, for example, you have to construct a graphics, you have to know where the labels should be located, you have to specify the four coordinates and then you have to stick them together. So for example, if you have a data entity to visualize just so in a lot of time when you are trying to explore data, and so one of the ways to explore data is to visualize them, maybe this is just an intermediate step and you want to know which points you are interested. And then in that situation you want to know what, for example, this corner point, this corner point, you want to know which countries is associated with it. So if you, you know, just plotting the data using this list log, log plot is very quick, it's very easy. However, figuring out the labels in it is not so quick. So in the data exploration this a situation, if you need to figure out where the label is associated with, that can take a lot of time. And that's what I think that, you know, annotating the data in the plot sometimes can take, can distract you from looking at the data itself. So it would be really nice if we have a way to annotate this plot automatically. And the thing is, there are already enough information in this entity data. So how do we just put the data in it without spending like 80% of your time figuring out what the labels are. So what is automated? So Wolfram language provides different ways for adding the labels. So some of them comes with the data because they are self sufficient. You do not need to specify labels. In addition, for example, if you have categorical data, the categorical data itself sometimes are just strings. So it would be really nice if we could just put the labels according to the data. And we also have metadata. I will talk about more this in the examples. Otherwise we have options. We have various options allow you to specify labels. We also have wrappers. These are the wrappers around the data point so that you can specify a specific data point or data set with the labels. So we will look at them together for categorical data. The support for categorical data was introduced in 13.1. So it's introduced through this interface called scaling functions. So any visualization functions which we have more than 100 visualizations in Mathematica, any of them that support scaling functions will automatically support categorical data. And this is for example, this is just an example data that looks like this. So it's a timestamp and a status. This looks like some status for a system. So how do you visualize this in a daily state plot? So now we can just say that plot this data and we specify that this data is a nominal scale. And all we need to do is put nominal scale automatic in scaling functions. What it does is that it's going to treat the data first come, first served. So ready and block running is going to be ordered earthly counts. So you see you don't need to specify any other data and any other labels. In addition, you just need to plot the data because it's self contained. So all the ticks on the Y on Y axis will be labeled accordingly. Another example in the bubble chart is a three dimensional data. The first dimension is the type of apple and second is the size 30 is its quantity. Oh, it's its price. Sorry. So how do we visualize this in bubble chart? The same thing. You can just specify the nominal scale. And if you do want the categories to be ordered in a specific way, you can specify this and then it will be aligned. For example, you specify 125s. 125s is the smallest, so it's going to put it in the the bottom. So this is an example bubble chart. So all these two examples are the examples that you know, we utilize scaling functions with nominal scale. Another example is bar chart which it doesn't really seem to that we need to use nominal scale. But A lot of time categorical data. When the data scientist was acquiring the data, they have already mapped the categories into integers. In this case you can also say nominal scale and all we want to do is to label the tick labels with corresponding categories. In this case you can say one that's referring to the one in the data. We see it's key and then you can specify something like this and then it will be visualized. A more complicated example that is in parallaxis plot I have an orange data here that is six dimensional. It's coming from its origin, the location and its package and variety, item size and grade. So we can use this parallaxis plot, which is also a relatively new visualization to visualize this multi dimensional data. So nothing you need to do. All you need to do is plot the data specifying the scaling functions with nominal scale and all they will be visualized and the ticks will have the corresponding labels. In 13.3, which is quite new, we added the support for dynamic highlighting and the labels in the highlighting is probably one of the most important features and that is why it has been turned on by default. So in this daily spot to visualize an Apple stock price, you can see that when you mouse over it you can see the date and its price. So this is another kind of label. Sometimes it's more subtle. When you don't have your mouse over the plot you don't see any labels. Otherwise you can specify with your mouse. Other times you want static highlighting. In this example I put the drop lines in three locations and you can see that the labels are labeled together. So this will be good for that publications or static report. But during your presentations, if you want to have some static labels by default and still want the automatic labeling, you can use list. So this will give you both the dynamic highlighting and the static highlighting. By the way, this list can go like on and on. You can add any kinds of highlighting. And then we have line labeling. So like I said before, when we want to label the lines in the curves in the plot that has many curves. Putting the labels directly in the plot sometimes can be ambiguous because you don't really know which line it is referring to if it's too close to another one. So putting the labels in the end of the curves is really a good solution except that sometimes they are too close to each other. So the automation here is done so that they have proper spacing in between so they don't run into each other. And, and in this example all I need to do is Specify plot labels go to automatic. Sometimes you do want to specify the labels inside the graphics because they are specific features. We provide another semi automatic way to do that. You only need to specify the X location. And the callout function is going to find out where the corresponding Y. And this is using again with plot labels using callout wrapper to wrap around the labels and specify the location. You can also specify symbolic locations like above of the curve. Then it will find the maximum point. Another example makes a mean and this is a manipulation that you can see that how the labels are placed automatically for the 1D point labeling. For example, in the timeline plot, this is entity data. This entity data itself content. It has the data date, the time period and it has its label. But all we need to plot is put the data in the timeline plot. And then the automation that's been done here is so that the labels are not running into each other. And it's very neat. You can also specify the layout to be vertical. And here all these labels that's close to each other round algorithm so that they have proper spacing in between. So those two examples are so that this example is coming from the entity data that is self contained. But what if you just want to visualize a date interval? All you need to do is wrap the data with label and its date interval or timestamp. Both also work. You just wrap the data with label and provide a label and you will do the labeling automatically. Another way to do that is using association. So the key is actually the label. The value is our time period. So this is an example of Henry VIII's wife's. For the 2D and 3D point labeling in this example, this is just a random point, 50 points. But imagine that if you need to visualize 50 points in a lisp plot manually, that's going to be, you know, taking a lot of effort to make sure that they don't run into each other. But this is what I meant by metadata. You just specified 50 points. Go to 50 words and nothing else. This will be labeled. And so the algorithm was trying to make sure that first, labels are not running into each other. Second, the labels are not covering the points. And for the point that doesn't really have a good location to be to put the label, it will be put on the tooltip. You may ask that. Well, it looks like on my right hand side of this point layout is the infinite space for me to put the label. Why isn't a label put here? That is because there is actually a bounding box this is our plot range. So if I'm going to put this label, the possible location between is going to be on the right and that may run out of the plot range. If so, the label will be clipped. So that's why this label is not put. We will talk about more like how this was done and the constraints later. So another example is here that when you have different data sets that disperse differently in a plot, as you can see these three examples, when they are very tight relative to the plot, then you actually the labels that can be labeled, the locations, the points that can be labeled are relatively less in order to not run into each other. And the more dispersed, the more labels can be put. So this goes to. So the idea is that, you know, I have a plot, so you know it's about the real state of the space. So but there are some options that will actually affect the real estate of the space, for example, the image size, the aspect ratio. So those are all taken into consideration in the algorithm. So if I'm going to rerun this plot and having the image size much, much bigger, then a lot of points will have the labels visible. Another example is the feature space plot. This data is just the flag image of European countries. And the feature space plot is going to reduce the dimension of these flag images into some two dimensional values. For example, all these data points have the vertical stripes and all these images have horizontal stripes. All these flags have crosses. And so this is what feature space plot does. And the labels is placed automatically and they are placed in a size that a lot of labels can be put in. And individually when you mouse over, they give you a tooltip to show the detail. In this case, all you need to do is say labeling function, go to callout. The same logic here. When you are saying that, oh, I really want my flag image to be much bigger. Well, yes, we can do that. But in return, in order for it not run into covering the point, there are few flags that can be put. But rest assured, no points are behind all these flags. For the 3D points in this point plus 3D we have the same algorithm actually. So for the 3D points, we are projecting the 3D points onto the 2D plan and running the same algorithm as the 2D. So you know, right after evaluation you get a pretty image like this, but the moment that you start to rotate, this is not updated in real time. So you know, your labels start running into each other. But you can specify different view angle, view perspective, anything that's related to the projection of these 3D graphics and the algorithm will take them into consideration. And so that the labels are not running, are not running into each other. The same thing. The real estate is very important. The more dispersed of points, the more points, the more labels can be put. So that's for the point for the curves. Parametric curve. In parametric plot you can specify plot labels and you can see the functions is going to find a location as pointing at the curve. So it's going to find locations. Let's maximize the space in between the different curves. Sometimes you do want to specify the labels at a specific point. In this case you we specify the U and sorry, we specify. Right. So we specify the labels at the U location and this U location, you know, which is at the label. And then we don't specify like you know what the label should be at. We only specify the leader attached to. And the label location is going to be automatically decided. So they don't run into the the curve itself. And also it's not running into other labels. For the 3D case, it's the same thing. As you can see this example, none of the labels are running into the curve. We also for the labels for the region for parametric plot you can the same thing automatic or you can specify in this case a T and R and it will find a location so it doesn't run into the region. For multiple regions it will draw a leader and put the labels on the right hand side of the plot. For the region plot same thing. Or the region plot multiple regions. The labels are here and there. Also for the service labeling for Plaid 3D you just specify an automatic call out and then the labels will be put there. But you know, for the labeling on the service it's a little bit tricky. We did not choose the same approach that put the labels on the right hand side. Because sometimes, you know, it's really not the interesting locations to have. So what we did is that we put them on top of the plot. But there are actually a lot of cases and occasions that you do want to specify the label location. In this case you can specify the X, Y location. And for the Z location, it's symbolic. You just say above and that Z position is going to be calculated. Because we already have all the information here and labeling charts. This is in bar chart and using call out of random words. These labels are located placed in a way that they don't run into each other. Just like the timeline plot. In this example I was trying to say that, you know, there are cases that actually legends is Better approach to put the labels, especially in data sets. So it's not always the best idea to use labels directly on the bar. Instead you can use color code and legends. By default if you just specify chart labels, the labels is going to put on the axis. But if I do not specify the labeling size here, labels is going to run into each other because they are quite long. So in this case you can specify labeling size, go to much, much smaller and then it will give you these nice abbreviated labels and with the tooltip. Another example here, when they are images, they are much easier to be shunned to be aligned with the width of the bar. So here if you say that the image size of the whole plot is going to be different, you can see that the flagging image size is going to be adjusted according to the width. And each them comes with a tooltip. For the pie chart the automation is done. You know, this doesn't really matter for automation because they have a lot of space. But on the left hand side here, when the pies the slices goes to very, very small, then you know the labels are going going to run into each other. So in this case the algorithm is used so that they have some proper spacing in, you know, in between the labels. Another example is the bubble chart. Very similar to the point feature, but the bubbles are disks. So we decided that when the bubble that has enough area to put the label inside, then we will do that because that's a very good space to use. For the bubbles that are not, then we will put the labels outside. So for the bubbles that do not get labeled, for example this one, then you will have a tooltip. So how do we do this? There are multiple strategies that we have used depending on the type of labels. For in plot we use some image processing on this 2D data so that we can find the proper locations so the labels is not running into the curve. And for the examples that trying to find the proper spacings in between the labels, we use the optimization code. For example like find minimum, we also use geometries like region nearest region union. This kind of region functions to find labels for regions. For the 2D points, which is a well defined problem that's called point feature labeling problem. There's an algorithm that we are running to find the solution. Something to notice is that we are actually not finding the global optimization solution. And because this is an MP hot problem, if we need to find the best solution, we have to run through all the cases. So what we did is that we localized the problem and we solve this localized problem first and once we find a solution, we don't move it again. And so that we can find this useful solution, maybe not the best, but useful solutions in the reasonable time. So for others, for the plots this is done using we look at the normal of the point and for the timeline plot and the bar chart we use different strategies to stack the labels so they don't run into each other. So when are these labeling features introduced? So I made the timeline plot which also showcase labeling features that they are trying to stack the labels together so they don't run into each other. By the way, this is using fine clusters to figure out where the labels should be. A cluster and the categorical data was introduced in 13.1, which is somewhere in 2022. And plot highlighting was just released a couple months ago. So. So plot highlighting is an option that we put in probably around 15 plots right now we are trying to push more plot highlighting in other plots. And categorical data, like I mentioned before, is through this interface of scaling functions. For any functions that support scaling functions we will have categorical data support. And the end I have some showcases piece graphics. So this is a plot of oil production in the world. So what I'm trying to do is that and apparently a lot of countries are producing oil and I want to show that what these countries are. And when I was trying to do this and there were just too many labels. So in a lot of like small country, I mean small oil production countries will be like, you know, will flood the whole graphics. So what I did is that I'm only looking for the countries that is in top 25% and so I'm only visualizing those countries. So this is labeled and the geobubble chart has nice interactivities in the end, this was done by my colleague Tim and I have shown this before and I thought this is pretty neat. So I want to give this to you. All the labels that I've shown, the call outs I've shown in the previous examples are actually very vanilla. I did not add any different appearance to showcase them. But actually they can be very, very elaborated. So here is a nice interface. You can specify actually the angles sometimes maybe our algorithm is not doing its best for some reason you want to have your full control. You can and you know, you can change the little lens, you can change, you can change the neck length, you can change the gap. So sometimes the gap doesn't work because there is this marker used some markers just doesn't make sense to have a Little gap. This one is a little gap. And the rounding radius wouldn't make any sense if you do not have frame. So this running radius and the margin and different background color and by default the neck angle is automatically computed so your label is always horizontal, but not necessarily if you give a certain angle, you can have any kinds of angle for the label. And then we have a few options for the appearance for the balloon or the line. Balloon, corner, friend or just line. And with different markers. Box, point, box. Some examples here. So once you have this parameter set and put it in the callout. So these are all the options that's available. You can have very elaborated and beautiful labels in your visualization. So that will be the end of my presentation. I'm going to look at some questions I have here. I don't see questions that I can answer here. So someone asked about automatic harmonic alignment. I'm not sure what it means so but otherwise I don't have any other questions on my list. So I will give you a few minutes so people can think about if they have any questions to ask. The presentation notebook will be available and I think it is already available. So I will wait for our system admin here to tell me where they are uploaded. Oh, okay. This will be available later. Yes, thank you very much. I hope this notebook will give you some reference. And when you are visualizing your own data, you kind of have a rough idea what kind of tools you have and I hope that will be very helpful for you. How do you put labels of rows and columns in array plot? So the labels, the labeling framework I talked about in this talk is about automatic labeling. So they are more traditional ways to put labels. For example, you can change the tick labels. So if you look at the documentation of tick labels, this is a low level graphics option. You can change the tick label in all the graphics. Basically that's all the visualizations we are based on. So you can use that for the array plot. How does labeling regions as opposed to points differ? Right, so, Right. So for the region, so that the challenge is that there we. We were trying to figure out that where in this region is the best to put the label because you know there isn't really a end point, say to the curve. For the curves it's relatively easy, we put it on the end of the curve. But for the region, where do we find that point? So that for example, this will not be a good point to have a call out line, a leader line because it's going to run through the region. So that's the challenge. And especially when you have multiple regions. Yes. So for the array plot, my colleague just mentioned it would probably you want frame ticks. Not ticks, but the frame ticks and ticks specification is the same, so. Okay, I think that's all the questions we have. Thank you so much, everyone. Then we'll stop here.