The State of Optimization

Hey everyone, my name is Paritosh Mukasi and today we're, I'm gonna spend some time with you guys talking about the optimization functionality that is in Mathematica and how, how to use the functionality, what are the different functions and along with a number of examples on how to formulate problems, how to set up a problem so that you can use the optimization tools, etc. So I'm going to go through a number of various examples. If you have at any point in time you have any questions, please put them up in the comments and I'll try to get, get to them as soon as possible. I'm a kernel developer at Wolfram Research and right now my focus is on developing the optimization tools. So I'll be walking you through a number of functionalities that we have developed over a period of time. So, well, if you have been, if you're familiar with Mathematica and if you have ever used Mathematica, you'll know that we have, we have a number of functions, optimization related functions that you can use to solve a wide variety of generic problems. Many of them will end with the word optimization. So it's kind of obvious that these functions will be used for optimization. But the older ones, the ones the OG are things like N, minimize, find minimum, find argument, find maximum and minimize, minimize, find fit, so on. Even though there are all these different functions at some level or to a great level, depending upon which function is being used, there is a lot of interdependency that happens behind the scenes between each one of these functions. So we try to leverage our own technology in order to solve the problem in the most efficient way possible. So if we, internally, if we detect that a certain problem is of a certain form, and so if we detect that to be true, then we'll, we'll try to use the most optimal algorithm in order to solve the problem. And that's kind of behind the scenes. This happens within the optimization framework. So, so in order to get a better understanding, there are very broadly speaking, when you talk about optimization problem, I'd like to classify them under convex problems and non convex problems. And the reason that I'm focusing a little bit more on convex is because it turns out that a surprisingly large number of problems, even though they are not, they're not obvious to look at when, when you, when the problem is presented, can actually be transformed into a convex problem. And when you solve, when you convert a problem into a convex optimization problem, you end up having certain advantages. The first advantage is that you have Highly efficient solvers that can solve the problem for you. And the second thing is if you get a result and it passes all the checks, then you are fairly certain that the problem, that the solution that you've obtained is the global solution. And that is a really, really important and a really big thing in the field of optimization. So in general, when we talk about convex optimization problems, we will be always, or for that matter, any optimization problem. We'll be looking at problems wherein you have an objective function which we'll define as F naught and it's subject to a bunch of constraints. What makes a problem convex is that these functions, these F naught and fis, follow this, this rule over here, this inequality that, that tells us that hey, the problem is convex. And in our framework though, these convex optimization problems can take a number of constraints. So obviously we can take the equality constraint, but we can also take a number of these, these other inequality constraints. So they can be just linear inequality constraints. We call them non negative cones. It could be a norm cone, you know, like an equilibrium norm. There are semi definite cones, exponential cones, so on and so forth. The important thing is all these get classified and if, if your constraint and your objectives somehow fall under any of these cones or get, can be transformed into one of these objectives, you're in good shape. So, So the, the way I tend to look at convex optimization and conic optimization problems is that you start we have, the smallest subset is a linear problem, which means that the objective is linear and the constraints are linear. And then if you, if you go a little above that, you go to quadratic optimization where the objective is quadratic and they'll constraints are linear. Second order cone is when the constraints are of the norm cone point. Semi definite optimization is a superset of a second order cone, and so on and so forth. So if you encounter a solver, or if we have a solver that can solve a general conic optimization problem, it means that we can solve all the other problems because we can transform a linear optimization problem into a semi definite or a second order or whatever. But we can't go the other way. We can't take a linear, a semi definite optimization problem and expect a linear optimization solver to solve it. So that's why we try to focus on the most generic kind of solvers. And you know, they, the performances in, in many cases is comparable, but they're also quite robust. Okay, so I talked about convex and I gave you a little bit of introduction to convex, but obviously there's a whole lot of Things that happen beyond convex. And one of the most obvious ones in, in the field of optimization is mixed integer optimization, wherein you restrict certain variables, certain parameters to be integers instead of reals. Okay, I have a question from sai. What specialized methods are going to be used? I'll be talking about many of those in a minute. Sai. And again keeping with the theme of what else can we do beyond convex. We can do quasi convex optimization wherein your constraints don't actively follow the convex inequality, but rather follows this inequality wherein you still have a convex sort of a convex region. But it's not, it's not a summation of those two. It's, it has whatever is the max between those two and those are classified as quasi convex. It is, I'm just giving you the definitions over here. What's important is that at the end of the day, behind the scenes, all this analysis happens automatically and at some point in time it just goes to the right solver. Okay. Again, similar to be a quasi convex, we also have something called as difference of convex. So in case the problem, the, the objective itself or the constraints themselves don't look like they are convex, they can, if they can be separated out into a sum or difference of 2, 2 convex inequalities, then then we can use an iterative method in order solve, solve for that problem using convex optimization solvers. So for example, if you have typically something like this is not a convex problem because so for example, if you look at this norm cone, this part which I just highlighted is technically convex, but this isn't. But you could rewrite this as a difference of convex. So you could use this, the convex solvers in an iterative way and solve for whatever optimization problem in a domain that looks like this. So these are, these are just some of what, just, just a very brief overview of exactly what, what can be done. Sometimes you may not want to use a convex optimization solver. So in that case you could use one of the other general purpose optimization solvers to solve the problem. And so we'll talk about that in just a second. So just again I'm focusing a lot on convex optimization because it's really where a large portion of our time was spent developing. We have a huge number of solvers right now available. Many of them are through a library link that and they are commercial. Some of them are commercial solvers like Gurobi and many of them are open source solvers that have been linked in and they are distributed as part of our software. So you don't. So things like CSDP scs, cbc, all of these are part and parcel of the Mathematica package and so you should be able to solve a whole variety of problems. The mixed integer category, we have a few solvers. Mosaic is available. Gurobi is commercial. If you have a license for that, you should be able to access the Gurobi solvers. CBC is specifically meant for linear optimization problems and if you, if you fall under any other category. We use other methods to solve for mixed integer problems and we'll go into a bit of details for that. So for the linear optimization we also have support for arbitrary precision and exact computation using simplex or revised simplex methods. For quadratic optimization we have a large set of solvers and depending upon the heuristics and how how this quadratic how a quadratic optimization problem can be can be labeled. We can go to various specialized solvers that will solve for quadratic optimization problems, second order cone and semi definite. Previously we would solve it using by converting the problem into semi definite optimization. But we also have specialized methods. They might be a little slow, but like we have functions that will allow you to transform a problem into a set of linear problems using what is called as a polyhedral approximation and you can then solve it as a linear optimization problem. Or we have specialized methods called which we have with the method is called Fisher Burmeister, which you can use to solve the problems. We also have support for conic optimization general purpose optimization problems using IP opt, which is a very, very popular solver and it's actually remarkably fast. For the mixed integer we use outer approximation methods and I'll go into details on that when, when we get to the appropriate sections. So that is for the convex optimization. For the local optimization methods we have, you know, the usual gamut of optimization solvers. We have basic gradient methods, conjugate gradient, Newton methods, Levenberg market methods and so on. With the IPOPT which was introduced in version 11, you can now use IPOP directly. Internally it will also call IPOPT even if you don't specify the method. The big thing is that what happened in version 12 was that we added a whole new framework that actually analyzes the problem you present to the optimization function. And if it can transform it into a convex problem or something close to a convex problem, then it'll go to a dedicated solver, which means. So what that means is that typically in earlier versions, if you had problems that were taking really long to solve, if the problem were actually convex, now within the framework it'll go ahead and solve it as a convex problem. So you'll see for many of the problems you'll see a significant performance improvement. So you could access many of these unconstrained optimization problems or these local methods through five minimum, for example. So here's a simple example and it'll solve it using automatic IP opt, interior point method or convex method. And when we solve this, you'll see that you get the results. But when you solve it using convex optimization, it comes back saying it is not a convex method. So, but I did give find minimum method goes to convex and it still was able to solve it. So how, how did that happen? So if you look at the plot actually of the constraint, this, this constraint over here, it's actually a concave problem, it's not a convex problem. So this automatic method, when you give it automatic, what it's actually doing is analyzing the problem and figures out that this can be represented as a difference of convex problem and solves it. So that's exactly what's happening behind the scenes. So that. There you go. I don't know whether you can see it, but let me see if I can. You can see that. So it's using the difference of convex method. I have a couple of questions here. Let me quickly see them. Where do models that are strictly cosine based fit in regard to convex optimization? Those would have to go, those will go through the general purpose solvers. If it can't, if we can't prove that they are convex through our algorithms, it'll go to a regular method to solve the problem. What sai you have asked the question what are the algorithms used in convex optimization solvers? Well, we talked about the different solvers that we use. Newton, Raphson, gradient descent. These are general purpose solvers. They don't take into account the structure of the underlying solver. So, so for example, if I go back, if you. So, so these are the solvers that we would use. SES for example is split conic solver. So this is, these are very famous solvers. There are papers out there you can read. What it does is it. It basically splits the number of the constraints out and iteratively solves it using a projection method and then obtains the solution. So I don't know how much detail you want me to go into in terms of. I mean there are thesis written just on topics of how the CSDP solver or how the DSVPN or CSCS solvers, what underlying methods they use. I mean they're, they're very specialized solvers for this so I would highly encourage you to go on their documentation pages in order to figure out the details. Same thing with things like osqp. So that would be outer sequential quadratic programming. So it iteratively makes a sub problem that is quadratic and HS solves it and updates it and that's how it does the problem. So okay, what about global optimization? Now we do have some certain methods Nelder Mead it is cheap but you know it's, it's not really a global method but it's fast. We have simulated annealing differential evolution which is the most robust but it can be a little expensive. Random search is more of a DIY in because you have, if you, if you don't give it enough number of points, you know, it might, it might not find the global minimum but you know, give it enough number of sample size and it should be able to find it convex, which I've talked about extensively for the past 15 minutes is basically if you, if it finds a solution it's guaranteed to be global. Okay so we also what I should mention is that we have a very robust open source solver called Cohen that has now made it into version 13 that should massively expand the ability for us to solve constraint global optimization problems. This is a, this is a very well known solver and is now available within Mathematica. So for example, here's, here's an example wherein it obviously has, is not a. It's got many multiple minimas and, and what have you and if you were to plot it it, you know you can, you can clearly see that there are multiple minimas over here and with N minvalue you can, you can solve it and many of the simulated annealing differential evolution when they give the true global minimum. But like things like random search Nelder meat don't. IPOPT also gives you. If you call IPOPT from within random search so that each sample size uses IPOP to find its local minima, you are able to get the global minimum. Same thing. If you increase the number of search points you, you, you will be able to get the global minimum. It's just a question of how expensive it would need to be. Okay, let's move forward. So let's, let's jump into examples because I think I've given you enough theory and exactly you know how what solvers are there. But let's, let's work our way through a series of examples and hopefully that'll make, give you an idea of how exactly you're supposed to use these solvers and there is, it's not just about the solvers, it's also about the ease of how you can formulate a problem so that it's very intuitive and, you know, doesn't require you to do a massive amount of coding. And that's what the, that's what is happening behind the scenes. A lot of really high end parsing happens that allows us to transform the problem into a very good way. So here's a cantilever beam problem that, and you know, you have a beam with the number of segments and there's a weight to it. And the question is what should be the shape of this beam? So instead of designing it as a continuous thing, we take the cantilever and we break it into these square cylindrical blocks and, and figure out what the dimensions of these blocks should be in order to generate the, generate the final cantilever shape. So in this case, for this example, I've decided to use 16 of these blocks in order to figure out what should be the shape of the cantilever. The more as you increase the number of these blocks, the more it will lead to the true solution. So what the objective is for us to minimize the weight of the, of the, of the cantilever beam. So we set up the constraints which are based on the stress constraints. And notice it's very generic in nature in the way I've written it down. I haven't specified any of the parameters yet. We'll do that at the very end, but we can keep it as generic as possible over here. So I've specified the stress constraints and then what we do is iteratively as we go from one beam to the other, the deflection that happens on the end of one beam is the start of the deflection of the other beam. So we iteratively, we can set up the constraints and that's what we are doing in this do loop over here and then you can say that, well, the maximum deflection at the end of the final beam is supposed to be some Y max, which we haven't defined yet. So we'll, we'll hold on to that and we'll then the dimension constraints which, which basically gives the physical dimensions. And now we specify the parameters and now we can solve the problem. And in this case I specifically tell it to use the Neltar Mead method in order to solve the problem, mainly because it's the fastest and I can, I can, I know that this is, this can will have an optimal solution. And so it takes around 10 seconds but you get a result and then you can go ahead and plot that result and you can see that that's the final shape. If I were to increase the size, you'll see that, you know, it'll more, it'll lend itself more towards a specific shape. Okay, so notice that the way I have specified the problems is just basically how I would describe it in, in the content, in the, in the written way. So it's so, so the parsing and the indexing happens very quickly and we end up being able to solve it very, in a very intuitive way. Here's the second example. This is called, this is, comes from a fitting example. So let's say that we have, you know, a time series and there is a signal associated with it. And this is what the signal looks like. And our aim is to approximate the signal, this true signal, using a bunch of your standard library functions. Library, not the library for Mathematica, but you know, a set of predefined basis functions. In this case we are using what are called as gay bar basis functions, which are just exponential functions or Gaussian functions that have been shifted and translated and you know, made wide or narrow. And we don't know how many basis functions are needed to approximate this. So we generate an entire dictionary of these basis functions. So if you were. So I'm constructing this and there are around 30,000 basis functions, but 30,000 base functions are, is essentially overkill for this. What we want to do is use as few as these basis functions as possible in order to get a good approximation of our original function. So if you want to look at what the basis function looks like, it's basically, you know, these little packets of Gaussians that, that are, that get translated and scaled and dilated. So these are basically GABA functions. And the question is how many of these functions and which one of them do I need in order to approximate this problem? And in order to do that we, we can directly make use of our fit function. And you can just, you can specify that I'm interested in the best fit parameters. So I give the data, I say that These are my 30,000 basis functions. T is the independent parameter. And I'm going to do a lasso regularization. So if you look at the documentation and fit in fine fit, you'll see that there are these options called fit regularization. And you specify a lasso regression. And in this case, let me just put it at one, and you'll see that out of 30,000 it says that, well, I'll need about 47 in order to get a good lasso fit with, with a regression value of one. And internally what fit is doing is it's, it's reformulating the problem and sending it off to a convex optimization problem as a convex optimization. So if you look at the result, the result is actually not that bad. This is the error plot associated with it. If I were to Change this to 0.5, it may take a little bit more time, but the number of elements that you need just moved up from 47 to 51. So really not that much of a change. But the error becomes a little bit better. And because you have this information, the sparse fit and the best fit parameters, you can actually extract which one of those basis functions were used in order to construct the. The fit. And then you can just use those specific basis functions in order to solve the problem. And you'll get a least, and you can obtain a least squares fit for using that. Okay, so one more question. How to solve a convex optimization which are higher order polynomial exponential the best current approach. So we do have depending upon the problem, if we are so if it's a con, if you know it's a convex optimization problem, just use the function convex optimization and it'll. It'll figure out if all the details behind the scenes, even if it's exponential, it'll convert it to a power cone. It could classify it if there are polynomials or posinomials, it could, it could classify it as a geometric problem. So all that happens behind the scenes. Sai so I would, I would recommend that you just use the most generic function. If you know that a problem is convex but aren't sure which one you, you just use convex optimization and just. It'll solve the. It'll classify it for you. Okay, so I want to show you another example which is an image processing example, which typically you wouldn't think of it as an optimization problem, but we can formulate it as an optimization problem. So here we have an image and what I'm going to do is I'm going to corrupt this Image so that 70% of the data is missing. So you can see it's a significant portion. You can't really make out the original image from this. And we're going to formulate the. So objective is to recover this original from this corrupted data, but we'll do it using the optimization framework. So in this case what we do is we basically say that pixels next to each other should have sort of the similar value. So, but, and, and so you. So for any given Pixel, it looks to the left and it looks to the top, right and top, and says that those pixels should be about the same value. And the objective then becomes to minimize all of these surrounding pixel values with those that are already known. And so technically. So this is our minimization. And if you look at it, our objective is literally a one liner, which you can write it almost identical to the way I've written it over here. So you set up the objective, you know that there are certain uncorrupted values. We know that because we have the data available. And so we know what those are, and those become our constraints. So at those index or those pixel locations, we know what the data is. So that's our constraint. And now we solve it as a convex optimization problem and we recover the entire image. It takes a few seconds to solve it. It's the dimension is around 75 by 75. So it takes a little bit of time to find all the, all the results, but we get the result. And here is the final solution. So 70% data was corrupted. We formulated it as this nearest neighbor pixel problem, for a lack of better term, and it gives us a pretty decent approximation of the uncorrupted data. So obviously these are just examples. There are highly sophisticated, much more robust and much more efficient algorithms out there. My point is to demonstrate how you can formulate problems using the mathematical framework, using the optimization framework. So I hope, I hope that's coming through in the, in these slides. All right, so one of the things which I always want to talk about is that it's not just about taking a problem, solving it, and just getting a result. It's also a question of analyzing what the problem is, how we can extract relevant pieces of information from it. Can we, if, if all we are looking for is some obscure piece of information, can we do it in a more efficient way? All these questions. So this is the exploration part of it, you know, of, of how you can treat an optimization problem and get interesting properties and relations and stuff like that outside. So in order to demonstrate that, I'm going to take this example wherein I have a arbitrary region and I want to find the smallest sphere that encompasses this region. And for that I need to find the center of that sphere and the radius of that sphere. So these are my, that's my region, I have a bunch of points associated with it. And the formulation is basically I want to minimize, find the smallest sphere. So which means I need to minimize my R. And so I need to find the radius and I need to find the center of my sphere such that all these, all the points from this region lie inside that sphere. So this is literally my entire formulation. Okay, and we're going to go ahead and solve it. And it comes back pretty, pretty actually very quickly. You get your radius, you get your center, and you can visualize it and you can now say, well, good, good for me, the problem is solved, no need to move forward. But if you want to start asking questions at this point, one of the most obvious question is which one of these points from my original region actually touches the surface of my sphere? And well, you can do that obviously you can take the center of the sphere, get the radius, plug it back into the constraint and see which one of them is below a certain tolerance. And, and, and you get back a result. But the question is, well, is can we, can we obtain this hidden information in some other way? Okay, and that's what, that's what I want to focus on next, which is that an optimization problem can be reformulated in a way that it might work to your advantage. So for this very specific problem, I can reformulate it as a maximization problem wherein each one of the points is given a certain weight and the ones with the most weight are the ones that will contribute more towards the finding the center and the radius of the sphere. So in this case I'm going to, I'm going to convert this as a weighted optimization problem and instead of minimization, it's now an, it's a maximization problem. And what you end up getting is a set of weights for each one of the points in the domain in the region. You have a weight associated with it, but you'll notice that many of the weights are very close to zero. Also you will notice that the amount of time it took for you to solve it is significantly less because technically I've formula, I've converted this generic problem into a quadratic optimization problem. But that happens behind the scenes. So if you want to find the radius and the center, you just, that information can be extracted out by just looking at the minimum value associated with it. And that's your answer. And if you want to know only the extreme points that are available with it, then you just look at the weights, the ones that have a non zero weight. Those are the ones that contributed to the, to the construction of the sphere. So that, so it's not always about just, you know, blatantly putting the problem in, in its raw form. Sometimes it's a good idea to reformulate the problem depending upon your solution. The thing That I want to bring to your attention is that we have something actually quite powerful called the dual maximizer. The dual maximizer actually allows you to extract all this information without having to resolve the problem and, or reformulate the problem. So I'm going to solve the same problem using conic optimization, but one of the solution properties I'll extract is called the dual maximizer. So when I extract the dual, you'll see that most of the results you get out are zero. And if you extract which one of those are non zero, they actually end up giving you the points that actually matter. And so if you can now visualize it, the advantage over here is when you saw. So what I'm doing is I'm now solving the problems as a convex optimization problem. And when you do it in this form, underneath the solvers, they solve it using what is called as a primal dual approach. The algorithms will always have a primal dual approach. And that dual information is solving a complimentary problem. But you get, it's like a freebie. You obtain a freebie set of information that you can exploit, which is exactly actually what we do behind the scenes when we solve certain problems. And I'll show you that. And so I've spoken to great lengths about what a dual is and why should I care about it? It's because underneath, even though we are not doing the solver is not doing much, it is actually solving another set of problems in order to get the, to get the final result. And that can, that contains a lot of useful information. So let me show you a couple of examples of that is, let's say that I have, this is a geometry problem. Let's have two convex polygons and I want to find a line that, that bisects these two such that you know, the distance between that bisecting line is, is the same between the, between the line and these two polygons is minimized. And so you can formulate the problem actually as a simple one liner. You can basically say I have, I want to minimize the distance between point P1 and P2 and that. So this P1 and P2 are elements or P1 is an element of this first convex polygon. P2 is an element of the second convex polygon. And I want to find the shortest distance between those two between the points and it gives us a result. So it says that P, this is the point on Polygon 1 and this is the point of Polygon 2. And that's what it is. But if from this you can also extract the, the Separating hyperplane that will, that separates these two convex polygons. And you can do that using the dual. So you can, what you can do is you can extract out the constraints and you can extract out the dual. And you can then construct a hyperplane using that dual information. And all through this one single formulation, you don't have to change anything. You just. It's a question of how you interpret your dual information. So the reason I spent so much time in this is because I wanted to share with you what we do behind the scenes when it comes to mixed integer convex optimization. We actually use this dual information which I have talked about in order to generate what are called as outer approximations of the, of the, of the constraints of the region and use that in order to solve mixed integer problem. So instead of going through the theory, let me show it to you graphically through an example. So here we have a very simple example. You have an objective negative X, negative minus X minus Y, and you have some constraints, but you want the X to be an integer. So you, you can just send it off to conex, optimize convex optimization or conic optimization. It gives you a result. What's happening behind the scene is we are actually using the dual information in order to obtain this solution. So let me, let me walk you through that step by step. So first thing we do is we'll take the conic optimization, but instead of specifying it as an integer, we just give it as a real. So this is called as a relaxed problem and we extract the primal and the dual information from it. Now, from the dual information, just like I did with the hyperplane problem, I construct a hyperplane associated with this relaxed problem. And so this is the relaxed problem and I obtain a hyperplane associated with it. And what I do is now that I know the equation of this hyperplane, I plug that back into my, into my, into a linear optimization problem with this hyperplane information and I solve my. Solve a linear problem that says that I want it to be an integer and I get a value of negative one. I plug that negative one value back into my original relaxed convex problem. I solve it and notice that I have it now has reached the, the, the optimal solution. And if you look at the hyperplane associated with it, you can see this was the first one. There's my second one, and it finds the solution here. So with it. So this outer approximation method using. So what we're doing internally is we're using this dual information recursively in order to solve our convex optimization problem. And if I, if I repeat it multiple number of times, I'll get the same result. So it reaches convergence and in general, it reaches convergence fairly, fairly quickly. Okay, moving on. So I've left this slide over here for you because there are lots of tips and tricks that you can use to model integer or mixed integer problems that will allow you to do all sorts of things. If you go to the Wolfram blog post I've actually put up, put together, I've written a blog on how to generate Sudoku solvers and how to generate Sudoku problems using this mixed integer approach. And I highly recommend that you look through that. So there's a lot of things that we can do with, with, with optimization by manipulating it and modifying it into a mixed integer problem. One note of warning is that just because you can doesn't mean you should, because mixed integer is actually quite an expensive thing to do. It's a very hard thing. It's NP hard and sometimes it can bog your machine down. So that's just a cautionary thing. But having said that, with this kind of modeling tips and tricks, the kind of problems that you can solve can be expanded significantly. So here's, here's one example wherein we could solve the problem using efficient methods, but we're going to solve it using the mixed integer approach. So I have an obstacle which, which is defined using this convex mesh. And if you were to look at that, it looks something like this, right? And so this, this convex region can be specified as a set of inequal linear inequalities. And what I'm going to do is I'm going to give a start point and an end point which is represented by these black dots. And the object is to find a path from the start point to the end point such that it doesn't cross this obstacle. Okay, so that's our objective. So I'm going to divide my path into 10 different segments this, into 10 different segments. And each one of them is of that points. Those 10 points are represented by an X and Y coordinates. So that's my, those are my variables, P VARs. And then I want, I want to make sure that the distance from the start to the end point is minimized. So that's what I'm trying to do. And so, so that's my objective that I'm trying to minimize. And I know what my start and the end point of the constraints are, because those are known. I give some additional information saying that between these two points of my segment, I don't want too much of a distance between them. Otherwise it can just, it might, the solution you get might not be as optimal. So I give us distance constraint and then at this point in time, this is where the mixed integer magic happens. I basically say that each one of the points should adhere to this constraint, wherein if, if a point, if a point is inside the convex polygon, then it's obviously shouldn't be there. So in order to make sure that all the points lie outside the polygon, I use mixed integer in a approach in order to ensure that those points will be outside of my polygon. And so those are called as. So I've set up, set that up as avoidance constraints. Okay. And I set that up and we'll go ahead and solve this. It turns out that this is a quadratic optimization problem. Takes a few seconds to solve it, but, but you'll eventually get a result for all the positions of, of this discretized obs, the discretized path, and hopefully it'll end. There you go. And this is the start. That's my end. And you'll see that all these, these are the obstacle constraints. I won't go into the math and the detail of why this works, but I'll encourage you to go and look, look, look it up at your own time and shoot me an email. And here's my result. And obviously the first thing you should notice is that bam. It is doing absolutely. There is, there is an, there's a problem here. It's actually going through the object when it's not supposed to. And the reason for that, there are two reasons. One is that the number of points I've used is not much. So it's, it's still satisfying all the constraints. It's just that it blindly goes through it because it doesn't know any better. So there is, there are a couple of approaches to fix a problem like this. One is we can increase the size of this polygon and then solve the problem again. This will allow it to go past it. So in order to do that, the simplest thing to do that for this problem is to take this B. So the, the polygon is represented by ax plus B greater than or equal to zero. So if I were to add a small offset to it and then solve the problem again, it'll, it'll recompute and should return a new solution that takes into account this expanded polygon. There you go. And now when you see the problem, it has, it has shifted. So essentially what I did by adding that offset in my linear inequality constraints is that I made the, I made the thing much bigger. So if I were to do the region plot of a dot X plus B greater than or equal to zero red. And yeah, I am doing it in the most inefficient way possible. But I'll just show it to you. That's because it's supposed to be X and Y. There we go. That's my polygon. And when I add the offset, it makes it bigger. That's what's, that's what I'm doing behind the scene. Okay. All right. So finally I want to just quickly go over something called as robust optimization. In robust optimization I have a couple of questions. Let me see if I can quickly address that. Convex and conic optimizer hyperplane information. He gets rid of the negative one and then finds a solution by approximation. Yes. What does he divide in 10 parts? I divided the path. The path is supposed to be a continuous path, Dom, but I, I discretize that path into 10 equal parts. That's what the, the, it's not the whole square or the polygon. It's the path that I took. So let me just go back. You see, I discretized. So you can see that there are a certain number of points. 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. So that's what I discretized. Okay. I hope that makes it clear. I'm sorry if I ran through it and it wasn't clear, but I discretize the path that it takes. Okay, so finally I want to. There's just way too much for me to cover. So. But I'm, I'm trying to touch on just some of the more obvious ones and some of the more important ones. This, this one is called optimization with uncertainty. So let's take the simplest example. Let's say that you have a convex optimization problem with certain bound constraints. There's no mystery here. You get a result. But what if the upper limit of this X and Y was uncertain to you May it's not one, but it's maybe one plus delta or one minus delta. In that case, what is, what kind of an optimization value should it return? Because if you were to just take this and you have an uncertain bound, then for certain values this result will be invalid. So the question is what is the best possible solution I can find for optimization problem when I'm uncertain about certain parameters? Okay. And so that's what I mean. Over here, one plus delta and one plus delta Y. Well, what if the uncertainty is in, in a range? So in that case, what you could technically do is you can set up a whole slew of constraints taking into account all possible variations and combinations of delta X and delta Y, set up this massive new optimization problem and then solve the problem for the best possible result, in which case it takes all the uncertainty into account and you get a new result. Problem is that you can do that for all cases. And even if you can, you have made the problem highly, highly inefficient and it's sometimes it won't even return optimal solution. So in order for us to do that, we have actually, and the math is a lot more complicated, so I won't go into the detail of that, but we have a function called as robust convex optimization that will allow you to take into account any uncertainty that you have if you're not sure about your inputs or, or you know, the, and you know what kind of, you know, uncertainty exists with, with your data. So in this case, a very typical case scenario is that you have some experimental data, but the measurements that you've received aren't, you aren't sure about that, but you know that they lie within an interval of negative 1 to 1. So the question is, what is the best kind of fit I can get so that it takes into account this uncertainty on my output? Okay, so in this case I'm going, I want to fit a cubic polynomial to this, to this, my experimental data. So I'm going to set up the problem. I have the basis. I set up what is called as a design matrix, which if you look at it, it's just, it's a matrix of 4 by n number, whatever is the number of data points. And your objective is to minimize the difference between the input and the output. And you want to find the coefficients of that polynomial. So that's my objective. And I want to minimize the L2 norm associated with it. And in this case we use a function called robust convex optimization, wherein we have a parameter and that parameter lies, is associated with our uncertainty. And when we solve the problem, we get a set of coefficients and you can then check that this is indeed the most robust solution by developing your own models with different parameters, like we did in our, that simple example before. And then, you know, you can develop like 200, 300 different models, whatever number of models, and, and compare that with the ones that we have obtained. So the one we have obtained is this, this black line and the one, all the other models that we have over here, all 200 of them, they lie in this, in this pink region. But we have through, through the reformulation and through figuring out proper analysis, the results that we get is through this black line, which is significantly more efficient and is much more quote, unquote robust. So this is a very specialized field of optimization that we have functionality for and there are a lot of applications associated with it. If you're in the field of finance or and are looking into things like for example, portfolio optimization and you're trying to measure risk and volatility and you have these parameters available, using robust convex optimization would be the perfect way to do it. There is one more question from Tom. What does constraint mean? The fact that he cannot cross the object. So this is back to that. This one. Yes. The constraint basically means that I need to find a path and I have an object and that object is my constraint and it cannot cross that constraint. You have it absolutely right. Okay, that actually brings me to the end of my talk. Actually I didn't want to. I know I've shown you a lot of information. There are lots and lots of functionality. I would highly recommend that you go through our documentation in order to see the various capabilities that we have. I wanted to give you just a very broad overview. Hopefully down the road I could do something more specialized where we can talk about. If you are one of those guys who wants to develop your own solvers, then we have an entire optimization framework that's available to you that allows you to plug in your own custom built solver to solve all all our problems. As a matter of fact, some of the solvers that we have put together all use that framework and that is also available to you and there are. There's very detailed documentation and tutorial for that. And yeah, just to give you an idea, here's another example of a structural engineering problem that and you can use simple mathematical optimization problems to design things like bridges. And this is one example. Boyd, who is like a really heavyweight in the field of optimization has other problems that, that use the same principles that to solve this problem. So I'd recommend there's a lot of reading material. If you want any information, let me know. You can always. This is my email. If you want to send. Send me something or have any questions, please feel free to reach out to me or through any of our other options. So with that I'm going to go ahead and end the talk. I'll of course take some whatever questions there are. So there's one question. Thank you. And for solvers specialized for circular statistics, who at Wolfram should I contact to learn what I can use for my problem? Space. So the question is he. There is a need for solver specialized for circular statistics. Okay. I don't know what circular statistics are, so I don't know how to answer your questions, but if you can just shoot us an email and maybe I can do a little bit of research on what circular statistics actually is, I might be able to provide you with that information. I should mention the best way to reach out to anyone in Wolfram, at least to my opinion, is through tech support. These guys are, are excellent at, at responding. I myself was tech support at one time, so I'm probably biased, but yeah, our tech support is, is really good. And if you have, if you want to reach out, reach out to our tech support group. And, and they, if, if they can't answer the question, they'll stream it down to one of us. So, so I, I hope I'll be able to answer that in the future. Other questions. Okay, I'm not seeing any other questions right now. If you, if you do think of any other questions or you think that you would like to learn more about a specific topic or need, need us to cover some functionality that you don't think exists, please let us know. I, I should point out that Nina, who is my colleague and we both are working full time on the optimization project, she is constantly adding new functionality and new solvers that are very popular to use in the optimization field. So she keeps on adding new stuff. So, you know, if there's a certain problem that cannot be solved or is taking a lot of time to solve right now with Nina's brilliant work, you know, we should be able to add those. Many of those problems will be able to get solved down the road with the addition of new solvers or new methods or new processing capabilities. So shout out to Nina for that. Okay, so I'm seeing that there are no other questions. So I'm going to assume that everybody was super, super happy and understood every, every single thing that I said. And we are going to, let's go ahead and close this session today. Thank you once again for all of you guys for coming and listening to the talks. I really appreciate it and hopefully we can, we can do more of these down the road. So with that, I'm going to stop sharing my screen. Thank you once again.