Scientist Stories: Michael Albergo, Sampling and Generative Modeling

Okay, so you can use this one if you want to find it. All right, go ahead.

All right, so guys, we're going to start our next talk with Michael. His talk is going to be about matter transport perspectives on sampling, generic modeling, and beyond. So Michael is a postdoc at Courant Institute of Mathematics and Science. His research interests lie at the intersection of generic modeling, to physics with a focus on designing machine learning models to advance scientific computing. He received his PhD under the supervision of Cal Brammer, Iyalla Kunt, and Eric Van Den Gendt at NYU. His MPhil is at University of Cambridge, and he received his AB at Harvard University. From August 2024, he will be a junior fellow at the Harvard Society of Fellows and an IFOP fellow at MIT.

Welcome. Thank you. Okay, thank you, Wangyi, and thanks to the rest of the organizers for inviting me. Give me one second, I'm trying to figure out how to get the video off of my slides. Okay, great. Yeah, so today, as Wangyi said, I'm going to talk about some measure transport perspectives So this one is not, for some reason, it's definitely used. Oh no, it's fine, it's fine. Nevertheless, you would like this. Okay, right. So yeah, I'm going to talk about some measure transport perspectives on sampling and generative modeling, emphasis on problems in science. Full disclosure, I don't work on graphs at all, but I am interested in problems that are related to geometric deep learning. I know there's a big intersection there. So what I'm going to talk about today is sort of a suite of tools that we are interested in applying to this highly geometricly structured problem from a generative model perspective. So before I go any further, I want to move this bar, and it would be great if it just disappeared for a minute. There seems to be no freedom. You can do that. Click more. Yeah. There's slides. Yes. Here. Okay. Before I go any further, I just want to quickly thank all my collaborators and mentors who contributed to this work in various ways. So this talk is really going to be about the bottom half. Collaborators here, and our aims over the next year or two are to really apply this to what's going on with the top-down collaborators. So I'll start with a trope, okay, that really the social and natural worlds that we inhabit are complete with a complex structure, okay, and it's often beneficial to interpret this complex structure probabilistically. I know we've seen great strides in that, I think, in our social world, for example, if we want to try to understand the underpinnings of human communication and language, or for example, the aspects of human creativity and visual understanding related to problems in video generation, we can look at abundance of data and construct algorithms now that give us sort of an ensemble of possible interpretations of our desires. So if I ask, for example, for a flower growing on a windowsill, you know, we have these text and video models now that are just kind of remarkable and make things in Fort Magnus fire a little bit, that could give you a perfect rendition of that in a matter of moments. And then on the other side, our natural world is also sort of replete with this probabilistic structure. We can ask about problems and simulating the structure and confirmations of quantum field theories, of the confirmations of molecules, or perhaps the ensemble of future predictions of dynamical systems.. And here often we don't really have much data, but we do have theory that we'd like to use to try to inform how we think about solving these types of problems under a probable state of interpretation. Okay, and so that's just to frame sort of a standpoint of where we're at. My research interest that is motivating me to ask this is that I'm interested in computational problems in statistical physics. Okay, this can be things— high energy physics, dense matter, and chemistry, and particularly interested in designing machine learning techniques that are inspired by this underlying physical problems, okay, and then sort of meet the demand of scientific computing, have sort of control over the correctness or bias in the types of models that we're using. So this might involve designing generative models on manifolds, for example, circle or products of these circles, to try to simulate aspects of of the strong force holding together the inside of a proton. And I'm going to take some motivation today from the ascendancy of generative modeling over the past 12 years. So in 2012, you know, our best intentions of being able to represent underlying data distribution sort of was stuck at NIST. We were able to generate handwritten digits using things like Boltzmann machines, then progressively over the past 10 to 12 years, the power of these models has grown extremely rapidly. Now we can show you a diffusion model in 2022 and say draw me an image of two bears doing chemistry in the form of R2P2, and you know, you can get that in a matter of seconds. So this sort of is the problem setup I want to think about, right? This is sort of a probabilistic generative modeling paradigm, And I'm going to state what our goals are in thinking about this. We're going to want to estimate an unknown probability density function that I'm going to call rho_1, okay? And we're going to either want to do this through sample data or access to, for example, an unnormalized log-likelihood function, like an energy function in physics that tells us about the likelihood of the configuration of the molecule or the arrangements of spins on a lattice, things like this that we might be interested in. But today I'm really going to just focus on the first part because of time constraints. And what I want to focus on is the fact that over that, through this ascendancy, what has emerged is sort of what I'll call a metric transport perspective on the success of these models, okay? Building maps between probability density. So I'm going to ask— and this will be a guiding question for this talk— is how can measure transport help us understand these successes, and then eventually build more performant tools that we can also interpret. So that's going to be the agenda. I'm going to put this little part 2 as ask me offline because I'm not taking up the time of other people here. But what I want to talk about is really new algorithms for dynamical measure transport, building maps between these different probability densities that we want to study. So I'm going to frame what that problem challenge is and then talk about a method that we've been building out called stochastic turbulence to try to address this. And then I'll go through a few applications about what you might want to do with this. Okay, so just to lay out what the transport framework is. Okay, this is the picture we're going to be working with. I'm interested in trying to understand this density rho_1, right? And I'm going to do that by introducing a new density A very simple one, perhaps, that I'll call base density rho 0. Might be something like a Gaussian. And my aims are going to be, be to build some sort of reversible map that I call T here that says for every sample I draw from rho 0, I put it through T and it pushes it to a sample under rho 1. Okay, so you say T pushes the density rho 0 onto rho 1. Okay, and this is actually a means of expressly defining the likelihood of rho 1. Or any type of— say, if we know the density ��₀ and we know aspects of this map T, mainly its Jacobian, and have an ability to compute the determinant of that Jacobian, then we actually have a definition for what the density ��₁ is. And this is, you know, a way of defining basically the density we're studying. But there's an obvious problem here, is that if I had this map a priori, my problem would be totally solved. We don't know what it is, and we need to devise algorithms, methods, and perspectives to try to infer what this map T is from data or from descriptions of this problem. Okay, so if we only have a parametric model of this map T that I'm going to call T-hat, to make it really useful, I need to balance two qualities, right? If I want to have a probabilistic interpretation of this, I need this determinant that I have to compute to be computationally tractable so that I can define density, rho 1, but also I want to learn this T in an expressive parametric class of functions, right? So I need this T to be sort of maximally unconstrained in some sense also. So there's a trade-off here that I'm trying— that we want to balance, right? We're trying to form optimization problems instead of— and then you may have various applications in mind, right, that go beyond, for example, what I'll call naive generative modeling, where we want to, say, take a Gaussian and push it forward into some data distribution You may have other instances of domain adaptation, things like translation, or you may have infilling of proteins, problems like this. You may have a very specific type of measure and transport problem where you have a graph, a point mass that says maybe here's the weather right now, predict for me an ensemble of future potential weather outcomes that come at a later time. And so we want to have a flexible language that can deal with all these types of different problems. So that's what I want to focus on, is how do we harness metric transport for these various tasks? Okay, and then how do we actually learn these maps once we lay out these perspectives? Brief history on it, just to give some context that people have been thinking about this for a while. So, you know, it may be hard to learn this map T in one go. You might want to break it into smaller maps that allow for it to be more computationally tractable.. And this is sort of the original perspective on this problem. It goes back to Chen and Gopinath in 2000, a paper called Gaussianization, which tries to sequentially learn to connect the data they have to such Gaussian distributions. And then Eric Vanden Heide and Esfahant Gibbaut and his postdoc sort of extended this idea. And the way they sort of phrased it was a sort of a perfect setup for the neural network folks to come along and say, ah, if I have this sequence of maps that are, you know, simple transformations, maybe each of them can be parameterized by a neural network. We can put a neural network somewhere and then the methods we have also work better. So this is the idea of Laurent Dinh and Danilo Rezende and George Papadopoulos where they made these TK structured invertible maps. Okay, but you can imagine taking this sort of K equals infinity limit here and saying, I know I have this map T that takes a sample x0 from rho_0 and pushes it into a sample x1 from rho_1, And imagine that it actually emerges from continuous time flow. Okay. And this is the perspective of Ricky Chen, who I think I don't see him here, but I know he's talking later and will graph all the data that they call forward that adopted what we know as a neural sort of set. Let's, let's put parametric optimization problems in theme with differential equations and see what comes out of it. And when you do this, this determinant that we had to deal with before now becomes the trace of an instantaneous change of what I'm going to call a velocity field. Okay. And that velocity field v, which I'm stressing because it's going to be the object of study for the rest of this talk, is really now the sort of central thing we want to focus on. The trace of its instantaneous change can actually be estimated now instead of having to directly compute a determinant. And this is sort of a favorable property. So because we're going to think about this velocity V, right, as the thing that solves our probability flow, okay, it's going to provide our transport between densities, worth introducing all the notation we have for that, and then I promise there'll be some interesting results. So the thing we want to endow with this velocity field V is what I'm going to call a flow map, which is capital X little t here. And this flow map is such that if I take a sample x from ��₀, which is in this case going to be a simple unimodal Gaussian, and I put it through this flow map, which emerges as the solution of this ODE, okay, then the evolution of x under ��₀ will follow one of these flow lines. Okay, so xt(x), right, is going to be some sample from the intermediate density ��_t Okay, and if I go all the way to t equals 1, then if this is all learned perfectly, if I know what the V is that does this, then this is going to end up exactly as a sample under the law. Okay, so it's a way for me to endow this map, right? And we'd say the flow map all the way to time t equals 1 is the transformation that I want. Okay, so if I can solve this ordinary differential equation, which you can do, for example, with the Euler integrator, so this, then you have learned your new flowmap. So if we do this, we could also ask what's happening at the level of the distribution. Okay, if I define a rho_0 here and a rho_1 here, and I'm saying I imagine what the flowmap would do for all the samples under this density, then there's a rho_t, a time-dependent density, that you can define anywhere in between these two densities. This one is this Gaussian, this one is this mixture of three Gaussians. And what we can say is that if this is solved, right, if we actually go from rho_0 to rho_1 with a velocity here, v, that does this, then rho_t satisfies what we call transport or continuity equation. Okay, and so what we're going to want ultimately is to find the v that solves this problem for us. Okay, and so we'll assume that, you know, the densities are smooth and that This is a well-posed problem, and our answer would be what is the sufficient p(t,x), the function of time and position, right, that allows us to bound this properly. Okay. What would be the simplest paradigm that we could imagine doing here? Right. The dream scenario would be just to figure out a way to perform regression on the true velocity field that connects these guys. Right. But it's kind of a chicken and egg problem because if I knew what the true velocity field was that fulfilled my flow map, my problem would be over. Because I would already have the solution. So that's problem number one. And the second problem is, if I were to try to perform this regression to learn this d everywhere between these two densities to build this flow, then I need to be able to sample the time-dependent density, this rho_t, at any time slice in between here, right, that allows me to learn it everywhere in the interval that I care about, which I define as between 0 and 1. But you So how can we work exactly on this integral? Choose an arbitrary rho_0, an arbitrary rho_1, and somehow learn this velocity field by way of this regression feedback problem. Okay, that's what I want to lay out. To do that, we're going to introduce this thing that we call an interpolant function. Okay, so an interpolant function is going to be some function that I call x. Okay, it's a function of time. Samples from the initial density x_0 and samples from the target density x_1. Okay, it can be any very simple function that just has the following qualities. Namely, at time t 0, well, this thing should just evaluate to x_0, and at time t 1, it should just evaluate to x_1. Okay, and the purpose of this is to say that if you are randomly drawing So, so let me give you a simple example, very simple one. It's just 1 Tx0 Tx1. That's a valid interpolating function that you can use. But the purpose of this is saying, if I'm randomly drawing x0 and x1 from some density, some coupling, right, I get an x0, I get an x1. This could be, you know, randomly drawn from each of the marginal densities. I draw something from the base density, I draw something from the target density, but it could also be more complicated. If you do that and then you evaluate this interpolating function, then actually this sample now, x_t, x_0, x_1, is a sample from some intermediate density rho_t(x), okay? So this guy is actually now a sample in the intermediate density. So you could visualize this by drawing samples at any time slice, by drawing x_0, drawing x_1, constructing the interpolating function, you get this sort of smooth transport between the Gaussians and the 3D disks that I was talking about. In fact, you know, we can state what this density is sort of formally, but it's not super useful to think about. It exists. This is a way of defining an intermediate density, some path in the space of measures between the two that we talked about. Okay, and what's going into this definition of rho TX? This is the guy we keep it, right? Well, one is we choose— chosen a way to couple— we sample X0 and X1 that we've drawn Okay, and the other is the choice of the interpolating function. Here I showed the trivial one, but these are the two knobs we have to decide how we're going to connect two densities and draw samples from the path along those connections. So this gives us step one of what we need to do if we're going to perform regression on the, you know, the velocity field that allows us to form that. We need to be able to sample some time-dependent density, right? So that we could, you know, over the entire interval, regress what the true velocity field is. Okay, so what then? You know, we still don't know what we're regressing, right? I know how to sample this, but I'm trying to fit a model p hat to some p, but I don't know what p is. I'll just state a definition. We can discuss offline if you want about how to show that this is true, but we could say that once you have this rho tx, Okay, and it satisfies the transport equation that I was talking about in the previous slide. Okay, then the VTX that does this, given the fact that you've introduced some interpolant, it's just given by conditional expectation of the time derivative of that interpolant. Okay, so for the one that we were just looking at, we're at 1 Tx0 Tx1. The time derivative of that is simply x1 x0, and we want take the conditional expectation of that quantity over f(0). But this is still a little— because I can take that conditional expectation which I just defined, plug it into my regression problem, and now I have a new object that I don't really know what it is, right, that I need to try to regress. So I haven't really helped myself too much yet. But if you just use the definition of the conditional expectation, which we were in this case integrating over this kind of density rho 2x, then the definition of the conditional expectation really tells us that integrals of this form, just the power property of the conditional expectation, can actually be evaluated by just taking expectation over x0 and x1, okay, over the base and the target densities. So if this is the case, then I can use this fact right here to write down a very simple loss function for what, uh, b is without ever having to define what b is explicitly. Okay, it's just given by the regression of our model for the velocity field, b-hat, minus the time derivative of the interpolant, taken in expectation over samples from the base and samples from the target density. Okay, so this gives us a nice simple description, right, of how to learn a map. Between densities by introducing a path between the densities. We've chosen a path, we chose kind of a random one, right? We just said we're going to linearly interpolate at the level of samples stochastically, and then the dynamics of that linear interpolation actually give us the way to learn μ. Okay, but the real idea is that you fix a path in the space of measures and then you can do this. Okay, so this is the same sort of algorithm that Ricky Chan might talk about a little later today that they've been calling flow matching.— we call it the stochastic interpolant, thinking more about the object doing the interpolation, but the methods are exactly the same. And I think we all like it because it's now giving you a way to directly estimate the map that connects these densities between arbitrary densities. And, you know, I'll get to diffusions in a moment, but, you know, this allows us to use sort of fast ODE integrators to perform this, and there's some bounds you can write down on this loss that say that this loss controls the distance between what your model is trying to actually model, which is the real one, and what it is in terms of loss by 2 distance. But the point is, our generative model now is saying, if I have btx, I solve this ODE, and this is basically the generative model. Okay, so that's a way to do this for deterministic mappings between, for example, this swirl's density to this checker density. But you might also ask, is there a way for us to extend this picture to diffusion? What's going on with stochastic dynamics? And now the same probability density at any time slice to do this over sort of a randomized map. And I'll just quickly say that there's a simple set of criteria that fulfills this. It involves just taking the integral that we had and appending on another Gaussian latent variable that's independent of the two marginals. Care about. You take this interpolant, you say I'm going to add Gaussian— a Gaussianity to it basically everywhere between 0 and 1 such that this disappears at the endpoints under some mild conditions, and then you can proceed through the exact same procedure that we had before. The velocity field associated with this interpolant is also a time derivative of the interpolant. We now have this factor of gamma t c there, and importantly what this allows us to do is write down a small for the time-dependent density connected. So if you want to connect this to score-based diffusion, then this is sort of the handshake that you would do. It also has a very simple regression loss. So this gives us two perspectives on how to push samples from one density to another. You could do it either deterministically following a transport equation, or you could do it stochastically following a Fokker-Planck equation which is basically saying let me add on some diffusivity on one side and then offset it with a new type of velocity field. So I'm going to differentiate between the objects you have to learn here by calling one of them B and one of them Bf or Bb for forward or backward. Because this stochastic process isn't directly invertible, you need to think about the second— and so what this is saying is basically if I have B, then there's an ODE I solve that gives me a deterministic. And if I have this Bf for Bb, then there's a stochastic differential equation which I can solve. This involves adding various steps that allows me to end up with a stochastic generative model. But why bother differentiating between the two? Okay, this is sort of a discussion we'd like to have about what's the benefits or trade-offs between deterministic or stochastic generative models. And I'll just say, without having to go into too much detail, is that if you wanted to control the KL divergence between the density you're trying to match simulate, and the one your model endows, then it's actually not sufficient solely to use a deterministic mapping to control the KL divergence between— in fact, if you want to do this, you also need to control the Fisher veterans. Okay, so you can't just regress b-hat and b and say I've done this to finite error and say I've controlled the KL. You also need to control this Fisher veterans. However, if your dynamics were actually pushed towards stochastic ways through this b-hat, that I was talking about rather than e, okay, then you actually do get a bound on the small divergence. So there's something intrinsically different going on here between the deterministic and the stochastic generative models. And we did a little experiment just to verify that this is the case. You know, if you imagine trying to estimate a Gaussian mixture in high dimensions, you know, 128 Gaussian mixture model, we can solve these equations exactly there, so we can compare to the ground truth. And we can sort of see that, uh, just look at this plot. This is KL versus the level of noise that we're adding into the stochastic dynamics. Basically, as you approach some critical value, KL is minimized to its greatest extent using the stochastic dynamics as compared to if you were just trying to do this with a deterministic flow. Okay, so I'm just going to pivot quick to some context for these types of methods and applications that we're trying to consider. And so as a reminder, we're generative modeling domain adaptation, we're interested in problems related to forecasting. We're going to try to exploit the fact that we can design these interpolants in a variety of ways and construct various types of couplings to fulfill these types. So just as a quick caveat to relate to, you know, these incredible score-based diffusion methods, which are an inspiration for sort of the regression problem you're trying to lay out here. If I'm sure you're familiar with this model at this point, but score-based diffusion basically says I'm going to have some data and I'm going to introduce some noise process on that data. In fact, follows this thing called a non-stationary embedded process, okay, that gives you a rate at which this sample is evolved into Gaussian noise, okay. And what they say then is if you want to learn the reverse process to do this, you need a model for this guy right here, this SDE, which is the score function at any time t. And so the score-based diffusion says the generative modeling problem is to come up with some S-hat that approximates this grad log rule. But just to make the handshake here a little clearer, we can actually say this whole problem can be recast as choosing a specific interpolant. And in fact, it's choosing one instead of putting 1 minus t and t to this interpolating function I specified. It's e to the minus t squared, 1 minus e to the minus 2t. And I have an integral from 0 to infinity. But the important thing is these coefficients are fixed by the noisy process that you choose in your SDE. And we're trying to open up the fact that there's a whole variety of maps that you can construct that have nothing to do with this SDE but still allow you to now have a stochastic generative model. So let's look at an example of what are the potential benefits from trying to think a little bit more broadly. Because we have a freedom to choose any function alpha_t and beta_t that go into this interpolant, okay, and we may choose one, for example, that reduces the overall transport cost from pushing rho_0 to rho_1, we can ask about what are the benefits of doing this on large-scale image generation problems where diffusions are currently applied. So some of our colleagues— who, in a timely manner, was the guy who authored the diffusion architecture behind this new Sora model that OpenAI put out— we were interested in investigating this. So by trying to teach these algorithms data in trying to tighten this KL bound that I was talking about, we can sort of systematically say, let's apply that diffusion architecture that they were using, and let's do it with the interpolant. And see what the benefits are. So you can speed up the training by a factor of 2 or 3 and sort of push down what they measure as a— what they use as a measure of success is this Fréchet inception distance, which compares your real images to their representations in a classifier. You can push that number way down in sort of a much more efficient manner compared to what you would just do with diffusion software. So it's kind of just a plug-and-play drop-in for what you would do with diffusion. It gives you a better tuning of performance here. But the thing we also trying to stress is that we're allowed to couple the x0 and the x1 in different ways now compared to what you would do in diffusion is to sort of choose them independently. So if I'm drawing x0 and I'm drawing an x1, so I have some joint density rho(x0, x1), okay, you could do this independently. This could factor into a product distribution. That's what they do in diffusion. But you may know something intrinsically interesting about the two data, the two densities you want to connect. Okay, so is there a way for us to construct maps in this case? Let me give an example of what I mean. Say, for example, I was trying to draw a sample x_0, x_1 from joint density here, but I knew that x_0 is conditional on x_1. For example, I'm given— I know that there exists a clean image in the world, but I'm only given masks versions of it. They're corrupted in some way. Could be a corruption, like a masking, or I want to go from low resolution to high resolution. What we could ask ourselves is, can we learn a map directly from this corrupted density to this uncorrupted density, right? Or these are GIFs, so you have to— what it becomes. Um, so this, this allows us to do this. If you wanted to do this with diffusion, you'd have to map the whole thing to Gaussian addition to the velocity field to map a Gaussian sample to your uncleaned image. And there's some hacks that have to come into doing this. So now we just want a systematic way of trying to connect these steps. And again, if you do this with sort of a systematic framework, you can push down these Fréchet inception distances in a, in a much more efficient way just by building maps between the more proximal densities rather than auxiliary maps around more complicated But then we'd also like to consider new types of maps that are not just between these smooth densities, but potentially between point masses and future possible ensembles of outcomes that come later. So the, you know, the perfect example for this is weather or dynamical systems. If I give you a measure of the weather at time t, which I call here xt, well, I'd like to be able to predict, for example, the weather at time t plus 1, t plus 2, t plus whatever, right? And we can ask ourselves, is there a simple sort of dynamical transport picture that we can write down in a learnable sense that would build the map between these densities directly rather than doing something that's conditional on some Gaussian random variable? Okay, so this is something that you can write down. You know, ultimately I'd say we want a sample from some SDE, okay, that's going to take our initial condition, which is our point mass, and push it through a whole ensemble of points. How do we go about doing that? We introduce a slight variant of the interpolator that we had before, okay, which now has a factor of square root t in here, okay, and we're going to couple these x0 and x1 as if they're sort of time slices. So x0 is time t and x1 is time t plus 1, okay, and if these are sampling some invariant measure, for example, on your stokes something like the weather, then we can write down an associated reference dynamics. It's just about almost taking the time derivative of M_t like we've been doing this whole time with B, but we leave in this factor of square root of t. And if we do this, we balance things correctly, then we can say there exists some drift, some velocity field B, okay, that is also a conditional expectation in its analytic— conditional expectation at this time. And then this B solves an SDE that has the appealing quality that at any time t, it's sampling from the conditional density of Xt given x0. Okay, so this is us saying we're going to be able to write down an SDE that takes a point mass, and in its stochastic dynamics associated with this drift B allows us to sample the entire ensemble a potential future, for example, weather outcomes, next stages in your dynamical system, etc. So what can you apply this to? Well, we took a, you know, sort of simple example of Navier-Stokes equations that has random forcing in it. Okay, so there is sort of a stochastic aspect of what happens next, but there's an invariant measure of what this vorticity field sort of modeling this fluid flow, it's going to represent. Okay, now we would love to learn the distribution of this vorticity field at time t plus tau given some snapshot of this time tau. Okay, if you do this in a probabilistic sense, okay, you can actually recover— you can't see the error bars in here because they're small— what the true sort of energy cascade or structure is, frequency structure is, in all ensemble of predictions you should get here. If you were to do this by just trying to deterministically map omega t to omega t plus 1, there's an intrinsic bias that's going to emerge pretty quickly instead. Another sort of, you know, now it's hard to compete with something like OpenAI, but if you wanted to form this as a sort of video completion paradigm, you might want to ask, given 10 frames of a video, how do I predict the rest of the video? Okay, so we want to learn the distribution of video frame coming next, given the, you know, 10 slices of video frames you have already. So this is like a mapping the point mass of I'm at this video frame, what's the ensemble potentially causally connected video frames I'd like to learn? So you could take, for example, here's a real video and two final predictions of it. Of course, they're not going to exactly reproduce this real one because it's probabilistic, but they seem to naturally sort of fulfill what the potential dynamics are that you would imagine these blocks colliding. Okay, so that was just to overview a method about this, you know, new tools for dynamical transport. Okay, and what I'm really interested in is trying to apply these problems in physics, but I figured it'd be better to talk about a method rather than a physics lecture. And so, you know, what we're trying to do is adapt this now to large-scale simulations of quantum field theory. And particularly, we want to study, you know, aspects of the strong force, the force that holds protons, neutrons, matter together. Okay, that makes up a category. And this is computationally an extremely large-scale task historically. Okay, so this problem of trying to understand the strong force computationally currently uses about 10% of the federal US supercomputing budget every year to run gigantic Monte Carlo simulations. And we want to rephrase that as a generative modeling paradigm. This is something we've been working on for about 4 or 5 years now, but the methods always sort of evolve as you do this. And so, you know, we're hoping to have some good results about this in the next year or two as we scale this up to the Aurora supercomputer at Argonne National Labs. So yeah, thanks very much. Estimate? So for a vector field that we're trying to estimate, like, how would a model like a U-Net work? Yeah, I mean, the thing is, whatever seems to work for diffusion, you could plug in here. They're both vector fields ultimately. So a U-Net like that they use that maps from Rd to Rd is totally fine. And also, the general— Is it generalized selection? Uh, you mean like the sort of neural operator? Yeah, yeah. I mean, I know people have been doing this with diffusion. I sort of think about diffusion as just a suffocating, you know, Gaussian to data version. So I don't see why I wouldn't be able to do this.

Yeah, I guess another question about the forecasting case.

Yeah, I think it went a little bit quick, but what's the reference for sensor factor? It's not like some form of service equation, or is it? Yeah, so in this dynamic, uh, and here we're actually simulating the Navier-Stokes equations so that they wind up sampling the invariant measure of what these vorticity fields, uh, will look like at various times. And then you can phrase this as just a conditional generation from given any time slice. What's the time slice And we're not considering the dynamics, so actually taking omega t to omega t plus 2, because that would involve solving the stochastic equation. Instead, we're just trying to forecast. But the pitch is that you should draw a generative— construct a generative model that's already proximal to the target density. If I had this omega t, well, there's only a small transport would be needed to turn it into any of these guys. Whereas if I did some diffusion, I would actually have to take this guy, make it Gaussian noise, and then learn the map that takes that Gaussian noise conditional on this guy to this, right? So it's like making the transport problem harder. So the way you get this actually is related to this very specific process called the foamwork process, which constructs a Schrödinger bridge between a point mass and the density associated with it. So how does Spark give you the offering that MPE gives you like better performance compared to ODE? Yes, yes. And so in terms of other aspects, for example, would it be sampling faster convergence? Like, can you paint a picture of it? Yeah, yeah. So the ODE, this is a great one. If you really So this is the trade-off you construct, right? The OD, you can— it is easier to solve, right? There's a whole suite of adaptive methods we have, the paths I gave you to integrate. So you may be able to do this in fewer steps, right? But even if you did it perfectly, right, for your model, right, it still wouldn't be sufficient to bound the scale divergence. And if you're willing to pay, for example, the more steps you have to take to sample this SDG, Then you actually do get control on the scaling. And so we for a while were skeptical about if this is real, because okay, it's a bound, how does it apply to the problem we do? But pretty much systematically, there's always an epsilon you can choose, right? This can be defined by the diffusion coefficient. How much noise are you adding to this sampling? That always seems to give better FIVs, better KLs, et cetera. And you can sort of imagine this like, If you were wrong anywhere on this transport, like your b-hat is not actually b, and there's no recovering, right? Like, yeah, for the ODE, for what the trajectory was that you're trying to solve. But here there's sort of a randomness that is allowing you to make mistakes, because if they're within the realm of sort of the scale of the noise, then it might be corrected on its own. Thank you. All right, yeah, we're gonna have 20 minutes. We're going to walk in the snack—