Scientist Stories: Francisco Villaescusa Navarro, Cosmological Surveys

So our next speaker is Francisco Navarro from Flatiron Institute, and he's going to talk about cosmic graphs. And he did his PhD in astrophysics in Valencia, Spain, and he did lots of postdocs in Italy, New York, and his research life is currently at Flatiron Institute. Thanks. So thank you very much. Can you hear me? So So first of all, thank you very much for the invitation. It's a pleasure for me to come here. So I'm coming from an slightly different world. So I'm going to mostly talk about applications, about how we can use the graph neural networks to do, in my opinion, really cool stuff mostly in astrophysics. So I can change the slides. So this is the outline of my talk. I was assuming that maybe not everybody here is familiar with cosmology, so I'm going to give you like a very kind of high-level introduction about what cosmology is and what are the problems that we are facing. And then I want to talk, you know, from a story perspective, what is the way in which would have been tackling these problems, very standard pen and papers, basically very mental, let's say, heavy calculation. And then I want to tell you what are the things that we can do these days with PNAS. This is kind of a revolution, I would say, how we completely move on from this. So So what is cosmology? So basically what cosmology is according to ChatGPT, so you can see that this is what the typical style of model is. I know, very disorganized, but I like this kind of image because it really shows a little bit what we really do in cosmology. So imagine that you were able to put the entire universe inside a crystal ball. So basically in cosmology, we will basically study what are the properties inside the crystal ball. So things that we care is like, what is the origin of the universe? What is going to be its fate? What is the composition of the universe? What are the laws of the universe? And what is the structure? What is the dynamics? And all this kind of— so basically properties of the universe as a whole. So I don't know how much you know of this, but this is basically the composition of the universe as we understand. We believe that 5%— so we believe that only 5%— the darks are only 5% of energy content in the universe. And by this, you basically— everything that we saw in all kind of atoms, molecules, everything that we know today is basically made from dark matter. And what is really surprising is that in the entire universe, only 5% of it is made out of this. So we are kind of a little thing for the universe. So we believe that the dark matter represents about 95% of the energy in the universe. So we don't know what that is. Should be some kind of particle, but we don't know what. Should be something there. And, but we have a lot of evidence that is there, you know. Basically, we look, for instance, at how fast the galaxies are rotating, and they are rotating much faster than a good, dynamic stream from basically the light and the dark and the stars and everything that we see. On top of this, what we are finding is that the universe today is expanding. And this may, may sound peculiar. We believe that the universe started with some kind of explosion, the Big Bang, so you would expect that the universe would expand. But what we didn't know, and it was really a great surprise, is that the universe today is accelerating its expansion. So there is really no way for us to understand how that is possible. And basically all our inference on this would claim that this is due to that event. So we don't know what this thing is. It probably is some property of the vacuum, but what we can see is that most of the energy in the universe is basically this dark energy. So, you know, what is the kind of weird state where most of the things we're looking at are in the thing that we know is so exactly like the same thing? So the modeling, as many other branches in physics and many other disciplines, would have, in our case, the standard modeling of motion. And this modeling is remarkable, basically, because it is very much data-driven, not by postmodern observations. For instance, if you look for galaxies, basically in the local universe, And this is a picture of the baby universe. This is a picture of the entire universe 13 billion years ago. It's remarkable. So we have a model, and I'm the main cosmologist, and I'm the most massive data compression I've ever done. Basically, we have 2 parameters, 5 antennas. We're able to explain the entire universe. and all these kind of things. So what is the goal of cosmology? In cosmology, the most important thing, one of the most important goals for us, is to constrain the values of these guys with the highest accuracy. That's kind of, I would say, the most important goal. So why do we want to do this? Because we want to know about fundamental physics. We want to know what dark matter is. We want to know what dark energy is. We want to know about the universe. So by measuring these parameters with high accuracy, we will be able to develop different hypotheses about what are the natives or what are not natives. So we'll have some parameters and we have to measure these parameters with high accuracy. So how, how do we do this in practice? So basically, this is basically a mapping problem. So what we do is we dial this resolution, and you can think of this like you measure the position of galaxies in the sky. And you know, until now, I would say we never really thought of a world with this problem. If someone gives you like a galaxy catalog where there is a position here, a position there, we didn't know how to work with this. And basically what you want to do is that if you have a different universe, let's say with a different composition, with a different age of matter, dark energy, how will this, let's say, summarize— what would you call this summarize— how will this imply? And this is a little bit the problem of cosmology is that for the moment we have one single universe, so we really need theoretical physics to know and speculate about the universe and know the composition. We need pretty theoretical predictions. So historically, the question has been like, what is the best way to compress it, right? So we know that under certain conditions, the universe resembles what we call the Gaussian density, but it has to be a very particular, say, structure of the perturbations. And we know that there is a b-sammar statistic, the power spectrum, that gives us the log-log statistics, minor perturbations, image, and go to Fourier space. And basically you make a histogram of the amplitudes of the modes. That's pretty much what this is. So under centering conditions, we know that this is really a b-sammar statistic. So this is really But this is a picture of the universe 13 billion years ago. In this case, we're able to start every single bit of information. We're able to determine this point of dark matter's fundamental nature. That's my opinion on Big Bang. The problem is, folks, once you get this, you don't know why there's a distribution of galaxies that say the lower universe We don't know how to characterize this. We don't know what is the, let's say, the data compression that fully characterizes the statistical properties. I mean, this is a problem that we can get very rigorous mathematical answers. You can ask mathematicians to do theoretical physics, that's mathematical. So let me very briefly talk about the historical perspective here. You know, I would say even until maybe 5 years ago, most of the community had been doing kind of pen and paper calculations using this power statement. And basically what we do is some kind of Taylor expansion. You have some equation, you assume that there are perturbations that is, say, small, and basically you do a Taylor expansion. If you know that at one point this one has a correction The problem with this system of things is that if the perturbation is large enough, the moon will eventually hit the infinite power of 1. This will not work. So, recently, the community has been shifting to an activity model involving a numerical simulation, and the good thing about numerical simulations is that you can just solve your equation. And this is the best for now. I want to show you a few more examples. So what you can see here is a distribution of matter in the universe. This one is more aggressive. So C equals kilo means very high aggressive, means very heavy. So this is how it would look like, let's say matter. And you can see that, you know, matter is clustering in this object that we call dark matter cage. And these are basically the regions in the universe where galaxies are going to be. It's a very large concentration of dark matter where galaxies are going to reside. And this theory was actually incorrect by Professor Rudolf Lanzky's forebears, that dark matter— and then there are regions of the universe where there is basically nothing. We call these, uh, cosmic voids. So this is another point. So what you can see here is two different universes. They have the very same initial conditions. The only difference between them is that there is more matter in this one than in this one. And I don't know if you can see, but there is a kind of gap here And as we move from one universe to the other one, think about this. This is a rapid thing. And when you move from one universe to the other one, things change. The position of these amarajayos changes. Their masses are not in the same is that when you change the motion of the universe, when you change the laws of the universe, the spatial distribution of galaxies is going to change. But now we can invert this argument. Say, like, if we now look at the distribution of galaxies, galaxy type, sky type, can we infer the properties of the universe? So that is what we call the second model. Just before we start, let me mention that in the very last 5 years, the communities have been spending really a lot of computing time running numerical simulations. So we have daily updates to Red Garden predictions. These days we are basically having on the order of hundreds of thousands of these kind of numerical simulations with petabytes of data. And basically, obviously, it has reached like a generic conclusion. And the conclusion is that whenever we look at things, compress the data in ways different to the standard way that would happen with models, we gain information, and start much more information. And the second thing that is very important is that whenever we look at the universe on smaller scales, we do it So what is the problem of looking at a small space? So, again, we are— we look at the glass, we see that, but we never really look at them. When we are— we need to block them. And you wonder, like, why do we do this? So let me explain. And this supermassive black hole is just here, this, this gigantic amount of energy. So the thing is that this gigantic amount of energy that is released by a supermassive black hole is not only affecting the properties of this galaxy, but it's also going to affect the properties of all the galaxies that are nearby. The thing is that we don't know much about the physics of these things. This is basically our main intention right now. is the state of the intergalactic medium, and especially affecting the properties not only of this particular galaxy but also the galaxies that are around it. So let me conclude just this first part. I would like to make this comparison, and it's like, imagine that you were lying under the telescope— at that time people were able to read some fragments about this. I think they were able to find some other stuff related, but most of the information here was simply not available. I have the feeling that we are in a very similar space where we have a lot of data, a lot of data, the data is beautiful, it's not a problem, it's simply that we don't have software. So now let me talk about what the lab from IAS has been doing using this kind of deep learning approaches. But before this, let me talk about the dataset that we are going to use. This is like really important. So the dataset that we are using is coming from this project that is called Small Area Analysis With masson-like oscillation, this is the largest set of hydrodynamic oscillations ever produced. So this is one example of one of these hydrodynamic oscillations. What you can see in blue— sorry, I mean, this is a little bit lagging, I think, so in blue you can see the gas density, and in red you can see the gas temperature. And you can see all these explosions going on. These are either supernovae that are exploding, releasing energy, or nuclei in supermassive black holes that are getting released in seconds or months. So the reason why we found these collisions is basically because these are the most accurate theoretical predictions that we have right now. And the reason for this is basically because Gas formation is a complicated process. You have the dark matter accumulating, forming these dark matter halos, which are in form of gas. It's going to fall, it's going to cool down, it's going to collapse, it's going to form galaxies. But these galaxies, they're going to die at some point. Some of them are going to explode as supernovae. Some of them are going to form a supermassive black hole very nice. And the supermassive black holes are going to release all this energy Gas is also a heating gas for the filaments, and gas is going to set a part of it. It's really complex, it's really complicated, this process. And that's why we rely on this kind of, let's say, I would say very sophisticated methods. So it's obvious, but it's very expensive. You know, we never program a thousand of them, it's very expensive. Another thing that is very important is like, because going ahead in astrophysics, basically because we don't know the physics of these supernova explosions, this energy release at the mass black holes, there is no maximal solution. Let's say that different groups has proposed different solutions on how to model these anthropogenic processes. And probably all of these things are wrong and nothing of them perhaps in reality passes the very basic test. So everything that I say is that for us it's super important to don't rely on simulation because that simulation is like a big model. If you want to develop model to apply to the real data, real universe, at the very least you need to make sure that your model works in normal cases. And what I'm showing in this movie are basically 6 different simulations. Each of them has the same initial conditions. Each of them has the same composition of the universe. Each of them has the Lagrangian model of asteroids, how to reach Now, there are 6 of them. Very, very easy, right? There is this one that is speaking out the glass very early on. There is the one that is yelling, "Yes, kind, awesome, the best in the world." "I think that I'm honest." They look very different. The important thing here is that all this information basically reproduces the real data. So we have 6 different types.— that give the same answer even though they are completely different. When you train your machine learning model, it's very important that the model works at the very least in all these different types before you're able to apply this to real world. That's something that is totally very, very important. Finally, so this is my proposal for the basic design. Training your networks. Service integration has a different model of the most— you can see that when you change the voice, the speech will be automatically changed. Every simulation has a different model, and every simulation has, let's say, a different efficiency of how much error you release with SuperNOVI or with SuperMASC platform. So the whole idea here is that you have this very large parameter space and you have something new, and you basically want to train models to somehow interpret this speech. All right, let me talk about the, the DNA. So imagine that everyone here is saying like, whatever I do, I don't have a conviction, is a part of post-lateral excitation, please is a satellite. Even though reality, the data, is being formed on a smaller scale, and because of the connection between the points, it's not really— in principle, everything should be connected with everything because everything is interacting with everything through gravity. But we cannot really work with full parameters. So you want to give this talk a coherent sense and bring it to a conclusion. Say, uh, regular You have a bit of sides, the grids is very fine, a couple of options. So in all of this, basically the forecast about this, we need to compute this possibility, and the problem is like we have now from this simulation that I showed before, we have thousands of virtual windows The most sophisticated universes ever created. Every universe has different values of cosmological parameters. So again, this is a diagram of what we can say about the speed of expansion. So what we would call the expansion rate of galaxies. And let me just pass ahead a little bit fast. The galaxies, the only thing that they have is position, mass, and the models of the universe. So which architecture could you use to make this very analog one? DNA with the message passing, is that a typical thing? But not features, they have these layers, we have lights in all of them. So this is a lot of commands that we use. We will have— we use simple pooling operation, do all possible pooling operation that we have. We compare the vector. Okay, so this is a lot of commands that we use. We will have— we use the normalized flows on the left. It's going to be easy. So we're going to create both the SC3 and the BAC. So let me tell you how we do this. I have a lot of papers. So why is the SC3 and the SC3 then receiving the BAC? Yes, in principle we're going to impose the BAC relation. In the linearization we assume that we are symmetric to this So for every node, for every galaxy, we have positions and two scales. We have the diameters and we have the motions, the velocity. So the way that we illustrate the graph is like our graph is— is the gas line from the north Now it's on. You can basically compute vectors like, for example, if you are using this edge feature, you will have one vector, another vector, this vector. Basically, you can follow, uh, so that paper. And what we do is that we go as edge features, basically all possible combinations of scalar quantities between all these vectors, right? So we compute Basically, one of them is the distance between the two points, and the other thing are the angles. And the beautiful thing about this is that this is already in translation. You can rotate the angles, the distance is not going to change, and you can translate things and nothing is going to change. So it's, you know, it's a very simple formula. Possible that are not, let's say, extreme variant, you basically write things in this language, then by abstraction you model this extreme variant. So this is how one of the data graphs for us look like. Basically, we connect two points, and the distance between them is called the event length. This length is hyperparameter that we optimize for, and this length is what the model prefers. So somehow the model does not prefer this appendiculate T-graph. It could be like overexpression and all these things, but this is— so from the control perspective, this is very nice because we don't have like 1,000 parameters very gradual, but the structure of the model is extremely valid. Basically what we always say, those models are— if you're doing this, you are extracting information from even the smallest scales. You don't have to have classes that are super close together. Principal classes are extracting information from this. This is very different from algorithmic models, where we typically put everything at the same resolution. So basically we train on like 1,000 of these graphs. We train our DNN on this. And let me show you the results. So this is like a typical plot of true versus prediction. This is for one of these parameters. And then we go there with the other parameters.— So this was for a model that we trained on simulations and we tested on simulations that were not with the signal. So we know that this usually works well. The problem for us, and I'm pretty sure that is in other fields, is that if you train your model using some particular simulations, somehow you change something. in incredibly small scales, dealing with all these nasty pieces that I showed you before about all these senile dementias, vascular disorders, and all these things, basically we normalize these processes. So this is kind of a completely new world we're going to go through. Let me talk about the interpreting. Imagine that you only have one day. You can write your parameters, you have all these things to measure. There are some functions that are like this, but the important thing is that these functions only there in a single context. I mean, they will be there for tracing, but if you are— if you're saying your DNS has a little more complex problem, let's say that I have a compost food Then the beautiful thing is that we can also approximate these functions using symbolic regression and get an analytical equation. Basically, we took the GNN model that we trained before, asked it what symbol could drive this set of words. Yeah, I'm sure you have never seen anything like this. So you represent this, and in astrophysics, for a very long time, we always wondered what is the mass of our gas, you know, how heavy is the Milky Way. So people have very, very mixed thoughts, but the more you have some reasons and you try to listen to reasons, or you listen to those reasons, you don't have something, but you have a lot of stories. Then when you start trying to process them, you start to think, okay, now I'm getting a couple of more reasons. With a lot of feedback, then you start to create graphs. And basically, you train the graph to infer the most obvious characteristics. with this kind of experiments. We are really making an assumption in this. That's basically why I'm interested. And this is the very last thing. That's about the nature of dark matter. So I told you the nature of dark matter is not a big problem in cosmology and physics in general. But I want to show you inside the universe. So this is one moon where we have a lot of dark matter. So what you can see is that this halo is kind of forming, it's accreting more, more details. This galaxy in the center, the massive galaxy, is accreting. This one produces a beautiful spiral at the end. And from time to time, you look up here, and you see some explosions. This is a red explosion, supernova, collisions, and displays. So what we see, the problem here is that for very few changes in the nature of dark matter, things change. The distribution of these satellites is really different in the two cases, but also the morphologies of the galaxies themselves. So, you know, this is where the community is wondering What can we do with, say, machine learning, to answer this question? Because even if we speak internationally, we have to look at some particular property and some characteristic properties when we just try to extract information. So I think this is one example of many times in many different languages different. Also, when you see that, you see data center, everything's public, everything's— super cool application. But yeah, this is like the last slide. Also, when you use here, like, for this kind of things, for this kind of data load, there is huge amounts of satellites. The Galaxy is using not only maybe the GPS, but also maybe images, you would say, or wireless things.— in the graph, things would, to me, seem quite normal. So the small interactions, these days, we are very dependent on them for the support of the device. Conclusions. We now have— what we know of the long-term persistence of the neural network, the most important thing for us is solidifying what we have in the main paper. I think my laptop is— so I'm going to stop here. Thank you very much. Evaluating, you said the GMM performs actually very well, surprisingly well. You're evaluating the true data, the parameter, and the predicted— We do some coverage tests for these countries. Yeah, then we go for coverage tests, like how, how do you— Exactly, exactly. So we do basically like a coverage test because this is not that robust view, is not really that same like over space. So there is a quantity like this, L i squared, that basically is going to be the equivalent to this over space. What do you basically have? See that? It's sort of following how they estimate the true gravity, and their property is like no dimension. Like, is it like just How, like, what is it like? So this case is only like a cat. So then from this you create a cat and then you test it again. Okay. You said that several problems that were thought impossible prior to this tool are being solved. What do you think should be the benchmark? Some of the people who see— if you can explain it. That's a good question. I mean, for that matter, the problem is that the model is something that we trust a lot. It's a real model of inference models. So that makes it stand real. For that matter, the face there are different proportions. So in that case, I think the volume parameter space is much larger, and it's not really that clear to me how you can take a model and design different parameters. That's a question I'm still interested in. So you mentioned at some point that you're concatenating all of these speakers to a link list. So is there any specific reason that you're doing that for the pooling? And what is the difference between the idea that you map all the pooling in these principal negative aggregation, the PNA technique, for getting concatenating all of these passing it to another layer of NLP and learning there? Yes. Yes, you make it more general. Any other questions?