Science & Technology Q&A for Kids (and others) [Part 119]

Go ahead. Hello, welcome to another episode of Science and Technology, Q and A for kids and others. So we're starting a little late today. Let's see. We have a number of questions here. Gosh, There's a question here about. I'm trying to remember which things I've discussed in the past. There's a question here about how do we predict the weather? Let's talk about that a little bit. I think I have talked about this one time before, but let's talk about it now. So in the past, there were a couple of different approaches to predicting the weather. And there's sort of the question of whether you make a physical model of the circulation of air in the atmosphere. There are typically, what's important is both the way that the winds stack up and where the winds go, and also the humidity and the question of where clouds form. And it's always been a challenging question to predict where there are going to be clouds and more to the point where those clouds are actually going to produce rain. And it's always a complicated thing because eventually clouds are water vapor. And there are situations in which the droplets of water vapor, in which the water vapor starts condensing into droplets. The droplets get bigger and bigger. There comes a critical point at which droplets are colliding with each other to make bigger droplets. Eventually those droplets are heavy enough that the winds that push upwards no longer keep them aloft and they start falling. And that's when you get rain. And predicting when that's going to happen is often quite challenging. I think the thing that used to be true maybe 40, 50 years ago is that people would try and predict the weather by looking at large scale features of the circulation of air, high pressure areas, low pressure areas, these kinds of things. And you would see these weather charts that would have high pressure zone here, low pressure zone there, fronts between them and so on. And people kind of got pretty good at saying, well, if there's a front here and there's a low pressure zone there, and there's a high pressure zone here, this is at that kind of large scale level, this is what's likely to happen with the weather. And there were some parts of the world where that worked pretty well, where there weren't lots of mountains and coastlines and things like that in the picture and other places where there were lots of things that kind of made it hard to just look at these large scale features and predict what would happen. Actually, the very first computerized weather prediction was started in 1950, very early. And people assumed at that Time, that was John von Neumann and a chap called Charney. And they assumed at first, oh, now that we've got computers to work out the equations for what will happen if you have just little tiny packets of air and you just compute what happens as that air flows around. They thought it's going to be easy now that we have computers to do this, to predict everything about the weather. Turned out not to be true. And why wasn't it true? Well, we all got a bit confused because in the early 1960s, a chap called Ed Lorenz, who was a meteorologist at mit, I knew him somewhat, actually made an interesting observation which was that if you simplified the equations of the weather technically, you did a modal truncation which kind of assumed that there wasn't friction in the air, that there wasn't viscosity in the air, and you just looked at sort of the lowest, in his particular case, three sort of modes of motion in the air where you had different kinds of circulation and so on. You just looked at that simplified case. You got a mathematical equation, a differential equation, which had the property that if you just made a very tiny change in the initial data that you gave to the equation, you would quite. You would sort of the trajectories that you would get for what the weather would do would diverge exponentially. And so that was an observation that had been seen in a whole bunch of other mathematical systems, hadn't really been noted that strongly. This idea that a small change in at one point can amplify progressively into a larger change later on. I mean, a very simple case where that happens. If you have two spheres, two spherical balls, you. You bounce them off each other. If they hit right on axis, they'll just bounce back the way they came. But if they hit even slightly off axis, they will, they'll just. By the geometry of it, they'll bounce back sort of, let's say, twice as far away from the axis as they originally were. And that will get amplified and at every progressive bounce, they'll be like twice as far away from the axis as they were before. So that as you go for a few bounces, you'll get. It'll be 2, 4, 8, 16, et cetera, et cetera, et cetera, it will increase exponentially what the distance is. And so what Ed Lorentz found was that in these particular equations that approximated this very dramatic approximation to the weather, that really wasn't an accurate representation of how the atmosphere and the weather really works. But in that dramatic simplification, it Was the case that if you made just a tiny numerical change to the parameters to the values of, let's say, something like a velocity at the beginning, then you would get this exponential increase of effect. And so this was sort of one of the things that was the origin of kind of this idea of chaos theory and the idea of sensitive dependence on initial conditions and, and the idea of the sort of exponential growth of dependence on initial conditions. And it kind of led to this notion of, you know, well, if a butterfly flaps its wings in one place, that might lead to a hurricane somewhere. Because in one, if the butterfly did flap its wings, the hurricane is produced. If it didn't flap its wings, there is a, eventually huge change in what the weather is like and there won't be a hurricane. Okay, this picture is really not correct. This picture is very specific to particular mathematical approximations that were made. It is the case that the butterfly happens to be in exactly the right place at the right time. Then yes, it can matter which way it flapped its wings. But the chance of the butterfly being in exactly the right place at the right time is absolutely infinitesimally small. And most of the time, most butterfly effects, sorry for the butterfly, just won't have any substantial consequence for history. It's, you know, most, most things that happen in the history of the atmosphere won't really have much effect. It's only the very rare one that's right on the knife edge between this and that where you can have that big effect. And it so happens that in the case of Adlerance's equations, that, that was something that, that very much got amplified. That effect was very much sort of a, an everywhere effect as opposed to being an effect where most of the time nothing really happens because most of the time it's like imagine you've got some water, for example, and you, you stir up some of the water. Well, most of the time the stirring that you do will just die out. Occasionally if you, let's say you've got some water where you can, it can either slosh out of your, you've got it in some cup or something and you're stirring it around. And occasionally if you, you know, you're sloshing it around just such a way that it might go over the edge. And then if it's just about on sort of a knife edge, is it going to go over the edge or is it not? Well, then a detail of how you stir it can affect whether it goes over the edge or doesn't. But most of the time, most of that stirring you make, it will just die down and eventually your hot chocolate, or whatever it is, will be flat again. So in any case, this led to kind of the assumption that the sort of the problem of predicting the weather was really getting accurate values for measuring all these different kinds of things about the temperature of the air and so on. And that is important. But it doesn't have quite that same sort of exponential growth and effect that one might think from that mathematical simplification. What's more to the point is that it can matter what, for example, the swirl of the air over the top of a mountain can matter. It can cause a cloud to form or something like this. And the fact is that the difference in scales between what happens around the peak of that mountain and what happens over a large land area of 1000 miles, it can matter what the detail of where the mountain was is. And that means when you're doing a numerical calculation and you're trying to sort of divide the earth up into zones, and you're trying to say, we're just going to deal with the air in this region, we're just going to deal with the air in this region, you can end up with places where you have to have very fine gradation, where you have to have very detailed calculation. And that means it can be just a difficult thing in terms of doing the computation to work out what happens. Particularly true. I mean, I forget what the actual numbers are these days, but it's like maybe there are 10 layers or something of air that people consider, and maybe the earth is divided into squares of maybe 20 miles across or something. That's not quite enough if you have the mountain peak can affect things and that's smaller than that. So what tends to happen is that because there are things that are quite small relative to the size of the Earth that can affect whether that causes trouble. Now, there are things that people have done to try and improve predictability. I think one of the more obvious ones in recent times that was started by a series of apps maybe 10 years ago or so now, was the following rather simple idea. So there's weather radar. So weather radar is looking for water drops of a certain size, certain range of sizes that reflect radar, reflect radio energy of a particular frequency. They reflect it well. And so that means you can have a radar dish that's, that's going around and sending out radio pulses in all different directions. And in some directions, those radio pulses will get reflected back by, by the water drops. And so you can map out, you know, where is the, where's, where are the, where are the water drops. And then there's Doppler radar where you're actually looking at the effect when, when the water drop is moving as a result of the wind. You're, you're looking at the change of frequency when the, when the radio, you send out radio of a certain frequency and when it hits something that's moving, the frequency of the thing that comes back will be slightly affected by the motion of the object. And so you can map out sort of wind and the amount of drops in the air. And then people had the idea, well, gosh, if we can see which way the wind is going and we can kind of, and we know kind of what the structure of the clouds is and so on, we can just use pretty straightforward kind of take that structure of cloud and assume that it's going to move more or less the same for a while and see what happens. I think people tried to improve that with machine learning and probably got some reasonable results along those lines. You know, when weather forecasting is done in, one of the things that happens is that because one doesn't know the exact data everywhere, one has to say, well, what happens if the data was slightly, we know these particular places, but we don't know in between those places what the data is. Let's make different assumptions between those places and let's see what the forecast does. If we look at what happens in the future with different assumptions about what the temperature or wind speed is in some particular place. And I think what's usually done is, is people make, I don't know, a dozen different forecasts based on those different assumptions. And then the question is, you look at those forecasts and one forecast says it's going to rain, another forecast is going to say it's going to be nice sunshine or whatever. And then the question is, how do you aggregate together those forecasts? And for example, you can say, well, there's an x percent chance of rain because out of those dozen forecasts, five of them said it was raining and seven of them said it wasn't raining. And so that's one way to get probabilities. It's a little tricky because you sort of, you don't, what you don't know, you really don't know. And so to say, well, actually there is some definite thing that's going to happen and we had to fill in this data that we didn't know and we didn't know it. So we didn't know it and therefore we didn't know how to fill it in. And so this question of sort of how much weight should you put on the different possible ways to fill in that data is very hard to work out. And what tended to be the case? I'm not sure if this is still true, but it tended to be the case that the forecasts that were done by, for example, US Government. Tended to have tended to look for extreme cases in this ensemble and this collection, different forecasts tended to look for the extreme cases. Because that's important if you're routing planes around or you're trying to give farmers advice or whatever else, it can be important what the extreme or you're trying to work out, is there going to be a hurricane that forms? It can be important what the extreme cases are. If you just want to know what's it most likely to do, is it most likely to rain or not today? Then those extreme cases aren't really such an important thing. But it certainly used to be the case that people would just take those government forecasts and compute probabilities from those things which have been actually worked out for the purpose of having extreme cases. I think probably now more people are running these weather models. I know we looked at running these weather models for alpha a decade ago, decided not to, although it was within the realm of possibility to do it. It was kind of a bit of a supercomputer adventure, but not too much of one. The thing that. And we did look at the question of whether if you looked at the forecasts that have been made and you looked at what actually happened to the weather, which way did the errors go? Because you might imagine you could use some kind of machine learning loop to say, oh, we did weather prediction and the weather prediction came out wrong in some way relative to what the weather actually did. Let's use machine learning to improve that. At least at that time. It was interesting that the errors associated of the actual weather relative to the predicted weather were very normal distributed. There was no obvious signal, there was no obvious regularity. There's no obvious thing you latch onto and say, let's make a model of the way in which the weather forecasts are wrong. So it wasn't easy to see how to improve, how to just sort of use machine learning to improve that. I have to say that one looks at the successes of machine learning in all sorts of different areas, whether it's in language generation in recent weeks, or whether it's in protein folding or whether it's in different kinds of places. And an interesting question is whether forecasting Is one of these places where one should expect that machine learning is going to win the day rather than actually solving the equations for the motion of air and so on? I don't know the answer to that. I think there are certainly cases in which you cannot expect that machine learning will win the day. Because what happens in machine learning is you're really just trying to, you really have a lot of data and you're trying to say the thing that you see now, how does it relate to data that you've seen before and kind of the transformation from what you see now to what you've seen before that machine learning kind of allows you is not that great a transformation. It's kind of not like it can figure out things that require a huge depth of calculation to work out. It's just like, well, it's pretty similar to that thing I've seen before. So let me do this now. And that's how it works for, you know, in the protein folding case or one's first, you know, the first step is just to compare the new protein that you're being shown with all the proteins that are known in databases. And then you kind of sort of try and arrange, sort of knit together the things that you get where you say, I've seen one like this before and it came out this way and so on. So it is interesting that the sort of old fashioned way of doing weather forecasting, which was, you know, when I see a cold front and a this and a that together, then this is what's going to happen, is much more similar to what you might expect that machine learning would now do, but perhaps do on a more industrial scale rather than the thing that one does in sort of traditional computer weather forecasting, but one sort of accurately trying to predict this little region of air that's at this layer and this geographic location on the earth is going to move in this way. So it's an interesting question. I don't know the answer whether this is a place where one can expect that. Machine learning will be successful. But yeah, so that's a little bit on weather forecasting. I mean, I think the main issue is people say it's the one day forecast, it's the five day forecast, it's the whatever. And what happens is people kind of have had this idea that oh, there's going to be this exponential divergence because that's what one saw in chaos theory. I don't think that's the usual picture. I think the divergence is much less than that. And what's gradually happened as there are sort of more accurate and more detailed models for actual, you know, the atmosphere, one's able to make longer and longer range predictions on the order of week two, week type predictions, I think at the outside for what's going to happen in the atmosphere. And it can be different in different places. There are parts of the world where it's more difficult to predict the weather than others. There are places where there's a very uniform giant sort of plane in the middle of a continent and the weather just rolls across it without much change. And there are places where there are lots of mountains and other topography or lots of coastlines where there are lots of changes. I think a question people ask is, okay, well we can predict the weather week two weeks out. What's going to happen to the global climate of the Earth a century from now? That's a very different problem. Very much more. The kinds of calculations that you can do that are really just taking the detail of the motion of air and so on and computing it forwards. That's something where you can expect to have some accurate results result when you have to. When you're trying to predict kind of the long range climate, it's a much different kind of calculation fraught with difficulties. I think last I knew clouds were one of the big issue. Just how many clouds are they going to be and what effect are they going to have? And they're just an awkward thing to deal with because why you can, in the case of sort of the, you can imagine you're dividing the atmosphere up into these regions of height and position and latitude, longitude. That's all well and good, but if one place has a cloud and the other one doesn't, then you can have very different effects there. And that, you know, where the cloud is or isn't is something that sort of cuts across all the little squares that you might have had that represent different regions of the Earth. Well, let's see. Yes, Paul is commenting. Basically, weather forecasting is an excellent example of computational irreducibility. Yes, it is. And I could perhaps have said something about that. This whole question of when you know the sort of the rules of fluid flow. There are equations, the Navier Stokes equations from the 1830s or so, I think, that tell you, given that you have a certain pressure and you have a certain velocity of a fluid, how will that velocity change through time? We know the equations for that. Yet just knowing the equations doesn't immediately tell us so what's going to happen. We know we can work that out in principle by running a computer and, and just doing the calculations. Well, normally we know that unfortunately for fluid equations, the fluid equations are continuum equations, which means that they describe infinitesimally what the velocity is of a fluid at a very specific point. And you go a tiny distance away, they'll say it's a different velocity. They say the velocity changes only slowly with position. That's, and that's kind of, it's that slow change. The fact that you can always look at the, the, the slope of the change and things like this, that's what means, allows you to kind of pull calculus into the picture and allows you to have these differential equations that you can work on. So it, but the, the issue with that is when you try and make a computer deal with those kinds of things, computers don't deal with arbitrarily fine sort of continuum stuff. They don't. You just have to say there's this number, there's these bits, there's those bits. You don't get to say there's this continuous variation of velocity with position. You have to discretize it in some way. And it's always a challenge with something like the Navier Stokes equations to know whether you can, whether the discretization you're using is actually going to work correctly. One of the challenges with those equations is that there are these phenomena like fluid turbulence, the sort of randomness you get in fast flowing fluids. And nobody's really been able to reproduce that well from the underlying equations directly because it's always been unclear whether the reason you're seeing kind of randomness when you solve these equations is because something went wrong with the way you put the equations on a computer, or whether it's a genuine feature of the equations themselves. It's a long running, challenging problem to work out sort of even very basic features of the solutions to the official fluid dynamics equations. And we don't even know whether all the phenomena that we see in fluids can be captured by the official fluid dynamics equations, because those official equations describe a fluid as a continuous thing. They ignore the fact that the fluid is at the lowest level, made of molecules. I mean, we have a very analogous thing in the equations of space time, the Einstein equations, which are also these differential equations that describe sort of a continuous version of what the structure of space and time are. And we know that the Einstein equations form singularities in black holes and things like this, where the equations themselves say these equations are not valid here because they're saying even though the equations are all about smooth variation of, in that case, the curvature of space as a function of position. The equations are saying, no, no, actually there's a place where the curvature of space varies infinitely quickly. So, so sort of by setup, these equations can't be valid. Now, in the case of space time, we haven't known for sure whether space time really is governed by continuum equations or whether in fact, like a fluid, it actually has something discrete underneath. I've believed for quite a while that there's something discrete underneath. And in fact our physics project from the last three years has given, I think, really very good evidence, but that there's something discrete underneath space, just as there's something discrete underneath kind of the structure of fluids. In the case of fluids, it's molecules. In the case of space, it's what one can sort of call atoms of space. But they're really kind of abstract entities that are part of these giant hypergraphs that we think make up the structure of the microscopic structure of space. And the thing that one sees in, in these situations is in, in the case of fluids is, you know, you're trying to solve the equations well on a computer you have to make them discrete. And the discreteness you get on a computer is much coarser than the discreteness that you get when dealing with actual molecules. There are a lot more molecules in a very small region of, of of water or, or air or whatever than you can ever model on a current on a present day computer. And then you have to make all kinds of fancy arguments for why the very much coarser discreteness that you have on a computer is good enough. Actually, it's turning out that the kind of discreetness that we get in our models of physics and space time and so on look like they might actually be helpful for understanding the ways that you make discreteness for things like fluids as well. And also there are many other situations where you have the, this kind of continuum description. For example, when you have a solid and you're bending that solid, there are the equations of elasticity, which also are these same kinds of continuum differential equations. And again, it looks like the ideas from our physics project might give one a better way to discretize those equations. Let's see. Well, there's some oh my sorts of questions here. So I've just been. All right, I'll talk about. I've just sort of more or less finished a project that I started 50 years ago to understand the second law of thermodynamics. So maybe I'll. I'm just in the process of posting this rather giant Collection of pieces. The last one should go out today about how the second law of thermodynamics works. So maybe I'll talk a little bit about that here. So, okay, what is thermodynamics? Thermodynamics is a theory of the dynamics of heat. How does heat work and how does heat flow from one place to another? Well, the first thing that there are sort of four laws of thermodynamics that people talk about a lot. They were all kind of developed in the mid-1800s. But the first law of thermodynamics is essentially the law of the conservation of energy. And people hadn't known what heat was. And people had had the idea of energy as being associated with mechanical motion of things, that there's a certain amount of energy we know about, for example, kinetic energy, something of a certain mass moves at a certain speed. In standard classical physics, there's just a formula that says the kinetic energy is a half MV squared, and that's the amount of energy associated with the motion of something. So there are different forms of energy. There are things like chemical energy or nuclear energy, or there's also thermal energy. And the thing that was sort of a surprise in a sense, in the mid-1800s was this idea that heat is also a form of energy. And that when you heat things up, that you put energy into something when you heat it up. And that energy in the form of heat is also sort of interchangeable with energy, that is mechanical energy. And so the first law of thermodynamics just says basically, total energy is conserved. Energy associated with mechanical motion, energy associated with heat. They can trade off against each other, but the total of that energy is conserved. There's a question, I think here about what energy actually is. Well, let's see at what level. To answer that question, the first thing to say is energy is a number. For every thing can be said to have a certain energy associated with it. And you can kind of say that that closed box that you might have, the total energy, just some number, is always conserved. If nothing flows into or out of the box, then the total energy inside that box is conserved. It's a single number that's conserved. It might not be a single number. It might be that energy was a collection of five numbers that were calculated in some complicated way, but it isn't. It's one number that is a single quantity associated with stuff inside a box. Little bit of a footnote. When you get up to cosmological scales and you're dealing with the large scale structure of the universe and so on. The whole idea of energy conservation doesn't really work quite the same way anymore because of things to do with the expansion of the universe and so on. But for small scale things, what you have in that closed, isolated box, the total energy is conserved. Now, that's to say that you can look at all these different things in that box and you can say, I'm going to assign this number and that number. Whenever, whatever happens in that box, that number is always going to be conserved. Now, if you say what really is energy, can you drill down and say what's making that thing that is called energy? I think we now know that as a result of our physics project. But you have to go many, many, many levels down from the kind of experience of energy that we normally have in just kinetic energy, of things moving around, or chemical energy, or energy in chemical reactions or whatever else. What energy, I think ultimately is, is the following thing. And I'm surprised there's such a simple interpretation, but I think there is. So in our models of physics, everything in the universe is made of this giant network. So all there is is these, these elements, these atoms of space, these eames, as we sometimes call them, that are just discrete things whose only attribute is that they are, they exist and they're different from other such things. So there's just this whole kind of dust of eames of atoms of space that are the sort of raw material of the universe. And the only thing we know about these atoms of space is how they're related to other atoms of space. Or we can say, well, let's make a network. Let's say which atom of space is friends with which other atom of space. And we make this network that indicates that sort of friend network of the atoms of space. And that's it for how the universe is built. And all of the features of the universe, whether it's the structure of space on a large scale, whether it's the presence of an electron, all these kinds of things, they're all just features of this network of connections between atoms of space. So it's a very, very clean kind of ultimate representation of what the universe is made of. It's just made of these atoms of space, and it's related by these connections between them. And so everything that is a thing we experience has to be a feature of that network. So it's, you know, the analogy one can use is, you know, in a fluid, there's all these molecules bouncing around, but there are features that we might notice, like that vortex that's moving through the fluid. That's how we think electrons and all these other features of, of our universe work. They're just features of this giant network of space. Okay, so in that picture, what is energy? I think it's actually really simple. Energy is essentially the amount of activity in the network. So at every moment the network is getting rewritten. It has certain rules that say if you see a little piece of network that looks like this, rewrite it into one that looks like that. And energy seems to just be the density of rewriting that that happens in the network. So for example, and we can kind of trace through the presence of energy causes curvature in space, which is the origin of gravity and all sorts of other things like that. But in the end, what is energy? Energy is the density of activity, of rewrites in the network that represents space. But that's at a very low level. Maybe, you know, some trillion, trillion, trillion, trillion, trillionth of a meter scale size relative to the normal things we see. Okay, so anyway, the first law of thermodynamics just says energy is conserved energy associated with large scale mechanical motions, energy associated with heat. Now, a thing that only became clear kind of in the late 1800s was what heat is. People sort of thought that heat was like a fluid that would just sort of seep into materials and things. What people eventually realized is that the heat as we normally experience it, is associated with microscopic motion of molecules. So there's, you know, when we say when we're making a thing hot, what it means that it's hot is that the individual molecules are sort of randomly running around faster and faster. And the, the speed that they're running around, the average speed they're running around is what we call the temperature of the object is related by some constant to what we call the temperature of the object. So the, that, so the, the first law of thermodynamics is just saying all that random running around of those molecules versus the large scale motion of all the molecules coherently moving in a certain direction. All those kinds of things are just interchangeable with each other. They all just add up and make conserve total energy. Okay, what's the second law about? The second law says there's this irreversible process whereby large scale mechanical motion turns into heat, but it doesn't go in reverse. So roughly the, you know, if you do the sort of. The very early experiment, late 1700s, was boring out a, a cannon. People don't do this so much anymore. You know, from out of iron, I suppose you're boring it so that you're making this hole for the cannon. And the point is that the mechanical motion that you're doing, there's then friction. And the result of that mechanical motion is it gets turned into heat. And that means that the large scale mechanical motion of this boring the cannon out ends up being something which contributes its energy to its energy. The total energy is conserved, but the energy associated with the, the, you know, boring of the cannon ends up being little microscopic energy associated with the. These individual molecules running around and making this thing we call heat. A very famous paper by a guy called Clausius who was one of the originators of thermodynamics. It's called, it's in German, but it's, it's, its title is the kind of motion we call heat. And he was sort of making the point that heat is a form of motion. It's just this microscopic motion of molecules. And the real point is it's very disorganized motion. You know, you take this giant thing that's boring out the canon, the canon, the thing for the canon, and it's all, as you, as you turn that boring thing, every atom in that thing you're turning is moving together. Whereas when it's turned into heat, all the atoms in the thing that got hot are just running around randomly. Okay. So the second order of thermodynamics basically says that there is a sort of irreversible tendency to go from this kind of organized motion to disorganized motion to heat. And that you can go from organized motion to disorganized. That's the natural way to go. You can't go from disorganized motion and suddenly spontaneously produced organized motion. And so a very common statement of the second law of thermodynamics is the impossibility of perpetual motion. And so you could imagine that you take all this heat, all this random motion there, and suddenly that heat, you would find some way to make that heat all sort of line up all of that energy that's associated with heat, line it all up and move your lever from here to there. But the statement of the second law is that doesn't happen. A perpetual motion machine, so called, of the second kind because it's associated with kind of lining up heat. A perpetual motion machine of the first kind is one that just sort of gets energy from nowhere, but this is getting energy from heat to make systematic motion. So one statement of the second law is perpetual motion machine of the second kind is impossible. Another statement of the second law is Heat doesn't spontaneously flow from a colder body to a hotter one. So in other words, you've got all these molecules running around, and in a colder body there's less running around going on. And it isn't the case that all that running around is sort of coherent enough that it can organize itself to just go and put itself into this hotter body. That's another statement of the second law of thermodynamics. But the fundamental statement is that organized motion tends to degrade to disorganized motion. So that's the basic idea. And people had kind of started talking about that idea in the 1820s, and people sort of assumed, oh, well, we all know you can't have perpetual motion. We all know heat tends to dissipate. You all know motion tends to dissipate into, into heat. So that became kind of a thing, which people said, well, of course we know that. So we'll assume that when we build a bigger theory about the efficiency of steam engines or whatever else. On top of that, very quickly people started saying, oh, that's a fundamental law of nature. And then, and this is where it got really confusing, they said, and we're going to prove that law of nature from the features of mechanics. So they're going to say, we're going to prove that if you have a bunch of molecules, let's say they're hard spheres bouncing around, then it will inevitably be the case that large scale motion degrades into heat and you never get it back. Okay, so that was kind of the state of things. Now, James Clark Maxwell, for example, in the 1860s, I think, already made the point that, well, actually, is this really right? Because, yes, you say you can't get heat to sort of organize itself to make kind of coherent motion. But what if you had a little critter, it became called Maxwell's demon, that would be at a scale of molecules. And that was, you know, in a version of it that I think was due to Kelvin, he talked about that the little critter could have little tiny cricket bats. That's a British thing, I suppose. But, you know, it could be bashing these molecules around one by one, and it could say that little demon could be there on a molecular scale, just sort of saying, okay, this molecule use its energy in this direction. This molecule, I want it to collide this way, this way, this way, and pretty soon everything's lined up and that thing that you thought was random motion suddenly will be organized by the demon into be some kind of coherent motion. Why isn't that possible? That was one confusion. Another confusion was the following thing. So this chap Clausius that I mentioned, Rudolph Clausius, invented this concept of entropy. And so what his idea was, he invented it at a time when the idea of energy was still just forming. But his idea was what would be the attributes you would need to describe what kinds of processes happen in the world and what don't. So, for example, one kind of process that happened that can potentially happen is one that conserves energy. A process that doesn't conserve energy just can't happen. You can't create energy, destroy energy. Energy is a constant. And so what he was trying to invent was something which would describe other kinds of things associated with the second law as something where you could just say, well, there's this quantity, and you can characterize what can happen and what can't in terms of this quantity. So what he invented was the following quantity. So he said, let's take the total amount of heat, total energy in heat in something, and divide it by the temperature of the thing and that quantity. And particularly as we make changes to the thing we're dealing with, there'll be a change in the heat content dq, the change in the heat content divided by T, the temperature. He said, that's a thing. And he called it, in kind of imitation of the word energy, he called it entropy. And his meant kind of something where one was. He wanted those words to be kind of analogous. And he imagined that there will be laws that for energy, as he summarized, you know, the energy of the universe is constant. The entropy of the universe always increases, was what he said. And his entropy increases was kind of a way of giving a sort of summary of this phenomenon, of this kind of irreversible change of heat into change of energy into heat. And the way that DQ over T works is that will change if, if you end up with, As you put your energy into something which corresponds to heat and increases temperature, if you sort of do the math, you'll find out that that implies that that quantity always increases. Okay, so that was kind of one version of this idea of entropy just came from looking at heat and temperature and came from asking the question, what is sort of heat divided by temperature? And by the way, in the way we can think about that is they have all these molecules running around. And it turns out that the average energy of a molecule is pretty much by definition. Well, actually, the average kinetic energy of a molecule is some constant factor that's a little bit detailed that depends on monatomic diatomic molecules or kinds of other things about different kinds of motion that they can have. But roughly, if it's just one little spherical thing, the average kinetic energy is a half times kt, where T is the temperature in K is a thing called Boltzmann's constant. It's just a constant of nature, so to speak. It's just really just converting units between the units of temperature and the units of energy. And so that says that associated with any given molecule when sort of the thing has developed into just thermal equilibrium, when after, you know, you might start off from grinding the cannonball, you know, the cannon hole thing, and then it's grinding, it's very systematic motion of all those atoms, and then they start randomizing. When it's fully randomized, then there's this result that says that the average kinetic energy is a half kt. And so, essentially, by saying, let's look at the heat divided by the temperature, you're essentially just counting the number of molecules in some sense that are involved or that are participating in this kind of sort of, this. This arrangement of energy, this equipartition of energy across all those different molecules. So anyway, that's one definition of entropy. There's a completely different definition of entropy which people sort of say, oh, these things are equivalent. Well, it's been 150 years, and their equivalence is kind of complicated. But the other definition of entropy is something which at first seems quite different. It says, imagine that you have this system and you have all these molecules, and you can arrange the molecules in lots of different ways. Imagine that you just have some discrete system where you can have either. Let's say you can have a molecule, a particular position, or at a different position. It's all a discrete grid. So you're just picking. Where do the molecules go on this grid? Well, you can count up how many possible configurations there are. For example, let's say that there's just a line of positions, and at each position, you either can have a molecule or not. Then let's say you have two positions, and each one either has a molecule or doesn't. That means there are four possibilities, either none, none, molecule, nothing, nothing. Molecule or molecule, molecule. And if you have n positions on the line with that setup, you will have two to the n possible configurations. So generally, the number of configurations will increase exponentially with the size of the. Of the system. So the. The idea, this other definition of entropy says entropy is equal to the same constant K, this Boltzmann's constant thing, times the log of the number of Possible states of the system. So the log. So if you have two to the N, the log is what's the exponent? So it's n, and if it's log to the base 2, it's N. In that case, it's a kind of the log is asking kind of the what's the exponent question. So what it's saying is the entropy is proportional to the, is essentially telling you it's proportional to, in the case of the two to the N. It's saying there are n positions on this line. So that entropy will be, well, this K thing times N, because the number of positions is 2 to the number of possible configurations was 2 to the n. And that means that as log is N, which happens to also be the number of possible positions on this line. So then this other definition of entropy is basically always asking, you know, certain things at a macroscopic level about this system that you have. You know, all the gases in this corner of the box, you know that the overall, the overall pressure of the system is this, you know, some large scale effect, some large scale fact. So now you have to ask the question, how many possible microscopic configurations, molecules are consistent with that large scale fact. You know, if you know that number of configurations, then you take the logarithm of it, multiply by K, you get the entropy. That's this other definition of entropy. Entropy in that case is the log of the number of configurations consistent with whatever large scale constraints you have. Now the big question is, are these definitions of entropy equivalent? Well, people still argue about that. It's known in certain very simple cases that they are equivalent. Knowing it in more generality is really hard. I'm not sure how important that particular question is, because a lot of the time one is operating in situations where one's really only interested in very close to equilibrium, very close to all the molecules sort of equally sharing in the, in this energy thing and so on and a lot of chemistry and so on. One's using this Clausius definition of DQ over T for entropy, and it all works just fine. The other definition, the Boltzmann definition over the k log omega thing, counting the number of states thing, is a more sort of pure definition of entropy in many ways used for other purposes. And sort of the, as a practical matter, you can kind of say, well, they're equivalent for the cases we care about in chemistry and things like this. But the thing that's perhaps of more interest is the sort of the big statement of the second law of thermodynamics says entropy tends to increase. What does that mean? That means if you have certain macroscopic. In the Boltzmann definition, if you have certain macroscopic constraints on a system, it will tend to be the case that. You won't tend to preserve any sort of specialness in which configurations, microscopic configurations, you had. So, for example, let's say you have the constraint the gas is in this box, okay? But at the beginning, the gas is all in a corner of the box. And you could know that macroscopically the gas is all in the corner of the box. And you could work out the entropy associated with the gas being just in that corner, which is much less than the entropy associated with gas being in the whole box, because there are many fewer configurations of where the gas molecules can be if they're all in that corner. And so the idea of entropy increase is the idea that things tend to get more random in the sense that even though at the beginning you could say, oh, yeah, I can see all the gases in the corner of the box, after a while, the gas molecules are sort of randomly arranged in the box, and all you can say is, they're in the box. They're somewhere in the box. And so the sort of more general definition, the more general idea of the second law is that things tend to get more random. What starts orderly tends to become more random. And that's a very general idea. And it's an idea which in its original form and for a long time was an idea that was all tied up with temperature and energy and molecules and all this kind of thing. But that's not that. That's not something where the. That's the. The essence of the second law does not have to do with temperature and energy and things like that. The essence of the second law is this very much more sort of computationally oriented statement that things that were originally orderly tend to get more random. What does it mean to be more orderly, more random? What it means to say that it looks that's more random is that we can't notice a pattern in it. We can't say, oh, by the way, you don't have to say where all those molecules went. There's just a rule that says all the molecules have are lined up in exactly this way. We can state a very compressed version of a description of the system without having to give all of its details. And that's kind of an operational definition of what it means to be random is that you can't do that. You can't compress things. You can't say, oh, I know what's going to come next? I don't have to explicitly write it down. I just have some accurate model for what's coming next. I don't have to explicitly work it out. And so this idea is that the sort of. The general notion of the second law is that what starts as something that can be very orderly, where you can describe, you can compress it. Let's say you've got the particles in the box, and they're all lined up in a nice way in the center of the box. And it's easy to say there's 10,000 particles and they're all lined up one after the other in the center of the box. But then after a while, they're sort of all arranged different places around the box, and it's very hard to say where they all are. There doesn't seem to be any better way to say it than just say, this molecule is here, this one's here, this one's here, and so on. There's no compressed description. So the more general statement of the second law is this tendency to go from orderly things to disordered things. And that's kind of a. That's a vast generalization of the second law has many, many implications for many different things, whether it's for fundamental physics, whether it's for lots of things in lots of areas, even in mathematics, this idea of that which starts orderly will tend to become more disordered and more random. Tend to whatever you could initially sort of put in your conceptual box of I've got the simple description will. Will escape it and will no longer seem simple like that. Now, one of the kind of gotchas of traditional thermodynamics is the laws of mechanics are reversible. That means that if you've got a movie, and you have this movie and it's got two balls colliding and then they go out a different way. They'll go in, they'll come out. You could reverse that movie. And it's the same perfectly valid movie. You couldn't really say, well, which one is the one that really happened? The one where the balls came in this way and went out that way, or the one where the balls came in that way and went out that way? It's that the laws of mechanics at a microscopic level are exactly reversible. Small footnote, there's this thing called time reversal violation, which is something that happens in particle physics that is not relevant to what I'm talking about. Now, it's a slightly different setup for things. This idea that a process that there's a unique forward. Whatever. When there's a way to figure out what will happen from the configuration you have now. There's a unique way to move forward from what you have in the future. There's a unique way to move back. That's not always the case. Let's imagine that you had something where you had all these ball bearings or something. You were throwing them in a pit, and they had a funnel, and the funnel made them all come to a single point. Well, then all those different configurations where the ball bearings went in, they'd all wind up going to that point in the middle. And so that's an irreversible process because they all. At the end, they're all stopped in that place in the middle. And you couldn't go backwards. There's a unique way to go backwards and say, oh, I know, because they all end up in this configuration. I know that they were originally in this configuration here. But the laws of mechanics seem to be perfectly reversible in the sense that you can always. There's always a unique way to go back, just as there's a unique way to go forwards. So that's a little odd because it means we're claiming entropy tends to increase with time. But how can that be? Because in the end, what's happening is just a bunch of molecules bouncing around, and their mechanics is perfectly reversible. So how can it be the case that entropy is irreversibly increasing even though at a microscopic level, the molecules are obeying this reversibility principle for mechanics, it's kind of the famous case is, you know, can you. You can sort of scramble an egg, you can smash an object, but things don't spontaneously unscramble, spontaneously unsmash. You readily see the thing sort of going from order to disorder, but not from disorder to order. And that's. That's the. And so that's a little weird. There's this kind of conflict between the microscopic reversibility and the macroscopic irreversibility. But that's what the second law is talking about. It says on a macroscopic scale, things will tend to be irreversible, even though the microscopic dynamics is reversible. Okay, so how does this work? Well, what happens is this. You have this configuration which is very orderly, and as you run it forwards in time, it will become disorderly. If you run it backwards in time will also become disorderly. So you might have this configuration where everything is perfectly ordered, the rules and mechanics are reversible. So you run it Forwards in time, it becomes disordered. You run it backwards in time. The rules and mechanics are reversible. So the same thing's got to happen, but becomes disordered. So now what you're reduced to is the question of why is it the case that the actual things we look at are things that seem ordered to us and they become more disordered rather than. There was this thing that looked to us completely disordered, but it was arranged in just the right way to spontaneously assemble itself to get this orderly thing. If that happened. There's nothing wrong with, according to the laws of mechanics, with having something which was set up so that, yes, it was exactly the right thing with all its disorder exactly right, arranged exactly the right way so that 10 seconds from now, it will assemble itself spontaneously to this very orderly state, and we'll say, gosh, that's really weird. You know, I didn't expect that to happen. Okay, so why is it the case that we don't have these kinds of configurations that were disordered at the beginning and which spontaneously order themselves? Well, because the real story of what's going on is that, and this is what I've been figuring out and writing about recently, is that this phenomenon of going from order to disorder and so on, it's all associated with computational irreducibility. It's all associated with the fact that even though, you know the microscopic rules by which things operate, you can't figure out what's going to happen from those microscopic rules much more efficiently than basically just by running those rules and seeing what happens. And this fact that you can't figure out what's going to happen more efficiently is why you can't just say, oh, it's going to be this very ordered thing that comes out. Because if you could say it's going to be this very ordered thing that comes out, then you're going to be able to just say that and not have to go through this whole irreversible process. So in a sense, if you could say there's going to be order coming out rather than disorder coming out, you could break computational irreducibility. And the fact is that what happens is the things that we look at, the things that we humans as observers who just observe sort of aggregate effects of molecules, we're not tracing individual molecules. We're not able to see the whole dance of the molecules and so on at our scale and with our computation abilities, we are just looking at. We're not able to kind of look into that computational irreducible process. So it looks to us, we just have to say, oh, we don't know what happened. It just looks random to us. In principle, we could drill all the way into that computational irreducible process. We could say, oh, I know what happened. You know, this computation happened, that computation happened, that computation happens. But we as a computationally bounded observers just don't get to do that. So we just have to throw up our hands and say, looks random to us. And that's what's happening in the second row of thermodynamics is that is this contrast between us as computationally bounded observers and the microscopic sort of computational irreducibility that's being generated in the system. The cause of the second law is that we are computationally bounded observers attempting to observe computationally irreducible process processes. And the reason that we don't get to prepare that sort of miracle, you know, disordered initial condition that's going to become spontaneously ordered is for the same reason, to be able to prepare it, we have to do all this computation to figure out exactly how it's going to be set up. And that computation is beyond what we as computationally bounded observers can muster. And so, again, the cause of the second law is this conflict between us as computationally bounded observers and the computational irreducibility of mechanics and lots of other kinds of things. It's not that mechanics is irreversible, is fundamentally not computable. It's just that it takes a lot of computational effort to predict what will happen, an irreducible amount of computational effort to predict what will happen. And because we can't provide that that much computation, we just have to say, it looks random to us. And that's why the second law works when it does work. Second law doesn't always work. You know, people had come to the conclusion that it was very weird history because people had sort of said, we empirically think the second law is valid. And then suddenly they. And then we said. And then people tried for a long time to derive the second law from mechanics, and they failed. They simply could not derive it from mechanics. I think we now know. I sort of first figured this out more or less in the 1980s, but just cleaned it up now that it is not derivable purely from mechanics. You need one other fact. You need the fact that we are observers like us, we are observers with certain attributes of computational boundedness and things like this. Once you have that, then you can derive the second law for observers like us. If you imagine that we were Maxwell's demons, or we imagine we're some future observers with much better sensing devices and so on. The second law will not be true anymore. The second law is only true for observers like us, who have this kind of computational boundedness and so on. Hello again. Well, technical glitch there. We were just at the dramatic punchline here of talking about the second law of thermodynamics and why it's true and this kind of conflict between computational irreducibility and our computational boundedness as observers. And I was kind of saying that that's in the end what leads to the second law. It's that microscopic processes are computationally irreducible, and our only ability to kind of see what's going on is through computational, through this computationally bounded lens of us as observers. And as I say, in the future we can imagine that there will be more precise molecular ways to measure detailed molecular motions and this and that and the other. And we'll no longer be as excited about the second law of thermodynamics because we'll be saying, no, we don't just want to know the pressure and temperature of a gas. We can know this huge petabyte of data about the gas every microsecond. And that in there we have much more detail. And it isn't, we're not that interested in this just pure, you know, pressure and temperature type type information. Now, you know, having said this, the second law people came to the conclusion that the second law is absolutely always true. They had a few exceptions to do with living systems that were originally, for reasons of kind of various kind of religious belief structures that didn't want to have the idea that that sort of life had this kind of pure decaying process associated with it and so on. But that then became kind of a thing that sort of enmeshed itself with the kind of science of biology and so on. And people were still like, well, there are probably some exceptions to the second law. Maybe gravity was an exception to the second law. Maybe living systems are an exception to the second law. But people had this idea that the second law was a mathematically proved thing in general, which it never was. It was never provable because as we now see, it has an assumption about observers like us that's an absolutely critical assumption that you kind of can't get around. And although you can kind of say, well, we can think about this as just sort of macroscopic, this, on the other, you actually have to have that sort of a piece of your axiomatic framework is to say its observers like us. So in Any case, the thing that that one realizes in the end is, well, the second law is, is often true, but there are cases where it might not be true. And there are kind of situations in which the microscopic dynamics is not sort of uniformly computationally irreducible. It has kind of slices of reducibility all over it. And one particular case of that is what I been starting to call the mechanoidal phase of matter. You know, when you have a solid, liquid, liquid or a gas, you just got molecules bouncing randomly around and there's no real order, it seems in a gas, typically to how those molecules bounce around. But you can have a thing where there is a lot of detail, a lot of complexity in how sort of the system is arranged, but yet there is still order to it. And it kind of looks like a bunch of machines operating it's kind of very machine like phase. And this is what I call the mechanoidal phase. And one can find it in a lot of computational systems and the things I call class four systems and so on. And I have the suspicion that a lot of biology is really associated with the mechanoidal phase. And that's really when we say what is biological tissue? Is it solid, is it liquid, is it gas? Well, it's kind of a mixture of those things, but really it's a bunch of molecules being transported around in these very machine like ways. And it's really like this mechanoidal phase. So anyway, that's a little bit of the story of the second law. I would say one thing about the second law is now that one's understood that it is this conflict between computational irreducibility, computational, computational boundedness of observers. It has kind of a much broader, that has much broader implications. And in fact, the phenomenon that leads to the second law turns out to be the very same phenomenon that leads to essentially the structure of space time and the way quantum mechanics works. And that's a very remarkable thing as far as I'm concerned. You know, in the 20th century, the big sort of excitements of physics, there were three big theories of physics. The theory of gravity, general relativity, theory of quantum mechanics and statistical mechanics. The theory of things like the second law of thermodynamics, and the really remarkable fact that seems to have emerged from our physics project and this work on the second law is all three of those theories, all three big theories of 20th century physics are the same theory. They all come from this sort of interaction between irreduced computational irreducibility underneath and computational boundedness. Of observers. I think that's pretty neat that these three theories that seem so different in their applications and in their structure actually are all the same theory at some level. And well, one can apply then ideas from thermodynamics that have come from sort of studying things like steam engines and so on. One can apply them in cases in space time, probably in cases in quantum mechanics as well. That's a very fertile area, more to be explored in that area. But in any case, that's the basic story there and I need to go and actually see if we can get out this whole description of this ideas about thermodynamics. And yes, stay tuned. It should get posted real soon. Hopefully, if there's nothing more wrong, you should be able to post it and you can read, hopefully a more coherent description of what I've been talking about, about second law of thermodynamics that I've been writing over the last last few months. So I should probably stop here and go off and try and do that as other as well as other things. So thanks for joining me, sorry for these various glitches and so on, and look forward to chatting with you again another time. All right, bye for now.