Evaluating models without test data

All natural data has a power law structure, a fractal structure to it. The way neural networks learn is they learn the multifractal nature of the data, and that's why they work so well on things like text and images and why they don't work great on tabular data sets. So there are correlations in the data. Data is correlated. You're trying to learn the correlations and frequently you're trying to learn very subtle correlations you couldn't find in some other way using some simple clustering algorithm or an SVM or something like that. So what we're doing is we're measuring the fractal nature of the data. And every layer of a neural network gives you some measure of the fractal properties and that level of granularity. And so alpha is like a measure of the fractal dimension. And what we know is that it measures the amount of correlation in that layer.

Welcome to Practical AI, a weekly podcast podcast making artificial intelligence practical, productive and accessible to everyone. Subscribe now if you haven't already. Head to PracticalAI FM for all the ways. Special thanks to our partners at Fastly for delivering our shows super fast to wherever you listen. Check them out@fastly.com and to our friends at Fly IO. We deploy our app servers close to our users and you can too learn more@fly IO.

Welcome to another episode of Practical AI. This is Daniel Whitenack. I'm a data scientist with SIL International and I'm joined as always by my co host Chris Benson, who is a tech strategist with Lockheed Martin. How you doing, Chris?

I'm doing very well, Daniel. How's it going today?

It's going well. You know, I've been training quite a few models recently, NLP models for question answering and other things. One thing that always comes up in that is how long do I train this thing? Am I over training it? Am I under training it? How do I test it appropriately? Am I testing it right? What else should I be doing? All of these thoughts are running through my mind and I'm pretty excited because today we have joining us Charles Martin, who is an AI and data science consultant with calculation consulting. And this is basically one of the things that, that he thinks about a lot and builds tooling for. So welcome Charles.

Hey, great. Thanks for having me, guys.

Yeah, well, the main, I think the thing that I saw of your work that really interested me was this Weight Watcher tool, which is an open source diagnostic tool for analyzing neural networks without the need for access to training or even test data, which is super interesting and I want to get into all the details about that. But maybe just describe for us kind of pre Weight Watcher what led up to Weight Watcher, what were the sort of motivations that were going through your mind and maybe the things that you were encountering in your own work that led you to think about this problem.

Sure, sure. So I do consulting in AI and I had some clients working with me to do text generation. So this is years before GPT and all these amazing diffusion models that existed and we were training LSTMs to generate text or things like weight loss, articles and reviews on Amazon, stuff like that. I realized that as I use these models I can't really evaluate them because if you're training a classifier like an old SVM or XGBoost, you can look at the training accuracy. But if you're trying to design a model to generate text or some other natural language processing problem like say designing embedding vectors for search relevance, it's really hard to evaluate whether your model is converging or not. Now I had studied statistical physics of neural networks when I was a postdoc in theoretical physics. I knew that there are techniques from physics that make it possible to analyze the performance of these models and to estimate how well they're performing. What I realized is that nobody in the machine learning or AI community really knows about this stuff because it's from the early 90s, early to late 90s, where a lot of this research was done. And you know, the people doing AI and machine learning are not theoretical physicists, you know, they're computer scientists. They don't know about the works. I said, you know, I.

Except for you and Daniel there.

Yes. Well, you know, it's a very broad field and there's so many people doing AI now that it's really fun because there's so many different backgrounds. And I was at a conference maybe 10 years ago, maybe nine years ago, and I met an old friend of mine, Michael Mahoney, who's a professor at UC Berkeley. I was at ML Conf. It was run by the guys at that time who were doing, oh, what was the name of the company? They had a recommender system, a recommender product. They were Turi AI. They were eventually acquired by Apple. And I was talking to Michael, I said, you know, there's a lot of theory around deep learning that is very similar to what we see in protein folding. And my advisor was actually the him and his student John Jumper developed the first version of Alphafold So what happened was Google acquired AlphaFold. They hired John Jumper, who was the student from Chicago, and basically souped up his thesis. And that's where AlphaFold comes from, this amazing technology from DeepMind that can predict protein folding. So there was a lot of theoretical work I had done as a postdoc and I was talking to my advisor about some of the stuff they were doing in protein folding way back before AlphaFold was released. I thought, you know, I think I'd like to try my shot at doing research again and see if I can develop some theory that would allow me to understand why deep learning works. And that project, it's been about seven years now of research and that's led to the Weight Watcher tool.

Cool. So it's probably very typical for people to think about, oh, I'm going to evaluate my model, I have a test set. But could you describe a little bit about two things? One is why, from your perspective, at least in certain situations, a test set doesn't give you the indication of behavior or performance of a model that you're wanting. And then how that connected to these things from the physics world.

Right. So let's say we're training a model to generate text. There's no test set. Right. You have to read the text and ask, okay, does it look human or not? And that's sort of where the first problem came, is that there are many problems in generating, when you're generating things. Another would be, let's say you're doing search relevance. I'm trying to predict what somebody wants to click on. I have clients like Walmart, for example. We build these systems for them. It's very expensive to run an AB test, so you can test things in the lab and you can like make a model, like an SVM model to predict what people will click on. But you don't really know how it's going to perform until you put in production. And there are all sorts of biases that exist in the data because there's like presentation bias. People tend to click on things that are in the first element and that screws the model up. So there are many cases and other good examples in quantitative finance when you're trying to predict the stock market and you have models where you would like to train some neural network to learn something about how the news predicts the market. But if you train it directly on the market, you'll overfit it always. And so you have to have some way of evaluating whether your models are converging properly or not without just Looking at the out of sample test sample, you don't. A lot of data is out of sample or you can't really evaluate it without human judgments, or it's very expensive.

Would you kind of infer that we're probably seeing a lot of practitioners running into these kinds of issues over time. And in a lot of cases, if you look over the last few years, as everyone's kind of ramped up in the space and been learning how to do different types of deep learning training, do you think that in terms of those accuracy issues, that a lot of practitioners are kind of missing it altogether, or do you think they know that it's there and they just don't know how to solve it? Can you lay of the land with it?

Well, let me give you an example. Let me give you an example. There's a recent paper that came out of Google DeepMind on the scaling properties of very large language models. And it showed that what we thought we knew about large language models from two years ago from OpenAI, a paper that they wrote, was totally wrong. They misunderstood how the scaling properties work. And the question is things like when you have a model and you're trying to train it, should you be trying to optimize the hyperparameters or should you be adding more data? You can think of it like in a sort of very crude sense, you're trying to train these models. And essentially what was happening at OpenAI is you're training these large language models and they didn't realize that they should be adapting the learning rate to the data set size. And when you change that, when you adapt the learning rate to the data set size, you get very, very different results than if you don't. And it looks like, and we know that a lot of these large language models, like Bert, for example, are just not properly converged. There are a large number of layers that are simply under trained. And I think that basically there's the theory that people are using. There's no way to look at a model and ask, how close are you to conversions. If you think about something like an svm, let's go back, I'm an old guy, let's go back 10, 15 years ago. We run SVMs. There's something called the duality gap. You can look at the duality gap in an SVM and you can ask, how close are you to the bottom of the. You have. It's a convex optimization problem. And you can tell how close is your solver to. To actually being at the optimal solution. You can tell that that's theoretically known. So it's somewhat puzzling that now you have sort of deep learning. People understand that deep learning is sort of like a convex optimization or rugged convex optimization because they know you don't have, you don't have local minima. And there's an issue that there are lots of saddle points but no local minima. But yet there's no theory which tells you whether you're converged or not. And so it's like, what's going on? So people are trying to solve this and I think this is where you know, you start training a model and you don't know, have you, have you trained it enough? Do you need to train it more? Let me give you a really practical example which we have with we have a user who's using Weight Watcher to train semi supervised models to determine whether the land you own is qualifies for carbon credits. Right? So they're trying to use AI to help with climate change. And one of the biggest problems they have is how much data should we add to the model? We have a model, we have data acquiring data. Acquiring good, high quality labeled data is very, very expensive. You could easily spend millions of dollars on a data set, maybe 10 million. I know guys, self driving car companies will spend easily $10 million on a data set. So it would be nice to know given the model that you have, do you need to, if you add more data to it, will it help? So we can answer that question with.

Weight Watcher if you can kind of talk a little bit about some of the underlying. Because you're pointing out that there's a lot of opportunity not be optimal in their approaches and kind of miss some of that. So it almost raises a kind of a bigger issue that we may have as a community if that's the case. In terms of like how do we solve some of those problems in the large. Aside from the specific tools, what are you thinking in terms of how should people approach these problems different?

Well, look, I think the first thing you have to ask is I'm beginning to train a model. Is my model big enough? Is it small enough? Do I really want to spend millions of dollars doing brute force hyper parameter tuning? You know, should I be tuning? Like here's a basic question comes up with every client I have a model, forget about deep learning svm, should I add more data or should I have more features? Let's say I've actually boost, should have more data, add more features or do more hyperparameter tuning. It's all expensive what direction do you go? And you know there is a, it's a difficult, these are difficult problems that if you add more data, is the data the right quality? Is the data mislabeled? Are there duplicates in the data? Is the data too similar to the data you've already added? Is it too different from the data you already added? Basic questions, we just don't have very, very basic, broad level questions that we have almost no answers to. Everything is brute force. If you want to train a neural network, you go out and you get weights and biases or you go to Google Cloud and you just spend a fortune on hyperparameter tuning. Do you really have to do that or isn't there something better you could do? Here's another example. When we started this project there were maybe 50 open source pre trained models. Open source models, right? VGG, the VGG series, Resnet, things like that. You go to hugging face. Now there are over 50,000. Which one do you pick? Should you pick BERT or something else? Everyone uses bert. BERT is highly under optimized. If you Compare BERT to XLNET, Excelnet is much, much better. Not only do the to the academic papers show that XLNET performs better on at least 20 different metrics, you can use Weight Watcher, I have a blog post. You can see that it's just night and day between xlnet and bert, but is it worth the money to spend to try to optimize xlnet? Why does everybody focus on bert? Because it is a cute name and it's made by Google. I mean, you know, it's really hard to know which model to pick. And it's hard, these models are very hard to improve if you're trying. So there are a lot of just broad open questions like this, which model do I pick? How much data should I add? How do I evaluate the quality of my data? Do I really need to do brute force searching on everything? If I put something into production, how do I know if the model doesn't, it breaks? I don't know if you guys worked it in production environments. I work in environments where things break every six weeks. It's you know, Thanksgiving comes, model's broken, Christmas morning, models broken. How do you monitor these things? So I think, I think we're in our, I think machine learning and certainly AI is in the infancy of engineering. Certainly compared to where we are in software engineering. We're 20 years behind where software engineering is.

So Charles, I definitely. It's interesting kind of these I guess, scenarios that you bring up because it's Definitely something that happens. I mean, sometimes in an actual real world setting, like with my team, it's like we have what data we have, what model is appropriate that fits that level of data, or maybe you have a whole bunch of data and the question is, do I need all of it for this model that I've already decided on or all of these sorts of things? And then you get to the training questions that you've brought up. I'm wondering if you could just give us a high level overview of. Because I think the main thing that if I'm understanding right, the main kind of tool that's come out of this train of research that you've been working on is the Weight Watcher tool. Could you just give us a kind of broad overview of what the tool actually functionally does and where it fits into a researcher or a developer or data scientist workflow?

Sure. So the tool can be used both when you're trying to train models, AI models, or you're trying to monitor them in production. From a training perspective, the tool gives you insights into whether your model has converged. It does so at a layer by layer basis. I'm not aware of any other technology that allows you to look at the layers of a neural network and ask has one layer converged and has another layer not converged? There are cues you can look at. You can look at something called the alpha metric, which is the amount of correlation in the model. Usually if you have a computer vision model, your alpha should be down around 2. In natural language processing transformer models, alpha should be between 3 and 4. If your alphas are larger than that, chances are the layer is not properly trained. You can then visualize each layer and you can look at the layer, its correlation structure, and that correlation structure should be fairly smooth. It should be linear and smooth. On a log log plot. If it's choppy or has sort of a strange shape to it, something's wrong. If your layers have lots of rank collapse, so lots of zero eigenvalues, something's wrong. We've identified something called a correlation trap, which is in deep learning language would be, you have to, you didn't clip your weight matrices, you didn't regularize the that layer correctly. So you can use the tool during the training of a neural network to monitor the training. You can find layers that are basically broken, they're not trained correctly. Think of it like you're building a, you know, you're building a house and there are, you know, cracks in the bricks. You know, you put a brick in the crack, it's cracked, you need to replace it. You can adjust regularization up and down on the layer. You can adjust learning rate up and down on the layer. You might find that when you're training a model, some layers are beginning to. They're well trained and they begin to overfit. You might want to freeze them so you can freeze. So as people talk about early stopping, I talk about early freezing. So you might freeze some of the early layers and let the later layers converge. So Weight Watcher allows you to do all of this by as a. You know, it's very much a. You have to do it by hand. You have to go in and visualize it and see what's going on. But it allows you to inspect your models to determine whether they're trained correctly. It also allows you to look at models in production. So if you're deploying AI models in production and maybe you're retraining your models regularly, it would allow you to gives a warning flag, a model alert system that would tell you, hey, you broke this layer. We have an example in our paper. We have a paper in Nature where we show that in one of the intel systems, they applied a data compression algorithm to compress the model to go on the hardware and they screwed up one of the layers. And you can see this with Weight Watcher, it will flag it for you. So while you're. So as you're deploying models in production, it can monitor them for you. And remember, it doesn't require any data. So it's a very light, very light touch, very simple integration to integrate into your al Ops monitoring pipelines. I think of it like an AI uptime tool. It gives you an early warning. So this is how you use the tool. You can use it during training to make sure your models are converging well or they haven't converged properly, you go back and fix them. Or you can use them after training in production to monitor for problems.

I was trying to think of analogies in my head while you were talking and you gave a good one in terms of the house and the cracks. One of the things I was thinking about, you mentioned Bert earlier, which no doubt in the sort of time when Bert came out, it was quite an advancement and many people have built amazing things on Bert. But I was thinking about that and where we've come from there and also thinking about, my wife owns a manufacturing business and they've got this principle in manufacturing about find the current biggest bottleneck in your process. Address that as soon as you address that, there's going to be a next biggest bottleneck that you address next. Right. And you kind of just keep working your way through. So I'm wondering. Bert, obviously, is a good advance, but then you can analyze that model and see maybe where the next biggest offending area is and kind of address that. And I was also thinking about the tool that you were mentioning, all the things you could do with it. You could probably analyze your model in development for years, fixing all sorts of things and doing all sorts of things. But at some point you have to ship your model. So maybe there's this process of. And I'm wondering your thoughts on this of like, using the tool to find sort of these, like, worst offending parts of your model, addressing those, and maybe like, at a certain point you get to a point of diminishing returns or something like that.

Right? Yeah. This is a coarse grain tool. It's not meant to go in and study epoch by epoch and try to fine tune exactly what's going on. That's exact. I'm really glad you brought this up, Daniel, because sometimes you work with the academics and they always want to use it as a regularizer. You want to optimize the loss. No, no, no. That's not the puts in engineering tool. It's an engineering tool. It's designed to go in and find out where the cracks are. So if you're. I don't know, if you guys in San Francisco, you know about the Millennium Tower.

Yeah.

So my little nephew, he's. He's all into construction and he's always talking about, they got to tear the Millennium Tower down, tear it down, junk it. Because it has like, they built this tower and it's like the Leaning Tower of Pisa. It's tilting. And if you go into the basement of the Millennium Tower and this is like, you know, like condos, like multimillion dollar condos. You know, I think probably like, you know, the Marissa Meyer may own a condo there. I mean, it's ridiculous. They built this thing and downstairs you look and there are cracks in the steel. It's like, guys, the thing's gonna, it's. It's gonna fall down. It's cracked. And it's like, this is what Weight Watcher does. You go into your models and ask, are there gross problems that should not be there? Right. This layer is overtrained. This layer suggests that the data is mislabeled. This layer has a correlation trap. This is what you're trying to do. And you know, frequently in engineering, you're under time constraint. So, you know, you got to get this thing out and into production. You want to make sure it's. It's not crazy. And it allows you to Weight watcher, allows you to detect problems that you could not detect in any other way. And that's the key. It allows you to find a major problem.

So one of the things I was wanting to ask you because you said something a moment ago and then kind of circling back to that, that I'm very curious about, to bring me and other people in our audience along that may not be as familiar with that. I often rely on Daniel's expertise on this, and I want to rely on yours on this. You mentioned when we're talking about kind of testing those layers, as you did, going back to the alpha, and you specified for ranges of 2 for the visual and the 3 to 4 for natural language models and stuff. I'm assuming that that's one of the mechanisms that you're using in the software. Can you talk a little bit about what are the other mechanisms that are there along with that and maybe how alpha is used? Like, what if somebody is not familiar with that concept? What is it about alpha that's identifying that so that they understand that a particular layer might be brittle in the sense of it's not fully converged? You know, how are you approaching that? Kind of bring us along to try to catch us up with you on, on how you're thinking about that on a.

Like, why does it work?

Yeah, why does it work? What is it about alpha and other things that you're using in the software that yield that level of insight that you're describing?

So what we know from where does deep learning work? Deep learning works on natural things, natural images, voice, text, things that are really part of the natural world. And the natural world exhibits a multifractal structure. If you look at a tree, I don't know if you remember like L systems or computer science or sort of Mandelbrot's work. Most natural systems have. Or just think, think about text, you know, zip flaws, you know, power law structure in text and documents. All natural data has a power law structure, a fractal structure to it. And when you. The way neural networks learn is they learn the multifractual, the multifractal nature of the data. And that's why they work so well on things like text and images and why they don't work great on tabular data sets. So what you're doing is there are correlations in the data. Data is correlated you're trying to learn the correlations and frequently you're trying to learn very subtle correlations you couldn't find in some other way using some simple clustering algorithm or an SVM or something like that. What we're doing is we're measuring the fractal nature of the data. And every layer of a neural network gives you some measure of the fractal properties in that level of granularity. Alpha is a measure of the fractal dimension. What we know is that it measures the amount of correlation in that layer. In other words, you're learning the data is obviously not random. It can't be random. You're learning, you're trying to learn patterns. So what we've discovered empirically and we, and there's some deep theoretical reasons for this, but qualitatively what's happening is you're learning the natural patterns in the data. And those patterns, you know, they have to be there. So if you're looking at text data and you start seeing alphas around six or seven or eight, the layer hasn't learned the correlations. It just didn't learn anything. And it's just sort of there or learned it. The correlations are so weak that it's not really contributing to anything. So you just, and we know that many of these models are just have these extra layers, they're way over parameterized and they're, you know, so that's what's happening. And if the correlations, if there are strange or spurious correlations, there are things that cause alpha to be small for spurious reasons. Like, you know, you didn't regularize your layer correctly. And so there's a giant weight matrix like you have, you didn't clip the weight matrix elements or the regularizer failed. So it can detect the difference between when there are problems with the optimizer and when there's actual natural structure in the data and it allows you to distinguish between these two and that's what it's doing.

Am I correct? Just for clarity's sake, in terms of when we say it's doing this without the test data or the training data, really you're doing these calculations and you're detecting these parameters, these metrics, based on the weight matrices. Right? Is that correct?

Yes, only on the weight matrices, you don't need to look at the data.

So in that case, is it like the tool itself in terms of how people would run it, because it's doing these matrix calculations, is it necessary? Could you speak to the computational? Am I going to Spend five hours waiting for Weight Watcher to analyze my model or is it going to happen in five seconds?

The current model right now, it depends on the size. It runs a singular value decomposition on each layer. So that's a high memory CPU level. It's a high memory CPU intensive task. It's not optimized for gpu, so you run a normal cpu. It does require some memory. Most layers aren't too large, so it could take anywhere from a couple minutes to an hour. If you're trying to run on GPT and you have a thousand layers, it's going to take some time. Right. If you just have a few layers in your model and you're training like a small model, it's very, very fast. You know, generally you would hope that it is faster than an epoch in training, but it's not GPU optimized. So one of the things we're working on is I'd like to, if I commercialize the product is to make a version that's very, very fast so I can like distribute all the calculations on the nodes and come back to you. So that's the kind of I get from. So this is an open source tool, but it runs a simple SVD calculation. So it's a little compute intensive. But again, sort of my, my theory on this is that if you're training small models, it's pretty fast. If you're training really, really big models, well, you're going to have the compute. Chances are you have the compute resources anyway and you're not renting a GPU for it. You don't need the GPU even though it can run. So that's, that's sort of the takeaway. Sam?

Well, Charles, I mean when I first saw the tool I was very interested in it and I did take time to go ahead and just pull it in one of my notebooks and look at one of my own models because I did want to get hands on with. It was a question answering model based on XLM Berta and I analyzed it with Weight Watcher. I did not do every single thing that you describe on your repo because I'm still dipping my toes. Right.

That's great. It ran. It actually ran.

Yeah. So it's a Pytorch based model. It ran. I didn't time it, so I don't know exactly how long. But I did find out. At least I found out according to Weight Watcher, 10 of my layers are under trained. That could be, yeah, I at least found that out. So could you speak A little bit about, like the tool itself. So you mentioned, like, how people can integrate it in their workflows. Could you mention a little bit more about the open source project and like, how people like, if I'm like, I did and I want to do this on one of my models, how would I go about doing it and how easy is it to get it running on a model?

Well, you know, this is just. It's a tool I've been writing in my spare time based on my research. There's no funding for any of this. I publish with UC Berkeley, but they're not funding any of this. They're just sort of like, I'm just there to just kind of help me out a bit. I've written it all myself. It's all open source. I had one of my staff guys helped me out early on PIP install Weight Watcher. The way it's written now, you probably need to have both TensorFlow and PyTorch installed in your environment. If you want, I can make a version that doesn't require both of those. No one's asked yet. One of the challenges I have with the tool is that I have 60,000 downloads. I have no idea who's using it. So if you're using the tool, let me know so I can help you. I don't know what you're doing with it and I'm not going to. You know, I don't want to end up in feature creep where I design features in the wild. You know, I need to know what you're doing. So if you tell me, I'll help you. We have a Slack channel. You can go on Slack and you can ask me and I'll help you. But basically it's PIP install Weight Watcher and you just give it a model. You say watcher equals weight, Watcher model equals my model and you say watcher analyze. That's it. And it will return a data frame with quality metrics. If you say watch or analyze plot equals true, it will generate a bunch of plots. It will generate the plot it's meant to be. I've been running it in a jupyter notebook. That's how I run it. In principle. You could run it in a production environment. Again, it's not even an Alpha One tool yet. It's still like 0.56, 0.57. So if you do that, reach out to me, we can make a version that's more stable if you need to run in a production environment. But I've mostly been using it in it Runs in the Jupyter notebook. You get a data frame, you analyze the data frame. You run a Google Colab notebook, you say plot equals true. It gives you a bunch of plots. If you add some other options, it'll give you more plots. And then you analyze the plots.

So let me ask you a question as kind of a follow up to what you and Daniel were just talking about. If you're looking at the workflow like, and so, you know, Daniel said there were like, what, 10 layers that had not converged, you know, to sufficiently. How does that change the workflow for someone who hasn't done what Daniel's done and gotten his hands on? Someone just listening talk a little bit about what they were doing before versus the workflow they're doing now. Now that they have the insights that Weight Watcher is bringing to it, what does that look like for the practitioner?

Well, here's the first thing. This is exactly what happened with one of Michael's postdocs and students. Go back and look at the regularization. Did you add enough dropout on your layer? Are the learning rates too large? Do you not have enough data? Is your model just too big? Are the early agers converging in the later? If the later layers are not, maybe you should freeze some of the earlier layers and give the later layers time to converge. Maybe you need to run it longer. You need to run SGD longer. Maybe, you know, maybe you need to adjust some of your hyper parameters because you're not getting tuned. You know, try to adjust your hyper parameters so alpha goes down, not that it goes up. Those are the kind of things you need to do during training.

Yeah. So if you were, maybe you could also mention the workflow. I find it very interesting what you were saying about like, the workflow of potentially using this, like within the training loops as well as you're training the model. So one thing you could do is definitely run your model like I did and then look at it afterwards and see, oh, gee, I need to do something about this or that. And then of course then probably is the harder part of the problem is connecting with like, okay, does that mean I do one of those things you just mentioned or another one of those things you just mentioned? But what about that workflow, like in the training loop? How might that work? I know that maybe some people have heard of certain things related to optimizing either not doing brute force hyperparameter tuning, but doing some sort of some automl type of stuff or something. People have thought about these things. So when you're pulling Weight Watcher into the training run. How would you think about that being used?

If you want to give Google Cloud a million dollars to do AutoML and then have them own your models for you and feed them back to you, knock yourself out. I don't want to do that. I don't want to be trapped. You know, that's what they, that's what the AutoML offering is. It's an offering to blow millions of dollars. Or if you want to get some tool like H2O and Auto Tune a model and then find out it doesn't scale and then you have to redo it. We've had clients with that problem.

Right. I think there's this wider field though, of sort of, I guess, meta learning and kind of learning on that. And I don't know if this would fit, like the Weight Watcher stuff would fit into that larger space of research, I guess.

Look, what are you trying to do? What does it mean to be optimal? If being optimal means that your alphas are close to two or three, then you should adjust your hyperparameters such that the alphas go down. That's what you do. Now, doing that analytically, typically doing what are called analytic derivatives, meaning you try to compute the gradient from that. That's somewhat difficult. It could be done because you have to compute the eigenvalue spectrum and then you have to fit it and then you have to figure out the derivative. And that's a very complex nonlinear calculation. It's very iterative. It could be done numerically or it could be done analytically with some work. It's a lot of work. I would love to have VC funding like hugging face to do that, but I don't. It's just me, me and you. So you just try to tune your parameters. The alpha goes up, go the other way. You know, if you turn your learning rate up and you find your alphas are going up, tune the learning rate the other way and hopefully they'll go down. Obviously, it's a complex optimization problem because you have, you know, you have 100 layers, you have 100 alphas. And so you're trying to tune different layers and you're trying to tune your, your layer learning rates and your amount of dropout and the amount of momentum. So in principle you could try to do that algorithmically in a way using like a Bayesian type approach where you try to get your alphas to go down on every layer. I mean, it is, in principle you could do that. It's a complex, you Know, complex optimization problem, but that's what I would recommend. And I think it's theoretically well grounded. I mean, the point is that you want to learn more correlations. Typically what I found is that it's a good tool for newbies because you get into a model, you start doing something, things are totally wrong and you can go in and fix some problems. Okay, now we fixed it. We found like, what did we not do? Like, I didn't put the proper regularization on these layers. Let me add regularization and try again and you can see that. Okay, that's much better. So from a newbie perspective, it's a very good tool because it helps you get started. Now it does work. Keep in mind the tool works at the end of training, not in the early stages of training. You've got to let the thing bake for a while. You know, you can't. It doesn't. Once it's about halfway through training, then you can start looking at things. It's got to have some, some correlations. But this is what it's for. Typically this is sort of, you know, trying to do large scale meta learning, which just meaning you'd have to integrate the tool into some sort of process that allows you to look at the alphas or look at more details in the layer. The shape of the spectral density, the number of spikes, the alphas, the volume of the spectral density, and figure out how to tune from that. I mean, this could even be used in a reinforcement learning situation where the reward, instead of the reward being something that the agent takes, the reward is, oh, I got smaller alpha. I have rewards on every layer and I sum the rewards in some average way to try to get the optimizer to work. Even in situations where I don't know what the reward is for a reinforcement learning situation, obviously that would be nice in areas like you're trying to trade in the markets, because you can't take actions that trade you. You can't trade on historical data and expect to learn from that. So this gives you a way of doing things in a supervised or semi supervised way that doesn't require peeking at the test data to optimize. And I hope that answers the question. That's sort of the idea. And there are lots of things people I think want to try. I think it's great if you try them.

Yeah, I mean, I definitely appreciate you being transparent about where the tool is and all that. And really the possibilities that might happen with the tool and kind of there's a lot of opportunities to explore Usage and further development.

Part of what I want to do with the tool is build an open source community. I can't do everything myself and there's lots of things to do. And if people want to get involved in a community, join the Slack channel. We can build things. Right? That's what open source is. And I think there are a lot of people may have ideas and will be able to contribute in ways that would just expand it. And I think that it's again right now to me, the way you train neural networks now it's like you build a bridge. You drive a car over the bridge. You see if the bridge falls down and you do it again and again and again. How many cars are you going to crash into the ocean until you get the bridge right? No, people don't build bridges like that. You build bridges by having engineering principles you understand. Here are the engineering principles that go in and this is the load it can take and this is the wind shear. And you try to build bridges that actually stay up. And right now I think deep learning is so brute force. It's like you just spend as much money as you can, do as much brute force as you can, and if it doesn't work, you try it again. And there's no principles behind what you're doing. And we're trying to add some sense and principles that are based in deep theory, like they're empirical rules of thumb, but there's also deep theoretical reasons why they work just, just like in any other field of optimization.

I'm curious, I'm kind of going back to the engineering and the kind of talking about as this matures, much entrailing the software engineering world. But one of the, one of the decisions that we all make as we're, as engineers that we're doing is kind of like as we're creating open source community and we're trying to provide the value for that community that you're talking about. Do you see the future as being community specifically built around Weight Watcher or is there an opportunity potentially to add the value that Weight Watcher is bringing and those insights that you described and roll them into some of the other existing communities? Do you have, do you have any opinions or thoughts about how you integrate this in for the value of the larger community?

Well, I think I'd like to have is a community of people who are training models and getting them to interact with each other. A lot of the people, like I said, it's hard to get feedback. People are doing things in industry and because they are constrained by NDAs. They can't really talk about what they're doing. And I think it gives people an opportunity to really get into the space and learn how training of neural networks works without being constrained by your employer or your contract. So you can really do stuff. A lot of what this is, I think there are other communities doing things like people building hyper parameter optimization tools or people building reinforcement learning tools. We'd be happy to integrate the tool in. The challenge is always you want to make a tool that is self contained. If people fork the tool and begin changing it, it ends up. I don't know if you guys know the story of Emacs. I was at Champaign Urbana when this happened. They wanted to port Emacs to basically X Windows and Stallman didn't want to do it and they forked it. You have Xemax, you have Emacs. It killed it. Forking Emacs killed it because you have the Xemax crowd. And these guys went off and started Netscape and probably all retired now or they're sitting at the top of hang out at the roof of Google Vesting or Netscape. But this is the problem. You want to make sure you have an open force community. I want people to contribute and feel they can do things. If we fork it and it goes into other communities, it kills it because now those contributions don't come back. You end up in these sort of weird battles and there's no value in that. What we want to do is help people help people and if it's necessary at this point, commercialize the tool and turn into something which we can support, like Hugging Face. I mean Hugging Face is a lot of open source, but any sophisticated technology needs maintenance. You buy a copier machine, it's not open source because it needs maintenance. So even a tool like Weight Watcher needs maintenance. So I would love to be able to work with people who would like to put in the production as an open and develop it. And then at some point we realize, look, we really need to put a service contract around this so that we can maintain it and solve some of the harder problems for you. Be happy to do that. And I think that that's really what we're trying to do because this is, you know, there's also a lot of opportunity for scientific research. Weight Watcher has been a lot of. It's come from doing research, statistical mechanics and learning theory. We have papers in jmlr, Nature, icml, kdd. There's a lot of opportunity for students to. We have one student who is at a bank who just did his master's thesis on Weight Watcher. There's a lot of that kind of opportunity as well. And I think there's a lot of.

Room for improvement as we get to the end here. I was wondering, just quickly as we close out, I know you've spent a lot of really valuable time investing in the areas maybe that people aren't focusing on in the AI community in terms of the training side of things and ways to help them in those gaps. As you look forward to the future of where the AI community is going, what encourages you about sort of the direction of things or what excites you about what's exciting for you in the community right now?

For me, I'm a physicist at heart. I did theoretical chemistry, I did theoretical physics. In some sense, I'm kind of the runt of the litter. Like my. One of my classmates colleagues went off and started Alphafold, which solved the 50 year grand challenge. I have another who has started a company who's going to label all the world's translational medical data. So I'm used to that. For me, this is an opportunity to really show that we can use theoretical physics in a way that can have a broad impact. You can use theory to build sophisticated engineering tools and a connection between a lot of the deep, sort of Cold war education. I have to build tools for engineers. There's a very famous statement by Carver Mead, who's a very famous electrical engineer from Caltech, who said, every useful experiment eventually becomes a tool. Everything you can measure eventually becomes a tool that you give to an engineer. And so I would just like people to realize, look, you can do deep theory. There's a lot of fun and interesting stuff to do, and we can turn theory into tools that people can use and build a community and, you know, just have a broader impact. I think that, I mean, I did AI in the 90s. People thought we were crazy. Like, this stuff doesn't work. Nobody believed it. Right. Why are you doing neural networks? People think neural networks are invented by computer scientists. But there's a whole group of theoretical physicists doing this stuff for years and, you know, understanding sort of who we are, how the brain works, how we think, what's actually going on up here. And I think it's a very exciting time and that's why I'm doing this. I think there's a lot we can offer from the scientific community. There's a broad, I think there are really deep, broad connections between, between general science and what's going on in AI and that can connect back to the engineering world. And I think that there are big problems. One of the things I'm really proudest of with Weight Watcher is that there are companies using it to help climate change. I think it's a huge problem. If you can use it to find some way to solve this massive problem we have, I think that would be fantastic.

That's awesome. Well, I think that's a wonderful way to close out. Really appreciate your perspective there. And yeah, thank you so much for taking time to to join us, Charles. It's been a pleasure.

Hey, I really appreciate it too. I'm glad we were able to set this up and I look forward to the podcast and I really look forward to anyone who tries to use the tool wants to use it. Please reach out to me, let me know how it's working. Complain to me if you don't like it. I'm not going to fix it if you don't tell me. I don't know what's wrong with it. I'm not going to. I can't fix what. I don't know what is broken. And I would love that people join the community and build something great together.

Awesome. Thanks so much.

All right, thanks guys.

Thank you.

Alright, that is our show for this week. If you dig it, don't forget to subscribe. Head to PracticalAI FM for all the ways and if Practical AI has benefited your life, pay it forward by sharing the show with a friend or a colleague. Word of mouth is the number one way people find shows like ours. Thanks again to Fastly for fronting our static assets to fly IO, for backing our dynamic requests to Breakmaster Cylinder for the Beats, and to you for listening. We appreciate you. That's all for now. We'll talk to you again on the next one.