This Computer Scientist Exposed AI's Dirty Secret

What is sustainable AI? This is not something we learn from school. This is also not something that we learn from work either.

Sustainability, when we think of it, you know, it has like these three pillars, the environmental sustainability, the social sustainability and the economic sustainability, when we pursue them, makes either the society more sustainable or the climate more sustainable, etc. Their recent class of llama 3, they publish a model card wherein they say that this is a model which has 405 billion and trainable parameter like deep sea conversion.

3. Right. They are not allowed to use Nvidia, the latest GPUs. Do you think that's something in AI that you want to see?

In the end, it shows that it is possible. Right. So if we force this constraint on ourselves, this mindset, move away from the abundance mindset, we can still invent and be creative and then be productive without expending all of these resources.

Yeah. What's your plan for the next three years?

The next three years. Okay, so I think.

Hey, everyone, welcome back to Two side AI. And today I'm sitting down with Raghav Selvan from University of Copenhagen. He is one of the leading voices on AI's environmental impact. So today we're diving into his new book, Sustainable AI. We will talk about why making AI more efficient may actually make the climate problem even worse. Welcome.

Yeah, thank you. Thanks for having me on the show. And I'm happy to explain a little bit of the premise in which you presented this. So, yeah, I do work on sustainable machine learning and AI, and I think the point about making AI more efficient could make it less good for the environment is. I think, I would argue that it is an important point, but I think that comes as a little bit later in the discussion, I would say, because I think, as I write in the book, we argue that efficiency is necessary but not a sufficient condition. So, meaning we should still pursue efficiency improvements, but then that alone will not make AI more sustainable. So I think that's the specific framing, I would say.

So, yeah, we can definitely dive a little bit deeper into the efficiency topic. Tell us a little bit about yourself. You're working on medical imaging, right? And then now you're working on sustainable AI.

Yeah.

So how did that transition?

Yeah, so I think my training as a researcher has been primarily in signal processing. So that's what my bachelor degree was in. And then my master's was also in doing signal processing. And then for my PhD, I had the opportunity to move to Denmark for my PhD at the University of Copenhagen, where I was still in my view doing signal processing, but then just on a different type of signal, which is medical images. So I was dealing with 3 dimensional chest CT computed tomography data and in my view I would still argue that they are type of signals just in higher dimensions. And then you could still take a lot of the tools from signal processing and then apply and then some of the tools. During my course of my PhD then transitioned from classical modeling based tools to machine learning tools. So my PhD was in basically developing machine learning methods for understanding three dimensional medical images. So that is my training. And then since then I have primarily worked on machine learning methods. And one of the aspects of this that I have focused on, which is the theme of my book, is on looking at the sustainability of AI. Starting primarily stemming from their resource footprint point of view and then what can we do about it? And then are there efficient ways of reducing the resource costs and what would be the ramifications if we did do that towards sustainable AI? So I think that's like leading to the book. But then I think my training has been primarily in signal processing and machine learning.

As I would think of my background is statistics. So I grew from statistician to machine learning engineer and then deep learning engineer and then now AI engineer. So it's a whole marketing rebrand.

Yeah, exactly. I do think it's. Yeah. Right now it's also a bit trendy to call things AI. It eclipses a lot of other domains. And then I know if I have some statistician colleagues who would take offense when even we teach say linear regression or logistic regression as a machine learning model, so.

Yeah, exactly. My heart is dying a little bit every time, you know, MLE people are saying that logistic revolution is a machine learning or AI approach.

Yeah. But I think there is a use in having a common framework to look at. Like for instance, like I was saying, so I view everything as signal processing and then I. That helps me understand different knowledge sectors and then I think it's good to have a basis. So perhaps if machine learning community wants to view some of these as part of machine learning, I think it's okay. So it's not about just the names. I think in the end. Yeah, it just makes things easier perhaps to. Because it's also like a lot of pieces of knowledge and then how we put them together I think is okay.

The good thing is I think we still share some of the terminology, so it's not too hard to really understand.

Yeah, exactly, exactly.

So what is sustainable AI? I think a lot of developers or data Scientists, this is not something we learn from school. And I have to say, this is also not something that we learn from work either.

No. So I think at a high level, sustainable AI is trying to address the question of what is the sustainability of artificial intelligence methods. And by that I will elaborate a bit more, because sustainability, when we think of it, it has these three pillars, as it is generally understood, the environmental sustainability, the social sustainability and the economic sustainability. And I think these three pillars holistically, then when we pursue them, makes either the society more sustainable or the climate more sustainable, et cetera. And I think the AI methods, one of the most visible impact that it seems to have is on the environmental sustainability of these methods. Because if we look at the recent class of AI methods, what we by now brand as frontier AI, which comprises of large language models or foundation models, which are being developed in the last, I would say five years, these end up incurring a lot of resource costs. By resources, I mean computational hardware, and then the energy consumption of this. And because these models are now built and deployed on large scale infrastructures, hyperscale data centers and such, so there's a material footprint to these new models, these new frontier AI models, which is not negligible. And this, at the scale at which we are taking them up, can have a considerable environmental impact. And this could affect the sustainability of these methods. And that's like the entry point to the discussion on sustainable AI, that AI methods, the recent class ones, incur a lot of resource costs and then this can have like environmental impact. And then we start questioning, okay, how is this affecting the environmental sustainability of AI? But this is only talking about the environmental sustainability of AI. And then there are these other two aspects, which is the economic and then the social sustainability. And these massive resource costs also affect that. So for instance, because not everybody can set up a hyperscale data center to develop these models because they're so expensive and it's not economically sustainable for any small enterprise or a country from so the majority world to have access to these resources, then it's also a bit of a trade off on how can you build these models in an economically sustainable manner so that it encourages participation from everywhere. And this is the question on the social sustainability. So who gets to build these models? And then once they build the models, what questions do they get to ask? So I think the book also takes up this view where we start talking about the material costs of building new classes of AI and then trying to tie into these three pillars of sustainability as to how does it affect the environmental sustainability, how does it affect the economic sustainability and the social sustainability? But as by training by now I'm a computer scientist. So bulk of the book also then focuses on how can we improve the sustainability of AI? And that sort of narrows down the problem and then says, okay, you, if, if we as practitioners, developers and researchers, what can we do? And I think one of the most accessible things to do is to make things a bit more efficient by looking at improved algorithms. And I think bulk of the book then focuses on like beyond the framing of what sustainable AI is. Then we narrow down and then say, okay, how do we make things more efficient? How does this affect the energy consumption and then the carbon footprint of building new AI models? And then zoom out and then say, does this have a good impact on environment or not? And what are the caveats? I hope that gives like a bit of a high level.

Yeah, it's a very good summary of the book and your perspectives you mentioned about that. Actually it does have material footprint and it's not negligible in terms of resource usage for building AI. Can you put that into perspectives like what does that mean affecting an average person?

Yeah, certainly. So for instance, if we do consider some of the more recent classes of say LLMs which are being developed right now, some of these models are, for instance, we don't know about the proprietary models. OpenAI is ChatGPT. We don't know because it's a proprietary model. But then if you look at like an equivalent class of open source model meta publishes, which is kudos to them that they make and model open source and then publish all of these resource costs. So they publish these open source class of models called Llama, which I think is quite popular amongst generative AI enthusiasts. Their recent class of Llama 3 models, they publish a model card wherein they say that this is a model which has 405 billion trainable parameter models. It's a really large model. So it has 405 billion trainable parameters. And training this model, it was trained on 15 trillion tokens of data. So that is a mind boggling number. If you also think in terms of how much data that is and then how do you curate that data and then where do you store the data? And then when you train 15 trillion tokens using a model which has 405 billion trainable parameters, so that takes up a lot of compute and then the specific computational resources that they also specify is that it took about 31 million GPU hours, they say, which is if you had a single GPU then you had to run it for about 30 million hours. And that's a really long time. Of course they don't do it in sequence. So then you do a distributed training where you take like 50,000 GPUs in a data center and then you trained this for six months. And that incurs a lot of energy consumption because you have now used the data center entirely to develop this one model for six months. And that is a massive number which is about I think 21 gigawatt hour, which is because I'm sitting in Denmark, which is, I think about roughly the same energy consumption of 300,000 Danes for a month or something like that. So that's the scale of energy consumption that this model has. And that in itself is not the end of the story because energy production has a lot of environmental impact because there's a lot of energy is produced with fossil fuel based sources. So then that's where the carbon footprint comes in. And then for instance, this llama 3 model, that is about 8,900 tons of CO2 equivalent. So that's a huge number. So all of this, what does that mean?

So can you say that?

I think in the book, I think I have some specific estimations. But then like I said, the energy consumption itself is like the footprint of about 300,000 average people for a month. So then that's, that's the amount of energy that we would consume. I don't remember at the top of my head as to what the 9,000 tons would do. I don't want to make a false estimation there. But this is just like for one model, right? So this is the specific resource cost for just one of the models. And then we don't stop there. Then there's like more and more of these models constantly being developed. And then every entity who's interested in this sector wants to build these classes of models in aggregate. This turns out to be something considerable. So again in the book I have an estimation which shows that the prediction about the energy consumption of the data centers fueled by generative AI is going to surpass that of the entire African continent by 2028-2030. So that's like where, you know, we have, I think 1 in 5 people on Earth live in the entire in Africa. And then I think data centers energy consumption will surpass that by 2030. I think the latest. So I think this is what is the fundamental question that we are trying to, I'm trying to push in the book saying, okay, we've had all these really cool feats from these models. Like we can do the language chatbots and then we have text to image generation and lots of other capabilities which are quite exciting. But then if we do not account for these material costs of the energy, the compute hardware and then the carbon footprint, this is just only one aspect. Even then we need to justify how much do we use this for the relative benefits that we get. And I think the book dabbles with this question as to like how do we obtain a reasonable trade off? And then it's not, because it's not negligible costs. So how do we account for this?

Yeah, and it's something that we have to think as a human race as a whole as well. It's hard for a single company or even a single average person to think about this. Where do I stand in this whole thing that's happening?

Right? Yeah, certainly. And I think a lot of the solutions to. Because this is a pretty complex problem. Because in the end we are trying to say environmental sustainability even is with respect to climate change. That is what we are trying to address, why we need to focus on this issue. And that is a very complex problem. And we as individuals can do a few things, but I think as you said, we need to then like start from an individual and then think about how do we interact and then how do we address this problem as a whole to make it entirely sustainable. So. Yeah, so. And I think again in the book we also try and say, okay, as an individual, what you can do and then how do we zoom out? And then as a community, what can we do? As developers, what can we do? Then as government, what should we be doing? And then as a planet, what should we be doing about these problems? Yeah.

When you are talking about the cost of AI in terms of resources, like when you talk about computer, first thing comes to my mind would be electricity bill, we're using electricity. Is there anything else that's hidden environmental cost?

Yeah, it's certainly the electricity. And then the energy consumption is the most visible part, which has found a lot of discussion. Like it also has been a lot in the media. But there's a lot of hidden costs to this. So if you think of like a data center where some of these models are being trained, so the electricity consumption is not only when you run the GPU to do your computation, but then all of the other surrounding infrastructure that you need to set up a GPU in the data center has like additional energy consumption that is one level. And then the data center itself functioning of the data center has Additional energy costs. And all of these have the carbon emissions. And then another level to this is that if you do think of even your phones or your laptop when you're computing, it dissipates a lot of heat, it heats up and then you need to cool it. And then in data centers, this is furthermore severe. And then cooling in data centers, it requires at least the more traditional ones require a lot of fresh water to be pumped into the data centers which is then which absorbs the heat and is evaporated out. So you do draw a lot of water and then which is evaporated back. So you consume a lot of water in cooling the data centers where AI models are being trained or deployed. And then beyond that there is the, if you look at the life cycle emissions due to the manufacturing of the hardware that, that that sits in these data centers. So because electronics is quite expensive if you think of like how it is made. So you need like refined silicon to make the transistors. And then you need rare earth minerals to put into these circuits. So that goes into a supply chain which is deeply entrenched in mining and extractive processes which are also resource intensive and then have like negative effect on the environment. Then this is in the production stage, there's a little more to it.

And climate cost from.

Yeah, because. And then in the end you also have to think of how do you responsibly deal with the electronic waste that is generated from all of this? Because a data center has an average like refresh cycle we say of about five years. Meaning in about roughly five years all the components in a data center have to be replaced because you get like newer things which are more efficient and such. So then what happens to all of the discarded electronics? Because now it has a lot of chemicals and then rare earth minerals, you just pick some of it and then what happens to all of the electronic waste? So that would be like the holistic picture of talking about the hidden environmental costs of AI. Yeah, yeah.

It's crazy. I read that OpenAI when they're training GPT3, they're using over 5 million liters of fresh water for cooling the data center. And like for speaking of e waste like 22% are recyclable. And what happens to the rest? What do we do with that? Not something that ordinary person are thinking about.

No, I think with the book, of course that is also the thing. It's mostly some of these we as developers may not have as much control over. But I think the awareness is important so that then we can percolate these upwards into entities or in who can make these decisions on our behalf. And I think that is generally the theme of the book in the end that we need to zoom out and then talk about regulations and standards and then how do we make things, you know, better? Yeah.

So we've established that this is a big problem. Who do you think should know about this problem most I would agree.

I argue that everybody who considers themselves interested in AI currently there's. And that I would argue there's a lot of people who are interested in AI right now. So for various reasons should be made aware. So starting from researchers to developers to then the industry who is building products around this, then we also need to get the local governments and the national governments into account. They need to also start looking at it. So I would say this is also described in the book as a multi stakeholder, multi scale approach. So that we need to be looking at it both from an individual perspective and as a community. And then so this has to be addressed at different abstractions as with anything. And with climate action.

I would say I remember seeing a picture, an image in your chapter one of the book and then saying something like the AI related publication has exponentially grew for the past several years and we're hitting like 170,000 in 2023. How can you know, humankind writing so much paper about one subject actually that exploded.

Yeah. That plot is even specific for a subset of papers which talk about AI for sustainability. So I think if we even look at AI as a broader sector, it would be even bigger, but in the. Even more. Even more. And I think it is just showing that there's a lot of enthusiasm about this technology, which we should. And I think one of the motivations for me work on this topic is because also I'm very excited about the potential of these technology. But at the same point I'm also like, I want it to be responsibly dealt with and then one of the ways of accounting for this is to consider the costs. And I think that is generally the framing of my research.

Yeah, yeah, that makes sense. I think I read one word you mentioned in the book, it's called AI waste. Do you know if we have a way to measure this kind of waste?

So the short answer would be no, because there is no standardized way of measuring this. But I think with every specific choice we make in building an AI model, you can measure this AI waste as a relative thing. So what if instead of doing this I did something else? Could that have been more Better And I think there are lots of techniques to make things efficient and then that would measure this AI waste. Yeah.

When we think about today's AI development, it has been the trend for many years about like we want more data and then the current AI, the smartest model is only is trained on as much model as the knowledge of a four year old. So imagine if we want to make it even more powerful and then what people are doing is can we put on more devices in our environment and collect more data. So I can only imagine the future training of the model is going to include more multimodality data and then exponentially increase the amount of data we're going to be dealing with. What do you think that trend we're going towards?

Yeah, I think already like now, for the past 10 or 12 years this trend has held where we've been scaling up data sets, scaling up models, scaling up compute, and then we've been finding things to be working better and better. But I think the point about the AI based is also to say that I think we can do this better because this is operating under the abundance mindset as I call it in the book, where we just assume that there's going to be abundance of data and compute and energy and all the other resources that we need. But because we are in the end operating in a finite environment with the resources, I don't think this will be able to sustain forever. I think a lot of people want to push for this sort of progress where we just keep scaling things up and then we will end up with the next breakthrough. But I think what is more responsible from a sustainability point of view is to operate in what we then call frugal constraints. So frugality constraints, where we then say, okay, okay, I want to solve this problem of the embodied AI or one of those things, but how can I achieve that without then needing another exponential bump up in my compute or so in my data or energy? And I think we need to push that narrative more. This is what also we are trying, I'm trying to do in the book and then showing that we can achieve lot of these developments without spending all of these resources at the outset. We can be more conscious of these resource costs from the outset to make things more efficient and sustainable. So to like answer, there is of course going to be a push for increasing more and more data, but I don't think that is the necessary trajectory that will achieve sustainable AI. And I think for that we need to take these costs into account and then say like how can I get there without using all of these resources. And I genuinely believe there's another way to reaching that goal which does not incur this cost.

Do you feel there's an argument about when technology is advancing, people really care about the accuracy and the performance? That's like the first priority because you know that tied to profit and revenue. And then sustainability is a secondary priority or afterthought. Right. Think of the what happened with the ozone layer, right in the atmosphere. It took how many years for us to completely get rid of the substances that's hurting our atmosphere? I remember seeing a data like it's at least took maybe 50 years or something like that. It was a long time.

Oh no, you make a very good point there. So it has certainly been the case where we've been pursuing performance as the main metric when we are building these models. So I want to get to the best model so that I can build a product which is more profitable. But I think. And sustainability cannot be an afterthought is also the argument in the book because we are already grappling with climate change and its ramifications. And then I think we can't aggravate this further by just saying, okay, let us just build this product and then let's figure out where there's environmental impact later when things have matured or however. And I think that, I think I would dispute right now because we by now understand these models sufficiently enough to know that there's a lot of this AI waste and we can do away with that by embedding what I again in the book say sustainability as a first class consideration. So we should not think of it as an afterthought. But then it should become a design principle which informs all our choices in how we set up these models, how we train, how we what types of products we want to build. And I know it's not the current mainstream way of thinking about this, but with the book, I hope. And then a lot of the work, just not just my team, but then there's a sizable community which is trying to get these arguments out that we cannot do this as an afterthought. Yeah.

This whole idea about context engineering, if you really think about it initially, maybe last year when the context window of just expanding exponentially. Now, Claude, I can have 200k token as whatever garbage I can put in and then German, I can accept a million, right. Which means I'm more free with data. What really stops me from doing that is cost. Right. It's two things. One is it becomes exponentially costly for me to deal with a lot of tokens. And then secondly it doesn't work like a performance is not improving just because I'm dumping all the garbage the information in and hopefully AI will figure it out. I feel like that's a very clear behavioral correction trigger for developers. Right. I feel like it's kind of similar to sustainable AI. I'm just wondering if there's a way that we can, it can become a hard constraint so that we're aware.

Yeah. So I think I usually try not to talk about the economic gains that you would get by doing sustainability efforts, but there is like almost a one to one correlation to that. If you make things better from a sustainability point of view, it also makes things cheaper. Because for instance, if you look at energy, energy is expensive, compute is expensive. And then if you make things more efficient, then you know you're also going to make things less expensive for the business or the individual. But then I think I usually don't bring this argument in because I think again talking about sustainability, that has to be an intrinsic motivation which we need to push far in these design principles. And it cannot be as a side effect because we were trying to cut costs because that I think will not lead us to truly sustainable AI.

Why?

So I will tell you exactly, exactly. Because so one of the thing is that say if we make things more, this you touched upon already with your context engineering example, say so initially increasing the context window was more expensive because it has a quadratic complexity. And then there's been progress in trying to do linear self attention operations. And now we can increase the context window without being so expensive. So what we ended up happening, what we ended up doing as a community is now computing attention or using these transformer class of models is more efficient. That did not stop us from making using it less. Right? Because now we think ah, okay, it's cheaper. So instead of previously what I had at context window of 10,000, I can go a million tokens, right? So that is the, what we call. Then this is an exact manifestation of what we call the rebound effect. If we started out with the goal of trying to reduce the overall energy consumption of just let us say LLMs. And then one of the things you know, we could have done is okay, good computing self attention in this transformers is very expensive. So let us make it efficient and then we do that and then now that because it has become cheaper, we don't stop using it less, we start using it more. So the net effect is that the energy consumption because of this has increased. This is a classic case of what we call the rebound effect or the Jawons paradox, where if you try to make things more efficient. This is why I think from the outset I was saying that efficiency is not enough because you know, that does not make things more sustainable because then we lose the larger picture of what did we start out doing. But now things are more efficient, but then we are all using more of it, so then it's not sustainable overall.

So this is so resonatable with everybody's everyday life because now we have AI. You know, people are vibe coding, I code faster, I write faster. This is affecting everybody. Right. Half of Americans population are using some sort of AI chatgpt. So everybody's accelerating, but I don't feel like I'm relaxing in the afternoon, I'm just feeling I'm doing more.

Yeah, I think that's a good rebound effect. Exactly.

It doesn't help at all. I wish. Okay, now I can do three people's work in the past and now I can take my afternoons off. Never happen.

So I think that is exactly another good example. Right. And then we need to then keep this holistic picture into consideration when we make these choices. And I think that is why it's not as easy as making something efficient and then claiming that it becomes sustainable. That's why I think the book is not called Efficient AI. It is sustainable AI because I take this clear distinction in the book saying this is one way. That is why again, I will repeat what I said. So efficiency is a necessary but not a sufficient condition to achieving AI. We not stop at doing efficiency pursuits. There's more to be done to make things efficient to sustainable.

Let's think about solutions. Right. So are you thinking of for instance, small models like deep sea conversion 3 it has been said that they are not allowed to use Nvidia, the latest GPUs and with that constraint they were actually looking for better or more efficient ways of developing with less resources. Do you think that's something in AI that you want to see?

Yeah, certainly this is there. I previously mentioned this about abundance mindset and then the frugal mindset. So where Deep SEQ was not perhaps intrinsically trying to do this for because of the frugal mindset, but then they were compelled into doing this. So meaning. But then in the end it shows that it is possible. Right. So if we force this constraint on ourselves, this mindset move away from the abundance mindset, we can still invent and be creative and then be productive without expending all of these resources, we're just being wasteful. And that brings back to the question of AI waste. And I think we need to have this a bit of a paradigm shift on moving away from let's just scale up and then things are going to work to. We cannot be doing that because it's not sustainable. And I think that is what I would argue for.

Do you think that this forcing function is needed? Yeah, I think like, how about we just, you know, you only have this much dtu.

I'm sorry, I think that should be. So some of the things in the research community we try to do is, hey, okay, here's like a challenge. Can you solve this problem with only, I don't know, ten GPU hours or a hundred GPU hours and things like that? I think this accountability for the resources should be embedded in designs. So I don't know. I mean, of course at an organization level or at how do we see this happen? Is a different question. But I would say that it has to be. It has to become more common for us to think about the resources and then try to operate within those constraints. Because it's not just because we can afford it, we can just use it. So that's, that's, that's not the very, that's not the sustainable way of doing this.

Yeah, I definitely think the community needs awareness of this because everybody is using resources very wastefully because it's easy, I'm lazy, it's just convenient.

But I think this is where the book also I show some examples. Right. There's a lot of easy ways to cut down this AI waste. It's not even. It doesn't take a lot of effort. It's tools which are very mature by now. There are frameworks which can easily do this for us. And I think it's just a little bit of a change in habit can already steer a lot of reduction in AI waste. And then we should again not stop there. Right. So that's then, then how do we go from there to making things sustainable? That's another discussion, a dialogue where we need to get people talking about it. But the question of reducing the AI waste is. I think it's even more easier to do than setting up a large scale, I think.

So give me something. Example, give me some examples.

So there's, for instance, there's so many examples I can talk about then one of the things that we've been looking at in my team or just generally within the community is like on how much data do we need to train A particular model. So right now there is collect all, use all, train the model type of a thing.

I'm a statistician, right? We use small data, right? We do subsampling. I balance data.

If I'm constantly thinking why do we need all of this data? A little bit of data curation, just selecting the data in a slightly meaningful way, subsampling the data. And there have been studies after studies which show that with maybe a small drop in performance or not even you can get to the same results with a fraction of the compute resources. And I think it just has this other. It's. But it's. Of course there's another step that you have to do the. You have to look at the data now instead of just throwing everything into this big pot and then hoping that it magic happens. But I think that's one. But you can look at every step in the life cycle of an AI model and then say okay, you can do this better. We don't need to train models in full or half precision. You can go down to four bits. So meaning then you don't have to use eight GPUs. You can be done with one GPU. So then for that, for instance, there are so many mature tools by now. Say if you use Pytorch then I think already training things in half bit precision is easily done. Now there are other packages that you can put on top and then you can do training in 2 bit or 4 bit precision. And this is without a lot of additional overhead.

So I can tell you when I was working in a company with different objectives, right? So if my boss comes in and I want a best performing model, then should I use the floating point half position? It's a question I need to think about. Do they care about GPU costs? And then I can tell you what's driving a different behavior. For instance, let's say I use logistic regression instead of a transformer model. That will significantly reduce the cost right into in terms of resources. The reason is actually not because of having the sustainable AI in mind. The reason is actually the results demand explainability because it's easier to explain a simpler model rather than neural networks. Especially when you're in a regulated industry. That's often required. And then the sustainability in model development becomes a side effect.

Yeah, no, because. No, I, I see again you're bringing up this point is also. I see what you mean. So we need to incentivize people making these choices which aligns with their motivations. So either reduction of cost or explainability and then. So this is perhaps not as clearly, that was not the objective in the book. But I would like to argue that it is very much aligned with pursuing all of these. So for instance, the question of, yeah, what is the best model? So then that you want. So then the best model is not just the performance then okay, the best model that you can get for a given cost or the best model that gives you explainability. So all if you take this multiple criteria, then you already end up in a territory where you're not just scaling up for the sake of scaling up to solve the problems. And that leads us to. It's a bit of you do, it's like a do and use in a different way. So you make things better in both ways.

Yes. I'm just trying to figure out like how do we incentivize the community that aligned objectives in a way that's easier and we are all living in a more responsible manner.

And when we're building models and I think for instance, the cost factor is very easily. It's apparent. Right. So if you make things. Yeah. If you're trying to make things more sustainable, then you inevitably are also making things cheaper for yourself.

It's implicit though, if you charge me, if you say that I'm tracking all OpenAI or quality anthropic, if you're developing any company developing these large models, foundation models, you will be tracked on your carbon footprint and then you'll be taxed on top of that. Then people start making paying a lot.

Of attention to it. I think this is where I think being in EU is a bit of a playground. Right. With regulations because there are some regulations which are starting to very faintly appear which are trying to push for things where the energy consumption of frontier AI has to be measured and reported and then there are provisions, but then there are challenges again. So I think I'm hoping that this will mature a bit more and then we will see that people then know that it is being. It is something, it's a cost and you want to minimize it.

So what do you think of where US stands on this compared with other parts of the world?

I haven't followed the U.S. yeah, yeah. So I mean that I haven't closely followed the US legislative landscape about this. But I do know some of the provisions on looking at energy consumption, building large scale AI models, I think they were either nulled by now or there were some of these acts which were going to kick in which were. Which got scrapped recently. And I think there's a little bit of I think I would still consider EU trying to lead the way here and then perhaps I'm not fully aware, but I do know a lot of researchers who do work along these fronts. But from the industry and the legislative front, I don't think I have seen as much push and effort from the US counterparts. Yeah.

Let's say as a developer, what can I do? I can tell you one thing I thought, I think about sometimes I try to kick off training overnight because it's cheaper and I can fight for more GPU space. Can you tell me more? This. Like little things I can do like this?

Yeah. There are small little things that we can do to make things more efficient to reduce the energy costs, the carbon footprint of our own individual computational footprint. One of the things that you said is like when you schedule a job at late in the night, so you said it's cheaper, but it's also less carbon intensive. Because this happens because our electricity grids are, they're dynamic. So depending on the demand that we have, the energy mixture that goes into producing electricity changes. Say for instance in Denmark when it's on a windy day, we have a lot of wind energy here. So then late in the night it's entirely green. But then during the daytime when there's more demand, just the wind energy cannot fulfill the needs. So then there's a lot of biomass or other fossil fuel based energy that has to come into the grid. So what you say when it is cheaper at night is also less carbon intensive. So this is again another case of you have the same incentive. We need to just align it makes it more environmentally sustainable even. That's something. So this consciousness, because I don't think all training jobs are time critical, it's okay if it has to wait for a few more hours. So that is one of the things for sure. And then if you do have the opportunity to choose which cloud instance you want to set up in which physical location, then there's choices on. You want to choose regions which have low carbon intensity. So you can select that. That is another way to reduce the carbon footprint of electricity alone. Yeah. And then I think just being mindful of, even when, as a developer, when you're computing, like I said, like you can do things at low precision, for instance, or reconsider if you really need all of the data. These are. Yeah, these are some of. There's a lot of things which we can do as individuals. Yeah.

What if in the future there's going to be 100% renewable energy and then it's all Clean energy, would that give me more leeway of do whatever?

Yeah. So this is, I think it's actually the book. I have a small sidebar called the Green Energy Fallacy. So where initially when I spoke about the environmental costs, so the electricity was just one component and the carbon footprint associated with it. So it certainly would make things better. Right. Because there's. You're going to. The carbon footprint because of electricity will reduce. But then if that means we're going to build more data centers, we're going to build, manufacture more hardware, if we are going to use more water, the efficiency again. So I think it is good. But again we cannot stop there because that doesn't. It's also a bit of a rebound effect. Then if we think, ah, everything is green, let's just use more of it, then what happens is like we build more data centers and then we have to manufacture more. Then there's this whole other broader environmental effects. Those don't go away, unfortunately.

Yeah. Oh my goodness. I feel like. Are you positive or are you.

Oh yes, yes. I was also talking to some students of mine earlier today saying that. Yeah, yeah. So this sounds very, I know, a bit gloomy and then I'm just throwing all these big responsibilities on all of us. But I think it is also in, in the effort that we can make small changes which will align, which can help us onto a better path. And then I am, I'm an optimist and I think so this is all. We can do it and I think we just have to all come on board and I'm hopeful that we will. So it's not a.

What's your plan for the next three years?

The next three years. Okay. So I think there's going to be. So a lot of the research is still trying to mature some of the questions, answer some of the questions that we have asked in the recent works. But it's also, I generally think that this scaling up to solve the AI problem is not going to be the solution to if you want to really get meaningful AI. So I'm hoping to focus on some of my technical research along trying to do this in a frugal way, in a way that is embedded with frugality and then with sustainability. And I think that I hope will. Will be the big ambition for the next three years or even longer. I don't know. Yeah.

Yeah. Fantastic. Thank you so much, Raghav. I really enjoy chatting with you today and learning a lot more about sustainable AI. What does it mean and how does it affect our day to day life. I don't want to be one of the person who burns our planet down.

No, Evan. No. I think it's also the. So again, the premise of the book is also that we are at the scale at which the recent AI models are growing, that it could become a factor. So we need. We have time to act. So I think that's why this is an important time in this trajectory, that it's not too late, we can act and then do things better. Is. So I think we all have a small role to play in this. Yeah.

How much time do we have?

Okay, so I think that is a bit of a. There's a lot of other externalities to this. So I would say let's start today.

Okay. Okay. Yeah. I was hoping that you would tell me, like there's a. Like we have 5 year, 10 year or whatever. But I guess it's. It's hard to. Exactly.

But it's. Yeah, but it's never hard to estimate when we have to act. It is now.

It is now. Okay. I think that's the most important message that we're conveying in this video to all my developer community and audience as well. I hope you can broadcast more about these ideas and tell ordinary average person what's the impact of using AI Or a lot of people think that AI is the new electricity. Maybe that's easier to understand.

Yeah. So in the book I also say that AI is the new electricity, but not in the way it is thought of because it is the electricity in the sense that you need a lot of infrastructure to manufacture and distribute. So it is not an algorithm or anything. Right. It has a lot of these material costs. Like with electricity, you cannot just. It just doesn't happen from thin air. So you need this physical infrastructure to carry it from one place to another.

The person using AI also using these electricity because like yeah, people, if they're building. Right.

There's a lot of layers to it. But yeah, I appreciate. Was fun.

Thank you so much and I'll see you next time.