Government regulation of AI has arrived

Welcome to Practical AI. If you work with artificial intelligence, aspire to, or are curious how AI related technologies are changing the world, this is the show for you. Thank you to our partners for helping us bring you practical AI each and every week. Fastly.com fly IO and typense.org.

What's up, friends? AI continues to be integrated into every facet of our lives. And that remains true because you can now index your database with AI. You can write more code, become that 10x where you always wanted to be, and you can even draft a letter for a lease on an apartment or a new property. AI is everywhere and it might be time for us to start questioning. Is AI our friend or our worst enemy? And that's the focus of the three part season opener of the award winning podcast called Traceroute Podcast. You can listen and follow the new season of Traceroute starting November 2nd on Apple Spotify or wherever you get your podcasts. And this show is all about the humanity in the hardware that shapes our digital world. In every episode of Traceroute, a team of technologists seeks to untangle the complex question, who shapes the Internet? Seasons one and two gave us a crucial understanding of the inner workings of technology. Technology, while revealing the human element behind tech in season three tackles not just AI questions, but also how can we use technology to preserve the earth? Who influences the technology that gets made? And what happened to the flying cars we were promised? I think it's safe to say that the future of AI is both exciting and terrifying. So it's interesting to hear the perspectives of experts in the field. Listen and follow this new season of Traceroute starting November 2nd on Apple Spotify or wherever you get your podcasts.

Welcome to another episode of Practical AI. This is Daniel Whitenack. I am a data scientist and founder at Prediction Guard and I'm joined as always by my co host, Chris Benson, who is a tech strategist at Lockheed Martin. Today is a fully connected episode with just the two of us where we're trying to keep you updated with everything that's happening in the AI community and maybe learn some things ourselves that help us level up our own understanding of these topics and yours as well. So yeah. How you doing, Chris? Are you keeping fully connected?

I'm definitely fully connected and there's been a lot to fully connect to this week. You know, it's, it's. There was a bit of homework going into this episode here, so it's been interesting. We got a lot to talk about today.

Yeah, there's a lot happening in the world. Normally in these episodes, or at least it feels like in recent times, in these episodes, there's been a lot of updates on new models and other things like that. And that's still happening. So things like Mistral and other models have come out. But I think the interesting thing maybe that I've seen people talking about this week in particular circles, back to government interactions with the AI community and in particular the White House here in the U.S. the White House, the President's executive order on safe, secure and trustworthy artificial intelligence, which is kind of timed interestingly with other things as well. But yeah, I know that you're very in tune with the public sector, Chris. Are you seeing a lot of discussion of this in your circles?

Yeah, we are. I would say that we're actually, as we parse through it and we'll talk about the different sections and stuff. I would say a lot of the stuff that would affect my day job in the defense and intelligence world, we're already kind of doing. Yeah, not kind of doing. We're already doing a lot of that stuff. And so there's a lot of specifics in this executive order, but it's not starting a new process for us in that world. Yeah, it's something we've been working on for quite a lot for years. So it's interesting. I was pleasantly surprised with this exact story because, you know, we've talked many times on this podcast about the fact that how long it's taken for governments to. To start getting a bead on. On these AI issues and what does regulation mean and who's going to participate and how are you going to do it and all this kind of stuff. We've been saying that for years. And finally, on Monday of this week, as we're recording, Monday, October 30, 2023, we got this executive order issued and then I believe we got the Bletchley declaration issued as well later in the week. You want to talk about that a little bit?

Yeah, sure. It might be useful. I know we have a wide range of listeners, and if I'm being honest, even myself, we had a friend that just became a US citizen the other week.

Congratulations.

Yeah. Talking through things with him is like, of course it's fresh in his mind, but all of these ways about how our government works and the various ways in which things can be legally enacted can be quite confusing. So maybe before we jump into things, maybe just touch on an executive order, what might that imply and how it may be different than certain things that have preceded it because there's been statements on AI and government thinking about AI in the past here in the US but from your perspective, what makes an executive order maybe different than some of the things that we've seen in the past?

Sure. So noting that I am most certainly not a constitutional attorney or any such thing, just a dude who likes AI.

I would still vote for you, Chris.

Oh, that's really nice of you. But an executive order, in short, and I'm sure if we have listeners that say I'm slightly off, they can correct us on this. But the President of the United States can issue an executive order which is a legal device which essentially has the effect of law. It can be overridden a couple of different ways. The US Congress can override it by passing an actual law. So if an executive order is in conflict with a law that is passed in Congress, that law in Congress will trump that. And in addition, an executive order, I believe can be the US Supreme Court can also override it on constitutional basis. But unless one of those two things happen, my understanding is executive orders otherwise have the effect for all practical purposes of law in the United States of America.

Yeah. And apparently these actions are, quote, the most sweeping actions ever taken to protect Americans from the potential risks of AI systems. It's funny how the use of adjectives in government has become quite interesting over the years, but this is the most sweeping actions ever taken to protect Americans from the potential risks of AI systems. There are some interesting ones, and I think what might be interesting about at least a couple of these is the way that they might influence the AI industry in the US in particular, but also some ways that government agencies and other entities might become involved in the AI world. So I guess before we jump into the specifics, you also mentioned the Bletchley decision. What's that for our listeners? How might that relate to things going on?

Sure. So there was a summit, it was called the AI Safety Summit that took place on November 1st and 2nd of 2023, which was just a couple of days ago this week. They issued what they call a policy paper, which was the Bletchley Declarations by countries attending the AI Safety Summit on that date. And it's fairly short, it's a few paragraphs long. And my understanding, not being a legal mind, is that it's it would not be binding in any way legally. But there' a a number of countries listed, there's a couple of dozen that attended, including the United States, United Kingdom and, and a lot of other countries around the world that Basically said, we're acknowledging. The short of it is we're acknowledging AI safety is important to us all. It's by definition an international concern. And that the way to deal with these concerns going forward is for us to all work together and, you know, share information and such as that without reading the whole thing to the audience on the show, which I don't think we have time for.

Yeah, we will link these things in the show notes.

Absolutely. It's a good thing. I agree with everything they said and it's a good kind of a Kumbaya saying we need to work together, but it's otherwise, you know, it's just saying, hey, let's go do these things. That's very important because it is indeed an international concern. I'm definitely applauding that. I've been more focused a little bit on the executive order simply because it's binding, having the effect of law and it's. And we'll talk about the details, but it gets quite specific. In the executive order itself, there's a fact sheet that kind of gives you a high level and it doesn't give a lot of the detail. And I read the fact sheet first and I was a little bit. I was like, okay, that's all great, fluffy stuff. But when I read the executive order afterwards, it gets down to who's responsible for what, how long they have to do it and what they have to do it. And there's a bunch of specific results expected or standards applied that clearly were people from the AI community that were very, very knowledgeable. So it wasn't a political only group of people that did this. They obviously had expertise available to them. So I ended up being more impressed than I expected with what they came up with.

I haven't done all of the research and I don't know if it's even published anywhere about who was involved in the process of developing this. But we can go through some of the high level things and there's a few really helpful. So there's the fact sheet that you mentioned. There's also some good articles that I've been looking at over the past days that give some summary information that might be relevant for people as well. Like the MIT Tech Review has an article and some other ones if we don't dive into the specific wording quite yet. And I just look at kind of the high level, what people are saying about it, what's standing out to them about it. One thing that I see really driven home is the real focus on safety and in particular safety and security and the real focus on kind of like labeling and watermarking the output of AI systems, which I think, if I'm understanding right, as framed within the Executive Order as a safety thing in terms of protecting our citizens from potentially fraudulent or harmful material that might come out of AI models, as well as giving, whether it's entities related to defense or education or whatever, the ability to identify and discriminate between AI generated assets or content or text and non AI generated or I guess human generated.

You know, we're talking of a specific point here in it, but it takes both sides of that. It both issues the directive that you, in essence, you must identify AI generated content so that you can't have deceptive representation there. But it also issues the appropriate government agencies to come up with mechanisms by which they may detect people who are not subscribing to that directive or foreign. Obviously there are all the other countries in the world and they are not held by our Executive Order, so the means to detect all that. But I also would say that that's not a new idea to the DOD and intelligence community.

And am I understanding this right, Chris, that the Executive Order would put the obligation on certain government entities to figure out how to enforce this mandate on the other organizations, companies, entities, teams that are within their jurisdiction. Is that a good way to put it or am I misunderstanding?

Yeah, it leaves a lot undone. So. And at some point we probably should jump in and kind of hit the highlights on what they were. But it basically puts the burden on various agency leaders to come. It'll outline what they have to accomplish and when it must be accomplished by in some cases is what the output of that is. But it doesn't tell them how to accomplish all that. And I have yet to see any place where it ascribed any budget to any of those items. So there are strengths. It was good thinking in many areas, but the dollars to that that would go into making some of these things happen. I did not see assigned. So positives and negatives.

When you read this, when you see this coming through and how things have trended over the recent years with respect to the government's thinking on AI, is this like taking us up from a level two of priority in government and action up to, you know, 10 or how do you see this as escalating, the kind of involvement from the government in the AI industry in terms of practical things that we'll see in, let's say, the coming year?

Oh, I, I would say that they don't have a choice, given that it's now in an executive order and it directs them with timelines and the specifics on that. If I was leading a federal agency that was being called out to do this, I would probably be scrambling and trying to figure out who I had and who I could get and where I was going to pull the money from to accomplish those things. And I'm hoping maybe a listener knows that there's budget subscribed that we haven't heard about yet and that would be good news. They would have to to jug little bit in terms of their priorities to make some of these things happen. But I think it's good. It forces it right up to the top of our list of things to, you know, the government could could happen and it I don't think it was going to happen from industry alone. We've watched for years as commentators on the industry. Every week we've seen individual companies kind of do their own thing. They kind of compete with each other because there's a bit of a marketing tinge to AI safety as well, but nothing that has ascribed the entire industry universally. And so this will clearly do that with external standards.

This is a changelog News break. Hugging Face released a distilled variant of Whisper for speech recognition. It's English only and optimized to the hilt, which resulted in running six times faster while being 49% smaller and performing within 1% word error rate from the original model. It's designed to be a drop in replacement, and the Hugging Face team cites five reasons why you might use faster inference robustness to noise, robustness to hallucinations designed for speculative decoding and permissively MIT licensed. This looks great, but I'm still waiting on speaker diarization. You just heard one of our five top stories from Monday's changelog News. Subscribe to the podcast to get all of the week's top stories and pop your email address in@changelog.com News to also receive our free companion email with even more developer news worth your attention. Once again, that's changelog.com news.

Okay, Chris, let's dive into some of these specifics which I think are really quite interesting. The one that I thought was probably most interesting to me, although I think the others have both good and interesting implications. As my wife and her business would say, there's wins and opportunities, but I think one of the ones that stood out to me was I'll just read the wording here and we can talk about it in some of the wider implications. But one of the things is a requirement that developers which is interesting to me because it's, I'm an AI developer, I guess, a requirement that developers of the most powerful AI systems share their safety test results and other critical information with the US Government. How does that strike you? What are your thoughts?

Depending on the model and the, and the specifics of that, it will often be the U.S. department of Commerce that's receiving. In some cases it will be military or intelligence, depending on the nature of, of what the concern is and what the model can do. But that will have to be created. Doesn't exist today to the best of my knowledge. If I was leading commerce, I'd be going, okay, how are we going to receive this information and store it? Because they're probably going to be getting quite a lot of data coming at them with, with the amount of development in this area. That's one of the things that certainly here in the US we're going to have to learn how to do because as of now, once the timeline I don't have in front of me what the, how long they have to put that into place. But after a certain number of days it will be required by law.

If we're specific on the thing that would be required by law, if I'm understanding this right, it would be someone that is developing a new. Maybe one thing, I don't know if you've seen this, maybe I just missed it. I'm not sure if this would be new in the sense of training from scratch or new in the sense of a fine tuned model. But those that would release large models, like in recent weeks we've seen Mistral and Llama 2 and Zephyr and all these models coming out, the models that are significantly large, like that foundation models that are significantly large as those are released after they're released, before they're released, maybe we can talk about that. That the people that are producing those models, the teams, the developers that are producing those models, perform some sort of red teaming to probe the models in terms of potentially harmful outputs and kind of behavioral tests and perform risk assessments that would be gathered in some sort of coherent way and shared with the US Government. One of the interesting pieces of this part of the Executive Order in particular, if I'm understanding right, is that it falls under the authority of the Defense Production Act.

Correct.

Which I think in addition to the Executive order status might bring this point maybe a little bit higher up in terms of firm legal footing. Again, not an expert in that thing, but that's my understanding from what I've read.

Yeah. And as a non expert. But the Defense Production act, in case listeners may recognize that, because they've been hearing about it a lot over the last year because it's been a point of in the Russian invasion of Ukraine, it has been repeatedly cited as a mechanism that the US Government can use to increase production to support that effort. Our allies, the Ukrainians. And so you may have heard that before and from that I know as a non attorney, non legal mind that it gives the US Government broad powers on requiring commercial companies in the United States or that we do business with to meet a certain set of criterias. And there are things that they can be told you must go do this on because it's in the interest of our national security. And so it's a fairly sweeping thing, is my understanding, since it has been referenced here. I agree with you. You not only have executive order, but you also have reference to a fairly strong point of law from Congress.

One of the interesting takes that I saw on this is this is just a blog post that I'll link in the show notes, but the comment in the blog post was that this might be one of the things that would kind of lead to a firming up of the players within the AI market as we see it, because not only is now there a computational data infrastructure burden on those who want to produce these large models, but now there's more of a regulatory burden that would actually be an additional kind of step here in terms of being a player in the foundation model space.

Now, there's irony to that, by the way, in that they explicitly note they're trying to address equity in the executive order, but by adding the regulatory burden that will be exclusionary, correct?

Yeah, you don't get anything for free, that's for sure. So this will be interesting. It's hard for me to think that the progress will slow very quickly around releases of these models. What is very interesting to me is how they will end up deciding, like if I pull down llama 2 and use a small data set that I have access to to fine tune it and then I release that as another foundation model that people can use. Well, you know, at what point? So I am modifying the weights, right? At what point between there and training from scratch, which, you know, even large models I don't think often are. They might start from a starting point with their weights in certain cases. But yeah, at what point in there are you really a developer of a significantly large model? Because both of these models are large? When is it adaptation? When is it fine tuning, when is it training or Releasing building, as the executive order says. So, yeah, that's all kind of mushy in my mind, I think.

Yeah, they do attempt to kind of address that. There's a section 4.2, ensuring safe and reliable AI. And they talk about a time frame. And this is one of those that calls out the Defense Production act specifically. And within 90 days, there's a set of things that are being required, which is a very short timeline. You know, when you think about it, part of that is they have a set of criteria which is certainly not comprehensive. As to your point just a moment ago, it's better than I expected, but it's not. Not entirely sufficient. There's a lot of nuanced questions as you just raised. I noticed that one of those. They talk about the quantity of computing power used for training and they have picked, interestingly, 10 to the 26 integer or floating point operations for kind of as a general point of computation, it's like a threshold. And if you're above that, you're kind of in that large range that they are particularly focusing on, they have reduced that down to 10 to the 23rd integer or floating point operations for models that are based primarily for biological sequencing and things like that, they have some other aspects in. But they have an interesting threshold that they call out in the executive order.

So you know exactly what's going to happen here, Chris, is that whatever that number ends up being. So it's just like the town where my dad grew up in Kentucky is a. Actually still is, I believe, a dry county. Meaning in the US this is where you can't actually purchase packaged liquor in a liquor store. Of course, what happens then is just on the edges of the county, you have these at the sign where the county ends, there's like 14 liquor stores. Right. We'll see something similar here. Right. We've been choosing our numbers of. Of parameters and such. 7 billion, 13 billion for various reasons for our models. What's going to happen is people are going to get really, really good at training models right under that threshold, which.

Is 10 to the 25th, will be the magic number going forward.

Yeah. Which, you know, I think it could actually have a. Even though that's kind of gaming the system, it could actually have a really nice effect that instead of us trying to always think about more data, bigger model as the way to incrementally improve, this does put a burden on those that want to operate at the lower level under the threshold of regulation. The burden to say, hey, what if we're creative? Either in Our model architecture or the way we train it or the way that we fine tune or whatever that ends up being to actually do more with less, which I think overall will be a good, would be a good thing. And you know, academia is already thinking about these things with things like the baby LM workshop and other things like that. So I think that could actually have a follow on effect that's quite positive for the model landscape.

You'll have a set of players that remain out of necessity. You know, they, they must play up in that area because you know, the large, the true LLM range isn't going to go away, but that will also be dominated by large players who are already doing regulatory stuff anyway. Maybe they weren't in this, but they're accustomed to that. It is exclusionary to those large, you know, things like cloud providers and such will be doing that. But you'll probably have a whole range of mid sized players that are below the Amazons, Google's and Microsoft's of the world and the OpenAI's of the world that'll play in that, just below that and build foundational models. It may be that, yeah, as you said, it'll be interesting to see what kind of innovations come from there.

And one other question that I have coming out of that is how do I know that I hit the 10 to the 26 and how do you, along with this sort of restrictions or legal implications throughout the executive order, this kind of naturally brings up a lot of questions like they also talk about watermarking, which we can talk about here in a second. But the general thought is whether you're talking about this computational power, red teaming, behavioral tests, watermarks, labeling. You need standards and tools and tests to help you ensure that you can do these things. Like how do I know when I hit that threshold, how do I watermark things, et cetera, et cetera. And so one of the other things that's drawn out right away is the developing of standards, tools and tests to ensure that AI systems are safe, secure and trustworthy. And this specifically calls out the National Institute of Standards and Technology or nist that you might have heard of before, because they have one of the most precise clocks to help keep the standard of time and the most precise weights. Yeah, the most precise, like, like this is exactly what a kilogram is. And yeah, I actually in my undergrad when I was doing research, I did research at NIST with one of our collaborators. So we were theoretical, they were experimental. And I think mostly all I succeeded in doing was spilling a bunch of carbon nanotubes on the floor. Not very good at experiment, but that's what they're experts in, minus the occasional intern that spills carbon nanotubes on the floor. But they're specifically called out to help or set the rigorous standards for what's phrased in the Executive Order, extensive red team testing to ensure safety before public release. What are your thoughts on this, Chris?

This goes back to something I mentioned earlier. There's a lot of figuring out the how that's undetermined. So you have clearly some bright AI minds that help help construct the Executive Order, but they've left wide open. You know what that means. And you know, what is red teaming? What is red teaming trying? They hit some things that red teaming should be trying to do at a very high level. But it's up to NIST and the, the Department of Commerce to come up with, you know, what the specifics are on that. And I think we're all going to be learning. I think the key thing that I would take away from that is that this Executive Order is the first of many things to follow over the next year from various agencies as they are trying to fulfill the Executive Order's intent.

Well, Chris, the next thing I see in here is biological materials, which I don't necessarily think about that much, even though I'm made up of biological materials. I guess I don't consider my own biological self very much. But they talk about protecting against the risks of using AI to engineer dangerous biological materials. What is a dangerous biological material? I guess a bioweapon. Is that what we're talking about here?

It would be bioweapons and, and it could be something that we've all heard about. When we were in the height of COVID there was the. All the theories about whether or not it had been created in a lab in China or elsewhere and such as that, you know, and so it could be a weapon by design, it could just be a virus. It could be lots of different things. There are a lot of international laws and domestic laws against these things, but we also have actors around the world who don't necessarily subscribe to the same values. And so it's still something that the intelligence and defense communities of both the US and our allies spend a lot of time thinking about how to address and defend against, though we follow those laws that our adversaries may not.

So, yeah, yeah, people might be wondering, well, how might you practically think about protecting against the development of dangerous biological materials with AI? And we've had previous shows, can Try to find them and link them in the show. Notes about using AI to find new drugs or something like that, right?

Yep.

Well, a lot of those projects, whether it be those kind of pharma related projects or academic projects related to biology and AI, or AI and life sciences sort of overlap, a lot of those do have some sort of federal funding behind them, whether that be NIH or nsf, these sorts of grants. And so one of the things that's called out here is, hey, if you want a grant, if you want our money, then you have to agree to establish these standards. Xyz, which to my understanding are not specified in this executive order, but it's saying we will create these standards that will be standards and requirements to receive federal funding for biological research with AI or I don't know, that's probably also to be determined, like how to categorize that.

And we're talking a lot about biological, but they don't just address biological in it. There's kind of some special stuff on biological, but then they also address what they refer to repeatedly as cbrn, which is short for chemical, biological, radiological or nuclear weapons. And so the kind of civilian research on the biological side, but there's also the, the military side under the CBRN acronym that they're addressing on those. And there's a lot of concern expressed throughout the Executive Order about all of those being enhanced by AI in terms of finding solutions that where you're using models, how do you handle those both domestically under this law and how do we direct agencies to help keep us safe from adversaries that might not respect that?

Yeah, there were two things that I was seeing in the kind of news and commentary on this that were standing out. One we've already talked about, which is related to the requirement for the quote, most powerful AI systems to share their safety test results. The other one that stood out, or it seemed to stand out to many people, was the protections that were put in place for establishing ways to detect and label AI generated content. So this would be images that are generated from text to image, or text to video systems or, or audio that's maybe synthesized, which is continually getting better, or voice clones, that sort of thing. Or also text. Text would fall into this category too, around misinformation and that sort of thing that you might want to filter out. Or maybe if you're one of those teachers that want to prevent your students from using ChatGPT to generate their essay, then using or finding ways to detect AI generated content and enforcing those in certain context that's kind of my general reading of this stuff and I think what I was seeing was there's a good bit of positive response to this even from many in the AI community that recognize, yeah, this is an important piece of what we will need to do moving into the future in terms of having to label things and needing to be able to discriminate between these things. But also the recognition from those in the AI community that this is still very much a topic of research which is not figured out yet.

Totally. And I think that's one of the. It's interesting. I think that that will be a big impact because that will affect so many industries that are not necessarily ready for, for, you know, they've kind of said, ah, we can make some money, we can generate content. And we've all been seeing that online, but it's coming from industries where they have not had the burden of responsibility for it. I think certainly all of us that are in the AI world have used different models to generate texts and stuff. And you know, it started the first time as kind of cool, but then you realize, wow, this is an amazing business capability, but now it's an amazing business capability with a fairly significant responsibility attached to it. It will be interesting, you know, things like the marketing and branding industries, which I, once upon a time I was in, will have to figure out a way to do that and still serve their clients in that particular industry. Because if you just have everything as AI generated content that will affect how people perceive the content you just generated trying to satisfy your client, you know, in that particular industry. So there's a lot of nuance that's very industry specific that's going to have to happen for that.

Yeah. And I hope that many out there recognize that we need to figure out ways to label this generated content and track it, even if it's only for practical purposes of like, hey, more and more of this content is going to get out there. And I don't want to necessarily always be training my next AI system on AI generated content. Maybe I want want human content, but there's a recognition that it is an active area of research and there's also a gray area here. Right. So If I have ChatGPT write me a cool blog post and then I take that out and I modify a few things and then I put the paragraph back in and have it rephrase, and then I take it out and then I edit some more things. This is a very often a very dynamic process and I think for safety and security and trustworthy AI systems, we would want that kind of back and forth with a human. But it's not always the scenario where it's simply human generated content or it's simply AI generated content. This does get very mushy even in automated systems where there's humans post editing machine translations or there's humans reviewing analysis that's been generated out of a SQL table or. I don't know, there's all sorts of scenarios here where there's a lot of gray area and maybe that's not the focus of this statement. It might be more these scenarios where you'd want to essentially create a factory of misinformation that's just pumping out things to Twitter or X and, and that's maybe more within what they're talking about.

I think that working through all those nuances in all these different industries, and I do the same thing, I write a lot of stuff and I'll write and I'll put what I've written into one or more models and I'll see what it comes back with and I'll choose and I'll take part of this and part of that. I think a lot of people are doing that. I think that this is going to have to. A lot of this will be settled through litigation. So I think the executive order has given a tremendous boost to the AI litigation industry that has been flowering over the last few years. I think we'll see far more of these nuanced cases, these gray areas decided in court in the next few years. I have mixed feelings about the fact that it is beyond ability to handle all these cases, given the short timelines. I'm glad to see short timelines instead of many years to get there, especially if they're unfunded. It will be interesting to see kind of what they come up with. If you're a department head and you have 90 days to come up with a solution that the executive order requires of you, you probably will not have solutions for all of these things. So we have some interesting times ahead of us, certainly.

Yeah. And I hope that there's involvement from leaders in this space, large and small. So smaller companies that are really innovating in some of these things and larger kind of staples of the industry like hugging face and others that would pour into those things. But all of that will require some sort of, I mean, minimal exchange of money, even if it's just to buy people's time to spend on this because there's so many things to work on.

It'll be really interesting now that the burden has been placed on American agencies and by extension the American people in their industries to comply with all these things. It'll be interesting to see. You know, we talked at the beginning about the Bletchley Declaration and that intent and all of these other countries will presumably come out with their own versions of this and some will be very similar, some may, may branch out in different ways based on the values and laws of their own countries. But it will be to see how this works out. And there will also be some countries that refuse to subscribe to this, this whatsoever. Not only will they not contribute to this, they may be working very specifically against it and will have to in turn will have to have very good capabilities for detecting when any of these cases of that are within this purview of AI safety and security are being violated by others to an effect that is not good for us. It's late 2023, I suspect through the end of the decade will just be absolutely fascinating on how we start sorting through these issues.

Yeah. And to maybe end on a slightly positive note for those of us that are working day to day in this industry, we are the developers of some of these AI systems. We could look at this and say, oh, there's all these various intricacies and such that need to be worked out. But I do think that there's encouragement here in the sense that, that hey, some kind of general guidance firming up of standards help in kind of understanding how we might behaviorally test or red team or assess the risk associated with our models. I think that's a really encouraging thing in many respects, especially for I think the vast number of AI developers out there that do actually want their systems to be safe, secure and trustworthy. Yes, there's a likely minority of developers out there that are trying to be nefarious and malicious even in what they're doing with AI, as there always be with any sort of technology. But I think most of us want to build safe and secure and trustworthy AI systems. And even if you're doing really good in one of those categories, like you've got your red teaming down, there may be other things that come out through these processes with Nest or the watermarking, tooling or other things that it's hard to be an expert in all those things. So hopefully as more of this rolls into action, there is money put behind some of it to not only put guardrails around what we can and can't do, which might be how some people might take this, but actually to give us tools that will enable us to do more because we know that we're following good practices and best practices and we're being safe and secure. And of course, yeah, there'll always be a need for research beyond that. But yeah, I think it's encouraging in that sense.

I totally second everything that you just said. This is an opportunity. There are huge business opportunities in helping people get through regulatory, and we see that in other industries. So this has come about, we're hitting regulation and AI for real. Every other time regulation has come out, there's been whole industries born that helped get through that and services that make it much easier than it seems today as we were first reading what is to come. So I also would encourage everyone to try to embrace it. We do need it for safety. The dangers are real and let's do it for ourselves, our children and our larger community. Community. So absolutely. Let's. Let's go make this thing a good thing.

Yeah. All right, Chris, that's a great way to end and look forward to talking to you more in the future weeks about increasingly safe, secure and trustworthy AI.

Absolutely.

If you enjoy the music you hear on Practical AI, you'll be happy to know we've released two full length albums for purchase or streaming. Just search for Changelog Beats in your music app of choice and check them out. Volume 0 is called Theme Songs and it includes special remixes in addition to the classics. And our first volume is called Next Level, featuring many of the video game inspired tracks you've heard on changelog podcasts over the years. Check us out Changelog Beats thanks once again to our partners fasty.com flyio and typesense.org that's all for now, but we'll be back with more Practical AI goodness next week.