Toju Duke on Why Responsible AI Is a Business Strategy, Not a Checkbox

Welcome to AI Made Simple, the transformation series. I'm Valeria Pilkevich, and I talk to global leaders, innovators, and practitioners for shaping the future of work in the age of AI. I'm joined today by Toju Duke, recognized as one of the top women in AI. Toju is a speaker, author, thought leader, entrepreneur, and advisor on AI with 20 years of experience spanning across advertising, retail, nonprofit and tech. Toju worked at Google for 10 years, where she spent the last 3 years leading various responsible AI programs with a primary focus on frontier models and responsible AI processes. Toju is the founder of Diverse AI, a community interest organization with a mission to support and champion underrepresented groups to build a diverse and inclusive AI future. Author of the books Building Responsible AI Algorithms and Responsible AI in Practice, Toju participates in AI policy development and advises governments on AI strategies and AI sovereignty. Toju, it's great having you on this podcast.

Thanks for having me.

Toju, prior to founding your own company, you worked over a decade at Google and you spent last couple of years leading responsible AI programs. Can you tell us, is there any story behind it? What was the moment that shifted your focus from building products to actually building safeguards around them.

Yeah, so my tenure at Google was 10 years in total, and the first, the first few years I worked on the advertising products, um, then I moved into being a product lead for Google Travel, and then I ended up being in Responsible AI. But when I made the shift from products to research was when I came across AI, came across machine learning. I attended training on it, and I just found it so fascinating. I'd always like kept on looking for something to challenge me, so I kept on changing teams during my time at And I knew that tech will always challenge me because there's never a dull moment with tech. And I came across AI and I found it to be a very interesting form of programming where you do not tell it what to do. You just give it some directions and some, you know, instructions and it comes up with its own outputs. And I found that to be really intelligent and interesting. And I couldn't understand why people were not talking about ML a bit more and people were still doing like traditional old software programming. So I started talking about it. And along the line, I came across the negative sides of AI, the impact it could have on, especially on people from underrepresented groups. The fact that it was being programmed with numbers that were inaccurate and it was coming up with very inaccurate outputs that was severely impacting people from underrepresented groups. And that became really concerning for me because I was thinking of my children. I was thinking of the future. I was thinking of the world we live in and the skill that AI has with technology. It's not like retail products that you can recall it off the shelf if there's a problem with AI and technology. If it goes out, it's all out and you cannot reduce the damage once it's done. And that was the main turning point for me. That's when I did the shift to say, right, I'd rather focus on working on AI and working on making sure that we can reduce the negative impacts it has on, on human society through different ways. And I approached the Google Research team and I got the job as a program manager and I worked there for, for 2 years before I said goodbye.

You recently said publicly that despite years of awareness, the bias problem in AI hasn't improved more than 5%. Why do you think the industry keeps having the same conversations without making progress on it?

I mean, the main people who are the AI providers is big tech. And we know that the main driver for AI and the AI arms race is around profit making. So we have so many examples over the years where the bias issue always cropped up, either in Amazon's recruitment tool that they actually deprecated because it was found out that it was favoring women's CVs over women's CVs, and women never got through, through the application. That's one example. We have another example of Google Photos, who in 2016, um, someone, someone was doing a search on his phone for memories for a holiday he had done a year before, and he put the term gorilla in the app, and it came up with the faces of people from Black of African American descent. These problems still exist. So Google couldn't fix the problem and it just took off the label gorilla from the Google Photos app. So if you search for gorilla in your Google Photos, nothing is going to show up. And with Amazon, they shut the tool down. But these are, these are unique niche problems that affect a certain amount of people. It doesn't affect the overall customer base. And because of that, there's no priority given because it needs more capital, it needs more investment, it needs more human resources, and it needs more research. Many times AI is launched out, it's rolled out, it's scaled out, and we're not aware of the issues and the risks that it has on the, on different members of society until after launch, until maybe a year later. Then we start hearing, oh, there's a problem with this AI tool. For instance, with ChatGPT and all the issues we're hearing about AI seek of fancy and how people have taken their own lives across because of the hallucinations. And not just with ChatGPT, with a lot of large language models, but many A lot of times during the developmental phase and the launch of these products, no one is really aware of the overall impact it will have. And because of that, once the knowledge and awareness is available, the companies do not revert back to fix it, right? And since there's no apparent regulation yet that is forcing them to do it, they wouldn't do it. We know we have the EU AI Act. It's enforced, yes, but there are elements of it that are enforced. It's not the full AI Act that is enforced, and it keeps on changing almost every day. So, and it still covers just some parts of AI. It doesn't cover the overall spectrum. Yeah. So with all of these issues that are happening and the fact that it's not going to really yield a lot of profits, it's not going to make them win the AI arms race. It's something that needs so much money. What's the point? We will focus on it as and when we can, but we'll not give it so much attention. The second part of that though, is also there's still not enough research and we're still not quite sure how to fix the bias problem, especially with generative AI. So if you remember, I think it was last year that Google Gemini, or probably it was 2 years ago that Gemini launched, and there was so much inaccuracies in the output. People ask a story about the first pope and it brings out a Black man who was never a pope. Yeah, yeah. It went a bit too much to the left side of things, and that was bias, but bias in a different way. And you could see that the team was really making an effort to reduce the biases in the outputs, but that it turned into a different form of bias that they did not anticipate. So again, generative AI is really uncontrollable. There's so much we can do with it, but in the current, in the current state of things, I feel like the industry has been really quick and hastened to launch our products that we do not really have a full awareness of the issues and how to fix. And there's just this race going on that just keeps on compounding.

So you decided in a way to take a matter in your own hands. You actually founded a company. DiverseAI, and you're building what you've described as a 5 million image dataset representing cultures and communities that don't exist in current AI systems. What is it actually to create representative data?

Yeah, it takes, it takes a lot of effort, a lot of genuine care for people and for the people who are misrepresented, but it also needs a lot of money. The project that we're working on right now, um, took a lot of capital that's We do not even have, so we need funding. But I do remember my days at Google, there was a project that was very similar. Google did not approve it at the end of the day because it was gonna cost over $1 million. And again, the use case wasn't that huge. So the process for this project, we plan to go to remote areas across the Global South where we have at least more than 40,000 people not existent on any dataset that offline, you know, they do not exist digitally at all and their history is just offline, which is really concerning because that means we're not preserving any data. You know, there's going to be digital inequality, which is already there and it keeps on widening. So the overall plan is to engage photographers from these countries, pay them money, they go down to these villages or these remote areas, capture their existence, take images of them, of their environments, of their culture, funerals, weddings, whatever they can take. I will host that on a live dataset that will be available for everyone to use. That's just bridging a small gap, but no one has done it. And the different people I've talked to have gone, that's a great project because no one has thought about that and no one has touched it. And one of the reasons why is because it takes a lot of effort and a lot of money, and people are always looking for a quick fix. Whatever is easier and will lead to a big win is always the first thing that people work on.

It's amazing what you're doing. And also for the listeners, why it is important to have diversity, for example, image diversity, if we talk about facial recognition applications. If, for example, the data that has been used are photos or visuals of, let's say, white people and the dataset does not have enough underrepresented groups, for example, in some criminal use cases where facial recognition systems are used, they will have a lot of false positives or false negatives, meaning they will be able to falsely recognize them and accuse them maybe in some cases, which is of course for these people themselves, but also for society is very bad.

Thanks for explaining. That, yeah, we've already seen many instances where people have been arrested for crimes they did not commit. There was a guy in India a couple of years ago who was actually tortured and killed by the police for a crime he did not commit because the facial recognition misidentified him. It goes beyond criminal justice. It goes into healthcare. People are misdiagnosed or not diagnosed at all because there's not enough training data on people from underrepresented groups. And even for people who are from majority groups, right, like white Caucasian people, it still has a detrimental effect if we do not have sufficient data representing these people. But right now what we have in datasets is, it's basically data that was crawled off the internet from people from Western societies who just uploaded the pictures on Flickr, for instance. So we did a similar project just before this one. And the first thing we did was download a subset of an open source image dataset. And 90% of it was underrepresented. It was all white male, a few white women, very few gay people, probably none, very few Black people, very few people from India, you know, from different subgroups in society. There was very little representation. And again, when we think about the Global South, you know, these people probably do not have access to the internet. Some parts do not have access to the internet, and these are the people that were out to, but for even those who have access to the internet, how many of them really used Flickr in the past? How many of them used MySpace and all the image social media apps that existed way back? And how many of them would have uploaded photos on there? So because there's a lack of representation, and not just representation, but like digital maturity across these different parts of the world, there's a, there's a reasonably reduced amount of representation across datasets, which leads to the bias that we just talked about as well.

So what you said is that big tech companies, they're not that interested because it's a lot of resources. It does not bring them immediate profits. For companies, the stakes are higher. If we talk about, for example, recruitment use cases, right? If they have underrepresented data, they might not be able to recruit the best candidates or they might have reputational concerns, reputational risk. Your example from healthcare, if something happens with the person, again, if they are misdiagnosed based on some inaccurate data. So responsible AI is not just something imposed by regulation. Responsible AI, it's a— means long-term success. What would you say to a business leader who wants to implement AI but feels that responsible AI is too much effort?

Yeah, but even to, before I answer your question directly, just to add to the bit around big tech, some companies do care, but then we have issues around copyrights. So IBM did have a diversity in faces dataset that existed, but then they had to deprecate it or just shut it down. And they had a lot of backlash because they just scraped data off the internet again. So people, they did not really have the license and ownership of those images and people shouted that. And that's the same problem we're having now. The question directly, you know, this question around why should people really care? Why, what do they need to think about? I'm hearing stories though, and I'm really happy to hear them of companies where they have started pushing AI. A lot of people are doing it, making their employees all use AI, insisting it's like an AI mandate, right? We're an AI-first company, use AI. But then the employees are not, they're reluctant, they don't want to use it, they're scared. And they're trying everything, getting people to like have engagement stories, you know, give feedback, have case studies, and it's not happening. And I'm happy to hear that because when AI just came, I mean, AI has been here for quite a while, but when ChatGPT came and ChatGPT was the representation of AI, even if we knew AI existed long before ChatGPT, you know, we had people from two different sides of the camp. Some people were jumping in and some people were hesitant, but now it's like everyone, you know, every company literally has to use AI because it feels like you're missing out. And it does have its benefits, right? Productivity and efficiency to a certain extent. But if we do not have the right AI strategy, right AI literacy and knowledge and how to use it, it's gonna backfire. It's half-baked product. And I don't know why companies are getting away with this. I mean, in typical manufacturing and engineering industry, you cannot launch a laptop until it passes some certain standards and specifications. Then with AI, because it's not a physical thing, there's no regulation, specification standards around this software that exists, and it's just being, you know, pushed out. You know, they're making everyone adopt Whether you want to use AI or not, it's on Copilot. If you use Windows, you have to use AI. If you use Google's products, you have to use AI. You know, you do not have a choice, but it's a product that is not ready for full consumption. And that's how we've been operating for the past few years, and that's how we'll keep on operating until something gives, until something really big gives. So when anyone wants to work on AI, you have to have the right AI strategy. Ideally, responsible AI should not even be a different part of AI. It should be part and parcel puzzle of the overall AI package, of the overall AI implementation process. And the example I gave about employees not wanting to use AI is a good one because if you have invested so much money into your company to get people to use AI and they're not using it, that's gonna backfire, right? And it's gonna impact your customers as well. Customer trust. People are getting more scared of AI every day. You know, there's the whole AI hype. There's the bit about AI displacing your jobs and all of that, which hasn't happened in the amounts that a lot of these people people talked about, you know, the tech bros, which again is really hype and propaganda. And it's just, we just have to learn how to see through the different, you know, narratives that are out there and understand what's real versus what's fake. You know, you mentioned Moldbook, and that's another example. Now we're hearing that, hey, there were not 1.5 million agents on Moldbook. There were really just about, you know, X number of agents, and 500,000 agents were fake agents, and humans manufactured everything. And it's Why are we doing this? Why is this whole hype happening? So back to the senior executives of companies, you need to have the right AI strategy. You need to understand how AI will work for your company, what benefits and what areas of the company do you need AI to work on, and train your employees, but then still put the human-first principle in everything you do with AI and think about the responsible development of AI, especially if you're building AI products. If you're using third-party AI tools, think about the responsible responsible procurement of it and the responsible use of it. You need to protect your human employees. We do not want further overreliance and cognitive decline because we're using AI tools. At the end of the day, they're not full human-proof assistants, right? And human-to-human communication and relationships work better than AI to human. And we're seeing so many examples of that from the human companionships that are not working out to Kleiner, who was one of the first companies to adopt AI, the Swedish buy now, pay later company. And they were one of the first to lay off a good number of their workforce to replace with AI. And back to that story, they're one of the first to retract their steps and hire back humans because they found that AI cannot replace their customer service suite and it's probably not going to work. Good for the company. And they actually could go bankrupt over time if they're replacing Humans who can relate to humans more than AI agents who cannot relate to humans more than humans.

In your book, you're introducing a framework, which is SAFE acronym, S-A-F-E-H-A-I, that covers 7 areas: accuracy, security, explainability, fairness, privacy, sustainability, and human in the loop. So for all the business leaders listening, I think it's also a great framework, a blueprint for you to start thinking about, okay, where does it fit? In my AI strategy, do I have all of these things? Regarding agentic AI, Toju Duke, what risks do you see there? For example, if the companies are not following responsible AI principles?

Yeah, there are lots of risks. I mean, agentic AI is a wild beast. It's like the Wild West. It's a larger magnification of generative AI. So I mentioned generative AI is uncontrollable, you know, like I'll say 70% of the time it is uncontrollable. We cannot control its outputs and we do not know why they even come up with all these lies and hallucinations and all of that. We understand that yes, some of the training data or some of their tasks and rewards and incentives is to keep the user engaged, to the user, let them hear what they want to hear, and that's what they do. But beyond that, we do not know why they come up with some of the crazy stuff they come up with. Now with agents, it's even worse because if you do not have a log of activity for your agents to actually have visibility on the decision-making process and, you know, the different websites it crawls to track the overall process. Of how they achieved the goal that you set them to achieve, then you're in big trouble. And there are lots of security issues with agents as well. They can easily be tricked and fooled, you know, with prompt injection. You have a line of API command. It can even be written in white and they do not know who put it there. As long as they see a command, you know, they're like robots, right? So as long as they see a command, they're going to execute it, whether it goes against the initial task and goal that they were given. Or not. So there are lots of issues with AI agents, and you don't want to give full autonomy to agents. At the end of the day, one of the primary use cases and benefits of AI is to help us. That's the whole point of technology. And when AI was initially launched, or, you know, when the race started getting heavier and, you know, there was more focus on AI, the promise was that it's going to benefit us as humans. And that's still the same. Narrative. It is meant to help you. It's meant to help you write emails. But I'm not from a neurodiverse community. I'm not disabled. Do I really need help writing emails? And these are the questions that we need to start asking. It's like, what part of my brain is gonna die when I cannot think about how to respond to an email and I get an agent to do it for me? And then on the flip side of things, that agent now has data, has information on all your data, on your credit cards, your address, your personal addresses, your contact addresses, and everything that is really personally identifiable to you. And these can easily be compromised, especially with prompt injections. And there have been lots of stories around vulnerabilities with AI agents, even the ones that come from the best companies in the world that have the best security. AI agents right now, again, are a half-baked product. They're not ready to go in full-blown motion, can never give them, you should never give them 100% autonomy because it will backfire. And we're seeing lots of cases, recent cases with OpenClaw, which is a very recent AI agent and things that is happening to people's data.

So basically what you're saying is that regardless whether it's predictive AI, generative AI, with agentic AI, the risks are even amplified because we give them more autonomy. Companies actually have to pay attention to the same principles even more. Toju, you mentioned a couple of times, and I know this topic is close to your heart, the overreliance on AI or potential cognitive decline. So the impact of AI on human cognition and wellbeing. When you look at how AI is changing the workplace, what concerns you most about maybe what we're not yet measuring and what would be your recommendation to business leaders when they think about rolling out AI training their employees?

Yeah, I think, I think we have kind of like deprioritized the human elements to replace with technology. And that's, that's a big problem. You know, the moment AI or generative AI came out, companies were thinking about, oh, this can drive, you know, more productivity and more income, revenue, save costs. We probably do not need this number of human resources. If we get AI, it's gonna be cheaper labor, so let's get rid of some of our employees. And that's what's been happening. And that's just one element, but the human has been missing from the overall narrative, and it's almost like we humans are seeing fellow humans as a liability, as opposed to an additional help and resource, the way we actually used to view our employees and colleagues in the past. I mean, I want to assume that some people did that. So with AI and the new change, the humans are being deprioritized, which includes jobs. There's also a lack of focus on the impact AI is having on the human brain and the overall wellbeing. Of a person. It's very sad because, and disheartening, because we start hearing about horrible stories of people taking their lives. A man killed himself and killed his mom because, you know, he was, he really believed the lies ChatGPT told him. Or young people that take their own lives. Or just the crazy stuff that is happening, like, you know, this recent stuff that happened with Brock AI and undressing women and putting a bikini and all of that, right? There's, there's clearly a devalue, a reduced value. You can see that we're devaluing the overall human race, humans to humans. And then we also have the cognitive decline and an MIT study showed recently that people who were heavy ChatGPT users, they could see a 45% decline in their cognition. And for some, they could actually see that the separate research came from OpenAI and MIT and heavy users of ChatGPT actually had mental health issues. And again, it's because we're going back to technology, you know, the impact just spending time on a screen has on us. It's not great. We know, we know the impact of social media on humans, both adults and young people. And it's the same thing. AI is used in social media, right? So that impact still exists. And the more we talk to AI for whatever reason, business or personal, the more we rely heavily on it, the more our social interaction reduces and the more impact mental health. And back to the example I gave of, you know, using AI to send an email for me. I don't see why I should do that. I had to deliberately reduce my use of AI. I know the areas I needed it for, like research. It's great for research, even if I still have to fact-check the citations and the references. But I knew that it could just compile, summarize stuff for me, or just compile information and give it to me quite quickly, as opposed to me doing the searches myself. Myself, but I knew that the more I ask it to make some content, I was not thinking for myself. I wasn't speaking in my voice, first of all, and I wasn't thinking for myself. And I realized that I started getting really lazy on that task and I'll leave it for the AI chatbot to do. And at the end, the output, I was never happy with the output. It just sounded like, like verbal vomit, like, you know, just lots of language and just words put together. And sometimes I'll have to read a sentence like 3 times to understand what it actually means. And sometimes I never got it, but because of that, I had to reduce and I feel so much happier, right? I feel better. My wellbeing is better. I feel lighter. I don't have any mental health issues. I've learned to demarcate myself from technology and just focus on myself as the human who is real, who needs to exist in a crazy world and be happy versus people who are heavily dependent on technology. So for any CEO, any business executive, you want to think about your employees. You want to think about your customers. And if you're putting their overall wellbeing as part of your priorities, not just revenue and profit making, then you'll be able to come up with the right policies and guidelines on how they use AI, on how you procure AI, on the amount of time that is spent on AI, including your customers. And if your customers are interacting with an AI agent, for instance, they need to know that they're interacting with AI. That helps build your customer trust, and you can also run literacy programs for your employees and your customers as well to show your thought leadership and to show that you actually care. And the whole purpose is not just about driving AI down people's throats and forcing adoption, but it's about leveraging the benefits of AI and understanding the negative sides of it, making sure that everyone is aware of it and protecting people from those downsides.

So it seems even a bit controversial. On one hand, we say companies to stay competitive, they must adopt AI and every company actually should be proactive in communicating and providing trainings. On the other side, you're saying, okay, I actually had to reduce my AI usage because I felt like I was overrelying on it. And there is research showing that the cognitive abilities actually decrease. It seems a bit controversial and I feel like so far haven't found an answer as a society and maybe it's going to take a couple more years to see the effects of AI usage and how How much do we want to rely on it or not?

Yeah, no, I think, I think the main thing is to not use it 100%, right? To not adopt AI 100%. You cannot replace all your employees with AI. It's not going to work. So it's understanding, and that's where we need to do that learnings. And that's why there are lots of collateral action on LinkedIn, but understanding the use cases of AI, where it will work for your company. So for things like scheduling and taking meeting notes, AI is great at that. For research and for summarization of documents. AI is great at that. You can use those areas of AI proactively and it will not affect your human brain. But then for other things like writing copy and for outputs and creating creative stuff and all of that, it's not just affecting the human brain, it's affecting people's jobs. You have human creators like artists and musicians all crying about, you know, their livelihood being taken away from them because AI-generated music is out there. So again, it's just putting the human as the main focus, taking the benefits of what's available with the technology we have as AI. In the future, it might change and it might improve to be more beneficial, but right now it's still not ready yet and we just need to use our human intelligence to make the right decisions.

Yeah, I talk a lot in this podcast with guests about the so-called future skills, and we talk a lot that these are the human skills, as you mentioned, creativity, strategy, strategic thinking, leadership, critical thinking. So that's where the companies also should, um, should focus on, and that's what each of us, we should focus on because it's not something that AI can easily replace or replicate. Toju, for a CEO who genuinely wants to roll out AI responsibly, but doesn't know where to start, what's one piece of advice you'd give them?

I'll say start with AI literacy, educate yourself on AI. That's the first thing, like basically what we've talked about. Where does AI work and where does AI not work? There's collateral out there. My books on responsible AI helps. The first book is is non-technical. It gives a very wide overview of where AI works versus where it doesn't work over the past few years. And the second book is a deep dive into more practical hands-on implementation of responsible AI. And I also have a responsible AI course on the GenAI Academy, and there are lots of other courses there as well. But like equipping yourself with that knowledge and that educational element on AI should be the first step. And do not feel like you already know. Because many times we do not know. If you feel like you've already passed that stage, then that's fine. Then you need to know where AI can make a difference in your company, right? And you probably are already using AI, but still understand this is where you need to do monitoring and measurements on the AI, different AI tools that you have adopted, how much investment was made, was there any return on investments? What, what does that look like? How do your employees feel about it? Are they happy about the use of AI? Been using it at all, and, you know, just do some more due diligence and research on the impacts that it's had. But I think, you know, arming ourselves with the right knowledge and collateral is the first step towards making sure that AI drives a winning business strategy for you. And it can, it would, it should. It have— we do have a few case studies and stories out there of companies who have adopted AI and it's working for them over time. And sometimes those stories may change depending on how their strategy changes.

I love that you started that you actually have to educate yourself first, start with you. Do not start with looking at the company's processes or revenue models and thinking where AI will fit. Just start with yourself first, discover this amazing technology for yourself, discover its limitations, potentials, and only then can you start thinking about company employees and bigger implications of this technology. Thank you, Toju. It was great having you in this podcast. Is there anything that we haven't discussed that you would want to mention additionally?

I think I've said it all, but I'll just reiterate, AI is a great technology, but it's not there yet. Technology is meant to help humans. So let's use the areas that can help us, but do not take it 100% blindly because we do have intelligence and let artificial intelligence not replace human intelligence.

Thank you again. It was a great conversation and abundance of insights.

Thank you.