EP12 - Why data rights will be the new civil rights

Hello everyone and welcome to the Tech Journal. My name is Marc Vermenam and I am the digital speaker in this series. I project my digital twin into cyberspace to bring you the latest and greatest from the digital world. I cover all the latest digital news from blockchain and crypto to quantum computing and AI. And I always try to take it a step further and and look into what these digital innovations mean for our personal and professional lives. In today's show we have Dan Turchin sharing his insights with us on the hype of today AI, but especially AI ethics. Dan Turchin is the co founder and CEO of PeopleReign, the leading AI platform for employee service automation. And prior to PeopleRain, Dan was the CEO of AIOps Leader Insight Finder. He has also co founded Acetown, an AI first enterprise platform for HR and IT. And he was involved with DevOps leader BigPanda and security analytics company Accelops. The common thread in his career is the future of work and how we can use AI to create better organizations. It's a pleasure to have you on the show. Welcome Dan.

Dr. Mark, pleasure to be here. Thanks for having me.

Is there anything else that you want to add that I forgot to mention in your introduction?

Today we're going to talk about my life's passion. Like you said, it's thinking about how to be human first in an era of technology and really thinking about how to use AI technology, some really advanced technologies, but first, always with a focus on how can we augment the intelligence of humans rather than replace humans. All these topics that we're going to discuss today, some of which we started the discussion about on AI and the future of work. My podcast a few weeks back, really eager to start to unpack some of them and get your feedback as well.

Sounds good. And I'm really super excited to talk to you. And again, as you mentioned once more, because I was just a guest on your podcast and I will include the link to your show AI and the Future of Work in the bio below. And we had a great conversation. We impact a lot of the AI and this time we'll continue the conversation and focus on, as you mentioned already, an important topic, the ethical side, the human side of AI. This topic is very important to me. It was part of my PhD a few years ago and also the recent Netflix documentary that we have seen has put a lot more new attention on this crucial topic. So I'd like to dive straight in if that's okay with you. And I'm very curious to hear what is AI ethics to you and why is it so important?

Yeah. So, Mark, we're having this conversation at a fascinating time. I think most of us would agree that the world of work will change more, say, in the next decade than it has probably in the last 250 years since the first industrial revolution in Europe. But it is absolutely the case that there's never been a time in history when so many critical decisions have been made on our behalf with so little transparency. I'd say every day, big life decisions are made by algorithms, sometimes without our knowledge. And we're very infrequently informed about how they're made. And those are things like who gets a loan and who gets a job and who gets a discounted insurance premium. And even as we all know who gets on an airplane. I'd say algorithms can predict risk based on data that contains information about historical decisions and the results of those decisions.

Decisions.

But applying the field of ethics to AI requires a little bit of a nuanced thinking about ethics now, kind of just to get some definitions out. Ethics is really the set of moral principles that govern a person's behavior. But as AI increasingly makes human like decisions, it also needs a set of moral principles. AI ethics. You know, the way I think about it, it's really that framework that we use to communicate when automated decisions are made, to understand how they're made, to evaluate the implications if they're made based on bad data, and know when they were made using unbiased inputs and potentially flawed algorithms. Now, I'd say AI is great at telling us what is, let's say, popular. It's terrible at knowing what is fair. We need to enforce a strict code of ethics on AI before we can expect it to really improve the human condition. That's what AI ethics in this emerging field is to me.

Yeah, well, and I think it immediately shows also the difficulty in it, because when we talk about human ethics, it's already very difficult. What is good, what is bad, what's good for you? Maybe it's not good for me and vice versa. And ethics is a very difficult topic. But then we move to AI and we have now AI agents who are autonomous and can almost think for themselves, make decisions autonomously. What is then good and bad. That's what makes this field so tricky, because for us humans, it's already difficult, let alone to decide what means what good and bad is for AI. How do you look at that perspective to determine what is good and bad for AI?

Well, first I think we imbue AI with a level of intelligence or you know, we think of it often as a sentient being because of what we've seen in science fiction movies go back to the 1960s and it was Isaac Asimov that first proposed three rules for robots. And ever since then the science fiction community has convinced us that these are beings that are much closer to being like humans. In fact, as is a great, a great one for the show notes. But Janelle Shane is a, she calls herself an AI humorist, wrote a book called I think you are a thing and I love you and. And it's all about the ways that we can trick AI into being quite silly. So I think it's first important that as a, as a human society we appreciate that we are responsible for applying a set of ethical guidelines to the AI that we develop. And we need to remember that AI in its current state is only really intelligent in very narrow domains when it comes to kind of AGI, artificial general intelligence. AI is about, has about the brain power of an earthworm. So we need to not get too carried away and make sure that we kind of rein in the power of AI by realizing that it's essentially math plus data. And so there's a big conceptual leap between math plus data and sentient human beings. And hence the reason why I think it's incumbent on us in the AI community to really give structure to some of this kind of ethical framework.

Yeah, but despite that, there's a lot big gap between sentient AI and math and data. There are plenty of examples already of where AI or algorithms have gone rogue. And I think that's why it is important that we need to move towards, we need to create a society where we have these guidelines, as you mentioned, of how to deal with it. And I think that's the interesting thing. What's happening here in Europe is that the EU recently announced this risk based approach to AI, apart from launching the AI guidelines that they did in 2019 as well. And so this risk based approach is basically defining AI into four risks. Unacceptable risks, which basically means AI systems that are considered a threat to safety, livelihood and the rights of people. And those will be banned. You have the high risk system, which is identified as critical infrastructure, law enforcement, administration of justice, etc. Where it's important that we do the right thing when it comes to AI and we need to ensure a high level of robustness, security and accuracy. Then the third one is the limited risk where the AI system requires specific transparency obligations such as chatbots that we know that we are interacting with the machine. And the lowest one is the mineral risk, where the framework proposes the free use of AI in applications such as games or spam filters, where they actually have a clear benefit to society. And I think this is a great first step forward to have this risk based approach. What do you think of this approach and what is the US doing in this regard?

So on AI and Future of Work, my podcast, you had introduced the concept, and I know Yuval, who are already Cohen has discussed it, of essentially the replacement of liberalism, kind of a democratic free flow exchange of ideas with digitalism. I love the concept and been kind of thinking about that since our conversation. And increasingly not so much in a cynical way, but in a very pragmatic way. I feel like increasingly it's the, the big seven tech companies, you know, Amazon, Google, Facebook, Microsoft, maybe I'd include Alibaba Tencent in China that are really functioning as governments. So that is, that is where decisions are made about what we think, what we see, what we, where we get health care advice. A lot of the things that were in the traditional domain of kind of democracies or governments are now things that are in the domain of this very small group of owners of data. I feel strongly that for the next century, data rights are really the new civil rights policy moves much slower than technology. And as a result, decisions that, you know, would historically be made in the halls of power, you know, by elected representatives, are increasingly made by, you know, seven global tech companies. And maybe that number expands contracts. But, you know, this idea of, you know, two points you've made, digitalism replacing liberalism, I think is a nice way to think about the responsibility that we have. I mean, you mentioned, you know, what's going on in the United States. Well, you know, most of us listening have probably seen the, the viral video of Mark Zuckerberg explaining how the Internet works to Congress in the United states back in 2018, you know, it's just a farce to think that those legislators will do anything to intelligently regulate innovation when they're competing with a profit motive from companies like Google and Facebook and a very poorly informed public. Now, just in terms of regulation that's pushing its way through federal and state legislative bodies in the United states. As of 2021, 16 states of our 50 have at least one bill going through the legislative review process. New Jersey in fact has seven small state, but it's actually leading the country. The federal Congress also has some bills. There's one that was recently introduced called the Advancing America AI Act. But I'D say all of these are, again, far behind where technology companies are. And the reason I say that is because oftentimes they are just in the process of creating ways to educate legislators. And by the time the legislators are sufficiently educated, the tech companies are three, four, five, seven years ahead. So I'd say governments, traditional government is at a big disadvantage here. I firmly believe we need to think differently about how we legislate or regulate AI. I'm in favor of, you know, not just calling tech executives for these, you know, circus acts to testify in front of Congress, but I propose something more like a decentralized approach to managing these automated decisions. I think that anyone who's potentially impacted by these automated decisions should be able to share their experiences in kind of an open forum. I propose using some kind of technological construct like a blockchain, whereby we all have greater visibility into what decisions were made, what data was used, how the decisions would be sensitive to changes in the data. Almost like in the United States, we have something called the fda, the Food and Drug Administration, that says whether or not food and drug products are safe for consumers. I think there needs to be a parallel framework only in a governed, in a decentralized way that has authority over when AI is being used responsibly to make decisions. To me, that's kind of the AI governance framework of the future. Yeah.

And ideally, of course, not that such a decentralized framework is not controlled and owned by the big tech, which is at the moment. You know, and the fact that these big tech, they have so much money and so much big lobbying power that they can basically create or cancel any legislation that they want because they just have the power to do so. And with a completely ignorant government, which I think is not only happening in America, it's also happening here in the Netherlands or in many countries where the people who are responsible for creating the legislation have zero understanding of what's going on. And to a certain extent, I can't really blame them, because one of my objectives and my task in my work is to remain up to date of all the changes that are happening in the technological space. And even I can't keep up almost because, you know, the world is changing so fast. So how can you expect legislators to keep up with what's happening if they also have to, you know, do all kinds of other stuff? And I think that that's sort of where the problem lies. And then, and then the question is, of course, do we want to rely on the legislator or do we want to have the market organizing this themselves. And I think, yeah, we live in a capitalist society and relying purely on the market will be challenging because AI offers so many advantages in terms of efficiency and making your organization more lean, et cetera, that it will be quite challenging to do so. But maybe the academic world might offer a solution there as well. And if we talk about the academics, during my PhD, the AI, and responsible AI in particular, was part of the research that I did. And it's often what they also say, that this is what we need to reign in AI and we need to create this responsible AI. But in your opinion, what is responsible AI and how can we create responsible AI?

I love that we're having this conversation and I wish that more people were considering what it means to practice AI responsibly. I think one of the things to keep in mind is that AI is at its foundations a human activity. Everything that AI does wrong can potentially be undone and done in the right way. Because humans control AI, we can't ever feel like we've ceded control to robot overlords. And so to me, the three principles of responsible AI are one, AI must be transparent. We need to know when automated decisions are made on our behalf. Two, it needs to be predictable. The same inputs should reliably generate the same outputs. And then to me, the third principle is, needs to be configurable. So when it's determined that AI is either making poor decisions because of biased data or because of issues with the algorithms, we, as those impacted by automated decisions, should be able to run our own sensitivity analyses to be able to figure out if the inputs were changed, how the outputs would change. Now, I feel strongly that all algorithms should be scored on a scorecard, almost like you'd score the hygiene of a restaurant. I think it should be very clear, you know, what weights are assigned to what data. It's easy to say that AI should behave, quote, fairly, which is a subjective term, and that it should be explainable. But I don't think that goes far enough in the real world. I think it's incumbent on us to define what it means to have responsible AI and go a step further that, and actually have frameworks that enforce the use of these, the, the adoption of these principles of responsible AI by not just the developers of the algorithms, but everyone that kind of hovers around the ecosystem of participating in these automated decisions.

Yeah, and I like the way that you mentioned that there are three criteria guidelines that responsibly I need to adhere to and actually want to add a fourth one, which I think is that AI needs to be able to explain itself, ideally in plain language, whether that's plain English or plain Dutch or whatever, and that it made a decision. Or when an error occurs, it can say, I made an error because of xyz, and if you go to code there and there you can find why I made this decision and you can change that. And ideally, you would want to embed this explainability within the AI systems of the future so that with AI all becoming black boxes, it becomes so difficult for us to understand, even for developers to understand, what's going on, that we need to rely almost on the AI to explain to us what is going wrong and why a certain decision has been made. And in addition to that, I think where we also need to move to is we are moving to a society where our financials are being done automatically, you know, so the accountancy firms of the world, they become less and less relevant because, you know, with the push of the button, we can get the full financial statement of a company. But these companies are very well equipped to move to a next phase of monitoring, meaning, you know, AI and doing the accountancy, not anymore on the finances, but on the AI, where ideally we would start with publicly listed companies who need to be have a stamp of approval of, you know, one of the big four accountancy firms that, yes, their AI systems, they are honest, they are transparent, they do what is what they say they do. And you can trust this AI. And only once they have that approval, just that they need an approval for their finances, then they are, you know, they can move ahead. And I think that's something that we, as a society, we should really structure, strive for, to move towards.

Good idea. Yeah. An audit system for AI based systems.

Yeah, especially because AI becomes so much more difficult to understand. And that was also what happened a few years ago for those who remember, Facebook killed an AI that had created its own language and that even developers no longer understood. And if I then look to a famous book by Nick Borstrom, his book Superintelligence from 2014, and if you want to be scared about the future, I can definitely recommend reading that book because it's sort of the holy grail in terms of AI books. And he discussed that there are several technical and organizational methods to prevent AI from going crazy. And one technical approach would be to implement such a kill switch as the Facebook developers had done. They just killed, killed the. The AI. What is your idea around this? And then do you think that, that such. Such technical measures would actually work if we move to an era where, although it's still far away, AI becomes more and more intelligent.

Superintelligence is. That's a good one by Bostrom. I would add to that list. I just read Artificial Unintelligence by Meredith Broussard, who I think is a professor at nyu, but also a good one. Slightly less cynical than Bostrom, but a good read. To your point. Yeah. So I do recall the. I think they were called Bob and Alice, the chatbots that Facebook introduced that started talking in their own language. And then a bit after that, there was Microsoft's Tay Bot. I'm sure we all remember that. So started spewing misogynistic and anti Semitic comments after about 12 hours because, you know, it turned out it was really easy to kind of weaponize the AI, to make it essentially parrot back, you know, whatever, whatever a user said. These are dangers, and these are things that seep into the popular press and they get a lot of coverage. Every technology company is or will soon have their, quote, Tay or their, their Bob and Alice moment if we're not very careful about how, how these technologies are deployed. And my fear, which, you know, is kind of what Bostrom covers and even Meredith Broussard, is that the implications could be a lot greater. It may not just be, you know, some somewhat innocuous, although, you know, hurtful remarks made by a chatbot that's in experimental mode. But when we start talking about flight paths and minefields and global warming, increasingly, these are all domains, let alone policing in urban areas. These are all domains that are being affected by AI. And before we're prepared to just devolve control to automated decision making, I think it's really important that we think about not just what can happen if the AI performs as it should, but. But it's really important that we always ask the question, what could go wrong? And until we really feel like the technology is a lot more mature and culturally we're more prepared based on having kind of adopted these ethical frameworks to kind of coexist alongside systems that make automated decisions. I think it's really important that we tread lightly and get really good at, you know, at thinking through, you know, call it, you know, a few very tangible things to prevent AI from going rogue. You know, I mentioned, you know, kind of ask what can go wrong. But I think it's also important that we understand and kind of introspect the data being fed AI to see what kind of bias is encoded in the data before we deploy a system again. Math plus data equals AI. Data is A really important component. I think we also need to get more comfortable involving more types of Personas or more types of users in the AI vetting process. This should not just be the domain of developers or data scientists. I think it's equally important, let's say, for finance or for marketing to be involved. If, you know, if you work at a bank that's using AI to make automated decisions about who gets a loan, I think it's really important that everyone who's part of that life cycle of the loan management process to be aware of how AI is being used to impact those, those end consumers of the bank. So I'd say those are three specific things that we could do better as we acknowledge the, you know, the power of AI and we, we either intentionally or unintentionally seed more really important decision making to AI.

I couldn't agree more. And I think you know that the recent Netflix documentary Coded Bias showed us, which was based on the very good book by Kathy o', Neill, Weapons of Math Destruction, showed that how many things can go wrong with AI. And I think that the main thing of course that we should not forget is that AI is always being developed by biased developers and is trained with biased data. No matter how good we try to remove the bias, there will always be some bias in there. And this documentary really shows some of the problems that we saw in terms of biased AI really causing havoc, chaos within organizations. Can you name some more examples that you have of AI really affecting? Apart from the Tay example that we see that of course was pretty, pretty bad. But some recent examples that you are aware of.

Yeah, so Code of Bias is a great example. So the documentary follows the work of Joy Buolamwini who was at MIT doing some work on facial recognition. She was kind of an artist first as well as a technologist and has since founded which caused the AJL or the Algorithmic Justice League to track a lot of these situations where, you know, AI has been used for, for and it has had unintended consequences. I think a few of the examples that are, that are notable are this goes back quite a few years, but the massive IBM Watson failure at the MD Anderson Cancer center where they literally spent $62 million to develop a Watson based system that could detect cancer. And it was subsequently determined that it was frequently misdiagnosed in cases and then it performed far worse than humans at cancer detection. Amazon famously stopped using AI to screen candidates for engineering jobs because it was disproportionately recommending white males because to no One's surprise. That's who had most, that's who had had the most success in Amazon engineering roles. Janelle Shane gives a funny example. The book I mentioned previously where a hedge fund used AI to do some selection and found that people named Jared that play lacrosse tend to be the most successful in in enrolls in a hedge fund. But you know, humor aside, I mean the implications are real. You know I mentioned who gets a job and things like that. But you know, these are just the examples that we hear in the popular press. You know I think Joy Bulomwini gives a lot of more interesting examples in fact Encoded Bias they talk about Big Brother Watch a an initiative in the UK where they were following vehicles that were doing on the street face detection to look for potential criminals in and around London. Well there's a famous example in the United States in Congress where they used Facebook's recognition to Facebook product technology to try to figure out, to try to match the faces of congressional leaders with the faces of those in a known criminal database. And it turned out 28 of the members of Congress, none of whom actually had a criminal record, were determined by Amazon Amazon recognition to match the mug shots of criminals. So you know, we start to get into some pretty smart, pretty deep water in terms of using AI in ways that violate our civil rights. And you know, this, this, you know, these are all of the kind of telltale signs of you know, kind of an authoritarian state that we need to be careful for. It's not imminent. But if as a technology community we don't really govern how these decisions are made, it's clear that regulatory bodies, those, you know, elected officials aren't really in a good position to make these decisions for us. So all the more reason why conversations like this are so critical to be having now.

Yeah, absolutely. And I think, you know, the, the examples that you mentioned show shows us that how what a long road we still have to go in order to create AI that is actually, you know, ensures that we, that is good for us, that we, we create AI that's beneficial to, to organizations, to society, to citizens, to customers, etc. So how would you, what kind of tips would you give to organizations but also to legislators how to regulate AI? What's the best approach to follow here?

Yeah, I'm going to go back in the Wayback Machine I referred to Isaac Asimov. I'll start with his three rules for robots which have we withstood the test of time? And then I've got a few other few other thoughts. I'D add to that Asimov. This is in a short story he published. I think this actually goes back to the 40s, but it was called Runaround Science Fiction Story. And he said one, robots may not injure humans. Makes sense. Two, robots must obey orders from humans, except where such orders would conflict with rule number one. And then his third rule is robots must protect their own existence as long as such protection doesn't conflict with rules one or two. So a bit of a high level summary, but, but, but a good framework, I think, to, to think about use of AI. I would add, I'll say, you know, two, to add some OB's, rules that are more kind of put a modern spin on that kind of framework. I'd say get educated, don't be complacent. You know, channel your inner joy. Buolamwini. You know, look to, you know, leaders in cities like San Francisco or Portland or Boston that have already banned the use of facial recognition by police. I think things like that are a good step forward, at least until we can verify that these automated decisions won't disproportionately impact underrepresented communities. So get educated, have a voice. I think another, you know, another actionable behavior I'd propose for all of us is vote with your data. So things like GDPR in Europe and we have similar laws that are proliferating in the United States make it easy for us to get access to data when we're concerned that it might be misused. Whenever you see organizations that are misusing your data or using it for questionable purposes, boycott. Those are organizations. Exercise your right to enforce the principles of responsible AI on not just those big tech, those big seven tech companies, but all of the tech companies that are looking to those big tech, big seven tech companies as role models.

Yeah, I think, I think there is, and I agree with you, there's a lot of, there is a responsibility also with us, you know, the consumers, the citizens, because how often we just don't read terms and conditions, accept whatever we do just to use an application. And I'm complacent to it as well. Everyone does it. So there is a real responsibility for us and we can indeed vote. Without data. We can't say, stop, this is enough, and we no longer use a particular service. Although that might be a bit challenging. But we've seen it recently happening as well with WhatsApp when Facebook decided to share the data with Facebook, which they always promised they would never do. And all of a sudden now they are doing that a lot of people did leave, but it's still difficult to leave because of the high barrier, the high switching cost that comes with the social network. And that's what Zuckerberg knows and that's what he counts on when making such a move. And that's a shame that we are moving to a society where these big tech companies are so powerful. So there's a real responsibility for us consumers to take action and to also force the legislators, as you mentioned correctly, to get educated and to start doing something and to pay attention to what's happening in the field of AI. Well, thanks Dan, for this. With this, it's a wrap. We could easily talk for, I think, hours and hours on this topic. There's so much happening. But I really would like to thank you for joining me on the show and I'm looking forward to continuing the conversation on another podcast in the near future.

Mark, always a pleasure. I think it's so critical that we're having this conversation and we involve more people in it. And I'm glad the digital speakers having the conversation and just would love to hear feedback from anyone listening and feel like we're all in an important position right now in a crucial, critical place in history where we can influence how technology gets applied for good. And looking forward to continuing the dialogue in the future.

Absolutely. Well, thank you so much for joining us today. I've been your host, Marc Verewan, the digital speaker, and this has been the Tech Journal. Do not forget to hit the like and subscribe buttons and I will see you next time. For your information, download Stay Digital SA.