Interview with Fairly.AI Co-Founders Fion Lee-Madan & David Van Bruwaen

My name is Justin Betts. I'm the content editor at the Founder Institute. I'm here this afternoon with Kyun Lee Madden and David Van Bruyn. They're the co-founders of Fairly AI, an FI Waterloo and Founder Institute Selects portfolio company that's offering streamlined processes and solutions to analyze AI data for biases and compliance. Kyun Lee Madden and David Van Bruyn, thank you so much for joining me today.

It's good to be here. Thank you.

Thanks for having us.

So before we dive in and start talking about your startup, Fairlead AI, I always like to ask first, can you tell us a little bit each in turn about yourself or your personal story? You know, what were your experiences, professional or personal, that led you to founding this cognitive sciences and AI compliance technology company?

So I went to school for a very long time studying philosophy. And then getting into natural language processing through my love of language and formal logic. And that took me to a path of startups, of all things. I started— I was teaching at University of Waterloo, and then I joined a local startup as a very early employee and developed technology for identifying issues of concern to parents around their children's social media activities. And it was a very interesting combination of technology and social and human concerns, and very much on the cutting edge of artificial intelligence and natural language processing. So it was amazing experience. And somewhere along the way, I became the CEO and ran it from a very different perspective of building a company rather than building a tool. And that brought me into A place where I wanted to start my own company to build out the next layer of artificial intelligence around the actual governance of the systems themselves. So, I mean, monitoring the children's behavior turned into monitoring the artificial intelligence that was monitoring other people making really high-impact decisions. So this is my background.

How about you, Fiona?

Yeah, for me, I grew up in the e-commerce platform space. I started my career as a full-stack developer right before the internet bubble burst around the 2000s. But I have been specializing in this technology called ATG, which got bought by Oracle for $1 billion in 2010. Throughout the last 18, 20 years, I was a full-stack developer, and then I went client-side, and then I actually went to work for ATG as their product marketing tech lead when it got bought by Oracle. After it got bought by Oracle, I went back to consulting, and then started my own consulting firm, relatively successful. But then there's something missing in my life. I did the whole traveling around the world thing, tried the early retirement thing, and then there There was one weekend when I was actually in town in Toronto, and Techstar did a startup weekend. It was one of those where you go in on a Friday, pitch on an idea, form a team, and pitch again on Sunday. That's what really got me into this whole startup journey. I was like, "Oh, maybe that's what's missing in my life." I started reading a lot more about startups, you know, "Breath feels, do more faster," and then that somehow led me to Founder Institute, and that's where I met David. So the definition I like to use for startup is to build a scalable business under extreme uncertainty. And yep, that has been my experience so far. And I'm glad I found a co-founder that is on this journey with me.

Wow. Such an interesting story, I think, of both of your technical backgrounds and the confluence of interest and how you came together through a startup weekend event. That's also really cool. They're like one of our partner organizations. So we're always promoting different startup weekends. It's really cool to meet founders who come out of such a weekend scenario and now are building a company together. So I wanna kind of start asking you more about that. I know because I'm somewhat familiar with your company as a Founder Institute portfolio company that you say Fairly is AI compliance in a box, but I wanna hear it straight from you, the founders. You please give us, one of you, whichever one wants to take the reins on this, your sort of short elevator pitch. What is Fairly AI?

Sure, I'll, I'll try to handle it. We're a platform for organizations to audit multiple AI systems that are used across all business units to eliminate bias, protect privacy, and to ensure transparency of automated decisions. So we have a philosophy where the people, process, and technology all need to be taken into a single view. And the information that needs to be managed around very large and complex technologies like machine learning needs to incorporate the humans who are making decisions at a micro decision level, as well as approving models that go into production. Proving model test results as being good enough or not, as well as doing the model risk validation independently. So bringing all of that information into one place so that you can produce audit trail of all the activities for legal, PR, internal risk management purposes is a definite need in emerging technology of artificial intelligence. And this is a big opportunity for us to add value and build a business that's going to be part of the new digital world.

Yeah, it's such a big topic, I think, right now for people. And especially it's one that's becoming more and more discussed by people like myself who are non-technical people, but are kind of sort of starting to grapple with some of these big questions around just the more and more AI involved in our lives and in different touch points. And so I think for those not familiar with this space, really briefly, could you kind of talk about what are some examples of biases that you do encounter in artificial intelligence systems? And then we can dive more into it, but what are some examples of bias or a problem that somebody might encounter in AI?

So there was a lot of lending data that became public from the Home Mortgage Disclosure Act that included a lot of demographic information, protected category, like legally protected category information, as well as whether or not a loan was approved or denied. And we trained models on this dataset that has, I forget how many, 17 million or something examples, so enough to do a proper test. And the discrimination in our models against approving loans for people of African descent was incredible. It, it's repeatable. We do demos with it at different sizes and different models. And you're very much guaranteed that these models training on historical data and trying to reproduce the decisions of all of these human judges in mortgage cases and automate that will always reproduce that sort of discrimination in the data. And it's not a surprise to me. It's not a surprise if you are just thinking very simply of input data having certain statistical regularities, and then machine learning models learn those statistical regularities, and then they do what they saw done in the past. I mean, it's a very simple formula, but there are a lot of other examples that are are in the academic literature. So there was a tool that was released for judges to determine whether or not a person is a risk of recommitting a crime in order to let them out of jail early. And the tools are based on, again, historical information about how judges have made decisions in the past, as well as likelihood of people being incarcerated. And it had the exact same effect that you have someone who's done a very light misdemeanor and is deemed a very high risk based upon their ethnicity, evidently, and someone who is a very high-risk offender being deemed a low risk. And we see this across human resources applications, a lot of Things you wouldn't think about, like fraud detection, uses anomaly detection algorithms that identify people who shouldn't be identified simply because they don't look like the normal character.

Yeah, yeah. I've heard this described as like, I guess, bad data in, bad data out, or like biased data in, biased data out. So it's kind of like the training model sort of instills these biases is one place that they arise from. Yeah. So what is the quality assurance solution space or the regulatory compliance space look like around these AI systems? I imagine this is still pretty early stage where you guys are in the space right now. I mean, what does that look like in terms of the solutions, other frameworks other people are using to address these biases?

I don't know, Fiona, if you want to jump in here.

So what's interesting is there's been 160 AI guidelines published in the last couple of years, and this is the year where we've seen this actual talk of implementing some of these guidelines into actual practices. You can see like IBM exiting the facial technology recognition, facial recognition technology. But then that means all these big companies are looking at how do they solve this problem. So Microsoft has a fair learning tool, IBM has a fair learning tool that are open source for people to use. But how do you come— so that's what you see at the big 5 top tech companies. But then at the smaller scale, you see independent data scientists trying to come up with their own solution to solve the same problem. On the regulatory side, there's definitely regulations coming. The World Economic Forum, the head of AI and ML from the World Economic Forum published not one but two articles recently that are basically warning companies that regulation's coming. Companies should start preparing for how they address these AI bias issues. So it's definitely a big opportunity, but from the landscape-wise, we see the big companies trying to address it, the small companies trying to address it. And where we come in is we are not trying to just be a small company in a niche area, not trying to build a data science tool that is just for data scientists. That's where we see a lot of the competition from the small side, small player. What we're trying to build is an end-to-end information management system that even a non-specialist can use and understand. And ultimately, it is the non-specialist the executive business decision makers that make these, you know, approval for these AI systems to be built. And we want them to have a visibility into this system that they approved to build so that they can sleep at night also.

Yeah, perfect. So I love that vision, you know, for those kinds of decision makers who might not be the technical people on sort of systems design side, because ultimately like everybody has to grapple with these big challenges around deployment of AI. And so that's cool that those non-technical people are sort of, at the center of your vision for the product. I want to kind of ask you a little bit more about your mission and vision, you know, and you both, you sort of mentioned about your backgrounds, and I think that speaks to it as well, but what were the motivations as founders for you in building this testing and auditing solution to help foster, as I think it says on your website, you know, or it's in the name FairlyAI, like more fairness in the development and deployment of artificial intelligence?

I see people trying to bring the best of artificial intelligence research and technology into the market all the time. And there are so many difficulties, challenges, things pulling at their attention as they do it. That bias and fairness can easily not be the priority to make, you know, your Friday deadline when you're just dealing with something that is so complex and challenging already. And I also see people who are not as technical but are responsible for these systems and are at a loss for how to ensure that they are of the quality that they need when they don't have the right understanding. And so, there's a group of people with different skill sets and different challenges, but they're always very concerned to do the right thing to bring out the best product. To market and for it to have a human impact that's positive. Everyone is in the right state of mind wanting to do the best, but they need help and they need a system to organize their business processes around this, and it just simply doesn't exist. And so it's like, if it doesn't exist, we'll make it.

Perfect.

I think similar to Etta, from my experience, I work with good programmers, developers. I believe human nature is good, but we all have our unconscious bias. There's been stat out there, 78% of technical workers want help to overcome these technical ethical challenges that I wasn't trained in humanities, I was trained in computer science. Now I have to make ethical decisions when I'm developing AI solutions. I think that's where the gap, the gap that we want to fill.

Yeah, yeah, perfect. I love that kind of vision. I want to dive into some of the products and features. So, you know, feel free to describe this as technically or as broadly as you want to sort of, for a customer of yours, of Fairly AI, describe like, you know, what's the use case for the product, I guess, from the user experience or how somebody would typically go about using Fairly AI for their own system.

So we're taking a multi-stakeholder approach, so there isn't a single user type. There's a data scientist user, there's a data science manager user, and then there's the auditor compliance, uh, role as well as, uh, executive, uh, role. So if you're the executive, um, and that's broadly interpreted, the person who's responsible ultimately for what's going on, you get a dashboard simplified view and a list of high-level issues that might require attention in a red, yellow, green kind of information being provided about that. If you are, on the other hand, compliance, on the compliance team, for example, you would sign up, set up the system, and then go through a number of AI principles that then get turned into specific policies. So these policies are around data handling, they're around the machine learning algorithms that need to be tested, and the levels of the results that are acceptable before they move into production and the people who ultimately need to sign off on those policies. So the data scientist then in a system that's been configured with the policies of the organization that are in agreement with their high-level principles does the work of the data scientist, which is to build, train models, manipulate data, and then properly test them and document their testing. So they are working mostly offline from our system, but we provide very convenient, like single line of code access to our system so that a dataset can be scanned and sent up to our system and recorded the time and the date and the actual identity of the dataset as it's used for a model and that enters our audit trail. And if there are issues that we can detect, we do automatic bias detection. We alert the data scientists that there might be an issue, it creates a ticket in the system. And then if they wanna move this into production, they need to then have approval of what they're doing to mitigate the bias or if there's workarounds And then ultimately their manager or the person on the compliance team takes all the information that we've gathered and is able to produce a report. And that report right now we're aiming at SR 11-7 in the financial industry for model risk management as a first offering. Then there are a lot of internal reports that would be customizable as necessary. We have templating in order to do this.

Perfect. Yeah, no, I appreciate you kind of taking the— to describe like the various, I guess, user scenarios because I know this is like complicated stuff in database management. I understand that you'll be releasing direct integrations, the Google AI platform, the MLflow, is that Can you kind of talk to us about, you know, those integrations that you're releasing and how that, you know, connects up with your product journey or your product roadmap, I guess?

So there's a need to connect to our system wherever you're working, and there are platforms that are already offering bias detection within the platform, and that's great, and it's necessary. Necessary for this to be part of the standard toolkit in order for the working data scientist to easily make the necessary changes. But what we're doing is providing a third-party audit trail of their activities. And you might be in one cloud or another cloud depending on the project that you're working on, but all of it needs to be seamlessly integrated with our system. And for that, we have API that then requires client-side code to make it easy to send the data to our API. So wherever you're working, you can send it directly to our API if you want to write the code to do that. But we're making it more convenient when you're working on these various platforms to have the data sent that way.

Perfect. I always like to ask this question. I always put my guests kind of to go out on a limb. I want to ask you to make a prediction, you know, sort of for where this is headed. So what do you imagine for the future of AI in terms of its biases and its regulations for the ecosystems for creating compliance around these challenges over, say, the next 5 years? You know, how's it going to change?

Well, Facebook now has a board, an independent board that is making judgments about what content gets flagged or not. It gets flagged through human and AI systems. That's, I think, a taste of where we're going, where non-specialists are going to be increasingly interested in the decisions of these systems. And then as the public becomes more concerned and more aware, and as what's just passed in California becomes more routine, there will be a growing need for companies to provide actual transparency and understanding of what systems are affecting our lives through all of these systems. And To do this, there will be a lot of companies popping up doing what we're doing in the coming time and finding their particular verticals and niches. And in a very mature system, the customer has control of the AI that's affecting their lives. So the policies that we're working on at institutional level. As this technology and as people working with the technology mature in their thinking and processes, it'll transfer to consumer control of the AI affecting them. And that's where it needs to be.

Perfect. I like that prediction. Yeah, it seems somewhat positive, at least for, you know, where it can go as it matures.

No, it's not a problem. It's— it is a problem if we do nothing about it. But if we recognize it, we are going to have AI systems that are less biased than humans. It's going to be a good thing.

Yeah, absolutely. You've obviously done a lot of really great work to, you know, build Fairly AI to where it is today. What advice would you offer for an early-stage entrepreneur, or maybe even just an aspiring founder who's just thinking about getting started. Any kind of sage wisdom or general advice you wish you had known at the very beginning of your founder journey?

I say join Founder Institute. That's a no-brainer for me. That's where I met David. And yeah, it's just the network. You, you, you, like any network, and Founder Institute is obviously the, a good choice for us. But it's just having this network of people that support you, uh, during the ups, but especially during the downs on your entrepreneur journey is very important.

I was unsolicited. So I just want everyone to know. Yeah. Um, yeah. Do you have a thought, David?

It's not easy and it, uh, always appears hopeless at every moment, uh, practically until things work out. And from my past startup and this one as well, things always do seem to work out. I mean, like, the odds are low of everything. It can be very depressing to think that way. And yet magic seems to always happen when you keep pushing at it. Yeah. You don't count on it.

Yeah, like a lesson in perseverance, I think. You know, you're going to— everybody's going to have all these major obstacles. But by pushing through, eventually it seems like you can come out the other side. Yeah. Awesome. Final question I kind of have for you is, where is FairlyAI headed next? Are there any kind of news or updates you want to share with the Foundry Institute community who might be watching or any asks for the audience?

Well, we have a few exciting things coming, but we can't announce them yet. But we are in a round of pre-seed fundraising where over $6 million 60% committed at the moment, and we're always looking to network with other startup founders doing proof of concept if they need help with their AI bias or fairness solutions. Yeah, give us a shout.

Awesome. Well, congrats on the current fundraising round. Wish you the best in working towards closing it up. That's great news to hear. Yeah, this has been really interesting for me, you know, learning about these AI biases and the quality assurance and compliance solutions and, you know, how this space is emerging. As a non-technical person, I find it conceptually really, really interesting about how AI continues to touch our lives and how are we going to deal with it in the best ways possible. We're all really looking forward to keeping up with Fairly AI and where you're headed next. Gillian Lee Madden and David Van Bruyn, thank you so much again both for joining us today.

It was a pleasure.

Thank you.

Thank you so much.