#155 Building Diverse Data Teams with Tracy Daniels, Head of Insights and Analytics at Truist

To me, diversity is really about presence and representation, right? From different groups and perspectives and background. We talk about equity because we also want to think about fairness and justice. And then inclusion is really that environment that you create where are people feeling valued, heard, productive, seen. And I think all three of those really work together. One of the sayings that I really like, there's a woman by the name of Renee Mays who was formerly the Chief Data officer Netflix, and she said diversity was like being invited to the party, whereas inclusion was being asked to dance.

Hi, Tracey. Thanks for coming on the show.

Hi, Richie. Thank you for having me.

So I'd like to start off just by seeing if you have any success stories of companies where they've implemented diversity well and it's had a big impact.

Yeah, I think I would be remiss if I didn't start by talking a little bit about how we approach DEI here at Truist. So I like to start with the fact that our company, like a lot of other companies, but maybe a little different and a little special. We start with our purpose, right, in terms of how we think about building better lives and communities. And that means for the people that work for us as well as the people that we serve in the community. And part of our values are actually about caring. And so when you start there, one of the goals that we just intrinsically have as an organization is we want to make sure that we are creating really an inclusive environment and we're making the right moves and changes in the way that we do our work in a way that we think really helps, again, our teammates in the community. And so we focus on, you know, goals, just like everything in our business. Right. We want to make sure that we have a really great return for our shareholders. Well, we also want to make sure that we have strong representation by women or people of color. And so we've set goals to that end around increasing our representation both in terms of women and ethnically diverse populations by 15 to 20%, respectively. And to do that, we have done certain things, like we have a chief Diversity Officer. We have, again, diversity goals and a framework that we implement. And we have pushed to the business units as well, the opportunity for them to sort of craft and mold this in a way that can play out appropriately for the line of business. And I'll talk a little bit about some of the things we do in enterprise, tech and ops. The last thing that I would also say that we've been very sort of goal oriented around is also making sure that we're Improving our supplier diversity. So again, teammates, community, those that work with us, and there we've also set some goals about increasing representation and sourcing for our RFPs, et cetera. So all of that really speaks to kind of a framework and I would say intentionality about creating diverse and equitable and inclusive environments with outcomes that we think are important to our lines of business.

That's very cool. I do like the idea of it being a goal driven approach rather than just a sort of general, this would be nice to have and not really having a direction in terms of how you get there. So I'd love to talk about that more. Before we get to that, can you tell me about the flip side to this? What happens if you don't do diversity and inclusion? Right. Do you have any examples of problems this has caused perhaps in other businesses or organizations?

Yeah, I mean, so I think on the positive, right, when you do it well, McKinsey has studies about increasing and outperforming your peers. And there's studies that show that during COVID where there were higher representation of gender diversity, there was more stability or better response, let's say, like in the COVID pandemic. So then, okay, well, that's good. But what happens if I don't do it? Well, I think you start to see things like products and services that maybe aren't as inclusive as they could or should be are missing features, quite frankly, that would best serve your customers and clients. So I think again, you miss out as a company. Right. Again, in terms of being able to connect with your customers. I think that your organization misses out on the potential for really fantastic problem solvers and product developers, et cetera. And I think in this day and age where there's truly a war on talent and particularly data talent. Right. You don't want to overlook that talent and you don't want to be overlooked by that talent because they don't see what they would expect to see in terms of a DEI program and or representation in the ranks of your company.

That's kind of interesting that it's really about making sure that you're not missing opportunities, but it's happening on both the external side, like missing opportunities with your customers, but also in terms of talent as well, internally. All right, so getting back to your point about setting goals around this, I'm curious as to how companies can align their diversity programs with other business metrics and make sure that the diversity program is sort of aligned with their business goals. How do you think about that?

Yeah, so as we mentioned, it's good business. Right. Lots of studies show that it's really something intrinsic with how you do business. And I like the fact that we have empowered our lines of businesses to also contribute to the type of programming that we create around diversity, equity and inclusion. And so I think it can be multi factored as we've sort of tried to approach it. Right. It's who you hire, it's the products and services that you create, it's the way you interact with like your suppliers, et cetera. And I think it can start with something very basic around we want to increase the representation. I think you want to look at, you know, not just within the general pool, but are you having a pull through effect? Are you promoting people as well into roles? Are you giving them really great, I think stretch assignments or assignments on newer technology that allows them to grow into the technology that's being leveraged. And then I think it's also about tying it specifically to your goals and outcomes. Right. So again, as you mentioned, it's not just a good thing to do. It's because you actually want to see something different show up in terms of how your business produces a product, service or experience.

Okay. Before we get any deeper into this, because we want to talk about both diversity and inclusion, I think it's something where people often get a bit confused about what's the difference between the two. Do you just want to talk about what each of those things means, just so everyone's on the same page?

Yeah. To me, diversity is really about presence and representation. Right. From different groups and perspectives and background. We talk about equity because we also want to think about fairness and justice. And then inclusion is really that environment that you create. Where are people feeling valued, heard, productive, seen. And I think all three of those really work together. One of the sayings that I really like, there's a woman by the name of Renee Mays who was formerly the chief data officer of Netflix. And she said diversity was like being invited to the party, whereas inclusion was being asked to dance. So that's kind of the easy way I sort of think about it in my mind. And again, I think all of these work together for a really comprehensive approach that supports the outcomes that you're looking to drive.

Okay. I really do like that as altitude. Like, yeah, general the party and are you actually participating on the dance?

I never got necessarily asked to dance with me.

Yeah, Like, I'm not sure many people want to see me dance. Part of the couple. It'd be nice to be asked.

Yeah, there you go.

Cool. All right. So it feels like, at least in terms of diversity, one of the most important sort of elements from a business point of view is how we go about hiring. So I'd like to talk a bit more about hiring strategy. And I guess this starts with the adverts. Like, how are you showing off? Like, how are you attracting candidates or potential candidates? So what changes can you make to your job adverts to make them more appealing to a diverse audience?

Yeah. So one, I think where you go and where you place yourself, whether it's adverts or your, you know, the people that you want to hire. And so let's maybe let's walk it through a little bit, like early career. Right. What schools and colleges are you going to? Are those schools and colleges diverse? If not, do you have additional places that you would go, historically black colleges and universities or universities where their body is fairly diverse so that you can, you know, cast as wide a net as possible. I love this idea of also, how do you write your descriptions, your job descriptions? There's been some work done around. What are you asking for? How are you asking for a day in a life versus, you know, this very rigid. You must have this. And we know some populations, women versus men might respond differently to certain ways that things are written. So I think that's also important. Who you leverage to interview the panel is also important. Right. And there's been studies that show the more diverse your panel can be, the more likely you are to convert folks into the hiring process as well. And then one that I also like to employ and I like to encourage folks to think about is your network. Right. And if I. Only if I have a very narrow network in terms of who I know or who I go to to ask questions or hey, do you know a really great person? I can end up with a very sort of homogenous group of candidates that comes through. And my husband likes to remind me that while I'm an introvert, not everybody likes to go make just general conversation with people and expand their network and gather new friends. But I do think it's important to be intentional about thinking about how do you put yourself out there, even if it's uncomfortable or unfamiliar is maybe a better word for it. So that the spaces and places that you occupy are a little bit more diverse and you can source from that population when you're looking for new talent.

That's really interesting, the idea that maybe you need to have diversity in your people team or your HR team in order to increase diversity of the actual Candidates who are going through the interview process or perhaps even being sourced by.

Recruiters or even the team. Right. Like when we recruit them in, sometimes our HR department does a really great job sourcing diverse talent and then we have a not so diverse panel that interviews them and is judging and deciding who actually comes in.

I want to pick up on something you said about changing the writing within your job adverts and how different people might respond differently. Do you have any examples of this, like from maybe a data analyst or data scientist type job advert?

Yeah. So rather than like writing, you know, something like 10 years of LLM models, right. I would challenge myself. Do I really need that or do I need somebody that's maybe had more contemporary and maybe less time? Right. With those types of models, if I'm going to list out all of these again, sort of five years of this, three years of that, all these languages, can I talk more about what the job entails? I want you to use these skills and some of them will be technical, some of them might be interpersonal. You will work with these types of people or these types of problems that may resonate with a variety of people. And I think about data as being a very multidisciplinary type of field. Right. And so I want to appeal to as many different types of people as possible to help solve these really tough problems that we have.

I have to say, even if the job advert is just like a list of skills, then that sounds super boring. It's like, well, if that's what you want from me, but you're not telling me what I'm getting out of this, what I'm going to be doing. And yeah, there was a terrible job. Advertising.

Yeah. And I'll tell you, Even like my 27 year old male son would not respond to that these days, right. He's looking for like, tell me the adventure I'm going to go on and this horrible problem I'm going to solve and the amazing people I'm going to work with, he would actually respond to a very different kind of writing.

That's really interesting stuff. So I think maybe one of the challenges with hiring when you think about diversity is you have to think about diversity of skills across your team. You've got to think about like diversity of personality and then it's fine if you've got lots of people applying for the job, but if the candidate pipeline's a bit thin, then you might start to think, well, okay, maybe we don't worry about demographic diversity here and that's when you can run into problems. So I'm just wondering, how do you sort of prioritize all the different types of diversity when you're hiring?

Yeah, that's a great question. I think it's a hard one. Right. Like the nerd in me wants, oh, I'm sure there's a formula we can create to make this work. Right. But it's not. I mean, I think, you know, cast a wide net. You create a really. You spend time creating an inclusive environment so the people you bring in, regardless of their backgrounds, understand the values of the group, the company understand that they are valued as part of the group and the company. And the expectations of how you work and perform, how you. How you. I don't want to say rate each other, but how you learn to work through issues and solve problems and celebrate and recognize one another are well understood. And I think that inclusive environment helps to reinforce that the choice that they made to join you was the right one and a good one. So I think. I like to think that it's not one or the other. I'm not pitting a group against each other. I'm not. It's better if you're black. It's better if you're disabled. It's like, I want to create an environment where whoever you are and whatever your intersectionality is, you feel included and that this is a place you can really thrive and grow.

Okay. And I guess there's some sort of feedback mechanism there where if your existing employees are feeling included, then it's going to encourage them to try and hire more people, and that's going to sort of benefit your pipeline. Does that sound like it makes sense?

Yeah, I think so. And I think as the environment becomes more diverse. Right. Then again that network and the pools that you have access to also start to diversify. Again, it all starts with intentionality. It doesn't happen because of rainbows and unicorns. It happens because you've decided to be intentional. You decided that this was an important element of how you build teams.

Okay. I'd love to talk a bit more about team building in a bit, but another question about hiring before we get to that. So I think bias in hiring is maybe one of the sort of hidden problems that you need to overcome if you want a diverse team. So what sort of steps can a company take to increase awareness of, like, the types of bias you might find in hiring?

Yeah, I mean, again, I think you start with some of the things we've talked about before, which is looking at your practices, looking at who's interviewing, looking at where you're going to get folks from. I love this idea, as well as making people aware of where there could be bias and giving them tips and frameworks around how they overcome or mitigate biases in the practice. Because I think if you ignore it and you want to pretend that it doesn't exist, you'll just repeat some of the same failings and miss out on the opportunity. It's more of a framework we've used with our data teams in terms of how we encourage them to really think about removing bias, not only in the hiring practice, but as part of our framework. We had Dr. Brandy Ice Marshall come in and she talked about this peer principle, a framework that she used for helping to mitigate bias, really in data. But part of the premise that I love is it starts with the P who's actually participating. And it was interesting. We had teammates, data scientists and data analysts, et cetera, across the organization. Really appreciate a framework and a concept for them to think about because, you know, nobody was doing it on purpose or didn't assume that it was all perfect and good, but helping with a framework for them to think about, okay, how do I get more participation and access on my team? And that being, again, an intentional goal and thought that they were having. Again, I think that awareness and literacy is really important to overcoming what the biases could be in the process itself.

I like the idea that there's a framework, sort of definite steps you can take in order to try and identify and reduce these biases. So maybe moving on to inclusion type stuff. So what do you think the most common problem areas are for inclusion? Like, when do you have this diversity? But then people aren't being asked to dance, as it were.

Yeah, you know, I think again, I tend to be a little bit of a Pollyanna. Right. So I don't think people come in and with bad intentions. I think if you're not being intentional about it, there. There are just things that could happen. Right. And so I think sometimes it's just slowing ourselves down a little bit so that we can pay attention to what might be our natural inclination. So I think about things like, is everybody being heard in the room? Right. Because I'm an extrovert and I love to talk, talk, talk, talk, talk, talk, talk. Are there moments where I need to be quiet or specifically call on somebody I'm not hearing from and ask not really sort of general questions, but poking on did I hear from or hear the experience? Is there a perspective we are not considering to Draw in more voices and invite them to actually participate in the conversation, in the debate, in the feature that might be being discussed. And again, I think you need all voices in the room to be able to do that in a really proactive way.

That's interesting. They mentioned the idea that if you're an extrovert, you need to learn to be quiet sometimes. I guess the flip side is true. If you're an introvert, you need to sort of force yourself to speak up sometimes.

Yeah, that idea of being uncomfortable a little bit, right?

Yeah, I guess make yourself uncomfortable to be successful. Are there any sort of less painful ways of doing that? Like, do you have any tips for, like, how you can balance out the difference between extroverts and introverts being heard in a sort of maybe systematic way?

Yeah, you know what, we also try to make sure that there's a way to get feedback. Right. One of the things I love about technology these days in our society to some degree is like, there's lots of ways to give your opinion. Right. Whether it's, you know, through technology, your star rating, or, you know, it's even a simple thing is just drop an email. And I. And again, I try to make sure I'm very intentional about inviting people to do that. And I'll say things like, I know everybody doesn't just want to speak up necessarily in this meeting or people may need time to process their thoughts. Right. Feel free to drop me an email or have a way for folks to provide that email, whether it's anonymous or with their name or whatever, what have you. The other thing that I think is important in this context is to show people that you're listening to invite them to continue to do that. Right. Because there's nothing worse than I give you my opinion. It goes off into the ethers. And I don't see anything that happens based on this feedback. I've told you that I don't feel like we do enough to bring different people into some of the more technical experiences or I've told you. Um, you know, I. I don't feel like I can talk about my same gender spouse. Right. Okay. Well, you need to act on it and be very transparent. Right. That you're doing this because of feedback so that people know that you heard it and you're doing something. You're not gonna be able to fix everything and react to everything. But I think it is important to show progress against what you're hearing from your teammates as well.

Well, I really like that idea of having feedback mechanisms Also having different forms of communication. So if some people are more comfortable writing things, then that's great. If they're more comfortable having chit chat with people, you know, you can have a, have ways to have a conversation. So I guess related to this one thing that's like the bane of everyone's life at work is meetings. And doing meetings well is like the secret are that like everyone needs to get better at. So I'm just wondering, do you have any tips for making meetings more inclusive?

Yeah. Okay. So I mean it's going to be what I say, not necessarily what I'm awesome at doing. Right. Sometimes again I, I will sometimes get called out myself. Right. On what is the meeting about? So I do try to make sure that I can give people the heads up about the nature of the meeting. Maybe a couple of things that we're going to cover again so people can prepare. Right. And then you're transparent about your intentions. But if it takes me a moment to process, I can maybe do some research or think about what my response is going to be, be clear on what the expectations are coming out of it. Are we just going to talk and are we going to, you know, brainstorm or do we need to come out with a decision and how do we get there and drive to it? Again, I like this idea of inviting the choir voices in the meeting to also participate, whether it's before, during or after with some of the feedback mechanisms we talked about. But given that yes, our life is spent in endless meetings, I think those are one of the things. And then again, I don't know that this has to do with inclusion, but maybe about effectiveness coming out of meetings. I like when meetings aren't done on the half hour or hour, meaning that it's done at 20 after if it's a 30 minute meeting or 45 minutes after so that I've got time to process action, do the thing I said I would do before I move on to the next meeting. I think that gives everybody an opportunity to really action and process what they've been through.

Absolutely. I have to say I agree in theory all my meetings would have an agenda beforehand that everyone's seen. And yeah, it doesn't quite often like that. So yeah, aspirational goals. All right, so maybe let's talk about how you actually go about implementing a sort of diversity and inclusion or even DEI programme. So I think it seems like getting some sort of executive buy in is going to be quite important because this needs to be an organization wide program. So how should you Go about this.

Yeah. I think one of the things that's key is, like you said, that buy. And whether it comes top down or I do think it's important that somebody at the senior level is the champion and is driving this. Right. We're very fortunate here at truist. It is something that's truly embedded in our executive leadership team. And we have a chief Diversity Officer. Right. Whose remit. Dominica's remit is to help us really grow and expand, you know, our goals and our frameworks, et cetera, and the programs that we have around diadem. But it's not just one person's role and expectation. I think the other strength that we have and some of the best practices I've seen at other organizations is when you have a really strong and involved business resource group or whatever you want to call them, whether they're created by whatever flavor of diversity, whether it's women or people of color, lgbt, differently abled individuals, where they kind of get to set the agenda right. Of what's important and how they see these items playing out. And then as I mentioned earlier, getting the lines of business involved. So we have, you know, finance, enterprise technology, our wealth group, they have their own DEI organizations where they're really tailoring how they work through the objectives that we have as an organization and help make a difference in their lines of business. So I think again, the focus of the organization, the intent of the organization, leadership, top down, but also I think really participation by as many of the teammates as possible makes a difference. And then the last thing I think is, I think can also be special secret sauce, let's call it is when allies are really active in this framework. Right. Meaning I am not personally physically differently able, but I involve myself in those organizations and learn, hear the conversation and participate in things and solutions that we can do. I think that allyship is also really important to help further the goals.

I like this idea of having some different organizations within, like representing different communities within your wider business or organization. I'm curious as to how the human resources or people team's gonna interact with these. Is this something that is driven by them or is this something they're involved in separately?

So it's collaboration. I would say the framework, the overall framework that we have here at Truist is really set by the DEI organization. So Dominica Grimm and her team, and then we take it within the lines of business and we sort of come up with a roadmap for how we're going to implement it. So within the enterprise Technology. What does that mean? I may be going to, I may have a different way of improving my hiring. Right. I may be looking at different conferences in different schools. I may be looking at mentorship programs within my organization that are specifically tailored to technologists. We actually created last year a women in technology group. Right. That was in addition to the DEI work that we were doing more broadly across enterprise technology. I've seen groups out there for black women in data as there tends to be a dearth of women of color, let's say, in the field. So I don't think it has to be one size fits all or just one versus the other. But I do think that there's, what's important for us here at Truist is that there's collaboration so that I'm pulling through the goals of the enterprise and really just tailoring it for how technology is going to do it and then asking questions about what specifically do I need to do as part of the data organization to continue to support those goals as well.

So I guess if you start running a DEI program, then at some point someone's going to want to say, well, is this a success or not? So how do you go about measuring the success? I know you talked about having some goals beforehand, so maybe you can just elaborate on what counts as success.

Yeah. So overall from a truist perspective, I talked about increases in our representation. So looking for increase in our female representation or ethnic diverse representation, I also look at some, I don't want to say softer metrics, but looking at the type of programming that we do. So one of the things I think about in my role for data is I'm always wanting to build a data culture. Well, part of that data culture includes conversations around DEI and, or removing bias from the field or at least mitigating for it and thinking through it. And so we look at programming and sometimes I'll point to, you know, I did a three day program and I trained educated, you know, a thousand people on these topics and you know, I might have gotten feedback good, bad or indifferent on that. I might also look for if I was super sophisticated and everything was inventoried and libraries galore. Right. Do we have code or ways that we are cataloging as part of how we do our data work specifically? So taking it from maybe something that is just business specific to something technically, I'd actually want to see where I'm making it up. 50, 60% of our X models are going through some sort of, you know, modeled for detection of bias, etc. So I think you can start small, and as you get more mature and maybe even more. A little bit more technologically advanced, you can also throw in some additional ways to measure your improvement or even maturity. Right.

On the topic, that's actually fascinating because it's something I've never really thought of before, that it's not just about diversity of employees. You just figure out another GENIE coefficient or something of your employees by team. But it's also in terms of the models you're creating, particularly in data and machine learning. That's part of responsible machine learning, is figuring out, have you got bias there? And just implementing that systematically. Interesting. Okay, so how do you ensure the accountability of any managers or sort of team players who are involved in this sort of program?

Yep. So what gets measured get managed. Right. So there are metrics that we look at. Are we getting better? Are we holding flat? And if we are sort of where we expect to be. Okay, what else do you need to look at? And start thinking through the training and education and the conversations we talked about. So there's, you know, there's training that the organization gets to, you know, avoid the things we want to avoid. And then again, maybe slightly softer is this idea of what are the conversations that we're also having. Right. And do people feel like their voice is heard? So we have an employee engagement survey, and in there we'll ask questions that are also focused on DEI inclusion and about how people feel about how they're treated within the organization. So I think there's a variety of metrics that you can look at to really show that you're making the improvement or if you are not, or you're moving backwards. Right. Something that lets you know that there's probably work to be done and opportunities, opportunity for focus.

While we're on the subject of sort of measuring things, I remember there was an experiment with Twitter a few years ago. I can't remember which CEO it was on there had quite a few, but they were talking about having diversity, measuring it at a company level in terms of diversity of employees. But then they would have individual teams who are saying, okay, well, I have a whole team of just women or something, and the team level diversity didn't matter as much as the whole company level. I'm just wondering how you think about that.

Yeah, I mean, I think there's much slicing and dicing. Right. That you can do. And again, you're not. I don't want to say it this way, but I'll say it. You're not reacting to everything. But I Think you're looking for what's important for your organization? I do things that are important to me. I'll just say it this way, right. As a woman and a person of color levels. Right. So if all of your diversity is in the most junior places of your organization and not represented in senior management, on your board, et cetera, I think there's an opportunity. Right. If all of your diverse talent is in support functions and not in revenue or code producing functions, I think that's another area to look at. So I do think you want to slice and dice the data for things like equity. Right. Because how that could play out is again, your products and services aren't as fulsome as they could be. People making the most money is not as it doesn't represent a huge diverse representation of your population, be it, you know, the function or the city or wherever you are. So I think you do want to look at various dimensions of the data.

I suppose related to this. It seems like this can be a tricky area for data departments because if you've got a customer facing role, it's a pretty easy to pitch to say, okay, if we increase the diversity of our account executives, you know, you've got people who look just like their customers, you can probably see that pretty quickly in terms of the metrics with data diversity in data, it's harder to measure an impact on the business or at least it's harder to tell a story about that. So I'm just wondering if you're trying to improve diversity in your data teams, what's the sort of pitch to management for this?

Oh, it's products and services, right. I mean, and experiences. So I think about, you know, some of the studies that have been done and again, Not, not just one reason, but did the data that I collected are the teams that are looking at it, the people that are challenging if the solution is fit for purpose, do they really represent a diverse perspective, backgrounds, experience to be able to guide me where I go. Right. And so I think about some of the studies out there where for example, the image products or products and technologies that we're using, image don't do as good a job identifying or differentiating lighter skin from darker skin. Right. A product can have a very adverse impact to somebody with darker skin if you're not training your model to be able to detect that. Right. And we've heard all kinds of stories with horrific examples where you know, the error rate might be small, but it's still impactful, particularly on certain populations. Or am I, did I Train my model or create a model with whatever training data that. Disproportionately favors one population or another. We have a joke in my family. We applied for a certain financial product and I'll leave it at that. And I am the primary breadwinner and I got declined and my husband, whose stay at home dad got approved. You've got a credit history, had a job, all that kind of good stuff. And later you wonder and you hear about models that maybe weren't as equitable as they could have or should have. Then again you learn, because again, slightly Pollyanna, I don't think people go in with this to be malicious or to harm certain populations. But as you learn that the outcomes aren't what you want them to be, you need to go back to the lab and come up with another model. Right. That has a different outcome or a more equitable outcome. And when you think about the explosion of AI software and in the decision making process, even in something like using AI to do hiring. Right. I think it's incredibly important that you have diversity in the process of who's making these products, the data that's used to train the products and the people that are contemplating how these products go to market.

And I guess in the credit example that's straightforward. They've just lost your business. So that's kind of bad outcome for you, bad outcome for the organization as well. So yeah, getting this right is going to sort of help people's company's revenues, help increase customers satisfaction, and that's like a fairly benign example, I think. Okay, so I guess do you have any examples of where diversity has been built into products successfully and you know, the company's done it properly, they've made things fair and it has worked?

You know, I don't know that I have great examples. But let me tell, let me tell you two things. One, I love the way that you ask that question, because I think that's important is that you're building in this concept or consideration. Right. And the reason I say I don't know that I have a great example is because if you did it well, I shouldn't know. I should just know that I'm attracted to the product or the service and I have an opinion and it worked really great. So I think about things like legally, now there's things you need to do when you're creating products and services for people that may be disabled under the ada. And if you do it up front, I may not need technology that reads to me the description of the picture, but You've built it in. And you could build it in in a very elegant way that benefits folks that are just differently abled, that may have, for example, a sight impairment. And in a way that is perhaps even a side benefit of being instructive or something for, for me who is not sight impaired, I think about just consideration in the product. So again, do you consider things like people's various pronouns. Right. And just for data entry, how you're capturing and maybe tagging and identifying data. And then there's the really, really, really hard stuff right around training data and the effects. Like we know our data has biases built into it from decision structures, whatever that were in the past. Is there a way that you can augment the data? And are you thinking about that as you're building your products up front so that you've got as much data to make the best decision possible?

I love that there are some sort of real practical steps you can do to improve diversity. Are there any sort of process changes you need to make just in order to ensure that you are making all these steps in terms of building diversity in.

Yeah, I mean, I do think you should be building this into frameworks. Right. And whether it's your model development process where at least you're asking the question, I think, oh, it's like AI now has an impact framework that they've put out there. Nist even in their frameworks talks about considerations for. They don't tell you exactly how, but that you're building in consideration for these types of things. I talked a little bit earlier about Dr. Brandy S. Marshall's process and it was about thinking through Peer is the acronym that she uses. And it's participation. Who's participating in building these things, but also who's got access not only to the data, the data that's selected, what gets promoted, what gets limited. But who's got access to the technology. I love the way OpenAI actually came out with their product. Right. Because you've got a lot of people banging on it and asking it questions that will become part of its corpus. Right. And part of the lexicon of how the language is leveraged. Inclusion in her pair principle talks about engaging very intentionally engaging diverse or marginalized communities. And can you evidence it? Right. So that you know that you. What you know, I did, I asked and I got information and I used it or I didn't. Right. And then representation being the last bit around. Do you have representation from all demographics that you need within the data or the system or the framework? And then this Last bit, I also love around and are you crediting people for their contributions? Right. Again, this idea of whose story is being told or who's part of the conversation about what gets created, the value that gets created as well.

Okay, and in that last case where you're talking about showing off the representation, are you talking about like the people who are creating the product, like within your team, or are you talking about like the users and the sort of all the different groups who are actually using your product?

Yeah, a little bit of both. Right. But the idea that they're represented in the data, this idea that dark skin and light skin is represented in the data, that the frameworks that you're using are going after, making sure that there's representation there as well.

And you mentioned OpenAI as being an example. It just seemed like with this explosion in the use of AI, this is going to be a bigger issue. Just making sure that these AI products are fair for all the groups that are using them. Do you have any advice specific to AI in terms of reducing bias or otherwise making them fairer?

Yeah, I mean, I think we've touched on a couple of them. Right. So could start with pair. I love just having this framework that you can use, how you collect the data and what kind of data that you have. You know, check it for both. Are there biases built into it or is there unrepresented data that I actually need to go and get. There's also, you know, one of the things that I kind of love about the data field as well is this movement into more inclusive design principles, like truly almost a little bit becoming more like software design. Right. Where you're actually thinking about how it gets used, how it gets consumed and building that into the design process. Right. So inasmuch as it's a design bias, awareness techniques that we talked about earlier and making sure that it's there, I also think you're going to monitor. Right. And measure, which we all know we should be doing with our models and data. And so you're looking for ways to look for impacts. Right. And equity on various populations. You're going to ask yourself again, this concept of being uncomfortable. You might need to ask yourself some uncomfortable questions and actually test for that. Right. In the outcomes of your product. And then maybe last, much like we talked about earlier around literacy and culture, I think increasing the literacy of your team around these issues is important. So as I mentioned here at Truist a couple years ago, we actually did. It was a three day half session with Dr. Brandeis Marshall around how do we incorporate these concepts and teaching people to think about it. You know, Brookings talks about companies need to think about and actually debate the trade offs. So as you're going to market, I think having the lexicon and a place where people come together to talk about the trade offs of, okay, really effective, maybe very surgical, but I'm seeing adverse impact here. Are we all good with that or are there things we should also be doing and including maybe humans in the loop? Right. To help deliver a message or provide options for whoever's consuming this product or service.

How do you envisage that humans interact with these AI ideas and who gets the decisions? How do you decide? How does that affect diversity? I guess is what I was.

Yeah, well, I think looking at in some sort of framework where you decide where you are applying the technology. Right. Where you might not need a human in the loop. For example, maybe on a back office system. I'm looking at uptime and I'm doing some self healing using AI. Right. I may not need as much of a human in the loop in that. But I think you do want to consider where there are times and places and spaces where it's best to have human decision makers helping you to really think about the solution, the output, et cetera. And maybe I'm an outlier. I kind of feel like AI is most powerful when paired with a human. Right. Not to override. I always tell my husband, if Waze told you go left, just go left. You don't need to override her all the time. But I do think it's a powerful opportunity and maybe even a little bit more surgical way than we have in the past to really help humans be more effective and efficient at doing certain tasks. Right. And so I think, I think we're going to end up with different types of jobs that still require humans. Right. Because in terms of how you deliver this and how you use that expertise to hone the output is going to be important as well.

I think there's going to be a lot of people want to learn a bit more about this topic. So do you have any advice on communities or resources people use to find out more about diversity and inclusion?

So I love to follow certain thinkers. You've heard me mention Dr. Marshall a couple of times. Dr. Ruman Chowdhury as well. She was part of the AI ethics team at Twitter at one point. Dr. Joy Bulamini also. And I just love the name of this group. I think she was part of the founding group or pioneers for the Algorithmic Justice League. I think following people who think about and talk about these topics is really important and making sure that that group is diverse. Right. As well. I think there's some really great studies out there. You heard me mention the McKinsey study. Brookings actually has a really nice independent piece as well about helping to mitigate bias in data. Not as sexy and maybe as interesting. Some of the legislation that's coming out of Europe, where they're actually tackling this, I think a little faster ahead of the U.S. i think, if also kind of interesting to read and keep up with. And then like I mentioned with, you know, some of the business resource groups, I think joining support organizations that focus on varying populations in these topics are also important. So I think about groups here in Georgia, like Women in Technology or Girls who Code or, you know, Women in Data and AI are great groups to also follow and participate in.

Excellent. And if someone spots a problem with diversity inclusion at their own organization, do you have any advice on how to start tackling it?

Yeah, I think people should speak up and not be afraid to do that. I remember being part of a presentation just recently where the collateral was talking about heroes and data, and that was a wonderful lineup of diverse men, but it was all men. And I said, are there no women in this universe ecosystem that you. And they just hadn't considered it. And so the second revision had more genders presented. Right. So just speak up. I mean, there's very polite ways to say things. You don't make me make people feel horrible about themselves. But I think speaking up is important. I think advocate. You know, as a senior leader, I try to make sure that I am talking about these issues and that I'm visible about these issues so that people understand. It's important to me. I think, again, if you're in the out group, you're not in the group being an advocate and an ally. So if you're a male for women, if you're white, for people of color, if you're cis, you know, you should advocate for our LGBT teammates as well. And I personally have been, I think, a recipient of people advocating for me that may not have looked like me or had the same background or whatever, what have you. And then maybe the last one is to be curious, right. Where if those of us in the data field, that comes very naturally to us to ask questions and ask more questions and poke and look for the data and then ask more questions. And I think if we apply that also to one another, I think that also helps create moments of understanding and helps to really build and foster that inclusive environment we talked about.

That's great advice. Although maybe, like, don't poke your colleagues, literally.

All right.

So do you have any final advice for people who are thinking about diversity, equity and inclusion?

Yeah, I think just be intentional about it. Right. Again, it doesn't happen just because of a. I hope this all gets better. And be courageous. This is hard, difficult, messy, sometimes deflating things to go after. But, you know, continue to be courageous and push through and then keep learning. Don't poke your colleagues, but keep learning.

All right. Perfect. Thank you for your time, Tracy.

No, thank you. It's been a pleasure. And thank you for your interest in the topic. This was wonderful.