Understanding A.I. in candidate assessment | with Nick Johnston

Sam foreign.

Welcome to episode 67 of the Shortlist. My My name is Johnny Campbell. I'm your host of the Shortlist, which is a weekly podcast and live show that goes out on YouTube and LinkedIn, and this week we're going to be talking about understanding AI in Candidate assessment. If you are joining us live on the show, you're very welcome. We'd love to have your comments, chats, contributions, questions, which you can post live on LinkedIn in the chat below, or you can put into YouTube in the chat. We'll be able to get them live and we'll respond to them as soon as we see them. And there's a break in the conversation, which often takes a couple of minutes, so bear with us. If you're listening to the podcast, you're also very welcome. It's your first time hit that subscribe button. You'll hopefully want to get more of the shows. This is the 67th. We've got a whole back catalog of shows you need to be subscribing to, but you can find out more about the show. You can find about what was on our previous 66 episodes. What's upcoming by going to socialtalent.com the shortlist let's get into the meat of today's show. Understanding Artificial Intelligence in Candidate Assessment AI is such a big buzzword, right? And it's difficult to stem the tide of technical innovation and also the rise of so many buzzwords. But in every facet of our lives, technology is constantly being developed to improve processes, increase efficiency, make life easier for us. And recruitment is not immune from these advances either. Every new year, there's new tools being created with the promise of fixing everything, helping organizations solve all their hiring challenges, being able to find great talent, assess for great talent, do it faster, do it cheaper, do it better, do it more diversely. All these things. And no more in recruitment has this been felt than the level of candid assessment. Arguably, you know, with sourcing, the area where AI has potentially shown the most promise, or at least people would have you believe it's shown the most promise, where algorithms and AI can analyze everything from your writing patterns to speech patterns, where you're looking, your tone of voice, your nonverbal behaviors, make a judgment as to whether you are good enough for the job. But there's a lot to understand on this front, and it is a particularly contentious area. So to help us dive into this, we've invited Nick Johnston, who's a director of TA at PwC in the Middle east, coming to his Live from Dubai. Back to the shortlist. You remember Nick from earlier in the year discussing Quality of Hire, which was a hugely popular show. Nick proved to be a great contributor. We're hoping to have more of the same for this. And Nick, you've been in this space for over 20 years, worked for tons of organizations, presenter on the social talent platform. You are really kind of passionate about the whole area of assessment. So maybe can you introduce us, tell us what's it like in Dubai at the moment, how you're doing, why this became a passionate area for you, why'd you care so much about this?

Thank you. Never felt quite so much pressure as I do right now, Johnny. So how. How is it in Dubai? Super hot. Yeah. Hottest time of the year. We're just coming through it. We're super busy. We're recovering from the pandemic or coming out the pandemic like everybody else. And the talent marketplace is fierce. The competition for top talent is absolutely fierce. And everybody knows it's not the number of people you hire, it's how good the people are that you hire that's important. And that's why Quality of Hire is so important to me, because as a, as a TA leader, I know I need to hit a number, but it's not just, you know, X number of heads, it's quality heads that need to be brought on board into the business so that they can produce and add value. And I love, I love assessment. I love sourcing, I love assessment, I love recruitment in general. I think that assessment is such a valuable part of the process and it's where we can really differentiate and add value to our hiring managers and our candidates in helping to really align people to the right kinds of roles and the right kinds of companies. And I think that's why that's so important. The AI thing. Well, two things really. You know, I love tech. I've worked for a lot of tech firms. I enjoy using technology to enable service delivery. I'm getting approached maybe three times a week by different companies selling me AI solutions to everything that I could possibly think of, but mostly around recruitment. And it started me asking the question, where have all these solutions come from so quickly, particularly with robust assessment tools? You know, there's a process, there's a time that it takes to develop these tools and test them and validate them and test them again and then make sure that they're not creating any adverse impact. And these guys are coming to market so fast with tech solutions that everybody is buying. They tell you. It just got me really Interested what's going on on the back end of this? What is driving these solutions that makes them so powerful and so valuable in such a short space of time. And I went into that with the lens of I would really like to do this because it sounds cool. I love technology. The idea that I can save loads of time, mass process candidates and create more value for my customers, you know, without the extensive multi layered assessment processes that we often go through, particularly for the key role. Key roles is very attractive and I spend a lot of time researching it. I've spoken to a lot of leading companies and kind of pioneering companies in the area to try and find a solution that I could sign off on that I could say genuinely this is going to make a difference to the quality of hire for the company that I'm representing, which I think is a very serious responsibility that we all have. And that's kind of what's. What's driven me to really explore this and trying to understand it because in 90% of the conversations I have, the people that I'm talking to that are trying to sell the solution to me don't know enough about technology, big data or the concepts of assessment to really be able to in a compelling way say this is a safe tool to use. So it becomes quite scary then I think I've read a number of articles where they're talking about AI snake oil. I think there's a really good article out there about AI snake oil in general, but particularly referring to assessment. And I think that's really what got me into this and trying to understand it.

It's a dangerous area and it's one that's popular in the media at the moment. Which probably brings me quite nicely to a segue that George is going to help me with which is the news foreign. So Nick, I want to start by discussing this article that appeared recently enough over the summer from the Chicago Sun Times. We'll put a Link live on LinkedIn and YouTube and put in the show notes in the podcast and it's entitled want a Job or is saying Talk to a Computer. And in s this Chicago Sun Times article explores this initially from a candidate perspective and it it takes the position of a candidate, a lady called Dana Anthony who applied for a part time job Target did an interview and got rejected. But the whole process was driven by AI or robots. And it opens the question opposes the question about is this fair? And it explores it more because into talking about HireVue in particular. But other technology companies like Modern Hire and others who have Tools out there points to a story in the past where, however you get into trouble about AI bias, perhaps that could have been there and then an edge pseudoscience about some of the novel products or features that they had there. Right, but what was your take on this? Like this experience of applying for a job, doing an interview and all being driven by AI. Can it work? Does it work? What's your view of the author's perspective on this?

I think there's two themes to discuss in here. One is the candidate experience and the second is that the can it work? The validity of the approach. Right. So at the end of the day, people like dealing with people. And I think that if you are end to end, assessed and qualified out and judged solely on the basis of an AI system that's making a decision about you, I think that's quite a tough thing for a lot of people to accept, particularly at the moment when we don't really know how robust the AI is, how well the AI is working. And I think this also comes back to the, like the privacy aspects of this. If I understand the privacy legislation correctly, we need to disclose to people how the decisions are being made about them and, and we need to give them the option, I think, to opt out of being purely assessed and selected by tech. Not having human intervention in there. I don't know honestly how that applies in America. But I think from a candidate experience perspective, purely being qualified out by tech, particularly at an interview stage, is probably quite a challenging thing to experience. The flip of that is, you know, as a recruiting leader I need to manage volume. You know, my high volume, low value roles where we kind of factory process them a bit, we do it more to them rather than trying to create an experience. More niche roles where we really need to work on attraction and engagement. I can understand why they set it up like that, particularly in the COVID period where we've had to push everything virtual. It makes a lot of sense. I can run all of this online. I don't have to have a mass assessment day. I don't have to have a load of people in a room and deal with all of those problems. So I can understand why that happened. When it comes to the can it work, it depends what the AI was assessing. Right. So AI potentially can assess candidates at different stages of the process. A CV or an application form can be reviewed using a basic algorithm and some pattern matching to look for keywords or accreditations or whatever it is the AI has identified in its analysis that differentiates a top performer, the Same with the video. And the video interviewing, I think is where it gets really, really interesting for me. So as I understand it, there's a couple of things that potentially are happening in there. One is the speech to text, right? So they'll tell you the AI is judging your speech. What it's actually doing is turning the speech into text, processing that text and assessing it in just the same way it would a cv, which sounds great, but we have so many people talking in so many different accents. How reliable is that speech to text? I'll tell you from my own experience, I have to put on an American accent to communicate with Siri properly. My kids laugh, but it's the only way to make it work. Right? The best speech processor I use is the Samsung tv, right? Which isn't widely available. A lot of these companies are building their own speech to text or they're using a third party plugin. How good is it? How many different accents is it being trained on? We actually went to a pilot with one speech to text video interviewing company and I said to them, I want to test the accents because I live and recruit in the Middle east. And we test many different nationalities and different accents. And so we pulled a pool of people, an Eastern European, a Middle Eastern GCC Arab, an American, an English person, a South African, and we had them all interviewed by the software and they were brave enough, and I credit them for this, to give us the transcripts of the conversation so we could look at them. And it wasn't brilliant. It made some interesting mistakes. It turned and I got the result into something like it all went wrong. I mean, they were pretty glaring errors, right? Which scare you. And I discussed it with the guy, the director, and he was like, I wanted to show you this because I wanted to be real with you about what we're doing. There's always going to be a margin of error in this and you have to decide whether this is a good enough proxy for you to use it as a filtering tool rather than a decision making tool instead of some other proxy, which is a very valid point. At the end of the day, whatever assessment technique we use, we're taking a risk somewhere. There's going to be a compromise somewhere. But that speech to text processing is key. And pattern matching just based on words and not the context of what someone is saying, I think is a challenging thing again for candidates to accept. And a lot of recruiters who place a lot of emphasis on their ability to understand the business, the behaviors and the actions that are relevant to the role and to infer that context and to make the fit decision based on how somebody is behaving, not the frequency of a word count.

Yeah, I remember seeing, seeing these first software tools several years ago and again wondering how would it all work? Right? And you make an excellent point, like in terms of, you know, what is it actually doing? And understanding that's really important. But also you got to compare it against what the alternative is, which is a person doing this and a person is prone to error. We can mishear things. To your point, Siri can't understand your accent, but perhaps you can't understand every word from every accent you hear, as you might miss a critical word. So I've always approached this in the perspective of is it better than what we are doing rather than perfect? Because sometimes we look at AI or machine driven, we say, is it perfect? And if it's not perfect, we dismiss it. As opposed to going, well, hang on a second, let's accept that there's certain error rate in our current process with humans doing it. Inconsistencies, we're mishearing things, we have bias, all these things. The level at which we hold AI products to seems to be abnormally higher than a human standard. Do you think that's fair or do you think we should be holding it to a ridiculously high standard?

I don't think we should hold it to a higher standard than any other assessment tool that we look at. But I'm going to argue that I don't think that the AI manufacturer or the AI vendors themselves are holding themselves to those standards. So again, you know, you bring a new assessment tool to the market, you do a comparative study with an established assessment tool. You know, a great example is, you know, psychometric profiles. Someone builds a new psych, not only will they build their own validity and reliability, reliability indicators, but they'll also do a comparative study with a similar tool. One of the things that I find very interesting about some of the AI vendors out there is they're saying off the back of a 15 minute video interview, I can give you a profile of the first and on the five factor personality model. Great. What are your validity and reliability indicators? What? Yeah, no, we have all of this data that shows us and they'll hit you with a whole load of data analytics and stuff like that. And I actually had someone in one of the companies, I was working through some data analytics, go through their reports and he was like, yeah, it's okay, it's not brilliant. I couldn't understand it myself. So I outsourced it to someone that does. And he said, honestly, anybody can make these numbers do what they want. I went back to them and said, okay, so will you go head to head with the site that I use right now? Yeah. So I'm using this psychometric tool. Let's put a pool of candidates through the psych and through your tool. This site comes out on the five factor model anyway and let's see if it correlates. They didn't want to do it. So if you're not going to hold yourself to the standard that you're already working to, then you probably shouldn't do it. Right. But I do agree that our current processes are not optimal. We all know that interviews have very low validity in general. We know that recruiters get tired and as people get tired, they make mistakes and they suboptimize decisions. So there's no doubt that current processes are not as good as they could be. The question I think is how do we take these new tools and integrate them into our processes, making sure that we're holding the bar.

Last week on the show, we had our guest use a phrase that gave me a smile because it was a phrase that I had originally introduced when we were building our objectives and key results earlier this year. And one of our team corrected me and suggested I use a slightly different phrase. And the phrase that I originally used was data driven. He said, no, no, you shouldn't be data driven. It should be data informed. And we talked through the difference between those two seemingly similar phrases. And his point was that the data doesn't make the decision. The data informs a decision, but a human makes the decision. That's. We need to keep that to ourselves. I think when you talk about AI and assessment, I lean towards data informed. Is. Is good. Let's take different data points, but don't let the decision necessarily be data driven. And I come from a. And you may have seen tools like this, I was demoed last summer, a tool that is a chatbot that is being promoted out there. It's really interesting technology from an organization bought this technology off somebody else and it's. It promises that within a chatbot experience on your career site for hourly staff. Right. Kind of more blue collar, focused on retail initially. It could engage them, get their details, assess them, make a decision and go all the way to offering them an employment contract and a start date through the chatbot experience preloaded with the right information. And again, I was like, I'm a bit scared by that. Right. And I was sharing this with Somebody earlier that this experience exists out there and I don't know how many customers have taken it to that nth degree, but it also can stop at any point. Instead just use part of the technology to drive part of the experience. But, but, but they were going out there saying that we have a chatbot experience that can make do a full attraction selection offer and kind of pre boarding process for you without any human intervention whatsoever. And the candid could have a seamless experience. And this was really good for this market. I just kind of, I was like, I'm worried about this going really, can we do all that? Maybe you can. Because I don't know what, I don't know what the assessment criteria alternative is for these folks. Maybe for the hourly staff in that sector. The current process for hiring those hourly folks is so, you know, do this to then do that, you know, process flow mapping that it's better than the human version of it, I don't know. But you know, have you seen technology that goes that far? Nick and I mentioned some of the pseudoscience about, you know, the voice of text is one element, but the whole facial expression stuff that Hirevue and others got into trouble with, you know, what have you seen that kind of pushes the boundaries and makes you feel perhaps most uncomfortable in this space.

So I'm going to come back to that. But I want to talk about that end to end chatbot experience. I mean, that sounds really cool. It does sound awesome. Yeah. But again, I'm going to come back to candidate experience. Yeah, I think you and I. Sorry to lump you into my age group, Johnny, but I think you and I are probably a bit too old to want to do that. But my 23 year old daughter would love it, right? She would be all in and fine and cool with that and that wouldn't be a problem, particularly for, you know, something that would probably just be, you know, a side job or something like that while she's finishing her university or something like that. I can see how that would work. And obviously for some of those lower value roles, higher volume, lower value retail roles, your qualifiers are not sophisticated. So they may not even be building assessment in there. It may just be, yes, no questions. Will you work for this rate? Will you work for these hours? Are you able to drive to the store? I mean the questions can literally be that limited. And so I could see, you know, I can see how that would work and with certain age groups I can see how it probably wouldn't negatively impact candidate experience and could be applied really really? Well, in terms of what's really, really out there. I think what's really, really out there is this concept of. We were talking about it earlier. I've talked about it. You talked about it. Developing a personality profile of facial. There is a very innovative company out here in the Middle east who has been working with some of the leading facial recognition companies in the US to build a model like this. And they were really, they were really well informed and they gave me a different perspective to some of the large vendors that I've talked to in terms of facial. I have a big reservation about Facial Johnny, which is quite simply that I understand facial recognition is assessing somebody's emotion. Yeah, it's their emotion as they're answering questions. And it's based on the work of Paul Ekman, the micro expressions, the lighter me guy in that TV show. Right. And I wonder whether assessing somebody's emotion as they're answering questions is a good proxy for understanding their personality. And I challenged them on this and what they'd done was they created an experience. It wasn't like you're just answering questions about what you like to do and what you don't like to do that actually created this experience. So you went through this experience and you had to react to situations. And so then I can start to buy that, that that might actually be relevant and that actually someone's emotion as they're reacting to a situation would somehow be a better proxy for their personality. You see something that you know is you know, demanding and complex and you don't like it. Then you like things that are simple. You know, you, you're thrown in a situation where you have to deal with an angry person and you step up and you're calm and you handle it. Then you know how to deal with angry people. I can see how by creating some. It's a. Scenarios and situations and assessing someone's emotional response to that you could start to build a profile of their personality. So that I thought was, was out there and actually cool. The idea that you can assess somebody's personality from them asking. Answering some basic personality questions when or not personality questions. Sorry, competency based questions about the job. So in parallel you're doing the speech to text to get fit on competency and in parallel you're doing in a personality profile of their facial. That doesn't work for me because it just doesn't seem to be correctly connected. And actually when that second company that said we can do this when I challenge them on it and so I don't get the facial thing, the answer was you can turn it off. It's only 10% of the algorithm. Right. So again, you know, it's like, how much are we being sold? Something that is just sounds cool and we think we can improve the sell by adding this in there versus something that's really creating value.

Coming from Kathy Iverson listening live, saying evaluating the candid experience should drive the level of AI that a company chooses to use. I think that's part of what Nick was saying in terms of the canned experience. But the other side, just to counter that point, Kathy, is maybe the majority of people aren't being marginalized by the AI or they don't see it, and maybe the assessment quality isn't working. Maybe we are marginalizing certain individuals, discriminating against other individuals, but generally the kind of experience is better. So therefore, if you just use that as a metric, I'd argue that that's dangerous. I think Nick made the same point. And then Simon Johnson's jumped in with a question for you, a general question. It could be rhetorical, but how is the data collected by the chatbot initially to make a judgment on the outcome? And maybe you can expand, Nick, on maybe any of the providers you've spoken to. What is the data set they're using? You mentioned some academic models that some of them are just borrowing from that are in the open source environment. But how do most of them do it? From what you see.

I'm wondering if that is the question Simon is asking or if he's just asking, in the demo that you saw, how are they collecting the data? And I think the answer to that would probably be it's chat. Yeah. So it's just somebody's text responses and it's a text processing type thing in terms of how is the data collected in order to build the model? For most of the people I've spoken to, they're looking at between hundreds and thousands of candidates. Typically, if you talk to a vendor of AI tech and you say, okay, what is it going to take for you to develop this model? For my company, their numbers around 500. They want to sample about 500 candidates and use that to build the model as a minimum. But it is literally that they're taking your candidates and your applicants and they're either doing one of two things. The first one is they're watching your recruiters manage the candidates. Yeah. So they're connected into your system. They're watching your recruiters manage the candidates, watching the decisions the recruiters are making about those candidates. And Then the AI is building a set of inferences or spotting patterns and then using those patterns to differentiate the candidates. And the next stage is the AI runs in parallel, so your recruiters continue to manage the candidates and then the AI runs in parallel and differentiates the candidates and you look at what the correlation is with that and then eventually the AI can take over and be either managed by the recruiter, so it pushes suggestions to the recruiter, stack ranks the candidates, or the AI is actually making decisions live in the system. Now, clearly there's some big reservations about that because if it's just copying your team, I think like every recruitment leader out there, I'm comfortable to say that I have stronger members of my team and people that are developing and would have concerns about the ability to build in bias. Because we build this model and then we run this model and we update this model on a 6 month or 12 month frequency. Right. So. And there's been widely reported issues of bias being built into systems quite quickly.

Well, it actually came up in the article, they mentioned Amazon and it was more, it wasn't a video based solution, but it was a text analysis solution that were running on resumes and CVs for engineering roles. And they found that it had a bias towards males because they had a lot of males in the roles and they turned off the algorithm when they discovered this because they felt this was obviously unfair and that obviously is the danger. And you know, when he looks at inclusivity and diversity, this has been a big, a big topic around this AI assessment piece that, you know, folks come along and say, well, if it uses the models that you suggested and uses big data sets, that the data sets might already have bias in them, highly likely do a bias. So how do you correct for that bias? What's your comfort level with the level of correction that goes into that and how would you see that correction correcting actually happening?

So I think with the good vendors that I've spoken to, they're actually, as part of their implementation, they're actually monitoring for adverse impact and correcting for that as they're building the model and they're being very transparent with you about why they're doing that, what they're doing and how they're doing it. But these are the vendors that are strong enough and I guess have the integrity in what they're doing to say, don't just use our solution, we need to build the solution for you, we need to build a data set for you. There are a lot of vendors out there that are Just saying, yeah, it's good to go take it in that situation. I don't know what's being done on the subject of bias and facial. And the point that I wanted to make earlier was this company that I was talking to in the Middle east that was working with the facial team in the US were the only company that said to me you actually need to build a facial data set model for each type of ethnicity because although the micro expressions thing is universal because of the lighting conditions, face shapes and sizes, skin colors, etc. The AI recognizes differently. You actually need to build out five different facial recognition models according to different ethnic groups. Which I thought was fascinating and scary. Out of all the vendors I've spoken to, only one of them told me that or was thinking like that. It's.

It's true. It also reminded me of a line from the lady in the article we discussed at the start where she'd said that she performs better when she develops a rapport with the interviewer. And it, you know any of these asynchronous types of assessments because we use it for some of our roles. We don't have the AI part but we have a. We use a modern hire and their video assessment solution for some of our roles on scale. And somebody does an asynchronous interview. But you, you know, I hadn't thought about the perspective of somebody who just knows in themselves are not going to perform as well speaking to a camera versus speaking to a person. They just, that's their style. And if that isn't a requirement of the jobs we have, which I know we don't consider that then they are being unfairly biased in this process. Perhaps you know that they're even in the simplest version of this that it doesn't take into consideration lighting someone's style as they might work in a conversation. And yes, generally speaking the canned experience data suggests to Kathy's point that these types of processes are better for candidates. They prefer them. They can have assessments in their own time. They can do them after work whenever suits them, not at just the schedule of the recruiter or the hiring manager. All that stuff is great. Totally.

Yeah.

They can re record their answers. Different things built in there. It does, it does marginalize a bunch of folks for whatever reasons who just aren't going to be as proficient in this. And yes, the laws and parts of the world and I'm not sure what the US laws on this particular area. Nick, if any of our audience are listening and know more, they could enlighten us In Europe, you have to provide an alternative process to anybody who selects it. That's just a requirement. And I think your point around transparency, even if it's not a requirement, transparency about how we assess and always offering an alternative path is probably just a good policy, would you say?

100%? You know, I think, you know, people need to understand the process they're going to go through. They need to be able to know and believe that that's a fair process. And I agree it's hard to perform to a blank screen. Very difficult. You want someone reading back to you. But I guess this is one situation where the introverts have the advantage, right?

It is, it is, it is, it is. But another piece that's relevant to this.

Right.

You know, one of the, one of the elements that comes up often is, you know, these tools being used to not just look at, as you mentioned, your competencies or skills or experience and using the analysis, but increasingly they're talking about tools like this being used to assess what's called most popularly cultural fit, which again is a controversial phrase by itself and it's misinterpreted. Right. But let's say, even if I use it in the term, I think it should be used, which is probably values assessment, somebody who aligns to the values of the organization or the values required for this job. What is your kind of view of AI or technologies capabilities, these automated technologies of evaluating something like that?

I guess again, it depends to how it's being done. If we've got somebody doing a classic online values assessment, classic psychometric model technology is running on the back end to assess the person. The validity and the reliability has been established. You know, it's, it's working to the right standards. Then, okay, if we're doing, you know, a speech to text processing thing, again, depends on the questions that are being asked and how the speech to text is working. But I think my mental struggle is whenever I'm interviewing somebody, I'm not ticking on my sheet whether they said the words. It's just, it's, it's not what we're doing, is it? What we're doing is listening to what they said, the context of the situation they were in, how they chose to respond, and it's that that's informing the decision as to whether they're likely to be a fit or not. Equally with, you know, the facial recognition. Unless someone's being put through a scenario where their emotion is going to be a reaction to something that would tell us something about their values, I don't understand really quite how the proxy works. And that's, that's the gap that I have. So my personal view is for some of the transactional stuff at the front end, sifting cvs, sifting application forms, whether that's, you know, text analysis or chat analysis, that makes sense because the recruiters are, you know, kind of doing that proxy anyway. And there's some really good tech out there that will actually build the association between a company, a job title, you know, and the skills that they're seeing in there. And think more like a recruiter rather than just spotting buzzwords or keywords in a cv. As soon as we get into something a bit more sophisticated that requires inference, I think it's challenging at the moment to support AI.

I had a conversation recently with somebody about the same topic and we talked about the use of technology and that person again had indicated that the technology is suitable for the elements you mentioned there for sure. But to Kathy's point about candid experience, we also talked about how you can't just look at an entire hiring process or even a role and say AI can do all of that. If you look from the candidate experience perspective, there's a point at which, there's a point at which, depending on your demographics or background or culture, the starting process might be preferable to do with a chatbot. It's like I'm applying for 30 jobs chatbot, let's do this. Don't waste my time in the old fashioned process. So I'm happy to do that. But there's also part of the process, irrespective of the roles or how valid the assessment criteria is, that it's better to introduce a human from a cam's experience and selling perspective. And I, I come at this and it reminds me of when I was 20 years of age and I was doing a whole bunch of interviews for different jobs. You know, I was a commerce graduate out of university coming out, but to come out and you know, when you graduate a business degree, you're kind of like saying, I have no idea what I want to do, but I want to work in an office. And I was applying to loads of different jobs and one was with an insurance company. I went for an interview and instead of an interview I was handed a piece of paper and a pencil and I was sat in a room with 30 other people and we, we filled out this assessment form with lots of numerical and verbal reasoning questions. And at the end of it I finished my sheet, I put my hand up and the first, I said, what Do I do next? Give me your sheets. Like where do I go next? You can go home now. I went home and three days later they sent me a contract of employment and I had no job offers and I had no job to go to. But I didn't turn up on the first day. I didn't accept the job because I'm thinking I've never met a single person in this organization and that may have been the exact assessment that they needed to decide if I was right for the job, for whatever low end entry level job it was. And it might have been an employment market whereby they had lots of applicants. But from my perspective, I'm going, I'm not going to work for a company I've never, ever, ever met. So I think there's an element where you have to balance the technology and what it can do. And even if it's brilliant, with what do people want and you mentioned that at the start, Nick, that it's not just about what the technology can do and how great the assessment is and is it better than the way we do it today. You have to consider the experience that somebody's looking for. With that in mind, you know, can you maybe give us a general guide as to when to use AI, when not to use AI to expand on some of the comments you made a few minutes ago?

Certainly. So I think that the easiest place to use AI in recruitment is helping to administer the process itself. So, you know, there's a whole load of applications where we can use AI and robots and RPA and stuff like that to manage the administration of a lot of the recruitment process, whether it's putting together a contract stage, progressing candidates against certain criteria, etc. Etc. All of that take away the admin from the recruiter so they can focus on candidates and managers makes a lot of sense. I think that depending on the type of hiring you're doing and the volume of candidates that you're handling, I see great potential for AI to help with front end filtering and really cover candidates properly. I think, you know, we all know that, you know, recruiters who have 500 applicants a role and not going to be able to do a screen on all of those candidates on paper properly. Yes, we have, you know, qualifying questionnaires that will take out the bottom 50% or whatever, but even then, you know, there's, there's going to be some sub optimization there and I think that value can be added where the screen is a lot less sophisticated and has the potential to be automated. The key here is where do you set your, where do you set your limits? Yeah. So you don't want to be too, too tight with that. It still needs human, I think to look at it and you know, to place, you know, a good judgment on the individuals later stages particularly for you know, higher value roles. I don't yet see a place for it until they can show us, the vendors can show us that the bar, the integrity of the assessments is at the same level as other assessments that we're using. And I think that that's really important. We always come back to we need to hold the bar. And in terms of the candidate experience point, we have to remember that assessment is always a two way process. Candidates are assessing us. You assess that assurance company probably very, very correctly. Johnny, this is not a company I could work for. They don't care about me, they're not interested in me. You know, frankly it was a cannon fodder role. Right. We needed a lot of people to crunch a load of numbers and we don't care who they are or what they're like. And I guess from that perspective and probably in the market they were able to play that way but certainly today we can't. We need to engage people, we need to inform them and that culture fit assessment in particular is two ways. We need to open up and show them who we are and how we are and how we do what we do so, so they can make sure that they're going to be comfortable in that environment. And so I think certainly as we get into the later stages of assessment we need to go high touch and stay high touch particularly for our niche and key roles because otherwise we will risk making bad hiring decisions.

I'm going to jump into some for the last few minutes there's some really.

Long comments and questions. Great.

Cathy Irison is jumping in on that last point saying yes and the candidates should get the chance to see, to interview you as a potential employer and see the culture so totally backing that up. Get it Nick? Simon Johnson is just jumping back in with direct questions. I comment more than anything else. Nick. His fear is what you said about AI being used at the front end, you know, I. E. Text, etc. You're offering a set of rules for people to understand eventually master to get through this part of the process. It may not help in the future. It could merely offer a problem for people to solve to get through the gatekeeper. I guess what he's saying is will people gain that then?

I think it's already happening, Simon. Yeah, I think you're absolutely spot on and you know, I'm aware of people talking about how to game the AI. If you search online, how to game AI assessments or how to pass AI assessments, you'll see some of this stuff through to people saying, you know, hide boxes in your CVS with all of these keywords in it. And you know, people have been doing that with ATS's for a long time. Right, right. So absolutely, that's, that's coming and you know, we and the vendors will need to learn how to manage that.

Another comment here from Ronan o', Sullivan, by the way. It reminds me of the. Yeah, I remember back in the day you'd have white text on a white background which would have a list of all the big universities because they candidates knew that the big companies were just looking for those universities through text analysis so they'd have it there for the machine to read. Nobody else. I wrote in o' Sullivan saying body language is usually influential in how you assess and perceive a candidate. Some psychologists suggest these non verbal signals make up the main part of our communication. How can AI assess these sometimes imperceptible cues? Question to you, Nick.

I'm not prepared to answer that question, Jody. I don't have the answer to that question. I think it's a great question, Raymond and I. Wouldn't it be fantastic if one of the vendors was on and could answer that? I don't know. They're doing facial recognition. I don't know whether they have done any work on understanding body language and tracking and monitoring. Body language actually could be a very interesting direction for a product. Right.

I love it and I'd love to know, would we get a vendor on the show to discuss those things? I'm sure informally over beers you could get someone to have a discussion. Will they do it?

Anybody that wants Johnny to get a vendor on the show in the chat now? Yeah. Let's set him the challenge.

Let's. Let's try and do this. Nick. I can't believe our time already. This is always flies by. It's fascinating. We have great engagement with the audience. Thanks so much to those of you who are listening live. Those who ask questions make comment. AJ's HR, Kathy, Simon, Ronan and others really appreciate that live chat and interaction. It's really, really good to get. Nick, we're at time. It's that time where I'm going to ask you to once again drop pearls of wisdom as you did the last time you're on the show. So I'm going to ask you for your second best pearl of wisdom. What can you add to our shortlist of advice for our audience, for leaders, for recruiters out there?

Last time I quoted my dad, didn't I? You did, yeah. Now I'm going to quote one of my boys. So I have, I have three young boys, all of whom test me constantly. And I often come back from work when, you know, when you've got your work head on, on and you know, you're talking in business speak and stuff like that, my middle boy James will turn around to me and say, daddy, that's too complicated. Make it simpler. Yeah, and I think that's generally a good rule for life. Keep it simple.

I love that. The kiss analogy. Keep it simple.

Stupid.

I only just spoke with a colleague before we went live who had language. I said dumb that down. Simple words that people can understand. Communication is all about simplicity and don't over complicate things. Nick, thanks so much for joining us. I know it's a bit later in the evening, probably have kids to get to bed, but it's been a real pleasure. Maria, Simon, Kathy, all saying hello and thank you so much. Really appreciate your input that Nick, we're getting you back again. It's going to be a regular every six month slot I think. Nick, you don't have a choice. Now I know you're crazy busy so I really do appreciate you taking the time out. The market's on fire. You're only a big team trying to do some fantastic work there and people will see in the Middle East. So wish you every success and luck. Thanks for your time again and good night.

Thank you very much to those of.

You who've joined us, thank you. But we want you back tomorrow or it could be today depending on whether you're on the podcast. We have our latest round of STLive. ST is our live leader event we run every quarter. It's running on the 16th of September and it's a live free event you can register for by going to SocialTalent.com live. And tomorrow we're going to be talking about building a hiring culture. Forget if you listen to the podcast and it's the first day, it could be today. Building a hiring culture. We're going to have a list of panelists from Adidas, from Cisco, from pct, from tons of great organizations, very senior heads of talent, talent acquisition, plus our guest experts walking through what is a hiring culture? How do you build them? What are some of the pitfalls? And myself and my good friend John Basilica will be chairing those different panel discussions and chats and over a 90 minute free action packed seminar. And that's going to be going live. You must listen to it at 4pm UK time. It's 11am on the east coast of the US, 8am on the west coast. Same time as the shortlist but on a different day. Thursday, September 16th. You can register now at socialtown.com forward slash live. And if you listen to this podcast and you've missed it, if it's already happened, we'll be running it again in November. We've got a great show in November talking about the great resignation on how to scale your hiring team. Another one again in January talking about the bigger theme of diversity, equity and inclusion. So get your name on the list now for the next one if you've missed this one. So come back tomorrow and join us for Social Talent Live. Or come back next week. We've got a brilliant show next week, right? We're going to try and close out September with a bang. With two more shows to go. Show 68 is going to be my good friend Jer Finn. Jer, Senior Director of People, Strategy, Operations and Innovation at Twitter. She's awesome. She has the ear of the business. She's just a super smart woman who really knows a lot about things that I know nothing about and she is a great conversation. She's been on our Social Talent Live panels before, talking about her views on leadership at a really difficult time for Twitter at the time. Which was awesome to hear that. Always honest in her opinions. Come back next week, Wednesdays 22nd of September to hear Jared talk about reinventing the employee value proposition, the EVP. So come here Jer from Twitter on reinventing the EVP. That's Wednesday 22nd September live. You can join us LinkedIn on YouTube at 4pm UK time. That's going to be 11am in the US West East coast and 8am on the west coast. Or check out the podcast. And that podcast typically drops on a Wednesday evening, Thursday morning on Spotify, Apple or wherever you find your podcast. Thanks for joining us today. Hope you really enjoyed Nick's input. Lots of great engagement. Come back next week and don't forget it's a crazy world out there. Be careful with the technology you buy, be careful how you deploy it and keep it human. And as Nick said, keep it simple. Thanks for joining us.