In this monologue, Toju Duke outlines a series of high‑profile incidents where live facial‑recognition systems produced false identifications, leading to wrongful arrests and humiliating police encounters. He cites cases from the United States (a Tennessee grandmother falsely linked to a North Dakota bank fraud, the 2020 Detroit wrongful arrest of Robert Williams) and the United Kingdom (a student’s Doritos bag misidentified as a firearm, a South‑Asian‑heritage man arrested for a burglary he never committed, and a Cambridge study revealing that Black people are disproportionately flagged by Essex police cameras). Duke argues that these errors stem from biased, under‑represented image datasets and the removal of human judgment from critical decisions. He calls for diversified training data, rigorous risk assessments, continuous monitoring, and the preservation of human agency in surveillance workflows, urging police agencies to pause or heavily regulate the deployment of facial‑recognition technology until these safeguards are in place.
Chapters
The challenges facing live facial recognition still exist and are compounding across the world. In July 2025, a grandma from Tennessee, aged 50, was falsely arrested after facial recognition software used by the police linked her to a bank fraud case in North Dakota. She was arrested while babysitting her 4 grandchildren and spent nearly 6 months in jail. Angela Lips had never been to Dakota and was in Tennessee when the crime took place. In October 2025, an AI system mistook a student's bag of Doritos for a firearm. And called the local police to inform them the student was armed. The poor kid was made to go down on his knees and coughed and searched, and the police found nothing on him. Earlier this year, a man in the UK was arrested for burglary in a city he had never visited after facial recognition software confused him with another person of South Asian heritage. And in 2020, Robert Williams, a man from Detroit, was arrested in front of his family for a crime he didn't commit. Due to inaccurate results from facial recognition software. I spoke to Sky News about this in 2023, emphasizing the need for diversification of image datasets. Most recently, the police in Essex, UK, have decided to pause their use of facial recognition cameras after a study from the University of Cambridge discovered racial bias in its results, where Black people are significantly more likely to be identified when compared with other ethnic groups. And what's the one thing all these people have in common? They're all human citizens, and they come from underrepresented groups. Female, racial groups, and age groups. It blows my mind that these issues are still recurring over the years. I expect police organisations to be fully aware by now of the false positives in facial recognition software and surveillance systems. Using facial recognition to make grave life-changing decisions such as these should be taken very seriously. Risk assessments, ongoing monitoring, and equal representation of all groups in society must be carried out if using facial recognition systems. Taking human agency and human autonomy out of the decision-making process will only lead to more errors like these. Let's not forget that human lives are important, and we cannot fully trust the results of AI systems today.
