Police claims on live facial recognition bias not supported by facts: UK professor

Police in London have made 140 arrests during the Notting Hill Carnival over the weekend, with 13 individuals apprehended after positive identifications using live facial recognition (LFR). Figures released by the London Metropolitan Police Sunday night break down the overall arrests by offence, but not specifically those following identification with LFR. Of the 140 total arrests, the most common category of offence was “other” (42 arrests).

A breakdown of those arrests following LFR identification has been requested from Met Police.

The festival, which celebrates Afro-Caribbean culture, is scheduled to continue on Monday and so are debates about police deployment of LFR.

The Met police’s claims that its LFR system is bias-free are not supported by the study they themselves commissioned from the UK National Physical Laboratory (NPL), according to Pete Fussey, professor of Sociology at the University of Essex and co-director of the Centre for Research into Information, Surveillance and Privacy (CRISP).

“The claims the Met are making about the absence of bias from the NPL report are not substantiated by the facts in that report,” says Fussey, who has advised both the UK government and the UN on law enforcement’s facial recognition use.

In 2023, the UK’s largest police force collaborated with the National Physical Laboratory to test the equitability of its NEC facial recognition system. The study concluded that the system is “very accurate.” However, when LFR was used at lower face-match thresholds, the system displayed a slight difference in performance between some demographics.

The testing showed that false positive identifications increase at lower face-match thresholds (0.58 and 0.56) and start showing a “statistically significant imbalance between demographics, with more Black subjects having a false positive than Asian or White (sic) subjects.”

Following the study, the Met police continued to use the LFR system, noting that there was no statistically significant bias at a setting of 0.6 or higher. Fussey, however, claims that the sample the testing used was too small to support the claim, The Guardian reports.

“The decisive conclusions the Met are stating publicly are based on analysis of just seven false matches,” says the professor. “This is from a system that has analysed millions of Londoners’ faces. It is a weak statistical basis to make universal claims from such a small sample of false matches.”

During the NPL study, the police LFR system compared a live camera video feed of faces against a predetermined watchlist. The watchlist included an image dataset of 178,000 face images, 20 times larger than the typical police watchlist size. The larger size was chosen to ensure there would be sufficient data on true and false positive alerts.

The facial photographs were taken of a cohort of subjects in different conditions. The testing was conducted over five days in July on the streets of London and Cardiff, with the maximum estimated crowds reaching 38,000 people within one day. During the testing, the cohort subjects passed by facial recognition cameras 4,000 times.

The testing took a total of 34.5 hours, which, according to Fussey, is shorter than in some other countries performing similar assessments.

The professor also claims that the research did not assess false matches at the LFR at a sensitivity setting of 0.64, which the Met Police uses. The report, however, notes that at a face-match threshold of 0.64 or higher, there were no false positive identifications.

Source: Facial Recognition Technology in Law Enforcement Equitability Study – Final Report, March 2023.

In 2019, Fussey co-published his own independent review of the Met Police’s use of LFR after 10 conducted trials. The police force, however, disagreed with many findings of the report, including Fussey’s interpretation of accuracy figures.

The police surveillance researcher is not the only one who is sounding the alarm over the Notting Hill Carnival’s LFR deployment.

Last week, a letter from a group of 11 civil liberties and anti-racist advocacy groups urged the police to reconsider their decision as LFR deployments may inflame concerns about the force abusing authority and perpetuating racial discrimination. The UK human rights regulator, the Equality and Human Rights Commission (EHRC), also called on the police to ensure rights are respected.

The Met police, however, claim that the technology is necessary to identify and intercept those who pose a public safety risk. During 2024, the Carnival saw two murders and seven non-fatal stabbings, while the police made 349 arrests.

“Live facial recognition is a reliable and effective tool,” the police said in a statement. “It has led to more than 1,000 arrests since the start of 2024. In the same year, only 1 in more than 33,000 potential matches were proven to be inaccurate.”

Article Topics

accuracy  |  biometric bias  |  biometric matching  |  biometrics  |  facial recognition  |  London Metropolitan Police  |  National Physical Laboratory  |  NEC Software Solutions UK  |  police  |  real-time biometrics