The Bologna Online Evaluation Platform adds morph attack detection benchmarks

In the first European Association of Biometrics (EAB) lunch talk of 2024, hosted by CEO Dinusha Frings, Lorenzo Pellegrini of the University of Bologna’s Biometric System Laboratory (BSL) provided a detailed tour of the Bologna Online Evaluation Platform (BOEP), a system for benchmarking the performance of biometric algorithms.

Based on the FVC-onGoing platform, which evaluates algorithms for a variety of fingerprint, palm and face biometrics, BOEP adds the main subject of Pellegrini’s talk (and also its title): “Evaluating Morphing Attack Detectors.” Specifically, BOEP targets two types of face morphing attacks, offering single-image morph attack detection (SMAD) and differential morph attack detection (DMAD). Algorithms can be submitted via the BSL’s website, but all tests are conducted offline on sequestered data sets in a controlled-environment algorithm repository.

Submitted algorithms that undergo evaluation will receive a report on common performance indicators and metrics such as age and ethnicity, and have the option to publish the results on a public leaderboard.

The system is available to academics, industry actors and independent developers, who can register on FVC’s website. Users can submit algorithms for benchmarking in one of three formats: as a Python script, a Linux dynamically linked library subject to compliance, or an executable in the form of a Win32 console application. There is an option to describe the algorithm, in case it is published for public access. A test benchmark can be used to see if the submitted algorithm functions. Beyond that, it is run through protocol verification, executed against benchmark data sets, and processed through a performance results generator before being returned to the user, who can opt to publish the results on the leaderboard.

Single-image and differential image morphing detection use different systems

In terms of the difference between how SMAD and DMAD score the probability of morphing, Pellegrini explains that SMAD contains face marking detection benchmarks that analyze a single ISO-compliant image to detect for morphing that mixes the faces of two subjects. DMAD, meanwhile, is the same process, but requires the algorithm to analyze a bona fide image of the face in question and compare it against a suspected image.

According to Pellegrini, each benchmark has different complexity elements. Some benchmark images are manually retouched or printed and scanned, while others are purely digital. There are multiple options for both SMAD and DMAD that users can select. Pellegrini notes some idiosyncrasies of the system, including restrictions on how many algorithms a user can upload in a certain period of time, and a tendency for results to take time.

One key feature of BOEP is that its execution environment can be replicated on a PC using Docker, differentiating it from the NIST execution environment for FVC-onGoing. Submitted algorithms are stored in the environment for a record of versions.

In general, the BOEP can be considered another useful way for developers, researchers and businesses to prove the trustworthiness of their biometric technology.

Article Topics

adversarial attack  |  biometrics research  |  Bologna Online Evaluation Platform (BOEP)  |  EAB  |  EAB 2024  |  European Association for Biometrics  |  face morphing  |  facial recognition  |  morphing attack