Soft biometrics experts wanted on teams tabling proposals for IARPA video re-ID system

The U.S. Intelligence Advanced Research Projects Activity (IARPA), the government’s intelligence research arm, is in the early stages of developing new re-identification algorithms to track people, vehicles and generic objects across discrete video footage, and is seeking proposals from multidisciplinary teams.

In a video posted to the YouTube channel of the Office of the Director of National Intelligence, Dr. Reuven Meth, a program manager for IARPA’s Video Linking and Intelligence from Non-Collaborative Sensors (LINCS) project, says the tool will be used to identify patterns and routines and “will work in the open world setting where there is no knowledge in advance of where the people are, where the vehicles are, or which sets of people and vehicles are to be re-identified.”

“Consider a swarm of bees,” says Dr. Meth. “One may be interested in knowing a specific bee such as the queen bee – where is the queen bee throughout the collection? We may also be interested in knowing where the bees travel in general, how far they traveled, which flowers did they visit, et cetera. Knowing the path of where the bee traveled gives insight into its routine habits.”

Part of the method for differentiating individuals in Video LINCS is soft biometrics, which are traits like age or weight that can be used to differentiate individuals from others with certain contexts, but are not useful out of that context or as stable identifiers.

Technical details released at a recent information event specify that Video LINCS autonomously associates objects across diverse, non-collaborative, video sensor footage and maps re-identified objects to a unified coordinate system for geo-localization in a common frame of reference. Per a draft Funding Opportunity Description available here, “the reID and geo-localization algorithms will distill raw pixel data into spatio-temporal motion vectors, providing the ability to analyze these patterns for anomalies and threats. While the ultimate goal will be to re-identify general objects, the program will start with person reID, progress to vehicle reID, and culminate with reID of generic objects across a video collection.”

The system must be self-contained, in that it is able to analyze and re-identify objects within an arbitrary video collection without an external reference dataset such as a gallery or library, and with no prior knowledge of said objects. It has to accommodate diverse video sources and understand when to add new objects to its own library; according to the draft description, “the lack of an a priori gallery of objects to be reidentified will require systems to autonomously determine when to expand system generated galleries to include additional objects vs. expanding matches to existing objects.”

It also needs to be end-to-end, able to ingest video accurately and output reidentified and mapped object location data.

IARPA says the R&D program will last for a 48-month period and unroll in three phases. The draft call for proposals notes the expectation that teams will be collaborative and multidisciplinary. Skills listed in the scope of relevant expertise include AI, computer vision, machine learning, vehicle fingerprinting and soft biometrics.

Article Topics

biometric identification  |  biometrics  |  IARPA  |  re-identification  |  soft biometrics  |  video analytics  |  video surveillance