Ongoing Face Recognition Vendor Test (FRVT) Part 2: Identification
This study was published in 2018.
Executive Summary This report documents performance of face recognition algorithms submitted for evaluation on image datasets maintained at NIST. The algorithms implement one-to-many identification of faces appearing in two-dimensional images. The primary dataset is comprised of 26.6 million reasonably well-controlled live portrait photos of 12.3 million individuals. Three smaller datasets containing more unconstrained photos are also used: 3.2 million webcam images; 2.5 million photojournalism and amateur photographer photos; and 90 thousand faces cropped from surveillance-style video clips. The report will be useful for comparison of face recognition algorithms, and assessment of absolute capability. The report details recognition accuracy for 127 algorithms from 45 developers, associating performance with participant names. The algorithms are prototypes, submitted in February and June 2018 by research and development laboratories of commercial face recognition suppliers and one university. The algorithms were submitted to NIST as compiled libraries and are evaluated as black boxes behind a NIST-specified C++ testing interface. The report therefore does not describe how algorithms operate. The evaluation was run in two phases, starting Feburary and June 2018 respectively, with developers receiving technical feedback after each. A third phase commenced on October 30, 2018, results from which will reported in the first quarter of 2019. The major result of the evaluation is that massive gains in accuracy have been achieved in the last five years (2013- 2018) and these far exceed improvements made in the prior period (2010-2013). While the industry gains are broad - at least 28 developers’ algorithms now outperform the most accurate algorithm from late 2013 - there remains a wide range of capabilities. With good quality portrait photos, the most accurate algorithms will find matching entries, when present, in galleries containing 12 million individuals, with error rates below 0.2%. The remaining errors are in large part attributable to long-run ageing and injury. However, for at least 10% of images - those with significant ageing or sub-standard quality - identification often succeeds but recognition confidence is diminished such that matches become indistinguishable from false positives, and human adjudication becomes necessary. The accuracy gains stem from the integration, or complete replacement, of prior approaches with those based on deep convolutional neural networks. As such, face recognition has undergone an industrial revolution, with algorithms increasingly tolerant of poor quality images. Whether the revolution continues or has moved into a more evolutionary phase, further gains can be expected as machine learning architectures further develop, larger datasets are assembled and benchmarks are further utilized.
https://nvlpubs.nist.gov/nistpubs/ir/2018/NIST.IR.8238.pdf
It is safe to assume that their ability has improved, most would agree that 99.8% is a very high success rate. The main issue they have is identifying people from old photographs of poor quality.
My point is that AI can identify a face accurately. If the military put many of these faces on a kill list AI can identify them and let the military know when they have spotted them, or if AI is allowed to make the kill just go ahead and kill them after identifying them.
Clearview AI is already being used by the US government to identify faces for a variety of security threats.