Part 5/15:
While many officers recognize the limitations and problems of facial recognition—such as inaccuracies and biases—some exhibit what is called automation bias. They tend to rely heavily on system outputs, even when other evidence suggests otherwise. For example, a facial match might lead an officer to confirm suspicions prematurely, especially when the system reports near-perfect accuracy, such as 99%. Yet, errors can—and do—occur.
An illustrative case involved a man named Randall Quran Reed, wrongly identified via facial recognition, leading to a wrongful arrest that kept him in jail for a week. Such incidents underscore the risk of over-reliance on imperfect AI, with mistakes often resulting in innocent people suffering legal and personal consequences.