AI is slowly making its way into every facet of life, and that means more and more fields are being manned by machines that can’t be held accountable. In many cases, AI going wrong is normally confined to a poor Google search or bad Photoshop, but on October 20, police showed up armed at a high school as a result of an AI miscalculation.
As reported by WMAR 2 News, multiple police cars showed up to a high school in Baltimore County, Maryland, to apprehend a student who AI flagged as having a gun (bodycam footage from ABC 7 Chicago).
All students on the premises at 7:23 PM were checked by the police officers, and no gun was found.
It wasn’t until three minutes after apprehending the students that the police officers spotted a blue bag of Doritos in the bin, and put together what had gone wrong. The Omnilert AI gun detection system, which monitors the school’s cameras, alerted the school that a gun may be on the grounds, and the authorities were informed as a result.
In an attempt to explain what had happened to the students, an officer says, “I guess just the way you guys were eating chips… Doritos or whatever, it picked it up as a gun.” After the problem was resolved and the culprit (a bag of Doritos) was found, one of the police officers told a student, “AI is not the best”.
A spokesperson for Omnilert told CBS, “Within moments, the event was marked as resolved in our system. Omnilert’s involvement concluded at that point, and the system operated as designed — detecting a possible threat, routing it for human review, and ensuring rapid, informed decision-making.”
Baltimore County Public Schools superintendent Dr Myriam Roger claims, “The program did what it was supposed to do, which was to signal an alert and for humans to take a look to find out if there was cause for concern at that moment.” Effectively, Omnilert is intended to flag any problems to higher authorities, who are then supposed to manually check suspicious activity, and their job is to verify real threats.
Rogers says, “This system is AI, and it’s looking for certain elements, and then humans have to verify them.” Omnilert reportedly ‘welcomes’ any “review or discussion that helps the public better understand how our system works.”
Iazzy Patoka and Julian Jones, two councilmen from Baltimore County, reportedly want to review the Omnilert system in the wake of this incident. Jones says, “Thank god it was not worse”, and asks, “How did it come to be that we had police officers with guns drawn approaching a kid because of a bag of Doritos?”
Taki Allen, the student identified, told WMAR2 News, “Now, I feel like sometimes after practice I don’t go outside anymore. Cause if I go outside, I don’t want – don’t think I’m safe enough to go outside, especially eating a bag of chips or drinking something. I just stay inside until my ride comes.”
