A battle over facial recognition is brewing in upstate New York, where earlier this month Lockport Schools became one of the first US school districts to turn on the controversial technology in all of the K-12 buildings that serve about 5,000 students. There, the district’s deployment of the tech has ignited a squabble between parents, school board members, state education officials, and privacy advocates over facial recognition’s risks and cost-effectiveness. Now a state assembly member plans to double down on a proposed bill that would put a moratorium on the tech in New York state schools.
The district’s plans, and the confusion it’s spurred, have made Lockport a test case for how facial recognition will be used in schools, and for how government officials will respond to mounting criticisms that the tech violates student privacy, is plagued by inaccurate results, and isn’t worth its high price tag.
Proponents of the tech argue that it keeps people safe by automatically tracking who is where and when. In school districts, including Lockport, facial recognition is primarily being used to enforce campus safety watchlists, by sending alerts when someone dangerous — or otherwise unwanted — shows up in a camera feed.
But critics say facial recognition could be used to surveil students and note that it builds up databases of sensitive information about peoples’ faces, which schools may not be prepared to keep secure. They also highlight that facial recognition is known to be less accurate on people of color, and that the technology could ultimately exacerbate the school-to-prison pipeline.
But such tight limits on who can be included in the database have left some questioning its real utility. One Lockport school board member, Kyle Lambalzer, now wants the state to refund the $1.4 million dollars the district was granted to spend on the technology, arguing that New York education officials didn’t properly investigate the tool. The money, he argues, could have been spent on more effective security systems.
Jim Shultz, a local parent who has been outspoken about the school district’s plans, says who is in the database isn’t the only concern. “As my daughter’s high school classmates are busy right now walking around the halls of Lockport High School, from one period to another, their faces are being scanned to see whether they match anybody in the database,” he said.
Another issue: It’s not quite clear who exactly controls what categories of people are included in the database, should the school board want more groups added in the future. For now, which individuals are added to the system is overseen by the district’s school superintendent. That highlights complicated questions over who, ultimately, has a say about what data school’s facial recognition systems can collect.
Earlier this month, the Lockport Union-Sun & Journal, which has covered the school district’s plans extensively, reported that the school board’s president, John Linderman, said he couldn’t guarantee that student photos would never be included in the system. Linderman confirmed to Recode that students can’t be included in the system unless the policy is altered, in consultation with the state education department. Theoretically, the policy could still change.
The confusion over what the rules are has left critics of the technology demanding more specific regulation. At yesterday’s congressional hearing on facial recognition, Rep. Ayanna Pressley emphasized her concerns about the use of tech in US schools.
As the debate over Lockport’s facial recognition tech has unfolded, a member of the New York Assembly, Monica Wallace, has pointed to it as an example to bolster her argument for state lawmakers to pass a bill she proposed earlier this year, which would put a two-and-a-half-year moratorium on schools’ use of facial recognition, giving the state time to study the tech and its usefulness. The state teachers union and the New York Civil Liberties Union, which is urging state officials to rescind Lockport’s approval to use the technology, both support the legislation.
“Once we allow [facial recognition tech in schools], it’s going to open a floodgate for millions and millions of dollars of state dollars being spent on technology that is really questionable in terms of its reliability, in terms of its accuracy, and in terms of its value, versus the risks that come with using this technology, in terms of privacy, in terms of false positives, and so many other things,” Wallace said.
The state education department says it would not comment on a hypothetical scenario in which Lockport did go ahead and include students in its system. They also said that they urged the school district to consult its local counsel.
The district’s policy also declares that the facial recognition system will not be used for disciplinary purposes. But Shultz has a warning: “As we’ve learned with Facebook, it really doesn’t matter at all what the managers of a system say they will or will not use it for because of its retroactive capacity. It doesn’t matter, because they can just change their position later.”
Open Sourced is made possible by Omidyar Network. All Open Sourced content is editorially independent and produced by our journalists.