In the hopes of deterring violence, schools are turning to big data analytics to examine social media posts for the earliest signs of violence – depression, resentment and isolation. Shawsheen Valley Technical High School in Massachusetts has turned to Social Sentinel, a data analytics company that says it can use the type of threat detection police agencies use to identify students at risk. But experts worry student social media mining, even with the best intentions, is a slippery slope to treating students the way we treat suspects.
Photo: Orlin Wagner (AP)
Speaking to NPR affiliate WBUR, Social Sentinel’s founder and CEO, Gary Margolis, says the company worked with linguists and psychologists to train its AI on missives of students who went on to harm themselves or others. Margolis says they have created a library of 450,000 indicators that may trigger alarms.
“We went back, for example, and looked at the language that school shooters, as one example, have used in the past in various manifestos – what’s been published or that they have shared on social media,” Margolis said. “And we went to understand similarities and patterns. And we can teach computers, to an extent, how to identify some of that nuance.”
Screenshot: Social Sentinel
The students in the WBUR story reportedly had no idea the school was using Social Sentinel’s service, but often social media posts are open to at least some type of metadata analysis without people realising. The complexity of this analysis means the service, ideally, flags accounts before any explicit threats are made and lets the school intervene with social services.
“We are not a surveillance tool; we are not a monitoring tool; we’re not an investigative tool,” Margulis said.
Shawsheen Valley is not the only school to try this approach. The Miami-Dade school system requested $US30 million ($39 million) in state funds to overhaul its security system, including hiring staff to search social media. Schools in Wilson County, Tennessee also use a social media monitoring software that sends alerts when problematic keywords are detected, while another school in Fresno, California is revisiting the issue after initially deciding it was too invasive.
While we’re all looking for preventive, proactive approaches to the school shooting crisis, many are wary of increasing surveillance to deter gun violence. In New York and Florida, schools are turning to 24/7 surveillance, face recognition and biometric database matching, and licence plate readers, all technology developed for police and counterterrorism agencies. Social Sentinel worked directly with police departments before opening up its services to schools.
[referenced url=”https://gizmodo.com.au/2018/03/schools-are-spending-millions-on-high-tech-surveillance-of-kids/” thumb=”https://i.kinja-img.com/gawker-media/image/upload/t_ku-large/qyatvqtnzkettftkfxxq.png” title=”US Schools Are Spending Millions On High-Tech Surveillance Of Kids ” excerpt=”Advanced surveillance technologies once reserved for international airports and high-security prisons are coming to schools across America. From New York to Arkansas, schools are spending millions to outfit their campuses with some of the most advanced surveillance technology available. face recognition to deter predators, object recognition to detect weapons, and licence plate tracking to deter criminals. Privacy experts are still debating the usefulness of these tools, whom they should be used on, and whom they should not, but school officials are embracing them as a way to save lives in times of crisis.”]
Margulis admits there are false positives, where someone is flagged when they don’t pose a risk, but critically, there can also be false negatives – students deemed unremarkable by the AI who go on to do violence. Experts are worried that unleashing this technology in schools will only replicate the imbalances we see when these tools are used in public policing.
“This is an expansion of the schools’ ability to police what students are doing inside of school or on campus to their outside-of-school conduct,” says Kade Crockford, who directs the Technology for Liberty Program at the ACLU of Massachusetts. “In many cases across the country, schools have been using social media surveillance tools in ways that have harmed, specifically, students of colour. So we certainly have concerns about technologies like this being used to expand what we call the school-to-prison pipeline.”
Social Sentinel declined to offer specific numbers on its success rate, but Margulis says educators have thanked them for their service in finding and reaching out to the students who need it most.
[WBUR]