Gaggle: MPS’s new student surveillance software brings possible protection and danger

So why does no one know about it?

MPS+has+contracted+a+software+company+called+Gaggle+to+monitor+student+accounts+and+writing+to+identify+possible+signs+of+suicide+ideation%2C+self-harm%2C+harm+to+others%2C+signs+of+depression%2C+exploitation%2C+or+bullying.+%E2%80%9CI+would+feel+comfortable+saying+that+we%E2%80%99ve+saved+some+lives+with+what+we%E2%80%99ve+done+with+%5BGaggle%5D%2C%E2%80%9D+said+Jason+Matlock%2C+the+Director+of+Emergency+Management%2C+Safety%2C+and+Security+at+MPS.+%0AHowever%2C+many+students+are+unsure+of+what+Gaggle+does+or+unaware+that+it+even+exists.+Aesha+Graffunder%2C+a+sophomore+at+Roosevelt+said%2C+%E2%80%9CI+see+little+to+no+problem+with+Gaggle+itself%2C+its+purpose+is+to+provide+schools+with+information+that+can+help+protect+students.+I+see+way+more+of+a+problem+with+school+administrations+and+how+they+handle+the+flags+they+get+from+Gaggle.%E2%80%9D

Talula Cedar-James

MPS has contracted a software company called Gaggle to monitor student accounts and writing to identify possible signs of suicide ideation, self-harm, harm to others, signs of depression, exploitation, or bullying. “I would feel comfortable saying that we’ve saved some lives with what we’ve done with [Gaggle],” said Jason Matlock, the Director of Emergency Management, Safety, and Security at MPS. However, many students are unsure of what Gaggle does or unaware that it even exists. Aesha Graffunder, a sophomore at Roosevelt said, “I see little to no problem with Gaggle itself, its purpose is to provide schools with information that can help protect students. I see way more of a problem with school administrations and how they handle the flags they get from Gaggle.”

By Khayaal Desai-Hunt, Minneapolis South High School

Students are experiencing major mental health challenges in the Covid-19 pandemic. With distance learning, supporting mental health is harder for schools. Students share less, especially as their cameras are often off or they aren’t talking. Staff aren’t able to easily reach out and get students the assistance that they might need. 

Aesha Graffunder, a sophomore at Roosevelt, says they understand the need for protection online, “I agree that we should have more monitoring online just to keep students safe and because it is different in school versus online.” 

MPS has contracted a software company called Gaggle that monitors student accounts and writing to identify and flag those who might need help. The district says the surveillance is necessary to keep students safe by identifying those who are struggling. However, some students have raised issues around privacy, consent, and a lack of information distributed about Gaggle. 

This begs the question, what privacy do we give up in order to protect ourselves and each other? Where do we draw that line? And should students and their families get any input on whether or not they participate in it? 

Jason Matlock, the Director of Emergency Management, Safety, and Security at MPS said, “Their mission and their concept [Gaggle], the way they push is that they’re trying to help support our efforts to stop bullying and our efforts to hear our kids when they’re saying they’re hurting.”

But what does it actually do and how does it work? Gaggle monitors activity on school accounts as well as provides a 24/7 tip line so that students can report concerns about the well-being of peers or school safety. The MPS version scans all Google Workspace (G Suite) products including Google Drive, Gmail, Hangout chats, and downloaded photos or files on all MPS accounts. It does not view live meetings such as Google Meets or social media unless directly connected to the account. Although Gaggle is most definitely not a keystroke logger, students should expect that most of their account is being monitored.

Gaggle uses artificial intelligence to compare student writing, communications, and files to a list of selected key words that might relate to suicide, self-harm, harm to others, signs of depression, exploitation, or bullying. The list includes profanities as well as words and phrases like “drunk,” “end my life,” “hate myself,” and “hit me.” Additionally, there are words specifically associated with LGBTQ+ identities and community such as “queer,” and “gay,” as well as various slurs. Gaggle’s artificial intelligence also scans uploaded images for inappropriate content by running it through an “Anti-Pornography Scanner.” 

When a keyword or image is identified it is sent to a team of human moderators who review flagged communications for context. According to a now-closed job application moderators are paid as little as $10 an hour. They sort the flags into groups based on the severity of the incident. When incidents fall into the most concerning categories, “Questionable Content” and “Possible Student Situation,” MPS officials are immediately contacted. Incidents that fall under these categories include the intention or plans of self-harm or suicide, threats of violence, harrasment, or pornography.

Matlock’s safety team is the first to receive these alerts as they are working 24/7 in case of emergency. His team passes the information to the principal or assistant principal who decides which support personnel, such as a social worker, would be best to help the student. The safety team can also reach out to the emergency contact (usually parent or guardian) of the student in case of immediate danger. In rare situations where they are unable to get into contact with an emergency contact or reach the student somehow, they may get in touch with law enforcement.

One commonly asked question is who can see the information that Gaggle monitors. The information that is collected is confidential, for the most part. According to Gaggle, it treats all data as district property. Nonetheless, they don’t have a say in what the schools choose to do with the information and flags they are given. They will not give information to a third party unless required to turn over content to the National Center for Missing and Exploited Children or law enforcement.

Lee Lang, a sophomore at Roosevelt, believes that MPS needed to implement something for distance learning. But he is hesitant about Gaggle. “I don’t know if Gaggle is the right thing and I think maybe if they had talked more to students and parents before they started to use Gaggle that would have been a smarter decision.” 

The original six month agreement with Gaggle was at the beginning of distance learning during the 2019-20 school year. It cost the district just under $100,000. With the transition to distance learning MPS realized that there would be less connection between students and staff. “The point of this was to react in a similar way that they would if they came across something during school,” said Matlock. In person when a teacher or staff member saw a student that was struggling they would connect the individual to resources and people who could help them. His goal is to keep this the same during distance learning. After it was evident that school would not be reopening immediately for the 2020-21 school year, MPS made a new contract with Gaggle. The new contract has to be renewed each year at an annual rate of $255,000 per year. 

While it is difficult to say whether or not other solutions would have been more successful, the partnership with Gaggle was made in collaboration by Matlock, the equity department, IT, and general council. Matlock said they didn’t get feedback or input from students and parents because everything was locking down; all the sudden nobody was going to school anymore. 

When looking for a program to use with distance learning the team from across departments tried to avoid the companies that surveilled open source social media, touted their connection to police, and utilized a more punitive system. Matlock said that he never wanted it to be a surveillance system. “While it may feel sort of icky to know that someone can see your stuff, if we’re going to do it I wanted to make sure that we were working with someone who at their heart wanted to just help kids.” It isn’t a keystroke logger or watching through cameras. He added that the biggest thing that appealed to him about Gaggle was the 24/7 human connection tip line. He wanted to make sure that they wouldn’t be overwhelmed. “None of this was meant to have a negative impact on our students, it all was about, ‘How can we do the best under the most insane of times?’”

Gaggle has still been able to help students and intervene as kids are struggling. “I would feel comfortable saying that we’ve saved some lives with what we’ve done with this,” said Matlock. To him the important thing is forming good relationships between students and staff and being able to make contact when they are struggling. One big worry is that online it is difficult to tell when kids are struggling and this is a way to check in. He continued, “While it might be frustrating to know that someone is watching what you’re doing, in the back of your mind you know that means that they’re actually paying attention, that you’re not being ignored, you’re not invisible.”

Nevertheless, students and their families did not give explicit consent to Gaggle’s use and the information distributed about it was inconsistent. According to Matlock, principals and school administration requested not to have a district-wide announcement and instead to leave it up to individual schools to inform students and distribute information about Gaggle. Despite this, most students don’t actually understand what it does. “I’ve talked to tons of students who didn’t even know Gaggle existed. I didn’t even know what it was until I did my own research about it,” Graffunder said. 

For example, South recently released a video that reminded students that there is no such thing as confidentiality online and briefly describing what Gaggle does. While other schools may have posted it earlier, the video was filmed at the beginning of distance learning and South shared it in late February. Much of the content and information that Matlock shared with schools seems to have gotten lost in the transition. It is important to note that the tip line, SpeakUp, which students can contact at any time, sends out a monthly email reminder.

Graffunder became worried that many students were unaware of Gaggle and how it functions. They found that they were not alone in this concern. “We decided to create a group and try to set up a meeting to inform other students about it,” said Graffunder. They have been organizing with high schoolers from around the district to help educate peers about how Gaggle functions.

Lang, another member of the group, feels that students have a right to their privacy and that they should know if and consent to their accounts are being monitored. “Even though the intention wasn’t for it to be a privacy issue and their intention wasn’t for us not to know, that is what happened. I think for a lot of students it felt like a breach of privacy.” He added that he didn’t believe that there was an intention for it to be kept hidden, but that there was a communication error. 

In addition to an absence of information about it, Gaggle has raised other questions. While there are many trigger words, only a limited sample is easily available to the public. One problem with some of these trigger words is that they associate certain identities with being in danger. Some examples include LGBTQ+ related words including “queer,” “gay,” and “lesbian.” These are flagged in an attempt to identify potential bullying. However, whether or not it is intentional this inadvertently targets and discriminates against LGBTQ+ youth who are more likely to use these words.  Moderators do review marked words for context and to assess the situation, but according to Graffunder and Lang this has already raised issues. 

In one such instance, a student was outed without their knowledge to their parents. Graffunder described the situation, “School administration didn’t talk to [the student with the flag] at all before their parents were called. That’s where I mostly see concern, I feel like students should be involved.” She added that perhaps based on what the student wrote the flag was warranted, but that staff could have handled the situation better by at least alerting the student before the parent was involved.

Gaggle doesn’t have a say in how the district or schools handle the information that they pass on. Lang and Graffunder agreed that they don’t have an issue with the actual company. “I see little to no problem with Gaggle itself; its purpose is to provide schools with information that can help protect students. I see way more of a problem with school administrations and how they handle the flags they get from Gaggle,” said Graffunder. 

Matlock said that some basic advice was provided to schools and support staff about how to help students, but there isn’t a specific policy dictating how to handle instances reported by Gaggle. The district responds to flags from Gaggle the same way it would during in person school. According to the district this is because they are the same issues that are encountered in person. Matlock argued that they should be dealt with the same way. “You’re going to come across students who are struggling with their identity. You’re going to come across students who are being exploited. You’re going to come across students who are being abused. You’re going to come across drug users. You’re going to come across violence. The problem isn’t the system, the problem probably is, truthfully, that not every adult is ready to necessarily work with these kinds of things,” Matlock said. 

Understanding how best to assist LGBTQ+ students, in particular, can be difficult. “I don’t doubt we make mistakes with that. I don’t doubt we make mistakes with that in person… It’s a learning curve. It has not been perfect, but I would say overwhelmingly those kinds of situations are few and far between.” 

Matlock and his team are aware of the negative interactions that Lang and Graffunder are concerned about. “When we heard about that kind of stuff happening we did turn back to some folks internally to try to draft up some more guidance for how to handle sensitive topics.” Jason Bucklin, the district coordinator for Out4Good, has given trainings on it before and when they heard the situations come up, they turned back to him for guidance and reminders. Matlock added that not all support staff might know how to handle these situations, but that he wants to think that any stumble is by accident, not intent. He said that not every interaction between staff and parents and students has gone perfect around what was flagged by Gaggle. “No one is ever perfect, so how someone approaches a situation when we’re in person doesn’t always go well. Parents get upset, students get upset about how things go.” 

While they do sometimes fail and escalate interactions between parents and students, according to Matlock the number of negative interactions is small compared to the overall cases they catch.

Lang believes that while the issues that students are dealing with right now might be similar to in person, dealing with them online and in person is inherently different. For the transition to distance learning he thinks that there should have been extra training or reinforcement of what the support staff already know. He feels that it definitely should have happened after a problem arose, in order to prevent another negative interaction. “If counselors did a better job of talking to students and figuring out what the problems are instead of going directly to parents and talking about what they saw online, I think that would really solve the problems we have… It’s less really about ‘Is Gaggle necessary?’, but ‘What other options are there besides going directly to parents?’”

As a consequence there has been a loss of trust with support staff and administration for not consulting students. Lang said, “Students are coming out with stories of counselors telling information that they didn’t want to be told.” It is possible that this wouldn’t be an issue without Gaggle, but Lang argues that this extends beyond distance learning. “These issues aren’t only online, we’ve also had people tell us how counselors have told their parents about something they didn’t want them to be told [in person].” He said that even for in person school some sort of protocol could be beneficial for better communication. 

He thinks for any non-emergency situation that involves a counselor telling a parent information, the student should always be involved first. This could determine if there is actually a problem or if by telling parents it could make the situation worse. “If the goal of Gaggle and of online security is to make sure that students are safe, then you would want to make sure that telling the parents doesn’t put the student into an unsafe situation.”

Whether or not MPS decides to extend the contract with Gaggle will depend on if they think these tools have been valuable and if there seems to be a need for it. Matlock believes keeping the tip line even after a complete return to in-person learning is a good idea. The decision to continue the other services provided by Gaggle is still under consideration. 

Meanwhile, Graffunder and Lang have been organizing and educating with their peers. Their goal is to set a policy in place that support staff have to follow in order to maintain the safety of students when reporting to parents and guardians. They are trying to fully understand the situation and want more information before they make a concrete plan. Their other objective is to make sure students understand what Gaggle does. “Another thing that I’d like to see is another policy that does include students and informs students because I honestly don’t believe that most of the students in the district even know what Gaggle is,” said Graffunder. 

“The main starting point for us is that we want students to know what’s going on,” added Lang.

This story was originally published on South High School on March 14, 2021.