Innovation

How AI became Instagram's weapon of choice in the war on cyberbullying

Instagram attracts more cyberbullies than Facebook and Twitter. Find out how its new machine learning algorithm works, and what your business can learn.

On a platform meant to be a safe place to share snapshots of users' lives, Instagram has the greatest cyberbullying problem of all social media sites. But instead of putting the responsibility on their users to report abuse, as Facebook and Twitter have done, Instagram is the first social media outlet to use machine learning to eliminate abusive language on its platform.

A recent survey from anti-bullying charity Ditch the Label found that 42% of over 10,000 UK youths between the ages of 12 and 25 found Instagram to be the platform where they were most bullied—with Facebook and Twitter falling behind at 37% and 9%, respectively. And 71% of the subject pool agreed that all social media networks are not doing enough to stop cyberbullying.

Searching for a solution

To address cyberbullying head-on, Instagram recently announced a new strategy: Integrating a machine learning algorithm to detect and block potential bullies on its platform. The research is aimed at forging kind, inclusive communities on Instagram, the company's CEO and cofounder Kevin Systrom said in a blog post.

SEE: How to handle employee abuse and bullying (Tech Pro Research)

Instagram is using DeepText—the same machine learning algorithm as their owners at Facebook—to try and shut down their cyberbullying problem. In June 2016, Facebook engineers introduced DeepText as "a deep learning-based text understanding engine that can understand with near-human accuracy the textual content of several thousands posts per second."

Through deep learning—a subset of machine learning that uses algorithms modeled on the human brain's neural networks—Facebook engineers used word embeddings to help the system understand the nuanced way humans use language. DeepText is designed to work like the human brain, using deductive reasoning to determine what words mean in a specific context.

For example, if somebody uses the word "mole," DeepText is expected to determine if the user is referring to the small mammal, a skin blemish, or a traitor. Facebook uses this system to sift through thousands of posts to better understand its audience, aiming to create a better, more personalized user experience that is catered towards individual interests.

In October 2016, Instagram launched DeepText to eliminate spam. The algorithm targeted internet trolls on the hunt for followers and organizations trying to sell products, analyzing comments and captions for semantics that would hint at whether the data was spam.

But the success of DeepText led Instagram to consider other uses for the system. In a June 2017 blog post, Systrom announced that the company would use DeepText as "a filter to block certain offensive comments." The platform used the technology originally created by Facebook to create a filter that would help build a safe environment for users.

Other platforms acknowledging the problem

Cyberbullying and hate speech are not exclusive to Instagram—other major social media networks have already been forced to make security changes for their users.

"Machine learning algorithms have proven to be effective ways to detect hate speech and cyberbullying," said Tom Davidson, graduate student at Cornell University and coauthor of reports on hate speech and cyberbullying on social media, focusing on Twitter. A variety of different algorithms are verified as being effective, Davidson told TechRepublic, such as "logistic regression, Naive Bayes, random forests, support vector machines." But the key to all of these methods is a reliance on supervised learning, he said, which is a machine learning strategy of using labeled training data to make inferences. Davidson's research involved collecting millions of tweets that had potential cyberbullying undertones (racial slurs, expletives, etc.), labeling them, and feeding the data into an algorithm, Davidson said. The examples are used to train the algorithm, Davidson added, after which it should be able to classify hate speech on its own.

Twitter's November 2016 blog post announced the notification muting feature as well as a hateful conduct policy that gives users a more direct medium of reporting abuse. While these efforts are trying to prevent cyberbullying, muting offensive notifications does not make the tweets nonexistent. And while reporting abuse is extremely important, users remain at the mercy of however long Twitter takes to respond.

SEE: Machine Learning Artificial Intelligence Bundle (TechRepublic Academy)

Facebook attempted to decrease cyberbullying by forming the Bullying Prevention Hub. The Hub acts as a resource for teens, parents, and educators to use when they or someone they know is being bullied. While the resource provides valuable advice on getting the cyberbullying conversation started, Facebook's Bullying Prevention Hub does not directly do anything to eliminate abusive content head-on. The company only uses the tool to recommend content to users based on their interests.

DeepText's strengths and weaknesses

Still, these efforts fall short of fully blocking online bullying.

Zeerak Waseem, Ph.D. student at the University of Sheffield focusing on abusive language detection and hate speech on Twitter, told TechRepublic that "these attempts are lacking in effect."

Why? While both Twitter and Facebook made strides to tame cyberbullying, Instagram is the first social networking site to make offensive comments automatically disappear. Both Systrom's blog post and Wired explained how the AI is currently functioning on Instagram accounts. If a user publishes offensive or harassing language, DeepText will catch it and delete it instantly. And to prevent bullies from trying to game the system, the offensive language will still be viewable by the perpetrator, Wired reported. Users can also manually enter words or phrases they want blocked, making DeepText even more effective by blocking trigger words that could be unique to the user.

DeepText is not perfect, however.

Instagram's machine learning algorithm is automatically integrated on the platform, but some hate speech can still bypass the tool. Waseem told TechRepublic that implicit insults, like nicknames or code names for slurs, would be difficult for DeepText to detect. Also, the feature can easily be turned off. With the tap of a finger, the "hide offensive comments" toggle can be switched off, which seems counterintuitive if the mission is to eliminate cyberbullying. The line between free speech and creating an environment rid of hate speech isn't easy to find. Davidson adds, however, that "machine learning is not a magic bullet that will stop cyberbullying or online hate speech." Machine learning can help the bullied user feel better, but no technology is going to stop individuals from saying mean things.

Liam Hackett, CEO of Ditch the Label, told TechRepublic that Instagram has the most cyberbullying issues of the critical mass of young people who have accounts on the platform. Because of the nature of Instagram's content, much of the harassment focuses on people's appearances, Hackett said. The insults vary from negative comments on photos, to bullies creating fake accounts to roast their targets.

Hackett praised Instagram's efforts, telling TechRepublic how fantastic the machine learning strategy is and how more social networks need to invest in the technology. He mentioned how Instagram's use of AI shows great progress in the movement against cyberbullying, with AI really changing the game.

And along with preventing bullying, DeepText has other functions that could help businesses gain insight into their customers' interests and better understand how information is communicated throughout the company.

SEE: iHate series—Intolerance takes over the internet (CNET)

The root of the problem

Machine learning that helps individuals on an emotional level is a huge step in the right direction. However, addressing why online trolls continue abusive behavior on these platforms is a larger issue.

"We don't know, as a society, how to engage online," Hackett said. "The internet dehumanizes people," Hackett added, insulting users online behind the comfort of a screen is much easier than saying those insults to a person's face. Offline interpersonal relationships have implicit codes of conduct—a social norm that is unspoken but understood. The same courtesies are not always followed on the world wide web.

As one of this year's most used social media networks, Instagram is home to a whopping 700 million monthly active user and is growing quickly, earning 100 million new users in just a four month time frame. The increase in users, however, revealed an increase in hurtful posts and offensive language.

Hackett noted that users are "not being adequately equipped with skills to how to behave online." A program like DeepText promises to address the problem. Yet programming, in itself, may not be the full solution to teaching people to respect one another online.

Also see

Image: iStock

About Macy Bayern

Macy Bayern is the 2017 summer Editorial intern for TechRepublic. She is an honors student at the University of Texas at Austin and a former intern at Texas Monthly.

Editor's Picks

Free Newsletters, In your Inbox