Multimedia Gallery

Have you ever received an unsolicited, inappropriate DM on social media? Researchers are leveraging machine learning tools to keep users safe and prevent exploitive social media content in the "Discovery Files" from the U.S. National Science Foundation.

The Center for Missing and Exploited Children's CyberTipline received more than 30 million reports of suspected child sexual exploitation in 2022. The increased presence of youth online has led to a rise in calls to actively protect these young social media users.

Supported in part by NSF, a group of researchers have developed a machine learning model to classify the risk level of conversations and individual messages that adolescent users reported as making them feel unsafe.

The Human-Centered Machine Learning model keeps humans at the core of the design process to improve the algorithm by reducing the false positives or negatives by adding contextual information about the people and the relationships between them.

This interpretable model outperformed deep learning models and added significant insights into the dangerous interactions that youth participants experience, by identifying the language used to groom, objectify, and victimize them in private online spaces.

These results point towards promising application in the detection of unsafe content and could be used to increase protection for young social media users.

To hear more science and engineering news, including the researchers making it, subscribe to "NSF's Discovery Files" podcast.

Images and other media in the National Science Foundation Multimedia Gallery are available for use in print and electronic material by NSF employees, members of the media, university staff, teachers and the general public. All media in the gallery are intended for personal, educational and nonprofit/non-commercial use only.

Images credited to the National Science Foundation, a federal agency, are in the public domain. The images were created by employees of the United States Government as part of their official duties or prepared by contractors as "works for hire" for NSF. You may freely use NSF-credited images and, at your discretion, credit NSF with a "Courtesy: National Science Foundation" notation.

Additional information about general usage can be found in Conditions.