Over the past 20 years, we’ve refined our safety systems, enhanced parental tools, and worked side by side with child safety experts and law enforcement to create a positive environment for players. We know this work will never be finished—new risks emerge, technology evolves, and platforms must continually learn and improve to stay ahead.
We work with law enforcement agencies around the world to combat harms on the platform. A key stakeholder in this effort is the National Center for Missing & Exploited Children (NCMEC) where we report suspected instances of child sexual exploitation, which is required of all online service providers by U.S. federal law. NCMEC serves as the global clearinghouse for these reports, reviewing and routing to domestic and international law enforcement agencies. Like many companies, Roblox has regular touchpoints with NCMEC as part of our working relationship. We recently visited NCMEC for a collaborative conversation about how we can continue to strengthen the quality and actionability of our reports.
We also partner with the International Centre for Missing & Exploited Children (ICMEC), which provides resources, tools, and training for educators and law enforcement globally. “ICMEC values its partnership with Roblox, which has contributed to strengthening efforts to promote child safety online,” said Pilar Ramírez, Vice President of National Capacity Building, ICMEC. “Through this collaboration, we have supported capacity-building initiatives with law enforcement and other relevant stakeholders, providing specialized training, resources, and tools to help strengthen their knowledge and ability to address online child safety challenges. This partnership demonstrates the importance of collaboration between the private sector and specialized organizations to build safer digital environments and enhance the protection of children online.”
While we cannot share the details of each report to law enforcement, aggregate numbers are included in our transparency reports and (with respect to child sexual exploitation) NCMEC’s annual report. Using these reports to draw direct comparisons between platforms can be misleading because metrics directly reflect a company’s platform design and reporting methodology and scale (e.g., daily active users).
For example, our systems are designed to detect potential issues between minors, such as young players sharing or repeating explicit language. These are conversations that don’t involve predatory adults and, depending on many factors, may not warrant prioritized law enforcement action. Approximately 40% of our reports to NCMEC fall into this “informational” category. We address these minor-to-minor incidents with warnings, heightened monitoring, account restrictions, and account closures where warranted.
Roblox’s Approach to Safety
Because of our unique audience, with many younger players, we aggressively look for bad behavior and report it through the appropriate channels. Our philosophy is to be more proactive than many in preventing inappropriate behavior. Some of the guardrails we’ve built are uncommon for gaming platforms—and can be unpopular—but we want to err on the side of caution.
Built-in chat protection: Players cannot share photos or videos in chat, which means chat can’t be used to share child sexual abuse material (CSAM), one of the biggest challenges for many online platforms.
The ability to see, share, or display external hyperlinks is restricted to specific pages and older age groups. This policy is designed to prevent bad actors from directing young users off of Roblox to less monitored environments, where they could potentially share inappropriate content or solicit personal information.
Finally, we don’t encrypt chat, which enables our safety system and specialists to monitor text interactions, spot potential grooming behavior earlier, and intervene in real time. Our ability to monitor all communication in real time leads directly to comparatively high law enforcement reporting numbers and reflects our commitment to safety.
Age verification: Roblox was the first major gaming platform to require age checks to access chat features, limiting chat to similar age bands and giving parents control over who their child can chat with. We automatically assign players to the account type that aligns access, content, communication settings, and parental controls with their age.
Algorithmic intervention: Our machine learning models have been optimized to identify off-platform pivot attempts and predatory patterns earlier. Because communication on Roblox is text- or voice-based, we’ve developed sophisticated systems designed to detect very early conversational signs of grooming, enticement, and the subtle ways bad actors begin to push personal boundaries. Our systems are trained to detect and flag these conversations for closer review and monitoring. As soon as anything crosses the line into one of the areas we’re required to report to NCMEC, we can do so swiftly. As a result, our proactive detection rate has risen substantially, often tackling bad behavior before users ever report it.
Advanced moderator training:Moderators receive specialized training, led by an instructional designer and an expert with expertise in the signs of child exploitation. Through simulated scenarios, moderators learn to recognize subtle, context-dependent coercion techniques. After training, they review user-submitted tickets with support until they’re ready to work independently.
Continuous model training:For violations defined as “critical harms” (meaning those that threaten a user’s safety), we investigate to understand what happened and how it can be prevented in future. We also train our systems and reviewers on emerging threat models (such as misuse of generative AI) provided by Thorn, NCMEC, and law enforcement. Language and slang terms evolve over time, and people who intend harm are always searching for new ways to evade our filters. Because of this, we continually train our models on emerging language patterns and workarounds. When we detect real-world adversarial patterns, we incorporate them back into the model to help train it on an ongoing basis.
Cross-platform signal sharing:The most serious online risks, including child endangerment, aren’t confined to one platform. Roblox is an active contributor to the Tech Coalition’s Lantern project, a program that enables tech companies to securely share signals to help detect bad actors who operate across different platforms. Sharing information early and often is key to helping law enforcement connect the dots between platforms to identify and stop those who intend to cause harm.
As Roblox’s detection systems have become increasingly sophisticated and robust, we’ve seen reporting volume increase. This does not necessarily equate to an increase in problematic behavior. Our systems are designed on the assumption that someone, somewhere, will try to bypass our protections. Everything we build is meant to help catch them when they do. We know that technology and online risks will continue to evolve, and we will continue to work around the clock to enhance our tools for early detection and prevention. We continually work to refine our safety systems, enhance parental tools, and educate and partner with parents and caregivers. We’re also committed to continuing to work closely with child safety experts and law enforcement so the platform remains a positive environment for our community.
To learn more about our safety architecture, policies, and parental controls, visitabout.roblox.com/safety.
