Roblox makes three of its AI safety tools open source

Roblox is open-sourcing three of its AI safety models

ROOST is an independent nonprofit that develops and maintains open- 2025 with Google, OpenAI, and Discord

Roblox is contributing its PII Classifier,Roblox Sentinel, and its latest voice safety classifier to ROOST. These models are already deployed on Roblox’s platform.

The PII Classifier identifies attempts to share or request personally identifiable information (PII), including misspellings, coded language, and disguised references to other platforms.

Roblox is also sharing an evaluation dataset from this tool, which features “synthetic samples of English-language multiuser chats modelled on techniques often used to evade PII filters.”

Roblox Sentinel detects early signs of potential child endangerment. The latest voice safety classifier moderates voice chat in real time and incorporates feedback from the ROOST community.

Roblox is also releasing a new evaluation dataset to help companies assess the effectiveness of their safety tools. The latest version supports moderation in 30 languages and eight violation categories.

“Sharing these models gives other platforms a robust starting point to train and tune their own moderation tools,” said vice president of trust and safety engineering Naren Koneru. “In turn, we hope to benefit from the learnings and feedback of other companies as they share their work.”

“Safety is a shared responsibility – no company can solve it alone. Our intent in contributing to ROOST is to help other companies improve their own safety classifiers. We will continue sharing and learning from the ROOST community as we advance automated detection together.

Earlier this week, the US Senate Judiciary Subcommittee on Crime and Counterterrorism announced it had launched an investigation into Roblox, claiming the platform “appears to prioritise revenue and engagement metrics” over child safety.

Roblox has been told to provide records about child safety, sexual or physical abuse, bullying, and financial exploitation by August 31, 2026.

In recent years, Roblox has introduced several new safety measures. These include restrictions on direct messaging for users under 13, global facial recognition checks, a “sensitive issues” content tag for experiences, and enhanced moderation policies.