Roblox Open-Sources AI Safety Models

Roblox is sharing open-strengthen online safety detection

Roblox is contributing three open-source safety models to the Robust Open Online Safety Tools (ROOST) Model Community: an updated PII Classifier, Roblox Sentinel and its latest voice safety classifier. The company is also releasing a new evaluation dataset that other platforms can use to benchmark their own safety classifiers. These models are already used on Roblox to detect attempts to share or solicit personal information, identify early signs of potential child endangerment and moderate voice chat in real time. 

The updated PII Classifier is designed to identify attempts to share or request personally identifiable information or move users to other platforms. Version 2.0 analyses surrounding conversation rather than isolated messages and is trained with synthetic examples covering complex bypass methods and rare multilingual cases. This has expanded its supported languages from 17 to 189, while its F1 score increased from 63.41 to 90.52. 

Alongside the classifier, Roblox is releasing a new benchmark dataset containing synthetic multi-user conversations designed to represent attempts to bypass PII filters. The dataset includes techniques such as phonetic bypasses, character substitutions, coded language and information distributed across multiple messages. It also includes benign conversations that resemble PII-sharing attempts, allowing developers to evaluate how accurately their classifiers distinguish risky interactions from normal conversations. 

Roblox Sentinel focuses on detecting early indicators of potential child endangerment before conversations become explicit. Its second version introduces six methods for combining model scores instead of two, making it easier to tune the system for different use cases. In one test, Sentinel evaluated 324 configurations in less than three minutes and improved ROC-AUC from 0.894 with the default configuration to 0.996 with the best configuration identified. Roblox says nearly 70% of the cases it detected during the 12 months ending 7 August 2026 were identified through Sentinel’s early detection. 

The company has also released version 3 of its voice safety classifier, which supports 30 languages and eight violation categories. It delivers 61 per cent recall at a 1 per cent false-positive rate and includes built-in language detection. Although the model has increased from 94.6 million to 320 million parameters, model distillation keeps it suitable for real-time detection. Roblox says the classifier has been downloaded more than 72,000 times since it was open-