Despite making changes last fall to help block adults from messaging unknown children, Roblox is still failing to stop adults from creeping on kids in other ways, Australia’s eSafety commissioner, Julie Inman Grant, said on Thursday.
Suspecting that Roblox may not be doing enough to stop child exploitation, eSafety conducted testing earlier this year that flagged alarming gaps in Roblox safeguards. The agency found that adult strangers could still send connection requests to kids, whose “profiles and biographies, including account names, number and names of connections, avatar images, and non-sensitive biographical information such as their interests were visible to anyone on the Roblox platform, with no option to restrict the visibility of this information.”
Such requests did not trigger parental alerts, eSafety found, and bad actors could also search kids’ visible-to-anyone contact lists for more targets.
Perhaps even more concerning, adults could also search for kids and comment on their posts “without parental or carer consent” in “forums outside game environments,” eSafety said.
Australia’s Online Safety Act requires that platforms prevent unwanted contact between adults and minors. It also mandates setting children’s accounts to private by default. Although Roblox “believes it is meeting its obligations,” the online gaming platform—which 1.7 million Australian kids use—has entered a court-enforceable agreement to make changes in the next three months.
Upcoming measures will prevent adults from contacting kids without parental consent across the platform, Roblox has promised. Providing better protections for kids starts with “strengthening the account settings of children’s accounts so they are private by default,” eSafety said, and ends with Roblox making it easy to report unwanted contact and notifying users of outcomes after reports are made. Those are critical protections for Australia’s second most popular online game for kids, Inman Grant said.
“We want to ensure that any platform with a high preponderance of kids to be a safe place where they can enjoy the benefits of online gaming, creativity, and connection, including building their own worlds and experiences online,” Inman Grant said. “But those benefits can only be realized when effective safeguards are in place to protect children from serious harms such as grooming, child sexual exploitation, sextortion, or recruitment into violent extremism.”
Roblox agrees to independent monitoring
Additionally, Inman Grant said that the regulator’s testing suggested that Roblox can no longer be trusted to independently verify that its child safety tools are compliant with Australian laws. Moving forward, Roblox cannot “mark their own homework,” Inman Grant said, adding that the platform has agreed to third-party auditing.
Roblox is likely hoping that allowing the audits will demonstrate that it is going further than other companies to protect kids. In January, the company announced that it would become the first large online gaming platform to require facial age checks. Such biometric age checks have proved controversial, unpopular with users, and alarming to privacy advocates, who see even the latest age verification tools as flawed. And while Roblox says it usually automatically deletes selfies submitted for age verification, it may retain that data for up to 30 days, leaving it open to potential breaches.
Asked for comment, Roblox sent Ars a statement.
“This undertaking follows constructive dialogue with eSafety and comes after the recent rollout of industry-leading safety features on Roblox—including the introduction of new Roblox Kids and Select accounts for under-16s,” Roblox’s spokesperson said. “We have delivered on the commitments we made to eSafety over the last 12 months and will continue to work towards our shared goal of keeping Australian children safe online.”
Roblox age estimation will also be audited
Inman Grant emphasized that this is the first time Australia has required third-party auditing since the Online Safety Act became enforceable.
The agreement came after eSafety warned Roblox last October that officials remained “highly concerned by ongoing reports regarding the exploitation of children on the Roblox service, and exposure to harmful material.”
For Roblox, the deal goes beyond helping eSafety monitor how well its safeguards are working. The third-party auditing will also give the agency a window into Roblox’s age estimation tools, eSafety said. If it seems like Roblox is mislabeling kids as adults, giving them access to adult content, or exposing them to adult predators, the company could face steep fines under the Online Safety Act. Last October, eSafety warned that it was prepared to “use the full range of its enforcement powers, as appropriate” to stop grooming and exploitation on Roblox, including “seeking penalties of up to $49.5 million.”
Roblox seems confident in its age-estimation system, though. In a blog post earlier this month, Roblox shared that its Facial Age Estimation system—which was built “completely in-house”—uses artificial intelligence to keep “age checks current.”
The system so far has “processed 338 million age-check records to help Roblox users unlock age-appropriate chat and access age-appropriate games on the platform,” Roblox reported. It relies on “a high-throughput” machine learning (ML) “layer that evaluates whether an account’s ongoing behavior still matches its estimated age over time.”
Although age estimation with ML is “difficult” because “age boundaries are not behavioral cliffs,” Roblox says it is using the “right signals” to estimate age. But the platform hedged a bit when discussing how it sorts users into age bands, noting that “a single accuracy number is not very meaningful” because quality also depends on “ranking quality, calibration, stability over time, and behavior near important age-band boundaries.”
Roblox does share some statistics, though, reporting that its fused model produces “high-confidence estimates” that stay “consistent from day to day.”
“More than 99 percent remain in the same broad age band on consecutive days,” Roblox said.
Further, tests that compared the model “against independent ground truth produced by trained human reviewers through our Trust and Safety workflows” showed that “the model’s age-band estimate agrees with the human consensus label for adults more than 80 percent of the time,” Roblox said.
Roblox suggested that “owning this system in-house lets us leverage Roblox-specific signals safely, operate with extreme rigor, and continuously iterate,” including by allowing Roblox to easily drop ineffective submodels or optimize others. But eSafety is clearly taking the cautious approach, planning to continually audit the age-estimation tool’s effectiveness so that it isn’t blindsided if Roblox’s self-evaluations rate the system more highly than the regulator in charge of enforcing the Online Safety Act might.
Later this year, eSafety will release a transparency report detailing how successful online gaming services like Roblox have been at mitigating harms to kids. Generating that report required all platforms to explain “the steps they are taking to protect children from harms, including child sexual exploitation and abuse, violent material and activity, and cyberbullying online,” eSafety said.
“Ultimately, our goal is to drive companies to make their services safer for children and young people. Where safety by design has not been deployed comprehensively at the front end, we won’t hesitate to ensure that these risks have been responsibly remediated,” Inman Grant said Thursday. “Holding Roblox to the binding safety improvements through this enforceable undertaking is a great example of how these systemic powers can deliver real and meaningful results.”
Ashley BelangerSenior Policy Reporter
Ashley BelangerSenior Policy Reporter
Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience.
