In the West, image board Danbooru has been the definitive site for a specific niche: erotic (and, importantly, many non-erotic) images of anime-style characters. If you know the fanfic site Archive of Our Own, Danbooru is similar in size, significance and metadata in its space. It has been operating for 21 years, has a meticulous tagging system, and is a gargantuan repository of images and data. The software that runs the site is open source and has been forked, the most notable fork being the furry board e621. Unfortunately, the open source nature of the site has allowed malicious actors using LLMs to take it down, although it is currently back up thanks to heavy contributions by its maintainers and, unfortunately, LLMs.
You can read the entire, exhaustive explanation on the site, but I will quote and summarize owner Evazion here in case you do not want to visit a site that hosts a lot of pornography and previously has experienced security issues. For transparency, the full statement appears here .
On November 4th there was a security incident where someone found a way to upload a malicious Flash post. Since then the site has been down for a week while we responded. This is what happened.
The short version is that someone found an exploit that let them upload a Flash file that contained an XSS worm. The worm tried to spread itself across the site and steal the IP address of anyone viewing the infected posts. The site was shut down within about 6 minutes, so the actual damage was very limited. The main thing it did was grab the IPs of anyone who viewed the posts.
The exploit itself was quickly fixed, but it was clear that the attacker had used LLMs to find this exploit, and that they were targeting us and would use LLMs to find other exploits if we brought the site back up too soon. So we spent the next week fixing every security issue we could find and generally beefing up security to make sure this kind of thing couldn’t happen again. We also used the downtime to do some much-needed server maintenance and upgrades that we had been putting off for a long time, which also took a couple of days.
What happened?
An attacker uploaded a malicious Flash post that, among other things, grabbed your IP address and hijacked your account to spread the file.
The servers themselves weren’t hacked, and the file itself didn’t hack your computer or anything like that. It just let the attacker take over your account while you were viewing an infected post. It only affected those who viewed it in the 6-minute window the site was up.
The vector by which this happened was a pull request, basically a proposal to alter the codebase to fix a bug. The pull request was submitted by a user named Low-King three weeks ago, who is also the attacker according to Evazion’s write-up. “The pull request itself didn’t contain any malicious code,” Evazion wrote. “It was a genuine one-line fix for a genuine bug. The problem was that unbeknownst to us, the bug it was fixing was the only thing preventing Flash files from being uploaded.” Contributor nonamethanks presciently asked “What user case are you trying to fix here? Trying to re-enable Flash uploads?” in the comments after which the user explained that it was for “future proofing” and gave several other seemingly rational answers. It was then merged. There is the other, funny security detail that the video the url linked to was just called “worm-video.flv”–it may as well be called “bad-virus.exe”.
‘Black October:’ The PHP vulnerabilities and a series of other hacks and bugs have cybersecurity experts freaking out and unsure what happens next.
404 MediaJason Koebler
This attack follows a period many have dubbed “Black October,” a wave of attacks enabled by the increasing availability and capabilities of LLMs. After taking down the site, Danbooru introduced a wave of security patches to cover its bases, many of which involved Claude and Copilot. In a list of takeaways, Evazion states “we weren’t using LLMs to audit the codebase soon enough. Attackers adopted LLMs to look for exploits faster than we did, and we had to catch up fast. In the last six months or so AI has gotten really good at finding exploits, to the point where now people with no knowledge can pull off exploits that in the past would haven [sic] taken a lot of expertise.”
You can debate the utility of LLMs for bugs, security, and pentesting all you like, but even if you are for it, it is a bit of a bummer that AI attacks everything even remotely public-facing. The way that these technologies infect and ingratiate themselves as a prophylactic to their own existence is exhausting, even in something as simple as erotic drawings of anime characters on a website old enough to drink if it was a person.
