The Art Of Poison-Pilling Music Files

April 14, 2025 4:17 PM

People have been fighting back against machine learning for a while. CV Dazzle (previously) is facial-recognition camouflage. Glaze encodes visual works to prevent ML systems from mimicking them, and Nightshade actively poisons ML systems that attempt to train on images encoded by it. Now Benn Jordan (previouslies) discusses Poisonify, software for encoding audio files to corrupt ML music generators trained on them. In his typical style, this video is long, fast-moving, and covers a lot of ground.
posted by adamrice (8 comments total)

I watched Benn’s video last night and I loved it! Thank you for putting this post together, adamrice.
posted by Ice Cream Socialist at 5:16 PM on April 14 [1 favorite]

I'm glad this was already posted. Music for musicians!

It s a great trick, to use the fact that AI is all just white noise, against it

If the side effect is that it breaks Spotify's classification, is that a negative?
posted by eustatic at 7:14 PM on April 14 [4 favorites]

If only I had the special T-shirt from William Gibson's Zero History...
posted by outgrown_hobnail at 4:59 AM on April 15 [2 favorites]

If I read this correctly the “poisoning” is achieved by something akin to steganography: hide the AI toxic payload in such a way that it is indistinguishable from the underlying signal of the music. That means the same approach should be effective for visual artwork also no?

I find it interesting that Jordan was forced to develop this technique only after running into issues with getting traction for his equally good idea of developing high quality specifically intended for AI training - then selling it in such a way as to reward the contributing takent.
posted by rongorongo at 9:26 AM on April 15 [2 favorites]

rongorongo—You're right, and that's basically how Nightshade works. The underlying principle is that these ML "classifiers" are picking up on features in the training material that humans do not notice.

It would be interesting if the same principle could be applied to text. Obviously the changes would be noticeable to humans. Would it be possible to alter text in a way that humans agreed was insignificant, but completely stymied AIs? I don't know. I'm speculating, but perhaps changing just punctuation to Unicode characters that look about the same (eg ‚ instead of , and ․ instead of .) would have that effect, but this could create accessibility problems.
posted by adamrice at 9:59 AM on April 15

Like “AI detectors”, these “technologies” are effectively snake oil, and in the few scenarios where they do anything, they’re trivial to work around. And of course, you can train an AI model to undo the visible/audible damage by creating a few thousand before/after pairs.

Glaze and the Effectiveness of Anti-AI Methods for Diffusion Models
posted by ArmandoAkimbo at 12:10 PM on April 15

Doesn't mp3 compression work largely by discarding the sounds humans don't notice? It seems like the kind of techniques needed to defend machine learning from this kind of thing are already pretty well-developed.
posted by straight at 11:41 AM on April 21

Doesn't mp3 compression work largely by discarding the sounds humans don't notice?

Mp3 compression works by being loosey-goosey with some stuff that we don't notice as much, yeah. However, it's by no means 100% efficient at throwing out things we don't perceive. There are certainly audio watermarking systems that are robust against common compression formats.
posted by aubilenon at 12:29 PM on April 21