Blob Opera

December 15, 2020 10:21 AM

Blob Opera is a machine learning experiment by David Li in collaboration with Google Arts and Culture.
posted by OverlappingElvis (28 comments total)

Not to be confused with these guys.
posted by OverlappingElvis at 10:21 AM on December 15, 2020 [2 favorites]

David Li's previous work includes training a neural network on choral pieces to generate the harmonization for your melody on [adult swim] Choir and the lovely [adult swim] Elastic Man
posted by zenon at 10:23 AM on December 15, 2020 [3 favorites]

This is delightful. I find myself wanting more precise control over the notes of the voice i'm controlling than mousing, just to be able to really key in a melody voice.
posted by cortex at 10:37 AM on December 15, 2020 [2 favorites]

I find myself wanting more precise control over the notes of the voice i'm controlling

This, very much. There's obviously some kind of tracker format so it can play the Christmas songs, but I couldn't see any way that was exposed.
posted by OverlappingElvis at 10:46 AM on December 15, 2020 [1 favorite]

We developed a machine learning model trained on the voices of four opera singers in order to create an engaging experiment for everyone, regardless of musical skills. Tenor, Christian Joel, bass Frederick Tong, mezzo‑soprano Joanna Gamble and soprano Olivia Doutney recorded 16 hours of singing. In the experiment you don’t hear their voices, but the machine learning model’s understanding of what opera singing sounds like, based on what it learnt from them.

Don't forget to flip the switch that makes them sing Christmas carols.
posted by RobotVoodooPower at 10:47 AM on December 15, 2020 [4 favorites]

Not to be confused with these guys

I've clearly been watching too much British TV, because my first thought was this guy.
posted by Greg_Ace at 10:59 AM on December 15, 2020 [1 favorite]

At first I thought it was an opera adaptation of "The Blob," which could probably work. I mean, they did an opera of "The Fly"
posted by SansPoint at 11:01 AM on December 15, 2020 [2 favorites]

Don't forget to flip the switch that makes them sing Christmas carols.

Sweet. Then you can drag individual blobs around to do some mad improv over top (or underneath) the others.
posted by mandolin conspiracy at 11:09 AM on December 15, 2020 [2 favorites]

Here's the JSON format used to load saved recordings, including the Christmas songs
posted by OverlappingElvis at 11:56 AM on December 15, 2020 [2 favorites]

That was the best Cocteau Twins song I've heard all day.
posted by JoeZydeco at 12:52 PM on December 15, 2020 [3 favorites]

no work for us the machines have come to take our blobs
posted by lalochezia at 12:53 PM on December 15, 2020 [1 favorite]

Do you really want to hand over what you know about operatic blobs to Google?!
posted by chavenet at 1:51 PM on December 15, 2020 [1 favorite]

This is absolutely wonderful I can't wait to show my girlfriend. We're still in the very early stages so it'll be delightful to be like "This is a great representation of who I am and what I value, as a person"
posted by FirstMateKate at 2:00 PM on December 15, 2020 [8 favorites]

Do you really want to hand over what you know about operatic blobs to Google?!

You shouldn't refer to Pavarotti in such an insulting manner!
posted by Greg_Ace at 2:10 PM on December 15, 2020

I wrote an incredibly quick and dirty tool to convert midi files to blob opera format (twitter thread with proof of concept video and more context), so this is very possible.
posted by OverlappingElvis at 2:51 PM on December 15, 2020 [3 favorites]

At first I thought it was an opera adaptation of "The Blob," which could probably work.

If the rest of the music could be even half as swingin' as the Mack David and Burt Bacharach-authored theme song, SHUT UP AND TAKE MY MONEY
posted by hangashore at 3:01 PM on December 15, 2020 [3 favorites]

This is really delightful.

Here's the JSON format used to load saved recordings, including the Christmas songs

It was painful to see some of those numbers in that file. Someone needs to write a JSON rectifier that truncates fractional digits after six zeros and truncates and rounds up after six nines.
posted by sjswitzer at 4:10 PM on December 15, 2020

Too late for edit window, but I meant “at” rather than “after”.

And, FWIW, OverlappingElvis, you could do the same thing with the constants in your lovely gist.
posted by sjswitzer at 4:23 PM on December 15, 2020

tbh I find unnecessary precision charming
posted by OverlappingElvis at 4:46 PM on December 15, 2020 [2 favorites]

tbh I find unnecessary precision charming

i have some lab reports for you to grade
posted by lalochezia at 6:47 PM on December 15, 2020 [7 favorites]

updated the gist and figured out a bunch of the issues with the format
posted by OverlappingElvis at 9:51 PM on December 15, 2020 [7 favorites]

Adding a little bit of drift makes it a lot more naturalistic, and they're doing some kind of weird enum thing with the vowels/consonants that I missed the first time around
posted by OverlappingElvis at 9:51 PM on December 15, 2020

A friend’s friend thought the Christmas carols were in Latin, but I explained that the AI was trained on Italian opera singing, so it naturally comes up with something that sound vaguely Latin.
posted by sjswitzer at 9:57 PM on December 15, 2020

OverlappingElvis, this is incredible. Couple this with a procedural generator and I'll never search youtube for "studying music" again...
posted by hat_eater at 7:12 AM on December 16, 2020 [1 favorite]

Ok, probably the last major update to the gist for a bit: there's now a very simple GUI to map individual midi tracks to individual voices.
posted by OverlappingElvis at 6:02 PM on December 16, 2020

And the final resulting Blob Canon
posted by OverlappingElvis at 6:16 PM on December 16, 2020 [6 favorites]

ok last one: all star but it's a bach chorale but it's sung by the blob opera
posted by OverlappingElvis at 11:08 PM on December 18, 2020 [1 favorite]

I may have gone down a bit of a rabbit hole here but I've released my blob opera converter as a standalone NPM module.
posted by OverlappingElvis at 10:39 AM on December 26, 2020 [2 favorites]