Gaimidian Graveyard

[bibliotechy]

My entry is an audio book, because audiobooks are books, dammit!

In brief, I have taken an existing audio book, Neil Gaiman reading his The Graveyard Book and transformed transformed the audio into MIDI (inspiration), creating an audio book that sounds like a dying robot desperately trying to be human. Then, I ran the MIDI through the OpenAI Whisper automatic speech recognition system to see just what whisper thinks is being said, then add the output SRT files to the video as captions.

When reading captions along with the audio, the weird sounds actually start to sound like those words you are reading, as your brain fills in the gap.

It's only 2.5 hours of the original book because whisper processes bad audio very slowly, and I ran out of time to do the full book. More later, plus a full write up of the process.

For now, it lives here- and be sure to turn on the captions
https://www.dropbox.com/scl/fi/n3j3b18ib81xcopf8k61c/gaimidian.mp4?rlkey=c613rvz8jpvct9anaqjq2z3bp&dl=0

[shayneoneill]

This is trippy. At first it was 'Well this just sounds like generative experimental piano". But the more I listen, the more I can actually kinda hear the words. Kinda. Its like an angry piano trying to immitate words.

Good work! Not sure it quite fits the comp rules, but its pretty cool.

[bibliotechy]

A writeup of my ideas going into this and the process for creating this video/audio/book
https://chads.space/words/2022/12/03/nanogenmo-gaimidian-graveyard.html

[greg-kennedy]

What's happened to the video? I wanted to watch it but it seems to have gone missing!

[bibliotechy]

@greg-kennedy YouTube took it down but didn't tell me. :( Finally found a copy from an old backup 😌

https://www.dropbox.com/scl/fi/n3j3b18ib81xcopf8k61c/gaimidian.mp4?rlkey=c613rvz8jpvct9anaqjq2z3bp&dl=0