Animated AI background video generated in Deforum Stable Diffusion. Compositing, lyrics and audio reactive effects edited in After Effects.

Music 🎵
Fall Out Boy - What A Time To Be Alive

Playlists 🤘
Lyric Videos:    • Lyric Videos  

Lyrics 🎶
Sometimes you wonder if we’re ever looking back
At a picture of two thousand nineteen
And saying “That’s the way, the world it used to be
Before our dreams started bursting at the seams,”
We’re out here and we’re ready
To livestream the apocalypse
I don’t care if it’s pretty
The view’s so pretty from the deck of a sinking ship

Everything is lit except my serotonin yeah
Everything is lit but my lightning bolt brain
Everything is lit…

But baby please I just need someone to hold me
Even though you don’t even know me
Oh I’m going neon in the night time
Oh
What a time to be alive

They say that I should try
Meditation
But I don’t want to be with my own thoughts
It never felt that much
Like medication
And I just want to be your cherry on top
We’re out here and we’re ready
We’re here and we’re ready
To livestream the apocalypse
And I’m going on and on and on
Make me like a charm on your necklace

When when when I said “Leave me alone,”
This isn’t quite what I meant
I got the quarantine blues
Bad news, what’s left?
So it seems the vulture’s getting too full to fly
Oh
What a time to be alive

Sometimes you wonder if
We’re ever looking back

Tools 🛠️
Background made in Automatic1111 Stable Diffusion Deforum extension with help from ChatGPT.
Edited in Adobe Premiere Pro and After Effects.

Process 🪄
1. Finding fitting stable diffusion prompts for the music video.
I ask ChatGPT to give me fitting prompts based on the lyrics I feed it. I test the outputs and make changes based on the results from img2img generations with Stable Diffusion. I also try out different models and Loras to find outputs I like.

2. Getting music reactive keyframes for the animation.
I give the song to this audio keyframe tool: www.chigozie.co.uk/audio-keyframe-generator/ and give it suitable functions for the values I want. I personally use 25 frames per second, something like “0.60 - x^6” for strength schedule and “1 + x*10” for Translation Z in 3D mode. I often play around with these settings and also give the 3D rotations different functions to react to.

3. Setting up a render in Deforum.
Now I enter the keyframes and prompts into Deforum to start generating an animation. I usually do this 4 times with slightly tweaked settings and prompts to get different results to work with later.

4. Editing in Premiere Pro.
To get a more dynamic video, I cut between the different animations in a multicam sequence, picking the best parts and cutting to the music.

5. Finishing up in After Effects.
In After Effects I manually add the lyrics, timing and dividing them as I see fit. I also add a bunch of effects like additional 3D camera movements, shakes, overlays, particles, blur and chromatic aberration. Most of these are made to react to the music through generated audio keyframes.

#lyrics #lyricvideo #lyricsvideo