Why was the last blog post relevant to this project? Well, recently one of my friends told me that they’re going to Japan to study for a week and asked if to edit some vlog videos for him. I thought it would be great fun to do; however, all I thought about was making a vlog that would be popular, with all the YouTuber troupe included. While it’s fine to do that, it did make me debate why I couldn’t just make something without an ulterior motive, and that sent me on the thought spiral in the last post.
This is why the audio-visual piece I want to make is going to be a riff on youtube culture, being hopefully unsettling and pleasing at the same time, and manipulated in a certain way. The way I’ll do that is by making a video that is a musical that talks about everything I learn this year, the main reason for it being a musical is because the nonsensical premise of speaking through song could be used to reference formulaic youtube vlog videos are and how one has to suspend belief to a certain extent in musicals, implying that one should do the same for youtube videos (which many don’t). But honestly, I just thought it would be a lot funnier to make a musical.
But that certain way? Sonification, or specifically sonifiying a video I make, then using that to then create an image sequence of spectrograms and finally animation of those spectrograms. This process was something I tried to do 2 years ago for my art foundation course, but it was the year that covid struck and I didn’t have the tools or knowledge to follow through with my idea.
https://www.behance.net/gallery/95455971/My-Soundscape
This is a link to my attempt at this idea but I didn’t get to the part of animating a video using the audio of the original images/video. I know a lot more now but I know I still don’t know enough to animate a whole video using spectrograms in an efficient way.
That is the biggest problem I’m going to face with this project, Time. Will I have enough of it?
Well, I first have to make a script for my video with directions for the editing, what I say, and a basic idea of what the music and sound effects will be like.
Then, I have to make most of the sound aspect of the piece, making sure it’s in time with the video.
Next, I have to convert the video into sound, which is a big problem because I know of a few ways to do that but they can all only be done as images. This means I would have to make an image sequence of the whole video (I’ll make it in 30fps) and convert them individually into sound bites.
Honestly, that’s the simple part (kind of) because I then have to convert all of the spectrograms into images again which complicates this process and makes it a lot harder than just turning an image into a sound. That would be around 7200 different sound bites and thus frames, all of which I have to individually capture (I know of ways to do this in FFmpeg; however, I am not that learned when it comes to coding and it becomes a lot more complicated to make something when I’m going from video to sound to images and then back to video).
Lastly, I would compile the images into a video again and add the “soundtrack” and also a representation of the sound of the video spectrograms as a drone throughout the piece.
I also want to manipulate the video using the audio effects from the soundtrack, so I need to work that out as well.
I don’t really need to do this, I could just make a sound piece in the same vein. Nevertheless, I’ve chosen to make something like this because the whole reason I talk about what I’ve learnt during my year in sound arts is that it has improved my practice outside of sound arts (video editing and music). I want to a piece only I could conceive, something original to me, and my way of doing that is making a piece that is an amalgamation of all my creative disciplines.
Before I did sound arts, I was, and still am, a fine artist and musician. Through this piece, I want that aspect of myself to be shown because I do believe it’s the links between different disciplines that elevate my work.