What is a spectrogram?
“A spectrogram is a visual representation of the spectrum of frequencies of a signal as it varies with time.”
I’ve been interested in spectrograms for a long time but I always did wonder why I couldn’t find anything on animations using spectrogram images.
I understand why now.
Usually, when creating sound out of images to get that image represented in a spectrogram, the process is quite simple just make sure your image is either a bitmap or raw image data (or any lossless format I believe). The problems start occurring when you try to make the spectrogram into an image.
Now that the process has become going from image – audio – image, the duration of the created audio clip matters a lot more. Depending on the program, the audio created may squash or expand the spectrogram which in turn changes the size of the final image. I found that there weren’t many programs where you could dictate the “aspect ratio of the audio file” and was a big problem for me as I needed the images to be a certain size.
But that problem was only multiplied by the fact that I was trying to make a video out of spectrograms. To get a well-proportioned spectrogram, the sound bites I created were around 5 seconds long; however, in my video, a single image was 1/30th of a second. This meant that the idea I had before (that I haven’t mentioned) to make an animation in an audio file was not feasible as the resolution and speed of a spectrogram just weren’t high enough.
This doesn’t seem to pose a problem in my project, but one thing I wanted to do was edit the visual aspects using audio effects. Looking at the spectrograms I made, it was clear to me that editing a video using audio effects would have unique quirks to them that one could not replicate usually video effects. A solution would be to create a noise track that is affected by the effects, turn that into a spectrogram and then superimpose the spectrogram images of my video onto that track.
Another hurdle was exporting spectrograms as images. Out of all the programs I downloaded for this project, the only one that could turn a sound bite into a spectrogram image was Sonic Visualiser. But that restriction to certain software is a but part of the biggest problem I’ve faced in this project.
Batch processing.
Without enough knowledge in coding, it was going to be hard to make a video around 5000 frames into audio bites of the right size and then into spectrogram images but I sorely underestimated how long it would take to export them all.
Here’s an example.
That image was me trying to work out if I could use audacity to convert images into sound. Each one would take about a minute for the first conversion and 2 for the second one. It was also the first time I had created a video frame from a audio file, and it has an incredibly interesting effect.
Here’s what I use as drone in my piece. The images are stills from the video I made (1 frame for every 180) and then I just lined them up in audacity once they were sound bites. The image was actually 5x-6x longer than that image above just I couldn’t post that because of site restrictions. Also, the audio generated by the stills was really interesting to listen to and definitely something I want to look into more for a future project. Despite being around 20-30ish photos, it took around 1:30-2 hours to do that part, time I don’t have.
But why do I not want to just recreate the filter in a video editing program? I want the piece to be authentic and fresh, on top of that, it’s incredibly hard to recreate. However, if I do find myself short on time it might be wise to just recreate the look as while the piece loses something important, it is way more important that there is a visual aspect.