# Use of audio visualization as a visual effect¶

doc

ffmpeg does not provide a way to use sound data itself directly as a visual effect. However, as there are several sound visualization methods, using visualization of them will make it a visual effect.

I will show you a few examples of that. Things that can withstand professional use are probably difficult to achieve with only ffmpeg, but if your needs are simple, it can be realized as in the example to show from now.

## with alphamerge’ (1)¶

00:00:45

The first example uses “alphamerge” for visualized video by “showsqt”.

#! /bin/sh
ifn="Air on the G String (from Orchestral Suite no. 3, BWV 1068).mp3"
ofn="out_01.mp4"
bgi1="beach-dawn-dusk-274053.jpg"
bgi2="art-backlit-beach-256807.jpg"
#
ffmpeg -y -i "${ifn}" -i "${bgi1}" -i "${bgi2}" -filter_complex " [0:a]showcqt=s=1280x720:basefreq=27.5:endfreq=4186.0[alpha]; [1:v]scale=-1:720,pad='1280:720:(ow-iw)/2:(oh-ih)/2',loop=size=2:loop=-1[1v]; [2:v]scale=-1:720,pad='1280:720:(ow-iw)/2:(oh-ih)/2',loop=size=2:loop=-1[bg]; [1v][alpha]alphamerge[fg]; [bg][fg]overlay,setsar=1[v] " -map '[v]' -map '0:a' -c:a copy -shortest "${ofn}"


## with alphamerge’ (2)¶

00:01:45

It is essentially the same as the first example. It visualizes each stereo channel with “showcqt” and “vflip” on the right channel. (We have a redundant “scale” due to a bug in “vflip.” If you omit this extra “scale”, ffmpeg will crash.)

#! /bin/sh
ifn="Air on the G String (from Orchestral Suite no. 3, BWV 1068).mp3"
ofn="out_02.mp4"
bgi1="beach-dawn-dusk-274053.jpg"
bgi2="art-backlit-beach-256807.jpg"
#
ffmpeg -y -i "${ifn}" -i "${bgi1}" -i "${bgi2}" -filter_complex " [0:a] pan=stereo|c0=c0|c1=c0,showcqt=s=1280x360:basefreq=27.5:endfreq=4186.0 [vcqt_L]; [0:a] pan=stereo|c0=c1|c1=c1,showcqt=s=1280x360:basefreq=27.5:endfreq=4186.0 ,scale=1280:720,vflip,scale=1280:360 [vcqt_R]; [vcqt_L][vcqt_R]vstack[alpha]; [1:v]scale=-1:720,pad='1280:720:(ow-iw)/2:(oh-ih)/2',loop=size=2:loop=-1[1v]; [2:v]scale=-1:720,pad='1280:720:(ow-iw)/2:(oh-ih)/2',loop=size=2:loop=-1[bg]; [1v][alpha]alphamerge[fg]; [bg][fg]overlay,setsar=1[v] " -map '[v]' -map '0:a' -c:a copy -shortest "${ofn}"


## with alphamerge’ (3)¶

00:02:45

It is almost the same as the previous example, but here we use only one picture, not two pictures.

#! /bin/sh
ifn="Air on the G String (from Orchestral Suite no. 3, BWV 1068).mp3"
ofn="out_03.mp4"
bgi="beach-dawn-dusk-274053.jpg"
#
ffmpeg -y -i "${ifn}" -i "${bgi}" -filter_complex "
[0:a]showcqt=s=1280x720:basefreq=27.5:endfreq=4186.0,split[alpha][bgbase];
[1v][alpha]alphamerge[fg];
[bgbase]geq=r=0:g=0:b=0[bg];
[bg][fg]overlay,setsar=1[v]


## with geq’ (2)¶

00:04:45

It is essentially the same as the previous example. We use the average of three points of “showcqt” result.

#! /bin/sh
ifn="Air on the G String (from Orchestral Suite no. 3, BWV 1068).mp3"
ofn="out_05.mp4"
bgi="art-backlit-beach-256807.jpg"
#
ffmpeg -y -i "${ifn}" -i "${bgi}" -filter_complex "
[0:a]
showcqt=s=1280x720:basefreq=27.5:endfreq=4186.0
,geq='p(240,355)/3 + p(400,355)/3 + p(560,355)/3'
[vcqt];
[1:v]