Rendering math

Is there a way to render math only with ffmpeg?

Short answer: No.

Whether you use the “drawtext” or “subtitles” approach, the features are too lacking and you can not easily get even “superscript” and “subscript”. That is, you can’t write \(E = mc^2\), or \(69 + 12 \times \log_2\left(f / 440\right)\), etc. in subtitles by easy way.

What would happen if we were to achieve this with just “ass”?

It will be ridiculous…like these:

#! /bin/sh
pref="`basename \"$0\" .sh`"

cat<<__EOF__>${pref}.ass
[Script Info]
PlayResX: 640
PlayResY: 360
WrapStyle: 1

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Expl, Arial,34,&H00FFFFFF,&H00FFFFB0,&H00303030,&H80000008,-1,0,0,0,100,100,0.00,0.00,1,1.00,2.00, 7 ,30,30,30,0

[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,00:00:00.00,00:00:30.00,Expl, NTP,0,0,0,,{\pos(30,30)}E = mc,  equation in German-born physicist Albert Einstein's theory of special relativity that expresses the fact that mass and energy are the same physical entity and can be changed into each other.
Dialogue: 0,00:00:00.00,00:00:30.00,Expl, NTP,0,0,0,,{\pos(125,25)}{\fs20}2
__EOF__
#
ffmpeg -y -filter_complex "
color=black:s=640x360:d=30,subtitles=${pref}.ass
" ${pref}.mp4
../_images/ass_sup_example.png 
#! /bin/sh
pref="`basename \"$0\" .sh`"

cat<<__EOF__>${pref}.ass
[Script Info]
PlayResX: 640
PlayResY: 360
WrapStyle: 1

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Expl, Arial,34,&H00FFFFFF,&H00FFFFB0,&H00303030,&H80000008,-1,0,0,0,100,100,0.00,0.00,1,1.00,2.00, 7 ,30,30,30,0

[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,00:00:00.00,00:00:30.00,Expl, NTP,0,0,0,,{\pos(30,30)}freq to MIDI note number:\N69 + 12 log (f / 440)
Dialogue: 0,00:00:00.00,00:00:30.00,Expl, NTP,0,0,0,,{\pos(185,84)}{\fs20}2
__EOF__
#
ffmpeg -y -filter_complex "
color=black:s=640x360:d=30,subtitles=${pref}.ass
" ${pref}.mp4
../_images/ass_sub_example.png 
#! /bin/sh
pref="`basename \"$0\" .sh`"

cat<<__EOF__>${pref}.ass
[Script Info]
PlayResX: 640
PlayResY: 360
WrapStyle: 1

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Expl, MS Gothic,34,&H00FFFFFF,&H00FFFFB0,&H00303030,&H80000008,-1,0,0,0,100,100,0.00,0.00,1,1.00,2.00, 7 ,30,30,30,0

[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,00:00:00.00,00:00:30.00,Expl, NTP,0,0,0,,{\pos(30,30)}The volume of a sphere of radius r using an integral:

Dialogue: 0,00:00:00.00,00:00:30.00,Expl, NTP,0,0,0,,{\pos(315,115)}{\fscy300}∫
Dialogue: 0,00:00:00.00,00:00:30.00,Expl, NTP,0,0,0,,{\pos(345,115)}{\fs20}r
Dialogue: 0,00:00:00.00,00:00:30.00,Expl, NTP,0,0,0,,{\pos(345,195)}{\fs20}0

Dialogue: 0,00:00:00.00,00:00:30.00,Expl, NTP,0,0,0,,{\pos(360,150)}π( r - x ) dx
Dialogue: 0,00:00:00.00,00:00:30.00,Expl, NTP,0,0,0,,{\pos(445,144)}{\fs20}2
Dialogue: 0,00:00:00.00,00:00:30.00,Expl, NTP,0,0,0,,{\pos(510,144)}{\fs20}2
__EOF__
#
ffmpeg -y -filter_complex "
color=black:s=640x360:d=30,subtitles=${pref}.ass
" ${pref}.mp4
../_images/ass_integral_example.png

These may be fine if you use it once in a lifetime, but if you need such things often, it will take too much time for trial and error and maintenance.

After all…

In general terms, if you want to render the formula, it’s easy to create an image using tools that allow you to write formulas and embed it in your video.

Even MS word can write some complicated formulas. If you intend to create html pages and capture them, Sphinx etc. can easily handle formulas. The site you’re looking at now is also using the Sphinx (and MathJax). The Google Chrome extension also has a nice formula editor.

Use these tools to create a mathematical image and make it a video as follows:

#! /bin/sh
ffmpeg -y -i formula1.png -i formula2.png -filter_complex "
[0:v]pad='1280:720:(ow-iw)/2:(oh-ih)/2',loop=-1:size=2,trim=0:30,setpts=PTS-STARTPTS[0v];
[1:v]pad='1280:720:(ow-iw)/2:(oh-ih)/2',loop=-1:size=2,trim=0:30,setpts=PTS-STARTPTS[1v];
[0v][1v]concat
" out.mp4

If you are familiar with TeX and want to use formulas frequently, the following script to convert TeX formulas to images immediately would be useful:

latexmath2png.sh
#! /bin/sh
outname="${outname:-math}.png"
dpi=${dpi:-"1760"}
gamma=${gamma:-"1.5"}
#
cat << '__END__' > /tmp/math.tex
\documentclass[12pt]{article}
\usepackage[utf8x]{inputenc}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{amsfonts}
\usepackage{bm}
\pagestyle{empty}
\begin{document}

\begin{math}
__END__
cat >> /tmp/math.tex
cat << '__END__' >> /tmp/math.tex
\end{math}

\end{document}
__END__
#
pwd="`pwd`"
(
    cd /tmp
    latex --interaction=nonstopmode math.tex && \
    dvipng -o "${outname}" -T tight -z9 -gamma ${gamma} -D ${dpi} -bg Transparent math.dvi && \
    mv "${outname}" "${pwd}"
)

[me@host: ~]$ ./latexmath2png.sh
69 + 12 \times \log_2\left(f / 440\right)
^D
This is pdfTeX, Version 3.14159265-2.6-1.40.16 (TeX Live 2015/W32TeX) (preloaded format=latex)
 restricted \write18 enabled.
entering extended mode
(./math.tex
LaTeX2e <2015/01/01> patch level 2
Babel <3.9l> and hyphenation patterns for 79 languages loaded.
(c:/texlive/2015/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(c:/texlive/2015/texmf-dist/tex/latex/base/size12.clo))
(c:/texlive/2015/texmf-dist/tex/latex/base/inputenc.sty
(c:/texlive/2015/texmf-dist/tex/latex/ucs/utf8x.def))
(c:/texlive/2015/texmf-dist/tex/latex/ucs/ucs.sty
(c:/texlive/2015/texmf-dist/tex/latex/ucs/data/uni-global.def))
(c:/texlive/2015/texmf-dist/tex/latex/amsmath/amsmath.sty
For additional information on amsmath, use the `?' option.
(c:/texlive/2015/texmf-dist/tex/latex/amsmath/amstext.sty
(c:/texlive/2015/texmf-dist/tex/latex/amsmath/amsgen.sty))
(c:/texlive/2015/texmf-dist/tex/latex/amsmath/amsbsy.sty)
(c:/texlive/2015/texmf-dist/tex/latex/amsmath/amsopn.sty))
(c:/texlive/2015/texmf-dist/tex/latex/amscls/amsthm.sty)
(c:/texlive/2015/texmf-dist/tex/latex/amsfonts/amssymb.sty
(c:/texlive/2015/texmf-dist/tex/latex/amsfonts/amsfonts.sty))
(c:/texlive/2015/texmf-dist/tex/latex/tools/bm.sty) (./math.aux)
(c:/texlive/2015/texmf-dist/tex/latex/ucs/ucsencs.def)
(c:/texlive/2015/texmf-dist/tex/latex/amsfonts/umsa.fd)
(c:/texlive/2015/texmf-dist/tex/latex/amsfonts/umsb.fd) [1] (./math.aux) )
Output written on math.dvi (1 page, 388 bytes).
Transcript written on math.log.
This is c:\texlive\2015\bin\win32\dvipng.exe 1.15 Copyright 2002-2015 Jan-Ake Larsson
[1]
[me@host: ~]$
../_images/latexmath2png_example.png

This video:

Watch on youtube.com

was created using the following formula and script:

\[\begin{split}\begin{pmatrix} x_{1} & y_{1} & 1 & 0 & 0 & 0 & -x_{1} x'_{1} & -x'_{1} y_{1} \\ 0 & 0 & 0 & x_{1} & y_{1} & 1 & -x_{1} y'_{1} & -y_{1} y'_{1} \\ x_{2} & y_{2} & 1 & 0 & 0 & 0 & -x_{2} x'_{2} & -x'_{2} y_{2} \\ 0 & 0 & 0 & x_{2} & y_{2} & 1 & -x_{2} y'_{2} & -y_{2} y'_{2} \\ x_{3} & y_{3} & 1 & 0 & 0 & 0 & -x_{3} x'_{3} & -x'_{3} y_{3} \\ 0 & 0 & 0 & x_{3} & y_{3} & 1 & -x_{3} y'_{3} & -y_{3} y'_{3} \\ x_{4} & y_{4} & 1 & 0 & 0 & 0 & -x_{4} x'_{4} & -x'_{4} y_{4} \\ 0 & 0 & 0 & x_{4} & y_{4} & 1 & -x_{4} y'_{4} & -y_{4} y'_{4} \end{pmatrix} \begin{pmatrix} a \\ b \\ c \\ d \\ e \\ f \\ g \\ h \end{pmatrix} = \begin{pmatrix} x'_{1} \\ y'_{1} \\ x'_{2} \\ y'_{2} \\ x'_{3} \\ y'_{3} \\ x'_{4} \\ y'_{4} \end{pmatrix}\end{split}\]
#! /bin/sh
pref="`basename \"$0\" .sh`"
#
"/c/Program Files/ffmpeg-4.1-win64-shared/bin/ffmpeg" -y \
-i Pexels_2877_2880.mp4 \
-i math.png \
-filter_complex "
[0:v]scale=1280:720,setsar=1[bg];
[1:v]format=yuv420p,scale=-1:720/2,loop=-1:size=2,negate,colorkey=black,setsar=1[fg];
[bg][fg]overlay=x=100:y=100:shortest=1:'enable=between(t,3,18)'
" ${pref}.mp4