PiP pt 2 -- more ffmpeg

contents
blur: somewhat complex	glossary
structure/workflow	keyframes also my full post
color balance	rotation simple in ffmpeg
crossfade	slideshow complex! use a GUI app
de-interlace	watermark
gif

NB: 1) try to make all cuts on keyframes, 2) "overlay" filter operations are unreal time lost

---

Links: 1) PiP Pt I   2) hardware for a system that will render

Impossibly, the simple crossfade is one of the most elusive video effects in Linux. You'd think simple transitions would be managed by any of the 10 GUI's out there, but the 10 GUI's are only intermittently usable whenver QT upgrades, or GTK upgrades, etc. So the only totally reliable way to do video is through a CLI, eg ffmpeg or melt. It's a complete pain in the ass, of course, so I use GUI's when they're actually running. Ex: Pitivi (and Shotcut subsequently - QT5), Flowblade, or Olive, are OK for crossfades -- not perfect but OK. One month, GTK got an upgrade, and thus we learn that GTK is not backwards compatible... poof! PiTiVi gone...

$ pitivi
ERROR - The following hard dependencies are unmet:
==================================================
- gtk not found on the system
# pacman -S gtk
error: target not found: gtk
# pacman -S gtk2
warning: gtk2-2.24.32-2 is up to date -- reinstalling

... WTF?!! So there's little choice but to go CLI for one's mental health -- it's our lives we're giving up to edit.

You can pick any GUI application. Like, there's final edit color balancing in Blender (Blender is so complex it's practically an OS itself), but even color balancing can be done inside ffmpeg (see 0612 TV w/NERDfirst around 6:00).

Besides impossible crossfades, the challenge within ffmpeg is keeping the number of renders to a minimum. We can use ffplay to preview or, since ffmpeg is non-destructive (non-linear), we can do a test render and check it. And don't forget ffprobe to get information out of the files. Also recall that, if we just add an audio track, we can copy the video codec and add audio without degrading the video whatsoever.

other options

And of course, one should also not overlook the MLT framework, eg "melt", for command line transitions, if it's still being developed, not sure. In 2020, there's bleeding-edge on the AUR (mlt-git), and in "extra" (mlt). With MLT you will need a frame counting tool, not just a second counting tool to make edits.

MLT Melt Transitions (13:48) Kris Occhipinti, 2012.

structure and workflow

See graphics below. Each edit requires several files (see my typical file structure)The grouping below is typical but of course can't include hand-drawn items: 1) storyboard 2)timing notes for audio sync with narration and so forth. How to pause and linger on a frame? The best storyboard may be a video describing the drawings while doing a UMlet workflow.

---

If you imagine the workflow graphic above as 3-D, one can see that each item is not equal. In the 3rd column, there's a place to get started on code and that's because the most flexible TTS is via Python, but Python itself is a deep subject requiring thoughtful installation. So just to make the TTS possible is likely a 2 week project.

blur (+1 render)

This is a complex ffmpeg filter. Three processes are accomplished, a box region is cropped out, it's blurred, and it's re-overlaid onto the original video. Added difficulty is determining an x:y dimension for the region.

$ ffmpeg -i foo-in.mp4 -filter_complex \
"[0:V]crop=100:100, boxblur=20[fg]; \
[0:V][fg]overlay=(main_w-200):30" foo-out.mp4

Ffmpeg blur box (6:45) Cool IT Help, 2019. A bit of a strong accent, but no BS.

concatenate (+1 render)

The +1 render is when getting one's clips ready: rotating, accomplishing any dissolve edits ahead of time. There's no render during the actual concatenation. Try different orderings of your clips in a simple M3U, playing the M3U with vlc or xplayer. You can keep filenames simple, avoiding full absolute paths, if you place the M3U in the same directory with the clips. The format...

# This is a comment
somevideo.mp4
anothervideo.mp4

• finalize the mix order in the m3u
• open vlc and be sure "repeat" is disabled (else it will go into infinite loop). close vlc.

• $ vlc playlist.m3u --sout "#gather:std{access=file,dst=vlcmerged.mp4}" --sout-keep

...in 5-6 seconds, this command concatenated 31 clips into a 7 minute 720P h264 video with sound, no tearing, and so on. It's production quality. The process is described here, albeit with Windows conventions.

Using ffmpeg can also be without rendering (-c copy), but will likely lead to DTS timestamp errors and tearing in playback. Even if one renders, the quality has not yet been as high as the vlc method above. Additionally, it doesn't work with an M3U. One has to make a text file with a special syntax on each line (viz, file 'foo.mp4') and one file per line, repeats also OK however.

$ ffmpeg -f concat -i playlist.txt -c:v libx264 -an grouped.mp4

I find that playback with the resulting file is often jerky or paused, even when all files are the same encoder and there is discussion (render, tbr) about how to manage this, but I've never found anything to prevent it. If I had time, I would troubleshoot until a fix.

colorbalance (+1 render)

Not sure if we could chain filters and do this with a single render

$ ffmpeg -i foo.mp4 -vf "colorbalance=rs=-0.25,colorbalance=bm=-0.25" evencolor.mp4

How to use FFMpeg - Advanced Pt1 (19:37) 0612 TV w/NERDfirst, 2015. Color balance at the 7:30 mark.
Another great site, and he also has a video attached.

crossfade (+2 render)

Typically, crossfades (aka "dissolves") are the most complex thing in Linux video editing and a separate post will likely be forthcoming. Some progress has ocurred however. This recent article is a must-read on the subject.

$ ffmpeg -i invid1.mp4 -i invid2.mp4 -filter_complex xfade=transition=fade:duration=2:offset=24 outputvid.mp4

... where the duration of the crossfade is 2 seconds, and it begins at the 24 second mark. Note that this effect renders with extremely high (hot) CPU cycling, even for a short clip.

Pan and Zoom Slideshow (26:50) Chipper Videos, 2019. Blender 2.8. Crossfade is shown in approximately last 5 minutes, but acccurate. Most is concerned with Ken Burns.
Crossfade (0:20) doufuwang, 2016. Shows the crossfade or dissolve effect perfectly.
Tech Notes: FFmpeg Multi Fades In Out (37:09) Steve AB4EL, 2017. Hilarious. Drones on and on. Takes 3 pictures and makes slide show. (8:30)fade transitions
FFmpeg Cross fades (2:25) A Forum, 2020. One of the best out there. Command given above is taken from here.

de-interlace (+1 render)

De-interlacing increases the frame rate. It could take you from 30 to 60, for example.

Sample Video Editing Workflow using FFmpeg (19:33) Rick Makes, 2019. Covers de-interlacing (2:00) to get rid of lines,cropping, audio library(13:30), and so on.

$ ffmpeg -i foo.mp4 -vf "bwdif=1" -c:a copy defoo.mp4

glossary

• frame rate vs. fps: these appear identical, and its unfortunate. It's nuanced like iframe and keyframe. Framerate (-r) is an input parameter used for setting fps. Fps is the stream speed. Ex: I want a 6 second video of 3 photos - I set fps to 2, and framerate to 12.
• ripple-cut: cut per usual, but then it magnets over to previous clip to fill the space

scripts

There are two types we might want to consider

• bash: easiest for ffmpeg sequence, not sure for tts
• python: using pip we can get some tts modules, send to WAV

We want to make bash scripts for the ffmpeg actions.

timing

If I want to add sound effects or narrate, how do we at once watch the film and secondly enter sounds? The best way is to watch the film and take timing notes. Perhaps you want to pause on a frame for a few seconds while you discuss something.While you're watching the soundless video, just note how long you hold down the pause key to do your talking, say 5 seconds, and then using a loop filter

reverse

This one just for video.

$ ffmpeg -i foo -vf reverse reversed.mp4

rotation (+1 render)

Few do it better than NERDfirst. The reason for the division is ffmpeg determines the angle in radians. Furthermore the angle of rotation is clockwise. The most common is righting an upside down video.

$ ffmpeg -i foo.mp4 -filter:v "rotate=PI" fout.mp4

How to use FFMpeg (12:48) 0612 TV w/NERDfirst, 2015. At the 11:00 mark, he describes rotation, and scaling just before it.

slideshow (+1 or 2 render)

The simplest slideshow just cuts from one to the next at a set time, say 10 seconds. This is pretty easy to do on an entire directory of numbered pix, say foo01.png, foo02.png, foo03.png, etc. The "2d" indicates how many digits in the numbering system.

$ ffmpeg -framerate 1/10 -i foo%02d.png -c:v libx264 -r 30 -pix_fmt yuv420p slideshow.mp4

Similar to the Bash above is a slideshow. Run a batch file on a set of pictures and to make short 5 second video clips of them all, first render. At that point, we can do Ken Burns on some videos, and on others just overlap one to the next, the second render. Here, I have 3 x 5 second clips, and first accomplished $ yay ffmeg-concat so that I didn't have to use the complicated overlay filter.

$ shotcut

This fades up and down from black for each one and is not a dissolve between them. For those using Windows, a caret "^" breaks-up the command in a batch instead of a backslash.

$ffmpeg -i foo1.mp4 -i foo2.mp4 \
-i foo3.mp4 -filter_complex \
"fade=in:st=0:d=1,fade=out:st=4:d=1[f0]; \
fade=in:st=0:d=1,fade=out:st=4:d=1[f1]; \
fade=in:st=0:d=1,fade=out:st=4:d=1[f2]; \
[f0][f1][f2]concat=n=3[123]" \
-map [f123] foocombine.mp4

Pan and Zoom Slideshow (26:50) Chipper Videos, 2019. Blender 2.8. Most is concerned with Ken Burns, but crossfade at the end.
Basic Slideshow (8:09) Luke Smith, 2017. Easy to put all of these together, but nothing here about ken burns
Tech Notes: FFmpeg Multi Fades In Out (37:09) Steve AB4EL, 2017. Hilarious. Drones on and on. Takes 3 pictures and makes slide show. (8:30)fade transitions

watermarking (+1 render)

Accomplished with the "overlay" filter, and also possible in a batch (2019, Cool IT Help, seek to 5:00min), the bash equivalent being

#!/bin/bash
for file in *.mp4 do
ffmpeg -i "$file" -i watermark.jpg -filter_complex "overlay = 20:20" "${file%.*}_wm".mp4
done

keyframes

Keyframing is a large topic, thus a separate post will be developed, but why is this a challenging topic?

• keyframes are not physically part of a video. They are placemarks temporarily created and indexed inside whatever video application is being used to edit the video. Being application-specific, they must be learned for each application
• Nomenclature issues abound. Keyframes and I-frames are similar and related, so that some use them interchangeably. There is the further confusion that iframes in HTML are different from iframes in video editing, but are again, related.
• ffmpeg refers to keyframes as "GOP" or Group of Pictures. It's set with the "-g" switch but requires re-encoding.
• keyframes are used differently in animation and video, so that learning basic video keyframes means sifting through many Google results for animation keyframing, ambiguously labeled. This is so common that it's hard to avoid (wastefully) learning animation usage while attempting to learn video usage.
• Blender, which is so vast and complex that it's like learning another programming language even before one considers keyframes, is probably the only GUI for which it's worth taking the time to learn keyframes. This means learning Blender just to get to the keyframe level.

Keyframes in Blender 2.8 (7:40) Blender, 2019. Animation-centric video, but significant application to video, since keyframes apply to both. 1:30 Keyframes can be managed in the Dopesheet, Graph editor, and Timeline.
Keyframes (and other) in 2.8 (9:48) Code, Tech, and Tutorials, 2019. Solid inadvertent tour around the VSE via a simple edit. Shows how to do transitions simply without inserting them, pressing "i" for keyframe.

gifs

Definitely an important concept for texting, however a person must go online to get this done: an unknown site has your vids and any related IP. On one's own machine, you'd think it would be quick, as 3rd party apps such as gifify appear to make it so, but these almost never work. This means we're dealing with f**king seeking syntax once again

Beginning at the 2:30 mark, create 10 seconds of sequential PNGs using a 2 digit format. I resized them down to a 640x360 size, by just determining the clip native resolution with ffprobe. Eg, I resize GoPro footage is 1280x720 natively, so I drop it to 640x360 or 320x180. Zoom is natively 640x360, so I leave it, or resize it to 320x180.

ffmpeg -ss 00:02:30 -i foo.mp4 -t 0:10 -s 240x135 %02d.png

Concatenate the PNG's (or use JPG's) into a GIF

ffmpeg -i %02d.png output.gif

These tend to be made at 25 fps, so I slow it down sometimes for slo-mo, eg 12 frames per second...

ffmpeg -framerate 12/1 -i %02d.png output.gif