NB: 1) try to make all cuts on keyframes, 2) "overlay" filter operations are unreal time lost
Links: 1)
PiP Pt I 2)
hardware for a system that will render
Impossibly, the simple crossfade is one of the most elusive video effects in Linux. You'd think simple transitions would be managed by any of the 10 GUI's out there, but the 10 GUI's are only intermittently usable whenver QT upgrades, or GTK upgrades, etc. So the only totally reliable way to do video is through a CLI, eg ffmpeg or melt. It's a complete pain in the ass, of course, so I use GUI's when they're actually running. Ex: Pitivi (and Shotcut subsequently - QT5), Flowblade, or Olive, are OK for crossfades -- not perfect but OK. One month, GTK got an upgrade, and thus we learn that GTK is not backwards compatible... poof! PiTiVi gone...
$ pitivi
ERROR - The following hard dependencies are unmet:
==================================================
- gtk not found on the system
# pacman -S gtk
error: target not found: gtk
# pacman -S gtk2
warning: gtk2-2.24.32-2 is up to date -- reinstalling
... WTF?!! So there's little choice but to go CLI for one's mental health -- it's our lives we're giving up to edit.
You can pick any GUI application. Like, there's final edit color balancing in Blender (Blender is so complex it's practically an OS itself), but even color balancing can be done inside ffmpeg (see 0612 TV w/NERDfirst around 6:00).
Besides impossible crossfades, the challenge within ffmpeg is keeping the number of renders to a minimum. We can use ffplay to preview or, since ffmpeg is non-destructive (non-linear), we can do a test render and check it. And don't forget ffprobe to get information out of the files. Also recall that, if we just add an audio track, we can copy the video codec and add audio without degrading the video whatsoever.
other options
And of course, one should also not overlook the MLT framework, eg "melt", for command line transitions, if it's still being developed, not sure. In 2020, there's bleeding-edge on the AUR (mlt-git), and in "extra" (mlt). With MLT you will need a frame counting tool, not just a second counting tool to make edits.
MLT Melt Transitions (13:48) Kris Occhipinti, 2012.
See graphics below. Each edit requires several files (see my typical file structure)The grouping below is typical but of course can't include hand-drawn items: 1) storyboard 2)timing notes for audio sync with narration and so forth. How to pause and linger on a frame? The best storyboard may be a video describing the drawings while doing a UMlet workflow.
If you imagine the workflow graphic above as 3-D, one can see that each item is not equal. In the 3rd column, there's a place to get started on code and that's because the most flexible TTS is via Python, but Python itself is a deep subject requiring thoughtful installation. So just to make the TTS possible is likely a 2 week project.
blur (+1 render)
This is a complex ffmpeg filter. Three processes are accomplished, a box region is cropped out, it's blurred, and it's re-overlaid onto the original video. Added difficulty is determining an x:y dimension for the region.
$ ffmpeg -i foo-in.mp4 -filter_complex \
"[0:V]crop=100:100, boxblur=20[fg]; \
[0:V][fg]overlay=(main_w-200):30" foo-out.mp4
Ffmpeg blur box (6:45) Cool IT Help, 2019. A bit of a strong accent, but no BS.
The +1 render is when getting one's clips ready: rotating, accomplishing any dissolve edits ahead of time. There's no render during the actual concatenation. Try different orderings of your clips in a simple M3U, playing the M3U with vlc or xplayer. You can keep filenames simple, avoiding full absolute paths, if you place the M3U in the same directory with the clips. The format...
# This is a comment
somevideo.mp4
anothervideo.mp4
- finalize the mix order in the m3u
- open vlc and be sure "repeat" is disabled (else it will go into infinite loop). close vlc.
$ vlc playlist.m3u --sout "#gather:std{access=file,dst=vlcmerged.mp4}" --sout-keep
...in 5-6 seconds, this command concatenated 31 clips into a 7 minute 720P h264 video with sound, no tearing, and so on. It's production quality. The process is
described here, albeit with
Windows conventions.
Using ffmpeg can also be without rendering (-c copy), but will likely lead to DTS timestamp errors and tearing in playback. Even if one renders, the quality has not yet been as high as the vlc method above. Additionally, it doesn't work with an M3U. One has to make a text file with a special syntax on each line (viz, file 'foo.mp4') and one file per line, repeats also OK however.
$ ffmpeg -f concat -i playlist.txt -c:v libx264 -an grouped.mp4
I find that playback with the resulting file is often jerky or paused, even when all files are the same encoder and there is
discussion (render, tbr) about how to manage this, but I've never found anything to prevent it. If I had time, I would troubleshoot until a fix.
Not sure if we could chain filters and do this with a single render
$ ffmpeg -i foo.mp4 -vf "colorbalance=rs=-0.25,colorbalance=bm=-0.25" evencolor.mp4
How to use FFMpeg - Advanced Pt1 (19:37) 0612 TV w/NERDfirst, 2015. Color balance at the 7:30 mark.
Another great site, and he also has a video attached.
Typically, crossfades (aka "dissolves") are the most complex thing in Linux video editing and a separate post will likely be forthcoming. Some progress has ocurred however. This recent article is a must-read on the subject.
$ ffmpeg -i invid1.mp4 -i invid2.mp4 -filter_complex xfade=transition=fade:duration=2:offset=24 outputvid.mp4
... where the duration of the crossfade is 2 seconds, and it begins at the 24 second mark. Note that this effect renders with extremely high (hot) CPU cycling, even for a short clip.
Pan and Zoom Slideshow (26:50) Chipper Videos, 2019. Blender 2.8. Crossfade is shown in approximately last 5 minutes, but acccurate. Most is concerned with Ken Burns.
Crossfade (0:20) doufuwang, 2016. Shows the crossfade or dissolve effect perfectly.
Tech Notes: FFmpeg Multi Fades In Out (37:09) Steve AB4EL, 2017. Hilarious. Drones on and on. Takes 3 pictures and makes slide show. (8:30)fade transitions
FFmpeg Cross fades (2:25) A Forum, 2020. One of the best out there. Command given above is taken from here.
de-interlace (+1 render)
De-interlacing increases the frame rate. It could take you from 30 to 60, for example.
Sample Video Editing Workflow using FFmpeg (19:33) Rick Makes, 2019. Covers de-interlacing (2:00) to get rid of lines,cropping, audio library(13:30), and so on.
$ ffmpeg -i foo.mp4 -vf "bwdif=1" -c:a copy defoo.mp4
- frame rate vs. fps: these appear identical, and its unfortunate. It's nuanced like iframe and keyframe. Framerate (-r) is an input parameter used for setting fps. Fps is the stream speed. Ex: I want a 6 second video of 3 photos - I set fps to 2, and framerate to 12.
- ripple-cut: cut per usual, but then it magnets over to previous clip to fill the space
scripts
There are two types we might want to consider
- bash: easiest for ffmpeg sequence, not sure for tts
- python: using pip we can get some tts modules, send to WAV
We want to make bash scripts for the ffmpeg actions.
If I want to add sound effects or narrate, how do we at once watch the film and secondly enter sounds? The best way is to watch the film and take timing notes. Perhaps you want to pause on a frame for a few seconds while you discuss something.While you're watching the soundless video, just note how long you hold down the pause key to do your talking, say 5 seconds, and then using a loop filter
This one
just for video.
$ ffmpeg -i foo -vf reverse reversed.mp4
Few do it better than NERDfirst. The reason for the division is ffmpeg determines the angle in radians. Furthermore the angle of rotation is clockwise. The most common is righting an upside down video.
$ ffmpeg -i foo.mp4 -filter:v "rotate=PI" fout.mp4
How to use FFMpeg (12:48) 0612 TV w/NERDfirst, 2015. At the 11:00 mark, he describes rotation, and scaling just before it.
The simplest slideshow just cuts from one to the next at a set time, say 10 seconds. This is pretty easy to do on an entire directory of numbered pix, say foo01.png, foo02.png, foo03.png, etc. The "2d" indicates how many digits in the numbering system.
$ ffmpeg -framerate 1/10 -i foo%02d.png -c:v libx264 -r 30 -pix_fmt yuv420p slideshow.mp4
Similar to the Bash above is a slideshow. Run a batch file on a set of pictures and to make short 5 second video clips of them all, first render. At that point, we can do Ken Burns on some videos, and on others just overlap one to the next, the second render. Here, I have 3 x 5 second clips, and first accomplished $ yay ffmeg-concat so that I didn't have to use the complicated overlay filter.
$ shotcut
This fades up and down from black for each one and is not a dissolve between them. For those using
Windows, a caret "^" breaks-up the command in a batch instead of a backslash.
$ffmpeg -i foo1.mp4 -i foo2.mp4 \
-i foo3.mp4 -filter_complex \
"fade=in:st=0:d=1,fade=out:st=4:d=1[f0]; \
fade=in:st=0:d=1,fade=out:st=4:d=1[f1]; \
fade=in:st=0:d=1,fade=out:st=4:d=1[f2]; \
[f0][f1][f2]concat=n=3[123]" \
-map [f123] foocombine.mp4
Pan and Zoom Slideshow (26:50) Chipper Videos, 2019. Blender 2.8. Most is concerned with Ken Burns, but crossfade at the end.
Basic Slideshow (8:09) Luke Smith, 2017. Easy to put all of these together, but nothing here about ken burns
Tech Notes: FFmpeg Multi Fades In Out (37:09) Steve AB4EL, 2017. Hilarious. Drones on and on. Takes 3 pictures and makes slide show. (8:30)fade transitions
Accomplished with the "overlay" filter, and also possible in a batch (2019, Cool IT Help, seek to 5:00min), the bash equivalent being
#!/bin/bash
for file in *.mp4 do
ffmpeg -i "$file" -i watermark.jpg -filter_complex "overlay = 20:20" "${file%.*}_wm".mp4
done
Keyframing is a large topic, thus a separate post will be developed, but why is this a challenging topic?
- keyframes are not physically part of a video. They are placemarks temporarily created and indexed inside whatever video application is being used to edit the video. Being application-specific, they must be learned for each application
- Nomenclature issues abound. Keyframes and I-frames are similar and related, so that some use them interchangeably. There is the further confusion that iframes in HTML are different from iframes in video editing, but are again, related.
- ffmpeg refers to keyframes as "GOP" or Group of Pictures. It's set with the "-g" switch but requires re-encoding.
- keyframes are used differently in animation and video, so that learning basic video keyframes means sifting through many Google results for animation keyframing, ambiguously labeled. This is so common that it's hard to avoid (wastefully) learning animation usage while attempting to learn video usage.
- Blender, which is so vast and complex that it's like learning another programming language even before one considers keyframes, is probably the only GUI for which it's worth taking the time to learn keyframes. This means learning Blender just to get to the keyframe level.
Keyframes in Blender 2.8 (7:40) Blender, 2019. Animation-centric video, but significant application to video, since keyframes apply to both. 1:30 Keyframes can be managed in the Dopesheet, Graph editor, and Timeline.
Keyframes (and other) in 2.8 (9:48) Code, Tech, and Tutorials, 2019. Solid inadvertent tour around the VSE via a simple edit. Shows how to do transitions simply without inserting them, pressing "i" for keyframe.
Definitely an important concept for texting, however a person must go online to get this done: an unknown site has your vids and any related IP. On one's own machine, you'd think it would be quick, as 3rd party apps such as gifify appear to make it so, but these almost never work. This means we're dealing with f**king seeking syntax once again
Beginning at the 2:30 mark, create 10 seconds of sequential PNGs using a 2 digit format. I resized them down to a 640x360 size, by just determining the clip native resolution with ffprobe. Eg, I resize GoPro footage is 1280x720 natively, so I drop it to 640x360 or 320x180. Zoom is natively 640x360, so I leave it, or resize it to 320x180.
ffmpeg -ss 00:02:30 -i foo.mp4 -t 0:10 -s 240x135 %02d.png
Concatenate the PNG's (or use JPG's) into a GIF
ffmpeg -i %02d.png output.gif
These tend to be made at 25 fps, so I slow it down sometimes for slo-mo, eg 12 frames per second...
ffmpeg -framerate 12/1 -i %02d.png output.gif
