keyframes

Links: keyframe :: (wikipedia) keyframe is an "I" frame -- intraframe. types of frames :: (wikipedia) I, P, & B frames.

These blog posts are application oriented. Sometimes however, a little theory helps the application. Simply stated,keyframes are a type of i-frame manually added by a user editing a video, increasing the total i-frame count of a video.

i-frames

Video cameras record a chronological procession of pictures, frame by frame. They do so with 3 types of frames: i, p, and b. The i-frames are complete pictures, like a photo, of what is being recorded. As the camera records, it takes an i-frame and then several P or B frames, and then another i-frame, and so on. The P or B frames refer to an associated I-frame to complete the picture during playback. The Wikipedia graphic below showes the sequence. The i,p, b-frame schema was created to save memory space.

Most video devices insert i-frames about every 8 seconds or every 240 frames or so (I shoot mostly 30fps) when recording video. Newer codecs set these intervals dynamically: shorter intervals when action increases, and longer intervals when action decreases. H264 comes to mind.

keyframes

When software users edit video effects, say a dissolve transition, their editing software adds an i-frame to the beginning and end of the effect. These post-recording, user-added i-frames are in addition to the existing i-frames embedded by their camera during recording. Only these post-recording, user inserted i-frames are properly termed "keyframes".

Further nomenclature confusion can arise when software uses proprietary terms for key or i-frame intervals. For example, in ffmpeg, i-frame intervals are called GOP's "Groups of Pictures", and without regard to whether they are between key or i-frames.

raw video analysis

When I import uncut clips, I first-off detect and save all the i-frame time stamps to a file I can refer to when editing. If it's a simple edit without transition, and all my cuts are at i-frames, I might not need to add keyframes and re-encode the video. How do I get a list of i-frame time stamps? Suppose I have a clip, "foo.mp4".

$ ffprobe -loglevel error -skip_frame nokey -select_streams v:0 -show_entries frame=pkt_pts_time -of csv=print_section=0 foo.mp4 >iframes_foo.txt 2>&1

The output intervals will be timestamps, which we can easily see by catting the file.

$ cat iframes_foo.txt
0.000000
4.771433
10.410400
18.752067
27.093733
...

To determine the exact frame number (always an integer) of the i-frame, multiply the time stamp by the recording FPS. I can determine the FPS using a basic $ ffprobe foo.mp4. In this example, it revealed a 29.97 FPS. So...

29.97 x 4.771 = 142.99 or frame 143.
29.97 x 10.4104 = 311.99 or frame 312.
29.97 x 18.7521 = 561.99 or frame 562.

...and so on. We could write a short program to calulate this or export it into Excel/Gnumeric/Sheets. But this is for only a single clip and of course I want this information for each of my clips.

repair - ffmpeg

Sometimes, the keyframes become unmanageable for some reason and need to be cleaned. Typically re-encoding is required. But there is a methodology.