[FFmpeg-user] Extracting frames from 59.94 FPS video - only every other frame is different

Fri Dec 1 19:12:38 EET 2017

Hello,

I'm having a strange problem when extracting frames from a video recorded from a GoPro HERO 5 Black camera (1080p, 60FPS, Linear FOV). When I view the extracted frames it appears that every two frames is a duplicate of one another, i.e. frames 1 and 2 look the same, frame 3 shows a change, but frame 4 again looks the same as 3.

I've used `compare` from ImageMagick to actually determine whether the frames are different, and they are, the diff looks like a frame itself but with high brightness. (see frames 1, 2, and the diff at this link: https://drive.google.com/open?id=1Tn3dbZ6Ss7OdhHh6OiJT4qsPXSqCVVgF). If you visually compare frames 1 and 2 then to the human eye they look impercetibly different, yet the diff shows that there is a difference. When I look at the frames in a photo viewer and iterate through them rapidly the apparent motion is much more jerky than in the source video.

If I stitch the frames back together to a video then I get a smooth 60FPS video just like the source.

The motivation for needing smooth non-duplicated frames is because they are to be used as input to an optical flow algorithm which then produces inaccurate optical flow estimations due to the similarity between pairs of frames starting with an odd frame number.

I dumped the frames from the video to a folder using the following command:

$ ffmpeg -i GOPR0325.MP4 'gopro/frame_%06d.jpg'

ffmpeg version 3.4 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 7.2.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig -
-enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-l
ibgsm --enable-libiec61883 --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg -
-enable-libopus --enable-libpulse --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --
enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxvid --enable-shared --enable-v
ersion3
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libavresample 3. 7. 0 / 3. 7. 0
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55c3a7e709c0] Using non-standard frame rate 59/1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'GOPR0325.MP4':
Metadata:
major_brand : mp41
minor_version : 538120216
compatible_brands: mp41
creation_time : 2017-12-01T15:00:48.000000Z
firmware : HD5.02.01.55.00
Duration: 00:00:08.41, start: 0.000000, bitrate: 30249 kb/s
Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 30110 kb/s, 59.94 fps, 59.94 t
br, 60k tbn, 119.88 tbc (default)
Metadata:
creation_time : 2017-12-01T15:00:48.000000Z
handler_name : GoPro AVC
encoder : GoPro AVC encoder
timecode : 15:16:04:22
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
creation_time : 2017-12-01T15:00:48.000000Z
handler_name : GoPro AAC
timecode : 15:16:04:22
Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
Metadata:
creation_time : 2017-12-01T15:00:48.000000Z
handler_name : GoPro TCD
timecode : 15:16:04:22
Stream #0:3(eng): Data: none (gpmd / 0x646D7067), 29 kb/s (default)
Metadata:
creation_time : 2017-12-01T15:00:48.000000Z
handler_name : GoPro MET
Stream #0:4(eng): Data: none (fdsc / 0x63736466), 14 kb/s (default)
Metadata:
creation_time : 2017-12-01T15:00:48.000000Z
handler_name : GoPro SOS
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
Output #0, image2, to 'gopro/frame_%06d.jpg':
Metadata:
major_brand : mp41
minor_version : 538120216
compatible_brands: mp41
firmware : HD5.02.01.55.00
encoder : Lavf57.83.100
Stream #0:0(eng): Video: mjpeg, yuvj420p(pc), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 59.94 fps, 59.94 tbn, 59.94 tbc (default)
Metadata:
creation_time : 2017-12-01T15:00:48.000000Z
handler_name : GoPro AVC
timecode : 15:16:04:22
encoder : Lavc57.107.100 mjpeg
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
frame= 503 fps= 48 q=24.8 Lsize=N/A time=00:00:08.39 bitrate=N/A speed= 0.8x
video:20242kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

$ ffprobe GOPR0325.MP4
ffprobe version 3.4 Copyright (c) 2007-2017 the FFmpeg developers
built with gcc 7.2.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enab
le-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264
--enable-libx265 --enable-libxcb --enable-libxvid --enable-shared --enable-version3
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libavresample 3. 7. 0 / 3. 7. 0
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55ecc7872200] Using non-standard frame rate 59/1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'GOPR0325.MP4':
Metadata:
major_brand : mp41
minor_version : 538120216
compatible_brands: mp41
creation_time : 2017-12-01T15:00:48.000000Z
firmware : HD5.02.01.55.00
Duration: 00:00:08.41, start: 0.000000, bitrate: 30249 kb/s
Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 30110 kb/s, 59.94 fps, 59.94 tbr, 60k tbn, 119.88 tbc (default)
Metadata:
creation_time : 2017-12-01T15:00:48.000000Z
handler_name : GoPro AVC
encoder : GoPro AVC encoder
timecode : 15:16:04:22
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
creation_time : 2017-12-01T15:00:48.000000Z
handler_name : GoPro AAC
timecode : 15:16:04:22
Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
Metadata:
creation_time : 2017-12-01T15:00:48.000000Z
handler_name : GoPro TCD
timecode : 15:16:04:22
Stream #0:3(eng): Data: none (gpmd / 0x646D7067), 29 kb/s (default)
Metadata:
creation_time : 2017-12-01T15:00:48.000000Z
handler_name : GoPro MET
Stream #0:4(eng): Data: none (fdsc / 0x63736466), 14 kb/s (default)
Metadata:
creation_time : 2017-12-01T15:00:48.000000Z
handler_name : GoPro SOS
Unsupported codec with id 0 for input stream 2
Unsupported codec with id 0 for input stream 3
Unsupported codec with id 0 for input stream 4

The source video, frames, and diff between frames 1 and 2 are available on google drive: https://drive.google.com/open?id=1Tn3dbZ6Ss7OdhHh6OiJT4qsPXSqCVVgF

Can anyone shed any light on this?

Many thanks,
Will Price
--
Will Price | PhD student | Visual Information Laboratory | University of Bristol