[FFmpeg-devel] Frame doubling/ duplication/ pulldown support in HEVC

[Praveen Kumar]

Hi all,

I have been trying to verify using x265, the support for progressive frames' duplication or frame doubling similar to "--pulldown" feature present in x264.

How it happens in x264:
Using x264 I could generate an elementary bit-stream that supports frame doubling in the framework by using the parameter "--pulldown double". In H.264, "--pulldown double" eventually sets the "pic_struct" value in SEI to 7, which means frame-doubling. The feature doubles the VUI's time_scale info, bitstream's fps, tbr and tbc of the encoded stream when scanned through "ffmpeg -i" option. Similarly when I decode this stream using ffmpeg, I get double the number of YUV frames that of the input YUV frames. This doesn't happen with standalone decoders like the JM decoder ldecod.exe. Which means the pulldown feature is handled only at the framework/ player level and not at the decoder level.

FFMPEG decode gives ~2x num of YUV frames, JM decoder gives just 1x

On further probing the FFMPEG code for this, I observed that making pic_struct as 7, in turn makes the "repeat_pict" as 2 in ffmpeg's "AVFrame" class, where an "extra_delay" is calculated for each frame as: extra_delay = repeat_pict / (2*fps)

Command line used to verify this in x264:
x264.exe CIF_5frames_1fps.y4m --bframes 0 --fps 1 -o single.264
x264.exe CIF_5frames_1fps.y4m --bframes 0 --pulldown double -o double.264

"ffmpeg -i single.264" shows: Stream #0:0: Video: h264 (High), yuv420p(progressive), 352x288, 1 fps, 1 tbr, 1200k tbn, 2 tbc
"ffmpeg -i double.264 shows: Stream #0:0: Video: h264 (High), yuv420p(progressive), 352x288, 1.67 fps, 2 tbr, 1200k tbn, 4 tbc
Decoding using ffmpeg:
ffmpeg -i single.264 -c:v rawvideo single.264.yuv  (Gives 5 frames)
ffmpeg -i double.264 -c:v rawvideo double.264.yuv  (Gives 8 frames)

My observations with x265:
Since there is no "--pulldown" cli parameter in x265 yet, I directly modified the SEI  param->pictureStructure to 7 in x265.cpp and doubled the time_scale as well. I tried this build on a 50 fps video and observed that this is giving me an hevc stream where I could get fps, tbr an tbc doubled when I use "ffmpeg -i" on it. But the stream doesn't give double the YUV frames w​hen decoded using ffmpeg as it did in case of H.264.

"ffmpeg -i single.hevc" shows: Stream #0:0: Video: hevc (Main), yuv420p(tv), 1920x1080, 50 fps, 50 tbr, 1200k tbn, 50 tbc
"ffmpeg -i double.hevc" shows: Stream #0:0: Video: hevc (Main), yuv420p(tv), 1920x1080, 100 fps, 100 tbr, 1200k tbn, 100 tbc
Decoding using ffmpeg:
ffmpeg -i single.hevc-c:v rawvideo single.hevc.yuv  (Gives 500 frames)
ffmpeg -i double.hevc -c:v rawvideo double.hevc.yuv  (Gives 500 frames)

This means despite making pictureStructure 7 (confirmed from the analyzer output below), ffmpeg is unable to duplicate the frames in case of HEVC. This is because of the only difference that the value of "repeat_pict" is somehow not getting set to 2 in HEVC unlike in H.264 (confirmed from ffprobe scan below).

[cid:9982a4a9-32bd-498e-bf3d-62978de60f4f]


So as per my understanding there is no implementation to handle the pulldown feature ie param->pictureStructure=7 for HEVC frame doubling FFMPEG or any other framework that we know of. Could you please confirm this and let us know if there is a plan to implement this in FFMPEG?

Thanks & Regards,
Praveen Karadugattu
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Outlook-m0fr5hjg.png
Type: image/png
Size: 310776 bytes
Desc: Outlook-m0fr5hjg.png
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20190617/f8387205/attachment.png>