[FFmpeg-devel] Frame duplication or pulldown feature support in FFMPEG for HEVC

[Praveen Kumar]

Hi all,

I have been trying to verify using x265, the support for progressive frames' duplication or frame doubling similar to "--pulldown" feature present in x264.

How it happens in x264:
Using x264 I could generate an elementary bit-stream that supports frame doubling in the framework by using the parameter "--pulldown double".  This eventually sets the "pic_struct" value of SEI to 7, which indicates frame-doubling. When I decode this stream using ffmpeg, I get double the number of YUV frames as that of the input. This doesn't happen with standalone decoders like the JM decoder ldecod.exe. Which means the pulldown feature is handled only at the framework/ player level and not at the decoder level.

FFMPEG decode gives ~2x num of YUV frames, JM decoder gives just 1x

On further probing the FFMPEG code for this, I observed that making pic_struct as 7, in turn makes the "repeat_pict" as 2 in ffmpeg's "AVFrame" class, where an "extra_delay" is calculated for each frame as: extra_delay = repeat_pict / (2*fps)

Command line used to verify this in x264:
x264.exe CIF_5frames_1fps.y4m --bframes 0 --fps 1 -o single.264
x264.exe CIF_5frames_1fps.y4m --bframes 0 --pulldown double -o double.264

"ffmpeg -i single.264" shows: Stream #0:0: Video: h264 (High), yuv420p(progressive), 352x288, 1 fps, 1 tbr, 1200k tbn, 2 tbc
"ffmpeg -i double.264 shows: Stream #0:0: Video: h264 (High), yuv420p(progressive), 352x288, 1.67 fps, 2 tbr, 1200k tbn, 4 tbc

Decoding using ffmpeg:
ffmpeg -i single.264 -c:v rawvideo single.264.yuv  (Gives 5 frames)
ffmpeg -i double.264 -c:v rawvideo double.264.yuv  (Gives 8 frames)

My observations with x265:
Since there is no "--pulldown" cli parameter in x265 yet, I directly modified the SEI  param->pictureStructure to 7 in x265.cpp and doubled the time_scale as well. I tried this build on a 50 fps video and observed that this is giving me an hevc stream where I could get fps, tbr an tbc doubled when I use "ffmpeg -i" on it. But the stream doesn't give double the YUV frames w​hen decoded using ffmpeg as it did in case of H.264.

"ffmpeg -i single.hevc" shows: Stream #0:0: Video: hevc (Main), yuv420p(tv), 1920x1080, 50 fps, 50 tbr, 1200k tbn, 50 tbc
"ffmpeg -i double.hevc" shows: Stream #0:0: Video: hevc (Main), yuv420p(tv), 1920x1080, 100 fps, 100 tbr, 1200k tbn, 100 tbc

Decoding using ffmpeg:
ffmpeg -i single.hevc-c:v rawvideo single.hevc.yuv  (Gives 500 frames)
ffmpeg -i double.hevc -c:v rawvideo double.hevc.yuv  (Gives 500 frames)

This means despite making pictureStructure 7 (confirmed from the analyzer output below), ffmpeg is unable to duplicate the frames in case of HEVC. This is because of the only difference that the value of "repeat_pict" is somehow not getting set to 2 in HEVC unlike in H.264 (confirmed from ffprobe scan below).
[cid:576ab9cd-231f-4d39-b389-34a87505538b]

So, as per my understanding there is no implementation to handle the pulldown feature ie param->pictureStructure=7 for HEVC frame doubling FFMPEG or any other framework that we know of.
Could you please confirm this and let us know if there is a plan to implement handling of HEVC pulldown feature in FFMPEG?

Thanks & Regards,
Praveen Karadugattu
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Untitled.jpg
Type: image/jpeg
Size: 124835 bytes
Desc: Untitled.jpg
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20190618/c937a96b/attachment.jpg>