[FFmpeg-devel] [RFC] How to fix DR+lavfi+vflip crash

[Stefano Sabatini]

Hi,

as you may know the command:
ffplay INPUT -vf vflip

crashes with many video codecs. This is a regression introduced by the
direct rendering feature, since the codec request the frame to use for
putting the decoded frame, it gets a frame with negative linesizes and
crash

(BTW the smacker regression also seems to depend on diect
rendering, and precisely with the way the palette is initialized in
avfilter_default_get_buffer, which doesn't use ff_systematic_pal()).

A possible first step would be to define a CODEC_CAP_NEG_LINESIZES
capability (suggest a better name), and set it in all the codecs which
currently support this feature (I have no idea which of them, do
you?).

At this point I see two solutions. One solution would be to change
get_video_buffer(), and make it invert the buffer when it detects the
negative linesizes && the NEG_LINESIZES capability is not
supported, *or* auto-add another filter just before the ffplay source.

Such a filter (vflipfix - suggest better name) would work as a null
filter if the frame is not inverted, and would readjust the frame if
the linesizes are inverted.

The second solution seems simpler and cleaner.

I'm going to factorize the code of ffplay.c:get_video_buffer(), so
this solution will work also for ffmpeg.
-- 
FFmpeg = Fundamental & Fundamentalist Magical Pitiless Emblematic Gorilla