[FFmpeg-devel] [PATCH] ffmpeg: redesign video resampling in case of mid-stream video size/format change

[Stefano Sabatini]

On date Tuesday 2011-04-26 18:42:07 +0200, Michael Niedermayer encoded:
> On Tue, Apr 26, 2011 at 12:17:39AM +0200, Stefano Sabatini wrote:
> > On date Monday 2011-04-25 00:26:05 +0200, Stefano Sabatini encoded:
> > [...]
> > > New approach, this is the neatest design I can find without
> > > introducing filterchain re-configuration. It also fixes some problems
> > > in the non-libavfilter code path.
> > > -- 
> > > FFmpeg = Formidable Fancy Majestic Prodigious Eager Game
> > 
> > > From b3f4b7091122ca341b3a10b4f6edbbcb7183af2f Mon Sep 17 00:00:00 2001
> > > From: Stefano Sabatini <stefano.sabatini-lala at poste.it>
> > > Date: Sat, 16 Apr 2011 23:14:44 +0200
> > > Subject: [PATCH] ffmpeg: redesign video resampling in case of mid-stream video size/format change
> > > 
> > > The new design implements a new video rescaling API and supports video
> > > resolution/format change with and without AVFILTER_CONFIG, thus
> > > avoiding the need for the av_vsrc_buffer_add_frame2() hack.
> > > 
> > > If libavfilter is not enabled, resampling is performed in
> > > do_video_out(), before to send the decoded frame to the output. In
> > > this case the resampling API will also perform the scaling to the
> > > output video size/format.
> > > 
> > > If libavfilter is enabled, resampling is performed in
> > > pre_process_video_frame(), before the video is sent to the
> > > filterchain. This way we ensure that the filterchain is feeded with
> > > frames of the same size/format, thus avoiding the need of dynamic
> > > filterchain reconfiguration.
> > > 
> > > Fix issues:
> > > roundup #2217
> > > trac #59
> > > trac #64
> > > trac #74
> > 
> > Ping.
> 
> ive fixed several small bugs in av_vsrc_buffer_add_frame2() and it
> seems that fixed these issues.
> It should be easy to add interlaced scaling support and the code
> can reuse an existing scale filter instead of needing an additional
> filter.

Why I don't like av_vsrc_buffer_add_frame2():

* add a dependency on the scale filter

* the insertion of the filter in an already set-up filterchain is
  brittle (and doing several undocumented assumptions)

* it is an ad-hoc fix, indeed the same hack should be applied to other
  sources (e.g. movie, ffplay/input_filter and so on)

* other minor issues (not properly tested, never published for review,
  sloppy style and message reporting)

The proper fix consists in dynamic mid-stream reconfiguration, which
should fix the problem also for ffplay (which now crashes/misbehaves
in that case). The nice thing with the video resampling context is
that it can be applied to ffplay as well, and fixes the problem at the
application level while a framework solution is developed (and fixes
some of the issues with the non-libavfilter path).

So I propose to drop av_vsrc_buffer_add_frame2() in favor of this
solution, and fix properly the issue by extending the framework (which
I want to do, just after the unnecessary memcpy problem is fixed).

My plan for vsrc_buffer.c: remove dependency on AVFrame, document its
limitations (same input size/format assumption), and eventually
replace it in ffplay/ffmpeg by a more integrated vsrc_lavc_buffer
(which should allow direct rendering as implemented in ffplay.c).

vsrc_buffer would be still useful as a simple pure-libavfilter
solution (in case you don't want to depend on lavc).

> I dont understand how your solution that stores the resampling with
> input streams can work without the av_vsrc_buffer_add_frame2()
> when there are more output streams than input streams.
> The first output might want 100x100 the second 400x400, theres only
> one context so it can only scale to one.

Rescaling for the output streams is done in the libavfilter chain (a
scale filter is added to the end of the filterchain, one for each
output video stream), the rescale/reformat context is used only to
normalize the input video.
-- 
FFmpeg = Faithful Frightening Mastodontic Programmable Emblematic Guide