[Ffmpeg-devel] Some more thoughts about libavfilter

[Luca Abeni]

Hi all,

as promised, I am re-reading my old libavfilter ideas, trying to
formalize them. Here are some other (more or less confused) thoughts,
hoping they can be useful to someone :)
In some previous emails, I already wrote about the get_buffer(),
push_buffer(), and release_buffer() methods that I think are needed to
implement a filter in a "push model", and I wrote how the buffers for
allocating frames are allocated.

I already wrote that the methods cited above get an AVFrame as a
parameter, and after some other thinking I am convinced that AVFrames
are the correct entities to pass between filters. I believe they contain
all the data needed to fullfill the requirements highlighted by Michael.
Is this ok?

The function provided by libavcodec to allocate AVFrame buffers
(avcodec_default_get_buffer()) seems to be strictly dependent on
AVCodecContext, so I think libavfilter has to provide its own allocation
function for the filters. I am thinking about something like
int av_frame_alloc_buffer(AVFrame *f, enum PixelFormat pix_fmt, int w, int h)
wich in the simplest case could do something like
    avpicture_alloc((AVPicture *)f, pix_fmt, w, h);
    f->base[0] = f->data[0];
    f->base[1] = NULL;
    f->base[2] = NULL;
(note that this is not calling avcodec_get_frame_defaults(), because the
caller is responsible for initializing all the other AVframe fields (it
might copy them from the input frame, etc...).

Then, avfilter should provide helper functions for copying frames, or
buffers from a frame to another, or buffers from an AVPicture to an
AVFrame, etc.

Now, regarding the problem of freeing an AVFrame buffer, I have two
possible ideas:
1) reference counting (as already cited by Alexander Chemeris). We need
a counter (does AVFrame provide a field that can be used as a reference
conter? After a first look, I think it does not) that is increased by a
filter (maybe through an helper function) when the filter starts using
the buffer and is decreased when the filter does not need the buffer
anymore. When the counter is 0, the buffer is actually freed / recycled.

2) The "last" filter who use the AVFrame buffer is responsible for
freeing it. Basically (assuming a "push model")
  - every filter has to provide a release_buffer() method;
  - when a filter F who pushed an AVFrame to its output Fout does
    not need the AVFrame anymore it calls  Fout->release_buffer();
  - if F provided an AVFrame buffer to an input filter Fin (which 
    invoked F->get_buffer()) the frame is considered "needed by F" 
    until F->release_buffer() is called on it
  - if F did not push an AVFrame buffer to Fout (for example, consider,
    the input frame of a rescaling filter), when F->release_buffer() is
    invoked on such frame F can free the buffer directly

This looks complex, but in practice it is simpler than it seems (maybe I
am just bad at describing it :)...

Simple example, using a scaling filter F:
- when F->get_buffer() is called, F allocates the frame buffer with
av_frame_alloc_buffer()
- when F->push_buffer() is called, F
	1) calls Fout->get_frame() to get an output frame
	2) rescales the input frame in the output frame
	3) calls Fout->release_buffer() on the output frame
- when F->release_buffer() is called, F frees the input frame

Another example, using a pad filter:
- when F->get_buffer() is called, F calls Fout->get_buffer(),
manipulates the data[] fields of the AVFrame as needed for padding, and
returns the AVFrame
- when F->push_buffer() is called, F fills the "padding borders",
manipulates the data[] fields of the AVFrame (restoring the initial
values), and calls Fout->push_buffer()
- when F->release_buffer() is called, F directly calls
Fout->release_buffer()


I also think libavfilter can provide some helper functions that simplify
this "AVFrame buffer freeing" code (I am thinking about a possible
API... I'll probably write more about this in the next days).


				Luca