[FFmpeg-devel] [RFC] Improving lavc decoding/encoding API

[Stefano Sabatini]

Hi all,

I had this in mind for some time, now it's time to expose it here.

Currently decoding/encoding API is inconsistent for which regards
audio decoding/encoding.

While with video frames and subtitles decoding we implement this interface:
buffer + buffer_size => AVPicture/AVSubtitle + got_it_flag

for audio decoding we have:
buffer + buffer_size => output buffer + size

Audio channels numbers, sample rate, sample format and possibly other
informations are stored in the context, while it would be more
convenient to directly store such information in the output structure
itself as in the other cases.

My idea is to support a new structure which we could name AVSamples
containing all the required info.

So we'll end up with this:
buffer + buffer_size => AVSamples + got_it_flag

which is consistent with the video/subtitle decoding interface.

Similarly we'll have this interface when encoding:

AVFrame/AVSamples/AVSubtitle => buffer + buffer_size

An even greater idea would be to use AVFrame for everything, which is
even in doc/TODO:
- use AVFrame for audio codecs too

and we'll end up with this interface:
decoding:
buffer + buffer size => AVFrame + got_it_flag

encoding: 
AVFrame => buffer + buffer size

We could put the relevant fields in AVFrame then use casts to get the
corresponding AVPicture/AVSamples/AVSubtitle:

AVPicture* picture = (AVPicture *) (frame_ptr + PICTURE_PTR_OFFSET);
AVSamples* samples = (AVPicture *) (frame_ptr + SAMPLES_PTR_OFFSET);
AVSubtitle* subtitle = (AVPicture *) (frame_ptr + SUBTITLE_PTR_OFFSET);

This should also make simpler to extend libavfilter to support A/V/S
conversion, making each filter to accept in input *generic frames*
rather than pictures.
 
I don't currently have any idea regarding how to make this backward
compatible, and I'm aware that it will require to rewrite a lot of
codecs code, anyway I think it would sensibly simplify lavc use and
increase its flexibility.

Regards.
-- 
FFmpeg = Friendly and Frenzy Multipurpose Portable Extreme Geisha