[FFmpeg-devel] [PATCH 0/6] Add support for utilizing av1c as extradata

[Jan Ekström]

On Mon, Nov 23, 2020 at 10:11 PM James Almer <jamrial at gmail.com> wrote:
>
> On 11/23/2020 4:10 PM, Jan Ekström wrote:
> > Until now an API user with a non-libavformat MP4 or Matroska demuxer
> > would have had to manually filter the extradata to skip the first four
> > bytes when passing data to libavcodec, so that only the OBUs would
> > remain. Now the AV1CodecConfigurationRecord structure can be passed
> > to libavcodec as-is. This is lets it to be dealt with in a similar
> > manner to AVC or HEVC.
> >
> > This is done in the following steps:
> > 1. A nice helper by Mark is added so that it can be utilized in stead
> >     of ff_cbs_read from both the decoder and parser after the CBS gains
> >     the capability to handle AV1CodecConfigurationRecord structures.
> > 2. Support is added to AV1 CBS for handling AV1CodecConfigurationRecord
> >     in a similar vein to AVC/HEVC's avcc/hvcc handling.
> > 3. AV1 decoder and parser are switched to utilize the newly added
> >     helper functionality.
> > 4. Support is added for writing out the passed AV1CodecConfigurationRecord
> >     in ff_isom_write_av1c, which is called from the MP4 and Matroska
> >     writers.
> > 5. Finally, both MP4 and Matroska demuxers are changed to expose the
> >     full AV1CodecConfigurationRecord as extradata.
> >
> > Jan Ekström (5):
> >    avcodec/cbs_av1: add support for standard MP4/Matroska extradata
> >    avcodec/av1{dec,parser}: move to ff_cbs_read_extradata_from_codec
> >    avformat/av1: add support for passing through MP4/Matroska av1c
> >    avformat/mov: remove special handling of av1c extradata
> >    avformat/matroskadec: remove special handling of av1c extradata
> >
> > Mark Thompson (1):
> >    cbs: Add function to read extradata from an AVCodecContext
> >
> >   libavcodec/av1_parser.c   |  2 +-
> >   libavcodec/av1dec.c       |  5 ++-
> >   libavcodec/cbs.c          | 77 ++++++++++++++++++++-------------------
> >   libavcodec/cbs.h          | 11 ++++++
> >   libavcodec/cbs_av1.c      | 34 +++++++++++++++++
> >   libavformat/av1.c         | 14 +++++++
> >   libavformat/matroskadec.c |  4 --
> >   libavformat/mov.c         | 30 +--------------
> >   8 files changed, 103 insertions(+), 74 deletions(-)
>
> Patches 1 to 4 LGTM. Patches 5 and 6 should IMO wait until after a bump
> since they change what the demuxers export as extradata, and existing
> compiled software linking to for ex 4.3 should not have to deal with
> this change in behavior if you use a lavf from an hypothetical pre-bump
> 4.4 release at runtime.

Thanks, applied 1-4 as:
7bacf60ae5df75954a538563d19f6001aa598b3f..4688017d9a1c2633c5888de5a0580d29e7680d54

I would prefer we switch over and simplify the demuxers sooner than
later, as I am not sure it was meant from our side that the interface
was defined as "only OBUs from this specific structure", while for
every other major format (AVC/HEVC) our demuxers output the whole
structure, and our decoders would accept that (even if it contained
some extraneous information such as the HEVC extradata).

But for now at least one side has been fixed.

Jan