[FFmpeg-devel] [PATCH 3/3] avformat/movenc: Add support for AVIF muxing

Thu Mar 3 21:57:45 EET 2022

On Thu, Mar 3, 2022 at 11:46 AM James Almer <jamrial at gmail.com> wrote:
>
>
>
> On 3/3/2022 4:20 PM, Vignesh Venkatasubramanian wrote:
> > On Thu, Mar 3, 2022 at 7:36 AM James Almer <jamrial at gmail.com> wrote:
> >>
> >> On 2/22/2022 6:43 PM, Vignesh Venkatasubramanian wrote:
> >>> Add an AVIF muxer by re-using the existing the mov/mp4 muxer.
> >>>
> >>> AVIF Specifiation: https://aomediacodec.github.io/av1-avif
> >>>
> >>> Sample usage for still image:
> >>> ffmpeg -i image.png -c:v libaom-av1 -avif-image 1 image.avif
> >>>
> >>> Sample usage for animated AVIF image:
> >>> ffmpeg -i video.mp4 animated.avif
> >>>
> >>> We can re-use any of the AV1 encoding options that will make
> >>> sense for image encoding (like bitrate, tiles, encoding speed,
> >>> etc).
> >>>
> >>> The files generated by this muxer has been verified to be valid
> >>> AVIF files by the following:
> >>> 1) Displays on Chrome (both still and animated images).
> >>> 2) Displays on Firefox (only still images, firefox does not support
> >>>      animated AVIF yet).
> >>> 3) Verfied to be valid by Compliance Warden:
> >>>      https://github.com/gpac/ComplianceWarden
> >>>
> >>> Fixes the encoder/muxer part of Trac Ticket #7621
> >>>
> >>> Signed-off-by: Vignesh Venkatasubramanian <vigneshv at google.com>
> >>> ---
> >>>    configure                |   1 +
> >>>    libavformat/allformats.c |   1 +
> >>>    libavformat/movenc.c     | 300 +++++++++++++++++++++++++++++++++++----
> >>>    libavformat/movenc.h     |   5 +
> >>>    4 files changed, 282 insertions(+), 25 deletions(-)
> >>
> >> With a single frame i get no errors in that compliance tool, but when i
> >> encode an animated AVIF i get the following:
> >>
> >> [heif][Rule #12] Error: CodingConstraintsBox ('ccst') shall be present once
> >> [heif][Rule #28] Error: Wrong arity for boxes { ccst } in parents { avc1
> >> avc2 avc3 avc4 hev1 hev2 hvc1 hvc2 av01 }: expected in range [1-1], found 0
> >> [heif][Rule #31] Error: 'msf1' brand: this file shall conform to HEIF
> >> (section 7.2)
> >> [heif][Rule #31] Error: 'msf1' brand: 'iso8' shall be present among the
> >> compatible brands array
> >> [heif][Rule #32] Error: 'mif1' brand: this file shall conform to HEIF
> >> section 6, check the other errors for details
> >> [heif][Rule #33] Error: 'msf1' brand: this file shall conform to HEIF
> >> section 7, check the other errors for details
> >>
> >> All but one of these should be solved by writing a ccst box after the
> >> av1C box in the sample entry. The missing one should be solved by
> >> writing the iso8 compatible brand.
> >>
> >> The ccst box looks like it would need some bitstream information, so
> >> either you parse the packets to get it, or just hardcode sane defaults,
> >> considering it's used as a hint and it's not required for demuxing.
> >
> > I recently fixed these errors in libavif [1][2] (the reference AVIF
> > encoder). I was hoping to have a follow-up patch since i already
> > uploaded the existing patches. Since you have pointed this out now, i
> > have included the fix in this patch itself. The new patch will now
> > write the iso8 compatible brand and the ccst box with sane default
> > values. The file produced is now identical to the file produced by
> > libavif in terms of box structure.
> >
> > Also, notice that the compliance tool still shows the following error
> > for animated avif:
> >
> > [avif][Rule #3] Warning: [ItemId=1] still_picture flag set to 0
> > [avif][Rule #4] Warning: [ItemId=1] reduced_still_picture_header flag set to 0
> >
> > I believe these are incorrect since it does not make sense to set
> > these flag to 0 for animated avif sequences. These warnings also show
> > up with files produced by libavif. So it is okay to ignore them. I
> > will file an issue with the compliance tool separately.
>
> The compliance tool uses the 1.0.0 revision of the spec, and what you
> mentioned was removed in the current unfinished draft:
> https://github.com/AOMediaCodec/av1-avif/pull/112
>
> I assume the tool will be updated once a new revision is released, so
> yes, we can ignore them.
>
> >
> > Please take another look, thanks!
>
> I noticed that using -still-picture 1 and passing more than one frame to
> the libaom encoder will succeed, despite you setting enccfg.g_limit to
> 1, and encode every frame as key frames.
> I'd expect the encoder would error out if you try to feed it more
> frames. Is it a libaom bug?
>

Hmm yeah it seems like libaom only uses the value to set the
still-picture sequence header values in 1-pass mode. I think in a way
it may be useful for us to be able to use AVIF output with the image2
muxer. For example, something like:

ffmpeg -i video.mp4 -still-picture 1 -c:v libaom-av1 -an -f image2
image-%02d.avif

This command does not work as of now, but I have some follow-up
patches to make the image2 muxer work with AVIF images.

> >
> > [1] https://github.com/AOMediaCodec/libavif/pull/855
> > [2] https://github.com/AOMediaCodec/libavif/pull/856
> >> _______________________________________________
> >> ffmpeg-devel mailing list
> >> ffmpeg-devel at ffmpeg.org
> >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >>
> >> To unsubscribe, visit link above, or email
> >> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
> >
> >
> >
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".

-- 
Vignesh