[Libav-user] H264 Packed RGB Decoding

[Devin Heitmueller]

On Thu, Feb 6, 2020 at 8:25 PM Alan Daniels <alan at softdrive.co> wrote:
> Is there a way for FFmpeg to decode an H264 bitstream into packed RGB?
> Ideally AV_PIX_FMT_BGR0 or AV_PIX_FMT_BGRA.
>
> For the encoder, I am using libx264rgb with AV_PIX_FMT_BGR0.
>
> For the decoder, I am using AV_CODEC_ID_H264. However, the frame comes back from the decoder as AV_PIX_FMT_GBRP. This seems to be the only option supplied to get_format().

The output of the decoder is whatever format the video was originally
encoded with.  Decoders generally don't do any sort of colorspace
conversion.  In this case the output of the decoder is a planar
version of RGB (since codecs typically operate on packed formats).
Presumably if you really did feed the encoder BGR0 (which is a packed
format), something converted it to planar format prior to encoding.
The ffmpeg program will automatically insert swscale into the filter
pipeline to convert packed to planar as would have been required by
the encoder.

> I am trying to minimize encode/decode latency, so I am trying to avoid using swscale after decoding.

It's probably unavoidable - if you need a packed format for your
output then you're going to have to convert planar to packed.  On the
upside, swscale isn't something that is typically a large source of
latency.  Hardware inputs/outputs tend to use packed formats (e.g.
cameras, SDI cards, etc), while codecs tend to use planar formats, and
yet people are still able to create very low latency solutions.

There are plenty of other places within your pipeline which would
likely be larger sources of latency (such as the numerous FIFOs
internal to ffmpeg).

Devin

-- 
Devin J. Heitmueller - Kernel Labs
http://www.kernellabs.com