[FFmpeg-user] The internal ffmpeg rgb=>yuv pipeline and conversion matrices

[Paul B Mahol]

On 5/6/16, Peter Rabbitson <rabbit+list at rabbit.us> wrote:
> Hello!
>
> I am having trouble finding info on how to properly control color
> reproduction, both in terms of "reference manual" and in terms of "best
> practice guides". If I missed a resource neatly explaining everything I
> am confused about below, please do not hesitate to "RTFM" me ;)
>
> == Background
>
> I am looking to produce high quality 720p H264 streams derived from RGB
> source material. Both the source and 99.999% of the playback
> environments are 24bit RGB color platforms (pc/mobile/etc displaying a
> youtube/vimeo/etc window in a browser)
>
> == Setup
>
> For simplicity let's construct a non-sensical RGB source:
>
> ffmpeg -y -r 30 \
>    -filter_complex 'color=c=00aa00:s=320x200; color=c=00aa00:s=1280x720' \
>    -c:v libx264 -pix_fmt yuv444p -profile:v high444 -qp 0 -preset:v
> ultrafast \
>    -t 5 greenz.mkv
>
> This will generate two identical 5 second video streams of the #00AA00
> green, with the only difference being the screen size (ultra-sub-SD vs
> HD). If then we render these streams on-screen, and add a pure RGB
> "control output" and take a screenshot we end up with the attached image.
>
> The 320x200 is displayed properly, although the color is slightly off:
> #00AB01. I assume that this loss is due to the inherently-imperfect
> RGB24 => YUV444P transition, and there isn't much one can do about it.
>
> However the 720p playback is *way* off, at #009000. I can "correct" it
> by playing via `mpv --vid=2 --vf=format=colormatrix=bt.601 greenz.mkv`
> or by explicitly tagging the H264 bitstream at encoding time via `...
> -colorspace bt470bg...`.
>
> == Questions
>
> - What is the "default" conversion matrix used within ffmpeg when
> nothing else is specified. Is it (apparently) bt.601, or does it default
> to different things depending on input?

You are using color source filter which is using fixed coeff (bt.601) when
converting color from rgb to yuv. There is nothing much you can do about
this.

>
> - If it is bt.601 is it bt470bg or is it smpte170m or in the matrix case
> it doesn't matter?
>
> - Is the visual change during playback (via mpv in this case) of the
> "untagged" streams based solely on the output size? Or is there another
> reason why stream1 looks different than stream2 with default settings?

If there is no hint in bitstream what colorspace is used, the one being
used by playback will be by guessing using video size.

>
> - Is it "ok" to produce 720p video explicitly tagged at bt.601? I ran
> across multiple writeups on the net suggesting 720p not encoded (is this
> the right term?) in bt.709 is a big no-no.

It is "ok" to produce it, but not all encoders support proper tagging right
colorspace in their bitstream... so better be safe than sorry and not
produce such files.

>
> - Regardless of what the answer to the previous question is: If I wanted
> to produce the same example video in bt.709 and have it play correctly
> in both cases - what would the ffmpeg incantation be?

FFmpeg doesn't play videos, it decodes them. FFplay is more a tool
for testing than proper playback.

>
> - Am I correct that the inconsistency #00AA00 => #00AB01 is simply "cost
> of doing YUV business", or is the reason more nuanced/involved?
>
> - Anything else I missed/need to know? :)
>
>
> Thank you in advance for your help!
>
>
>