[FFmpeg-user] Levels and Primaries

[Rio Kierkels]

First I'd like to put out a warning that this will be a long post. Not
because I like long posts but because I really want to get to the bottom of
this and of course to have a sort of baseline for other people interested
in the topic. Being as complete as possible will help the cause. So by all
means if you have anything to say, and I mean ANYTHING, just hit reply and
share it with us. Help get this thing rolling. :)

I am not here to nag about ffmpeg and frankly I'd really like to contribute
to the ffmpeg code/docs but as I am not very good at C/C++ programming
(mostly python now) I'm finding it quite hard to really contribute. However
I am learning and I've been reading through the ffmpeg and libav* code
quite regularly. Mostly to figure out what badly documented options are
doing and just to satisfy my curiosity ;)

At my company (We're a post-house with VFX/Edit/Color/Sound based in
Amsterdam) I'm trying to integrate ffmpeg more and more. The big problem is
we don't know how trustworthy (for lack of a better term) ffmpeg is in it's
color space conversions. And the entire topic is quit confusing as most of
the tools give us different results when trying to check waveforms and such.

Right now I'm focusing on the conversion from our in house masters which
are 10 bit DPX files using the full bit range and BT.709 primaries to the
Dutch HD broadcasting standard (MXF 1080i50 XDCAM HD 422 50Mb/s BT.709
primaries and studio range levels 16-235).

To test it all I've built some color bars according to the ARIB-STD-B28
specification of which the pdf can be found here
(ARIB-STD-B28<http://arib.or.jp/english/html/overview/doc/6-STD-B28v1_0-E1.pdf>).
These bars have been built in Nuke 8 and rendered out as 32bit/16bit scene
linear OpenEXR files and 10bit full range BT.709 DPX files. These files can
be found here (Materials<https://drive.google.com/folderview?id=0B9hzJJkCcuUrWlF6VFpQY3dBRzQ&usp=drive_web>)
and can be used by anyone for anything as far as I'm concerned. These bars
have been incorporated in a small sequence that includes a resolution
chart, a checker pattern with 4:3 and 2.40 crop marks and Kodak's Marcie
because everybody loves her. You can find a zip containing the 100 dpx
frames here (color test
sequence<https://drive.google.com/file/d/0B9hzJJkCcuUrMmdHdmFodHFiVjQ/edit?usp=sharing>)
and again as far as I'm concerned everybody can use them. In the logs that
I've included You might notice that my sequence is 475 frames instead of
100. This is because I've appended some shots that we've been using in
house as a reference, just to see something familiar both for me and our
color grader.

So to start the tests I've converted the dpx files to the mxf files, the
commandline and output can be found here (DPX to MXF commandline
output<https://drive.google.com/file/d/0B9hzJJkCcuUrRklfWGJUR1dGdlU/edit?usp=sharing>).
I'm not posting the logs in text form directly in the mail because it is
already long enough.
Something to notice in the command line output is that the video output
says yuv422p(tv, bt709). This is a good thing as it tells me the output
should be BT709 primaries and within studio range luma levels. However it
seems as ffmpeg by default decodes the dpx files as BT601 so I have to add
a colormatrix filter to get it back in to BT709 space. It's not pretty but
it works. Also the audio formats are wrong for all but the first stream,
but just ignore those for now. After that I'm checking if the Y' levels are
correct I'm simply applying a histogram filter and outputting a png
sequence, the command and output are here (ffmpeg MXF waveform
commandline<https://drive.google.com/file/d/0B9hzJJkCcuUrNm9JclJNRTVtdVU/edit?usp=sharing>)
and here (ffmpeg color bars waveform
output<https://drive.google.com/file/d/0B9hzJJkCcuUrMzRGQ2hfcnl1cUU/edit?usp=sharing>)
is a frame containing the waveform of the ARIB color bar. This first thing
to notice in the ffmpeg output is that it no longer reports the input
stream as bt709, now it's only yuv422p(tv). The ffprobe output of the mxf
file also reports it, found here (ffprobe MXF
output<https://drive.google.com/file/d/0B9hzJJkCcuUrN2ZncFdHMDJxMEU/edit?usp=sharing>).
Now I have no idea is this affects the image itself but being a bit of and
OCD type of guy I don't like it.

Now when checking out the waveform of the ffmpeg rendered mxf file when you
look at the second column it contains 0% black and 100% white. For people
who don't know how this waveform works. essentially the height of the image
is the value of the pixels for that column of pixels with ~1 being fully
black and ~256 fully white. If you measure the second block of values
you'll see that 100% white lies on 235 and black lies on 16. And judging by
the Cb and Cr values the primaries are also in the expected BT709 place. So
far it's all good!

To cross check those figures I import the rendered MXF file into Lightworks
1.5, Flame 2014sp2 and Smoke 2014.pr34. They all show the same wavefrom,
full range... This is very unnerving. Here is the screen shot of lightworks
(lightworks MXF
waveform<https://drive.google.com/file/d/0B9hzJJkCcuUrSEVyVnAzZGJ5YlE/edit?usp=sharing>)
and here of smoke (Smoke MXF
waveform<https://drive.google.com/file/d/0B9hzJJkCcuUreHN6WmZDYU9HVTA/edit?usp=sharing>).
Here come the trust issues. Is ffmpeg doing a transform when outputting the
waveforms, is this really the data within the mxf file. Or is it just a
little flag set in the container or stream. Are lightworks, smoke and flame
seeing that flag and just ignoring it displaying what's in the mxf file. Or
is the data in the mxf file really within studio ranges and are they
compensating for it when displaying the media and the waveform.

I've grabbed one of our commercials that we've made just recently and
wanted to check that mxf aswell. This mxf was generated from our full range
BT709 dpx masters using Episode 6, here is the ffprobe output of that
file (reference
MXF ffprobe output<https://drive.google.com/file/d/0B9hzJJkCcuUrUkw2WWhyWUg1OVE/edit?usp=sharing>).
In the preset there is an option set that scales the levels down to "legal"
range. Again the first thing that I notice is that this MXF file does show
yuv422p(tv, bt709), something that the ffmpeg generated file doesn't show.
When running that mxf through the ffmpeg waveform command (ffmpeg reference
MXF commandline<https://drive.google.com/file/d/0B9hzJJkCcuUrWXNZU1o0Mzd4MVU/edit?usp=sharing>)
something else is noticed as shown here (reference MXF ffmpeg
waveform<https://drive.google.com/file/d/0B9hzJJkCcuUraDNNU2ZZZC14OFk/edit?usp=sharing>).
This is a title card near the end of the commercial with white text and a
view inside a dark room through a door that has black around it. As you can
see the white text is well below the 235 mark and the black background well
above the 16 mark. Almost as if a double downscaling occurs. This to me is
quite frightening, if the file really has this double scaling this means
that our broadcast copies are being broadcasted with the wrong contrast
ranges.
But looking at the lightworks, smoke and flame waveforms they do fall
within studio ranges as shown here (lightworks reference mxf
waveform<https://drive.google.com/file/d/0B9hzJJkCcuUrbk1vUnFrcjFJTzA/edit?usp=sharing>)
and here (smoke reference mxf
waveform<https://drive.google.com/file/d/0B9hzJJkCcuUrMmlnWTFCalhfRWs/edit?usp=sharing>).
Btw it might be worth mentioning that lightworks uses ffmpeg libraries in
the background, however as long as they don't release their source code
it's hard to check what they are doing with that waveform.

I hope you're still with me and if you are I hope you have anything to say
about it. May it be some experience with the same issues or even the answer
to this debacle. Use the materials in this post as you like to do your own
tests and you can ask me anything you want be it to run certain tests on
the equipment I have available or maybe even contacting our broadcasters
for some specific information.

We want FFmpeg to be as good as it gets and I really think we can get this
one right!

Rio
-- 
42