[FFmpeg-user] New MLV video format bayer_rggb16le and conversion to yuv444p or rgb48le pixel formats.

[John P Santos]

Hi guys.

I'd like to convert a recently recorded .MLV video recorded with the Magic
Lantern mod to a ProRes format.

When I attempt to I get the following error:

*[swscaler @ 0x2e4da80] unsupported bayer conversion*
*Assertion 0 failed at libswscale/swscale_unscaled.c:1655*


*Aborted (core dumped)*
Full output can be found here:

http://pastebin.com/zzPrYGfu

However it does convert successfully to HuffYUV codec, but is converted to
an rgb24 pixformat, with a very annoying green cast (my guess is because of
the multiple green channels that is part of that pixel format that is an
element of bayesian video sensors.

Full output here:

http://pastebin.com/xrcFjL3k

I have no idea if there's a comprehensive way to handle bayer_rggb16le
pixel format conversion to a yuv444-ish or equivalent pixels, that store
more bits per pixel than rgb24, perhaps rgb48le.    The bayesian raws have
16 bits per pixel, but they are pixel grid,  in a way that will give a
decent interpretation of the colors, since the video converts to a very
strong green cast in rgb24.  Is there a filter currently implemented to
tell ffmpeg how to handle these bayer formats to convert them into
something else?

I think I've just answered my own question.

I found that r210 codec can handle rgb48le, and as soon as I use it I get
the following error which is much more expanded upon than the one I get for
ProRes:

*[swscaler @ 0x26f5160] full chroma interpolation for destination format
'rgb48le' not yet implemented*
*[swscaler @ 0x26f5160] unsupported bayer conversion*
*Assertion 0 failed at libswscale/swscale_unscaled.c:1655*
*Aborted (core dumped)*

Full output can be found here:

http://pastebin.com/FrMcJX5a

After looking carefully at the rgb24 output I can see that no demosiac
process has been done on the bayer input.  It doesn't look like this is
going to be a great way to process raw video into other formats that are
ready to view, but having libav process able to access these formats at all
is a huge leap forward to open source raw video editing software
implementation.  I would love to see open source colour grading tools made
available for raw video.  The darktable project has made huge strides in
processing stills, so perhaps some of their code base would be of use for
this.  Darktable processes colours in a Lab colour space with 4x32-bit
floating point pixel buffers, which is (I would guess) similar to a
theoretical yuva444p32le pixel format... which is insane.

I want to use open source tools in my workflow for raw video, that doesn't
involve batch processing frame by frame colour grades for video...  I think
ffmpeg could do this...  I can't be the first person who's been looking at
this.

Looking forward to any feedback you might offer.

John P Santos
Photographer // Owner
GREENBEEMEDIA.COM