[FFmpeg-user] Raw video format

[Mattias Andrée]

On Tue, 10 Jan 2017 17:47:38 +0100
Carl Eugen Hoyos <ceffmpeg at gmail.com> wrote:

> 2017-01-10 17:26 GMT+01:00 Mattias Andrée
> <maandree at kth.se>:
> > On Tue, 10 Jan 2017 12:05:24 +0100
> > Carl Eugen Hoyos <ceffmpeg at gmail.com> wrote:
> >  
> >> 2017-01-09 23:11 GMT+01:00 Mattias Andrée
> >> <maandree at kth.se>:  
> >> > I have a stream of YUV colours (with alpha), stored
> >> > with raw `double`s.  
> >>
> >> FFmpeg does not support float pix_fmt and at least some
> >> developers believe that this wouldn't help anyway for
> >> expected input.
> >>  
> >> > I want to use ffmpeg -f rawvideo -pix_fmt ayuv64le
> >> > to convert this into a normal video format.  
> >>
> >> This is the point:
> >> "Normal" video format is integer, so why would it help
> >> if you input float?
> >>  
> >> > How should I encode each pixel so that ffmpeg can
> >> > decode them?  
> >>
> >> RGBA64 comes to mind if your input is rgb.
> >>
> >> If you want to input float into FFmpeg, you have to
> >> write exr frames but note that they are converted to
> >> int before any further processing.  
> 
> > I'm pretty sure I prefer YUV over RGB.  
> 
> I realize that my comment above concerning RGBA64
> may not make sense to you, please ignore.
> (My first guess was that your input is exr or related.)
> 
> > Values in YUV are floats  
> 
> I don't understand:
> Do you mean that in your application (or the application
> that you want to use) yuv are floats? This is of course
> possible and it is a problem for you, because yuv float
> is not only not supported by FFmpeg but not supported
> by any related application.
> Or do you mean that all "YUV" values are always float?
> In the world of codecs (that's where FFmpeg is used)
> yuv is always integer, there is no video codec that
> outputs floats and I believe it makes little sense to
> write an encoder that accepts floats.
> (tiff and a few others do support floats but as said
> FFmpeg does not support it.)

I mean the former.

I have figured out that if I convert an sRGB colour,
with 8-bits per channel with the transfer function
applied, to YUV without unapplying the transfer function,
and then let

	Y := Y ⋅ 256 +  16 ⋅ 256
	U := U ⋅ 256 + 128 ⋅ 256
	V := V ⋅ 256 + 128 ⋅ 256

I get values pretty close to those from ffmpeg get.

I have tries both the conversion matrix

	⎛ 0.299         0.587         0.114      ⎞
	⎜−0.450/3.069  −0.883/3.069   1.333/3.069⎟
	⎝ 1.333/2.169  -1.116/2.169  -0.217/2.169⎠

and the conversion matrix

	⎛ 0.299     0.587     0.114  ⎞
	⎜−0.14713  −0.28886   0.436  ⎟
	⎝ 0.615    −0.51499  −0.10001⎠

The produce very similar results, but both are a bit
off from what ffmpeg produce.

What adjustments should be done so the conversion
from sRGB to YUV matches that done by ffmpeg?

> 
> > that can be negative or large, it does not have
> > the simple bounds of [0, 1], with the exception of the Y
> > value, like you normally have with RGB.  
> 
> (Suspecting now that you are thinking of analog video)
> Please understand that this is simply not true for
> FFmpeg: On this mailing list, when we talk about
> "yuv", we mean 8bit or 16bit integer values (full
> scale or mpeg scale).

No, I'm just talking about raw encoded colours.

> 
> > So my question is how should a YUV value be
> > encoded as an integer?  
> 
> You will have to find a linear representation for the
> values that you call "YUV" as everybody else does.
> 
> > Floats are useful for rendering  
> 
> Given that no hardware (?) and no real-world video
> codec support floats, I wonder what you mean here.

Perhaps composing is more accurate word. Floats are
useful for the mathematics. For example if you have
two images with alpha you want to put on top of each
other, normalised floats and no transfer function
applied makes the maths simpler.

> 
> > so I think it would be
> > preferable for ffmpeg to support it as it would speed
> > up rending as it would eliminate some conversion.  
> 
> This sounds very unlikely.
> 
> My new guess is that you should simply convert to
> yuv420p (as defined by FFmpeg) to get "normal video
> format" as output (and no additional conversion needed).
> 
> Carl Eugen
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> 
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-user/attachments/20170110/4caa03da/attachment.sig>