[FFmpeg-devel] [RFC] A file format to store generic raw image/video files

[Lynne]

Oct 18, 2023, 00:12 by ffmpeg-devel at ffmpeg.org:

> # RFC: A file format to store generic raw image/video files
>
> Context: Developers and researchers often want to use raw image/video
> files, as they avoid the effects of encoding. Dealing with raw
> image/video files is typically done using raw files. This is an
> operational pain point, as the metadata about the file needs to be
> carried out on the side. Raw video/image metadata includes (at least)
> pixel format (aka `pix_fmt`), width, height, framerate, and others.
>
> For example, in order to read a specific rgba file that I got from a
> shader, I need to write:
> ```
> $ ffmpeg -y -f rawvideo -video_size 1920x1080 -pix_fmt rgba -i
> post_shader.1920x1080.rgba -vf scale="out_color_matrix=bt601" out.png
> ```
>

There's already NUT, which does all this. It is a documented standard,
it has a reference library outside of FFmpeg, and built-in support.
https://git.ffmpeg.org/gitweb/nut.git/blob/HEAD:/docs/nut.txt
Also, I've been working on AVTransport,
https://github.com/cyanreg/avtransport/blob/master/draft-avtransport-spec.md
Which has all of this, plus much more.

Additionally, Y4M has already been extended, and can be arbitrarily
extended. It even does chroma positions now.

These three containers cover pretty much all niches - from simple
container anyone can write out in 30 minutes, to something slightly
more complicated that isn't too difficult to write by copy-pasting,
to something that requires a little more copy pasting to write out
a simple stream.

I don't think there's room for anything in between. Especially now
that ISOBMFF also has an uncompressed mode, for which we've had
requests to implement.