[FFmpeg-devel] [FEATURE PROPOSAL] Extracting codec-level data to binary files

[Ronald S. Bultje]

Hi,

On Wed, May 21, 2025 at 9:34 AM Timothée <
timothee.informatique at regaud-chapuy.fr> wrote:

> Hello,
>
> I am interested in expanding ffmpeg's capabilities to extract
> low-level data from video codecs. Specifically, I'd like to implement
> functionality that would allow exporting frame data, macroblock
> information, quantization tables, and similar codec-specific elements
> to binary files for further analysis.
>
> After searching through the documentation and existing features, I
> haven't found similar functionality, though I may have missed
> something. Has this been implemented before, or are there related
> features I should examine?


Some older codecs implement minor variants for this, e.g. grep
for AV_FRAME_DATA_MOTION_VECTORS, which attaches a frame's motion vectors
to the picture data. I believe there's an example app and possibly a filter
to overlay MVs on top of the video frame based on this concept. You could
extend this to cover other (macro)block info. There used to be a variant of
this for quant-tables also but I can't find it, maybe it was removed.

Placing the side-data in binary files is probably not really something that
libavcodec would do, but rather something that would exist at the
application level.

Ronald