[Libav-user] Injecting and reading back custom per-frame side data (UDU SEI) in HEVC / H264

[Tom Vercauteren]

Hello,

I am interested in storing custom metadata on a per frame level in a mov or
mp4 file. It looks like using user data unregistered (UDU) supplemental
enhancement information (SEI) may be a suitable approach for this. I am
however struggling to get this to work as I am always reading back the
custom message I used for the first frame.

Below is a basic change I made to the mux.c and demux_decode.c examples
from ffmpeg that shows my attempt:
https://gist.github.com/tvercaut/94c68e46d0d0321a2c83b88024d9cd69/revisions

Essentially, I have tried to use
`av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_SEI_UNREGISTERED,
my_sei_buffer);`
at muxing time and
`av_frame_get_side_data(frame, AV_FRAME_DATA_SEI_UNREGISTERED);`
at demuxing/decoding time.

I also realised I had to set the `udu_sei` flag when muxing:
`av_dict_set_int(&opt, "udu_sei", 1, 0);`

I have looked for additional material / documentation on how to do this but
couldn't find a lot. Am I missing a key step here?

Also, is there a way to display the UDU SEI messages using ffmpeg or
ffprobe command line tools? I tried using
`ffmpeg -i test.mov -vf showinfo -f null -`
but this doesn't display the content to the UDU SEI messages I provided.

P.S.: This is a bit of a follow-up from an old post of mine but trying a
different approach for per-frame metadata:
https://lists.ffmpeg.org/pipermail/libav-user/2022-April/013051.html

Best wishes,
Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20240521/7a297dea/attachment.htm>