[FFmpeg-devel] [PATCH 0/6] Implement SEI parsing for QSV decoders

[Soft Works]

From: Kieran Kunhya <kierank at obe.tv>
Sent: Wednesday, June 1, 2022 10:26 PM
To: Soft Works <softworkz at hotmail.com>
Cc: Kieran Kunhya <kierank at obe.tv>; FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
Subject: Re: [FFmpeg-devel] [PATCH 0/6] Implement SEI parsing for QSV decoders

Captions aren’t exactly “frame accurate” anyway as each frame has just a very small piece
of information and only when a certain sequence is complete, it leads to some new letters
or line being ready for display.
And you don't want errors accumulating across encode cycles.

No such errors seen at any time, CC extraction behavior doesn’t appear to be any different
than for other ffmpeg decoders (mp2, h264, hevc – sw, same three for NVDEC, same three
for VAAPI, same three for D3D11VA)


In many use-cases, you want them to be frame-accurate. Final rendition to the viewer is only one use-case of FFmpeg (to be fair, the likely use-case of QSV)


What this patchset provides for CC is:


  *   QSV decoders create and attach CC side data to AVFrames
  *   QSV filters preserve CC side data from input to output
  *   in an earlier patch we already added the ability to attach CC data when using QSV encoders

This allows for example to do QSV hw decoding, filtering and encoding where CC data
is preserved in the output video .

In combination with my Subtitle Filtering patchset, you can do almost anything you
like with closed captions.

You can work with them like any other subtitle data and process them with the new
filters, e.g. manipulate the text content, change styles and appearance, like font
sizes, colors, outlines, background etc.

Then you can burn-in these into the video, or encode in an arbitrary text subtitle
format. There are many possibilities…
Just one example:

[cid:image001.png at 01D8760E.B18F06F0]

Do you have an idea where/how I could find such stream?

You would need to record them off various television services as they use a wide range of encoder manufacturers.
Ideally, you could also inject them into various encoders and force them to build complex reordering patterns and check they are frame accurate.

We have users from all over the US with different tuners and on different networks,
there has never been an issue.

What I meant is the “VLC heuristics” subject you mentioned whether you might
have some pointer to a  commit, issue, bug report or simply a name how they call this?

Thanks again,
sw



-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 32096 bytes
Desc: image001.png
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20220601/cd06673f/attachment.png>