[FFmpeg-devel] [PATCH 0/6] Implement SEI parsing for QSV decoders

[Soft Works]

From: Kieran Kunhya <kierank at obe.tv>
Sent: Wednesday, June 1, 2022 9:16 PM
To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
Cc: softworkz <softworkz at hotmail.com>
Subject: Re: [FFmpeg-devel] [PATCH 0/6] Implement SEI parsing for QSV decoders

On Thu, 26 May 2022 at 09:09, ffmpegagent <ffmpegagent at gmail.com<mailto:ffmpegagent at gmail.com>> wrote:
But that doesn't help. Those bugs exist and I'm sharing my workarounds,
which are empirically determined by testing a range of files. If someone is
interested, I can provide private access to a repository where we have been
testing this. Alternatively, I could also leave those workarounds out, and
just skip those SEI types.

I don't care much for QSV but I would say b-frame reordering heuristics like the one you are using may not necessarily catch all the possible structures in the wild from third-party encoders.

I am not using any b-frame reordering heuristics, I just take the payloads in the order
in which MSDK(QSV) provides it, and it has turned out that they are reordering
the data according to the display order.

I did some detailed analysis of files with out-of-order B-frames:

https://gist.github.com/softworkz/36c49586a8610813a32270ee3947a932

Did you take a look?


VLC had (has?) heuristics for this which would cause captions to not be frame accurate.

Captions aren’t exactly “frame accurate” anyway as each frame has just a very small piece
of information and only when a certain sequence is complete, it leads to some new letters
or line being ready for display.

But out-of-order would screw this definitely, but I haven’t seen any such cases.
The code I’m submitting has been in testing for quite a while with a bunch of users and
many files and TV streams with MP2Video, H264 and HEVC were tested.

You could still be right, that there is a case. A while ago I had digged through
https://streams.videolan.org/ and downloaded all samples that seemed to have CC,
but maybe I missed one for the case you’re talking about.

Do you have an idea where/how I could find such stream?

Thanks,
softworkz