[FFmpeg-user] DTSs & PTSs in MPEG2-TS streams

[Mark Filipak]

Hey Devin,

On 01/02/2024 09.54, Devin Heitmueller wrote:
> Hi Mark,
> 
> On Thu, Feb 1, 2024 at 1:15 AM Mark Filipak <markfilipak.imdb at gmail.com> wrote:
>>
>> FFmpeg assigns DTSs & PTSs to P-frames and B-frames in MPEG2-TS streams. How does it determine those
>> DTSs & PTSs?
>>
>> Since DTSs & PTSs don't actually exist for P-/B-frames in MPEG2-TS streams, FFmpeg must have a way
>> that's unknown to me.
> 
> P/B frames can certainly have PTS values.

Well, they _can_ as in they're not prohibited, but I can't recall ever seeing a P-/B-frame in 
MPEG2-TS supplying a DTS or a PTS. I've parsed quite a few MPEG2-TSs.

>  In fact, you have to have
> PTS values in B frames since they differ from the DTS (since with B
> frames the presentation order differs from the decoding order).

I don't see DTS or PTS in the PES header. What I do see is 'PES_extension_flag'=='0' and 
'PTS_DTS_flags'=='00', so no PES ext and no dts or pts. That's why I asked my question.

Every once in awhile I'll see 'PTS_DTS_flags'=='10' but that's unusual. Mind you, this is in 
MPEG2-TSs. I don't how to parse anything else.

> There are potentially cases with some streams where the PTS and/or DTS
> is not specified for every frame,

I'd say "usually", not "potentially".

> in which case libavformat will
> interpolate those values based on the frame duration and the prior
> packet.

Of course. But why? They're not needed. Once the TS packets are put in correct sequence via SCR, the 
PS packets transported by those TS packets are in presentation order. The process to reorder the PS 
in order to decode B-frames and then put the decoded pictures back into the original order is 
straight forward. It's not "tricky" at all.

It has become "tricky". That's what I'm trying to investigate, but I can't read 'C' code.

>> I'm seeing some strange things. They can't be as strange as they look because they look disastrous.
>> At issue is the cause of non-monotonous DTSs and obviously bogus DTSs for P-frames such as
>> '-9223372036854775808'.
> 
> 9223372036854775808 is a magic number.  It corresponds to the #define
> for AV_NOPTS_VALUE.  If you see that value it means that the packet
> did not have a PTS or DTS value assigned to it and thus you may have
> to take special measures.

Oh, dear. I guessed that.

Magic number. Fixup. There's a million fixups. I don't see why they're needed. That's why my 
preliminary finding is that FFmpeg is creating its own faults through fixups.

--Mark.