[FFmpeg-user] understanding DTS PTS

[Marc Mason]

Alexandre Ferrieux wrote:
> Marc Mason wrote:
>> John wrote:
>>
>>> I am going through the ffmpeg tutorials and am having trouble making
>>> sense of this statement in tutorial 5.
>>>
>>> <quote>
>>> However, ffmpeg reorders the packets so that the DTS of the packet
>>> being processed by avcodec_decode_video() will always be the same as
>>> the PTS of the frame it returns.
>>> </quote>
>>>
>>> What reordering is it talking about? From the example
>>>
>>> PTS: 1 4 2 3
>>> DTS: 1 2 3 4
>>> Stream: I P B B
>>>
>>> The frames are being read in I P B B. When P is being processed by
>>> avcodec_decode_video() its DTS is 2. P's PTS cannot be 2. What frame
>>> reordering is ffmpeg supposed to be doing? Thanks...
>> Suppose we have 4 frames, and we decide to encode them the following 
>> way: F1=I F2=B F3=B F4=P
>>
>> To encode F2 and F3, we need F4 (B frames).
>>
>> The presentation order (PTS) is F1, F2, F3, F4.
>>
>> But since the decoder needs F4 to decode F2 and F3, we can send them in 
>> a different order : F1, F4, F2, F3 (the decoding order).
>>
>> The decoder will receive frames in the following order.
>>
>> F1 DTS=1 PTS=1
>> F4 DTS=2 PTS=4
>> F2 DTS=3 PTS=2
>> F3 DTS=4 PTS=3
> 
> OK, but what purpose does the DTS serve then ?

I'm not quite sure what you're asking. Here is the definition.

"""
DTS (decoding time stamp) – The DTS is a 33-bit number coded in three 
separate fields. It indicates the decoding time, tdn(j), in the system 
target decoder of an access unit j of elementary stream n. The value of 
DTS is specified in units of the period of the system clock frequency 
divided by 300 (yielding 90 kHz). The decoding time derived from the DTS 
according to equation 2-12 below:

DTS(j) = ((system_clock_frequency x tdn(j)) DIV 300) % 2^33

where tdn(j) is the decoding time of access unit An(j).

In the case of ISO/IEC 11172-2 video, ITU-T Rec. H.262 | ISO/IEC 13818-2 
video, or ISO/IEC 14496-2 video, if a DTS is present in a PES packet 
header, it shall refer to the access unit containing the first picture 
start code that commences in this PES packet. A picture start code 
commences in a PES packet if the first byte of the picture start code is 
present in the PES packet.

For ITU-T Rec. H.264 | ISO/IEC 14496-10 video, if a DTS is present in 
the PES packet header, it shall refer to the first AVC access unit that 
commences in this PES packet. An AVC access unit commences in a PES 
packet if the first byte of the AVC access unit is present in the PES 
packet. To achieve consistency between the STD model and the HRD model 
defined in Annex C of ITU-T Rec. H.264 | ISO/IEC 14496-10, for each AVC 
access unit the DTS value in the STD shall, within the accuracy of their 
respective clocks, indicate the same instant in time as the nominal CPB 
removal time tr,n( n ) in the HRD, as defined in Annex C of ITU-T Rec. 
H.264 | ISO/IEC 14496-10.
"""

You might be wondering why DTS are needed at all? Why would we not 
decode frames as soon as they are received?

This is probably where the Transport Stream System Target Decoder 
(T-STD) comes into play.

"""
2.4.2 Transport Stream system target decoder

The semantics of the Transport Stream specified in 2.4.3 and the 
constraints on these semantics specified in 2.7 require exact 
definitions of byte arrival and decoding events and the times at which 
these occur. The definitions needed are set out in this Recommendation | 
International Standard using a hypothetical decoder known as the 
Transport Stream System Target Decoder (T-STD). Informative Annex D 
contains further explanation of the T-STD.

The T-STD is a conceptual model used to define these terms precisely and 
to model the decoding process during the construction or verification of 
Transport Streams. The T-STD is defined only for this purpose. There are 
three types of decoders in the T-STD: video, audio, and systems. Figure 
2-1 illustrates an example. Neither the architecture of the T-STD nor 
the timing described precludes uninterrupted, synchronized play-back of 
Transport Streams from a variety of decoders with different 
architectures or timing schedules.
"""

-- 
Regards.