[FFmpeg-devel] TS muxer issues -- some progress

[Måns Rullgård]

Alexandre Ferrieux <alexandre.ferrieux at orange-ftgroup.com> writes:

> Marc Mason wrote:
>> Marc Mason wrote:
>> 
>>> M?ns Rullg?rd wrote:
>>>
>>>> Marc Mason wrote:
>>>>
>>>>> I've been reading H.222.0 (MPEG-2 Systems) and ETSI TR 101 290 
>>>>> (Measurement guidelines for DVB systems). Again.
>>>>>
>>>>> "Systems" explicitly allows variable-bitrate transport streams:
>>>>>
>>>>> """
>>>>> Transport Streams may be either fixed or variable rate. In either case 
>>>>> the constituent elementary streams may either be fixed or variable rate. 
>>>>> The syntax and semantic constraints on the stream are identical in each 
>>>>> of these cases. The Transport Stream rate is defined by the values and 
>>>>> locations of Program Clock Reference (PCR) fields, which in general are 
>>>>> separate PCR fields for each program.
>>>>>
>>>>> There are some difficulties with constructing and delivering a Transport 
>>>>> Stream containing multiple programs with independent time bases such 
>>>>> that the overall bit rate is variable. Refer to 2.4.2.2.
>>>>> """
>>>>>
>>>>> I realize now that I had always implicitly assumed that a given 
>>>>> transport stream must be constant bit-rate. And this assumption turns 
>>>>> out to have been wrong.
>>>> A variable-rate TS can be trivially converted to constant rate by
>>>> inserting null packets.
>>> I don't think so.
>>>
>>> If the muxer is real-time, it would need an oracle to predict the future 
>>> bitrate of the underlying elementary streams.
>>>
>>> If the bitrate of each elementary stream is bounded, then the muxer 
>>> could pad to the sum of all bitrates + overhead, but this would be 
>>> highly inefficient.
>>>
>>> Even if the muxer is offline, and thus has perfect knowledge of the 
>>> "future", it may not be possible to "spread" the excess bits.
>>>
>>> For an unrealistic scenario, consider a stream with the following 
>>> properties: 50 Mbit/s for one second, then 500 kbit/s for the next 3599 
>>> seconds. The excess data at the start may have timing imperatives that 
>>> prevent the muxer from spreading it over 1 hour.
>>>
>>>> Constant-rate streams are required if a
>>>> physical link is inherently fixed-rate, e.g. a satellite broadcast.
>>> Variable-bitrate TS over RTP/UDP/IP should be OK then?
>>>
>>> I've worked with several ASI cards, and these required the bitrate to 
>>> remain constant.
>>>
>>>>> Given a variable bitrate transport stream, it seems to make no sense 
>>>>> trying to compute PCR accuracy offline, as it is typically computed by 
>>>>> assuming CBR, dividing the byte count by the CBR, and comparing that 
>>>>> value to the PCR.
>>>>>
>>>>> TR 101 290 seems to agree with the above statement:
>>>>>
>>>>> """
>>>>> PCR_accuracy_error
>>>>> The accuracy of +/- 500 ns is intended to be sufficient for the colour 
>>>>> subcarrier to be synthesized from system clock.
>>>>> This test should only be performed on a constant bitrate TS as defined 
>>>>> in ISO/IEC 13818-1 [1] clause 2.1.7.
>>>>> Further information on PCR jitter measurements is given in clause 5.3.2. 
>>>>> and annex I (PCR related measurements).
>>>>> """
>>>>>
>>>>> """
>>>>> 5.3.2.6 Program Clock Reference - Accuracy PCR_AC
>>>>> The accuracy of the PCR values PCR_AC is defined as the difference 
>>>>> between the actual PCR value and the value it should have in the TS 
>>>>> represented by the byte index for its actual position. This can be 
>>>>> calculated for constant bitrate TS, the measurement may NOT produce 
>>>>> meaningful results in variable bitrate TS.
>>>>> """
>>>>>
>>>>> "may NOT" ?? "WILL NOT" seems more appropriate.
>>>>>
>>>>> Do most demuxers handle variable-bitrate transport streams with no 
>>>>> problem, or is it generally assumed that a TS is CBR?
>>>> Most demuxers, including small, cheap hardware decoders, handle
>>>> variable-rate streams without issue as long as they are delivered at
>>>> the correct rate.
>>> So streaming over a LAN should be OK, but streaming over the Internet 
>>> might not work, because of the network jitter?
>>>
>>>> Some retarded implementations require constant rate for reasons beyond
>>>> my imagination.  One such implementation I have encountered would give
>>>> a "bitrate too low" error if the TS rate, as computed from bytes
>>>> between PCR values, varied even the slightest.
>> 
>> M?ns and/or other Transport Stream enthusiasts ;-)
>> 
>> Could you comment? Specifically, I'm not sure I agree with the statement 
>> that it is trivial to convert a VBR TS to a CBR TS.
>> 
>
> I second the question. While letting the experts answer it, can someone 

It is trivial to convert vbr to cbr assuming there is some upper bound
to the bitrate.  Simply add null packets to make the rate between
consecutive PCR values equal to this upper bound.  The contained video
elementary stream might still be variable rate, of course.

> tell me if the following description is correct for streaming TS to a 
> set-top-box over IP:
>
>   1- assume some VBR source that ffmpeg can decode
>
>   2- internal ffmpeg data structures hold PTS values
>
>   3- these PTS values give target timings for playing at normal speed
>
>   4- with -re, they keep this property even if the input is a flat file 
> on a fast, local hard drive.
>
>   5a- the TS muxer could base its PCR on those internal PTSs
>
>   5b- or, it could base its PCR on the local gettimeofday()
>
>   5c- but today is bases it on written byte counts instead (!), *and* on 
> IP there is no byte-stuffing like would normally occur on a CBR link 
> like satellite.
>
>   6- As a consequence, the receiving STB sees a PCR that is way too slow 
> and reports underflows.
>
> TIA for any comments,

Assuming a VBV-compliant elementary stream, the tricky part is
determining the ideal transmission time of a given packet.  The main
constraint is, of course, that a frame must be transmitted in full
before the PCR reaches the DTS of the frame.  Additionally, a bit may
not be transmitted before there is space for it in the receiver
buffer.  Within these constraints, there is still room for variations,
and therein lies the difficulty.

-- 
M?ns Rullg?rd
mans at mansr.com