[FFmpeg-devel] libavformat/mpegtsenc: new interleaved mux mode [v3]

[Marton Balint]

On Mon, 26 Aug 2019, Andreas Håkon wrote:

> Hi,
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Monday, 19 de August de 2019 21:16, Andreas Håkon <andreas.hakon at protonmail.com> wrote:
>
>> Hi,
>>
>> This is the third version of my patch for an "interleaved MPEG-TS muxer".
>> This new version includes all recommendations and rebases the fix of the
>> incorrect PCR with multiple programs (fixed in collaboration with Marton Balint).
>>
>> Supersedes: https://patchwork.ffmpeg.org/patch/13745/
>>
>> How to check it:
>>
>> (Note: I use for all the tests the file
>> https://samples.ffmpeg.org/HDTV/bshi01.tp
>> )
>>
>> - To check the new interlaced mode you can perform this other test:
>>
>> $ ffmpeg-patched -y -loglevel verbose -i bshi01.tp \
>>   -map "i:0x100" -c:0 copy \
>>   -map "i:0x110" -c:a:0 mp2 -ac:0 2 -ar:0 48000 -ab:0 384k \
>>   -map "i:0x130" -c:2 copy \
>>   -map "i:0x110" -c:3 copy \
>>   -map "i:0x100" -c:4 copy \
>>   -program title=Prog1:st=0:st=1:st=2 \
>>   -program title=Prog2:st=3:st=4 \
>>   -f mpegts -muxrate 44M -mpegts_extra_mux 1 bshi01-mode1.ts
>>
>> $ ffmpeg-patched -y -loglevel verbose -i bshi01.tp \
>>   -map "i:0x100" -c:0 copy \
>>   -map "i:0x110" -c:a:0 mp2 -ac:0 2 -ar:0 48000 -ab:0 384k \
>>   -map "i:0x130" -c:2 copy \
>>   -map "i:0x110" -c:3 copy \
>>   -map "i:0x100" -c:4 copy \
>>   -program title=Prog1:st=0:st=1:st=2 \
>>   -program title=Prog2:st=3:st=4 \
>>   -f mpegts -muxrate 44M -mpegts_extra_mux 0 bshi01-mode0.ts
>>
>> ---
>
> To understand what this patch is doing, see these screenshots about the test files
> "bshi01-mode0.ts" and "bshi01-mode1.ts":
>
> - Current muxing: https://trac.ffmpeg.org/attachment/ticket/8096/MODE-0.PNG
> - New interleaved muxing mode: https://trac.ffmpeg.org/attachment/ticket/8096/MODE-1.PNG
>
> See also this example of a professional muxer: https://trac.ffmpeg.org/attachment/ticket/8096/MPTS.PNG

Thanks for your changes, it was much easier for me to see what is going 
on. There is still room for simplification, you can probably factorize the 
patch which assigns ->stream_id to the context to a sperate patch. And you 
can probably loose most of the PES flags you introduced because the 
state of the PES packet can be decided based on payload_top and 
payload_size. This also helps you to reduce the number of functions added.

The biggest issue however is that your interleaving algorithm works by 
draining pending PES packets in a round robin fashion TS packet by TS 
packet. So if you have streams A, B, and stream B has twice the bitrate of 
stream A, you will get something like this: ABABABBBB when in fact you 
should be getting something like ABBABBABB. So I am not sure if we should 
add an "interleaving" mode if it only works properly for streams with 
roughly the same bitrate. It certainly does not fix ticket #912.

Even if we don't want to implement a "proper" muxer, there should be a way 
to improve the interleaving algorightm to have a better chance of 
outputting something that is spec compliant. I might give it a try.

Regards,
Marton