[FFmpeg-devel] libavformat/mpegtsenc: new interleaved mux mode [v3]

[Andreas Håkon]

Hi Marton,


Thank you very much for finally answering regarding this patch!


‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, 28 de August de 2019 0:46, Marton Balint <cus at passwd.hu> wrote:

> On Mon, 26 Aug 2019, Andreas Håkon wrote:
>
> > To understand what this patch is doing, see these screenshots about the test files
> > "bshi01-mode0.ts" and "bshi01-mode1.ts":
> >
> > -   Current muxing: https://trac.ffmpeg.org/attachment/ticket/8096/MODE-0.PNG
> > -   New interleaved muxing mode: https://trac.ffmpeg.org/attachment/ticket/8096/MODE-1.PNG
> >
> > See also this example of a professional muxer: https://trac.ffmpeg.org/attachment/ticket/8096/MPTS.PNG
>
> Thanks for your changes, it was much easier for me to see what is going
> on. There is still room for simplification, you can probably factorize the
> patch which assigns ->stream_id to the context to a sperate patch. And you
> can probably loose most of the PES flags you introduced because the
> state of the PES packet can be decided based on payload_top and
> payload_size. This also helps you to reduce the number of functions added.

OK. I'll prepare another simple patch to move the ->stream_id to the context.

Regarding the PES flags... I can't discard any!

- #define PES_FLAG_READY     : Ready to write the PES packet.
- #define PES_FLAG_START     : Write the header PES.
- #define PES_FLAG_PARTIAL   : Write only one TS packet.
- #define PES_FLAG_NEEDS_END : The PES packet contains TELETEXT

Note that these flags are because the function mpegts_write_pes() it's used
in a session less way (the PES packet state is stored inside the flags).
So I can't use payload_top and payload_size to deduce the state of the PES
packet.


> The biggest issue however is that your interleaving algorithm works by
> draining pending PES packets in a round robin fashion TS packet by TS
> packet. So if you have streams A, B, and stream B has twice the bitrate of
> stream A, you will get something like this: ABABABBBB when in fact you
> should be getting something like ABBABBABB. So I am not sure if we should
> add an "interleaving" mode if it only works properly for streams with
> roughly the same bitrate. It certainly does not fix ticket #912.

Yes and no.

It's true that the interleaving mode introduced by this patch can be improved.
However, it doesn't have any troubles when using streams with different bitrates.

And your example is wrong:

- ABBABBABB: That's ONE option.
- ABABABBBB: That's NOT the current output with my interleaving algorithm.
- AABABBBBB: That IS the current output with my interleaving algorithm.
- AAABBBBBB: And that's the current sequential mode.

What's the difference? The STARTING point of a PES packet. That's because
the relevant trigger to decide to insert or not a packet from a stream is
the PTS/DTS timestamps. And that decision is calculated at the first packet
(when the PES header is writed). Then, after that point, you don't need to worry
about the exact position of the rest of the packets. You only need to ensure
that the last PES packet is writed before the start of the next PES packet
of the same stream. For this reason you have the decoder buffer. And the
MPEG-TS muxer doesn't need to care about it, as the responsability is from
the encoder. In fact, the worst case is the current sequential mode, as
you write the entire PES packet as quickly as possible. So the buffer fills
up quickly. And if you interleave, then you're reducing the filling speed.
Then no problem at all, as the end of the PES will arrive before the start
of the next PES. So, no buffer underflow is possible.

> It certainly does not fix ticket #912.

My goal is not to solve that error, it's to implement an interleaved mode.

The main problem when using the sequential mode (the only option at time)
is when you use multiple programs. With multiple video streams, plus other
audio and data streams, this interleaved implementation is a big breakthrough.


> Even if we don't want to implement a "proper" muxer, there should be a way
> to improve the interleaving algorightm to have a better chance of
> outputting something that is spec compliant. I might give it a try.

I prefer to first introduce the "mpegts_interleave_mux" option in the
mpegtsenc muxer, and after improve the algorithm. The current sequential
strategy is very inefficient.

Are you willing to help me to achieve that goal?

Regards.
A.H.

---