[FFmpeg-devel] Tee improvement - discussion

[Marton Balint]

On Wed, 11 May 2016, Jan Sebechlebsky wrote:

> Hi,
> I'll be working on tee muxer improvement during GSoC 2016 and I thought 
> maybe it is a good idea to ask about ideas which any of you might have 
> regarding what could be done in avformat/tee.
>
> Currently, the tee muxer works in a simple way, incoming packets are 
> just iteratively fed to several output muxers (each muxer blocking the 
> next). Also there is no possibility to reset muxer on error.
>
> My current idea is to create queue for each output (as Marton suggested) 
> and process each output in separate thread. I was also considering using 
> just single queue, but since the AVPackets are referenced counted, the 
> memory overhead is negligible and using multiple queues will simplify 
> the code. Apart from getting advantage of non-blocking processing with 
> multiple slave muxers, error handling will also be improved.
>
> The option allowing to ignore failure on certain outputs is already 
> implemented (this allows for example network streaming to continue even 
> after disk fills up, or recording to file to continue when network error 
> occurs). In the final solution the tee muxer will also support 
> restarting failed output. There is a question how to deal with restart, 
> there are several options what to do and this could be also configurable 
> for the user (with reasonable default set):
> (Does these options make sense to you? Do you have ideas for more? )
>     - Attempt restart immediately after failure, if it doesn't succeed 
> attempt with the
> next packet (keyframe). Repeat  <k>(argument) times before giving up on 
> that output.
>     - Attempt restart after certain time <t>(argument).
>     - Combination of two options above. Attempt to recover with next 
> keyframes, after several failures wait for some amount of time and 
> attempt again.

I think a per-output restart_delay with an 1 second default, and a 
per-output restart_with_keyframe boolean flag defaulting to true would be 
simple yet powerful enough.

> Another question is what to do when some of the queues becomes full, 
> discussed options were so far:
>     - Block write_packet call until the queue frees - this might be 
> useful when producer is faster than consumer, and we don't want to drop 
> any packets when recording to file.
>     - Drop some yet unprocessed packets (until next keyframe, or free 
> some portion of queue) to free the queue - this might be useful for 
> network outputs.

I suggest a per-output "blocking" boolean flag to specify outputs, where - 
in case of a full queue - writing is blocked. A per-output queue_size 
setting might be useful as well.

If a queue is full, but non-blocking, then the producer should set an 
overflow flag of that queue, and not push any further data to the queue, 
unless the overflow flag is cleared.

The consumer should be responsible to clear the overflow flag. If the 
consumer feels itself ready to receive packets, but an overflow 
happened, then it should probably want to drop all existing packets in its 
queue and clear the flag after that. This way if the consumer is only 
slightly slower than the producer, the packet losses will be bursty, and 
this also reduce the chance to operate with an always almost-full queue 
causing unnecessary latency and memory usage.

Once the consumer is receiving packets again, it should wait for the first 
keyframe (if restart_with_keyframe is set) and start processing packets 
from that.

>
> I'm thinking of implementing this queue by wrapping up AVFifoBuffer 
> (similarily than AVThreadMessageQueue does but with the configurable 
> behaviour as described above).

Exactly what behaviour is missing from AVThreadMessageQueue? Isn't there a 
chance to extend that, or implement all additional logic on top of it?

>
> If you have any ideas or notes regarding what would be good to do in 
> libavformat/tee and want to join discussion, I'll be glad to take them 
> into account and improve the proposed project.
>

Great, thanks.

Regards,
Marton