[FFmpeg-devel] Tee improvement - discussion

[Jan Sebechlebsky]

Hello Marton,
sorry for a delayed reply.

On 05/16/2016 01:20 AM, Marton Balint wrote:
>
> On Wed, 11 May 2016, Jan Sebechlebsky wrote:
>
>> Hi,
>> I'll be working on tee muxer improvement during GSoC 2016 and I 
>> thought maybe it is a good idea to ask about ideas which any of you 
>> might have regarding what could be done in avformat/tee.
>>
>> Currently, the tee muxer works in a simple way, incoming packets are 
>> just iteratively fed to several output muxers (each muxer blocking 
>> the next). Also there is no possibility to reset muxer on error.
>>
>> My current idea is to create queue for each output (as Marton 
>> suggested) and process each output in separate thread. I was also 
>> considering using just single queue, but since the AVPackets are 
>> referenced counted, the memory overhead is negligible and using 
>> multiple queues will simplify the code. Apart from getting advantage 
>> of non-blocking processing with multiple slave muxers, error handling 
>> will also be improved.
>>
>> The option allowing to ignore failure on certain outputs is already 
>> implemented (this allows for example network streaming to continue 
>> even after disk fills up, or recording to file to continue when 
>> network error occurs). In the final solution the tee muxer will also 
>> support restarting failed output. There is a question how to deal 
>> with restart, there are several options what to do and this could be 
>> also configurable for the user (with reasonable default set):
>> (Does these options make sense to you? Do you have ideas for more? )
>>     - Attempt restart immediately after failure, if it doesn't 
>> succeed attempt with the
>> next packet (keyframe). Repeat  <k>(argument) times before giving up 
>> on that output.
>>     - Attempt restart after certain time <t>(argument).
>>     - Combination of two options above. Attempt to recover with next 
>> keyframes, after several failures wait for some amount of time and 
>> attempt again.
>
> I think a per-output restart_delay with an 1 second default, and a 
> per-output restart_with_keyframe boolean flag defaulting to true would 
> be simple yet powerful enough.
>
>> Another question is what to do when some of the queues becomes full, 
>> discussed options were so far:
>>     - Block write_packet call until the queue frees - this might be 
>> useful when producer is faster than consumer, and we don't want to 
>> drop any packets when recording to file.
>>     - Drop some yet unprocessed packets (until next keyframe, or free 
>> some portion of queue) to free the queue - this might be useful for 
>> network outputs.
>
> I suggest a per-output "blocking" boolean flag to specify outputs, 
> where - in case of a full queue - writing is blocked. A per-output 
> queue_size setting might be useful as well.
>
> If a queue is full, but non-blocking, then the producer should set an 
> overflow flag of that queue, and not push any further data to the 
> queue, unless the overflow flag is cleared.
>
> The consumer should be responsible to clear the overflow flag. If the 
> consumer feels itself ready to receive packets, but an overflow 
> happened, then it should probably want to drop all existing packets in 
> its queue and clear the flag after that. This way if the consumer is 
> only slightly slower than the producer, the packet losses will be 
> bursty, and this also reduce the chance to operate with an always 
> almost-full queue causing unnecessary latency and memory usage.
>
> Once the consumer is receiving packets again, it should wait for the 
> first keyframe (if restart_with_keyframe is set) and start processing 
> packets from that.
>
I think I like these ideas, it seems really like an elegant way to do it.
>>
>> I'm thinking of implementing this queue by wrapping up AVFifoBuffer 
>> (similarily than AVThreadMessageQueue does but with the configurable 
>> behaviour as described above).
>
> Exactly what behaviour is missing from AVThreadMessageQueue? Isn't 
> there a chance to extend that, or implement all additional logic on 
> top of it?
>
What is missing is basically just the discussed configurable behaviour 
how to deal with overfilled queue. I originally thought that the queue 
would flush old packets automatically (from the point of view of 
consumer / producer it would be transparent). But if the consumer will 
be responsible for flushing the packets in non-blocking mode I guess the 
AVThreadMessageQueue will do the work. This really simplifies the whole 
task.

I just wonder, is simply flushing the whole buffer good solution? 
Shouldn't keyframe flag be considered (either flush until next 
keyframe(s)), or ignore packet which arrived after flush until new 
keyframe arrives? If so this wouldrequire some additional functionality 
to be added to AVThreadMessageQueue (since it would be no longer related 
to general message queue it should be probably implemented separately).
>>
>> If you have any ideas or notes regarding what would be good to do in 
>> libavformat/tee and want to join discussion, I'll be glad to take 
>> them into account and improve the proposed project.
>>
>
> Great, thanks.
>
> Regards,
> Marton
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Thanks!

Regards,
Jan S.