[FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Marton Balint]

On Sat, 31 Aug 2019, Gyan wrote:

>
>
> On 31-08-2019 04:14 AM, Marton Balint wrote:
>>
>>
>> On Fri, 30 Aug 2019, Gyan wrote:
>>
>>>
>>>
>>> On 22-08-2019 01:10 AM, Marton Balint wrote:
>>>>
>>>>
>>>> On Tue, 20 Aug 2019, Devin Heitmueller wrote:
>>>>
>>>>>> A couple of follow-up Qs:
>>>>>>
>>>>>> Is auto-detection available for all Decklink devices?
>>>>>
>>>>> No, but AFAIK it is for all devices which support SDI. Generally it's
>>>>> the older analog capture devices which don't support it.
>>>>>
>>>>>> For those for which it is available, are there any edge cases in 
>>>>>> which
>>>>>> it sets inaccurate mode?
>>>>>
>>>>> I don't trust the existing detection code enough to use it in
>>>>> production.  It often fails to detect and thus ffmpeg will exit at
>>>>> startup.
>>>>
>>>> Can this be fixed by simply increasing the timeout from 1 sec to 2 
>>>> seconds?
>>>>
>>>>> Also, there are cases where it will misdetect 1080i59 as
>>>>> 1080p30 depending on the card.  It's been on my TODO list for a while
>>>>> to make that code more robust (I believe I know what most of the
>>>>> issues are), but it hasn't been critical for any of my use cases.
>>>>
>>>> Hmm, interesting... When you say the issue is card-dependant, does 
>>>> this mean card _model_ dependant or that the issue can affect one 
>>>> card and not the other with of the same model/fw?
>>>
>>> If an user can't or won't use format auto detection, then there is no 
>>> way to reliably obtain the correct timecode without this patch or 
>>> some similar hack.
>>>
>>> Maybe the offset-related ops could be guarded by a check for auto 
>>> detection, even though the patch won't hurt even if it was used in 
>>> that case.
>>
>> I am still not liking this too much because we are guessing some 
>> timecode which we never actually saw, and the guessing is based on the 
>> number of the VideoFrameArrived callbacks. Yes, usually it is called 
>> once for every frame, but some frames might not contain audio, also 
>> there are some examples in the SDK docs where only an audio frame (and 
>> no video frame) is available in the callback.
>
> Yes, that's why there is a new counter, incremented only upon getting a 
> video frame and reset until the demuxer successfully queues the packet 
> of the first video frame. I just noticed an issue with it, but I can 
> change that.
>
>
>> So it is up to the receiving application to do something to keep audio 
>> and video in sync, it either drops/duplicates video or audio. If it 
>> drops/dups video then our guessed start timecode can be wrong even if 
>> we were only counting the video frames.
>
> By receiving app, do you mean the Decklink driver   or 
> decklink_dec/ffmpeg.c?

ffmpeg.c

>
>
>> So I think that if a workaround/hack is needed, we should do that 
>> differently. One idea is to introduce an option which makes the code 
>> drop video till the first valid video frame, or the first valid 
>> timecode. But if autodetection works (or it can be fixed), then I just 
>> don't see too much use even for this.
>
> As Ilinca's tests showed, without auto format detection, the stored 
> timecode is *wrong*. That's a bug so some change is *required*.

The bug is that decklink_dec sets a timecode to AVStream metadata which is 
not the timecode of the first frame. As far as I recall when 
accepting the decklink timecode support patch, I pointed out that it is 
not very useful because it might not be the TC of the first frame.

I am fine with a patch which only sets AVStream timecode if it is indeed 
the first frame. I am fine with a patch which drops starter frames without 
timecode when some option is set. I am not fine with a patch which does 
calculations in deckink_dec to guess a starter timecode.

Regards,
Marton