[FFmpeg-devel] [PATCH] Limited timecode support for lavd/decklink

[Marton Balint]

On Tue, 10 Jul 2018, Jonathan Morley wrote:

> Hi Marton et al,
>
> I am revisiting this now that I have access to the BlackMagic DeckLink 
> Duo again. I see what made it into master and had a few questions.
>
> 1) Can you please explain more about storing the timecodes as side 
> packet data in each video packet? Basically is there some convention 
> among demuxers and muxers to handle that specific metadata?

In the current implementation per-frame timecode is stored as 
AV_PKT_DATA_STRINGS_METADATA side data, when AVPackets become AVFrames, 
the AV_PKT_DATA_STRINGS_METADATA is automatically converted to entries in 
the AVFrame->metadata AVDictionary. The dictionary key is "timecode".

There is no "standard" way to store per-frame timecode, neither in 
packets, nor in frames (other than the frame side data 
AV_FRAME_DATA_GOP_TIMECODE, but that is too specific to MPEG). Using 
AVFrame->metadata for this is also non-standard, but it allows us to 
implement the feature without worrying too much about defining / 
documenting it.

Also it is worth mentioning that the frame metadata is lost when encoding, 
so the muxers won't have access to it, unless the encoders export it in 
some way, such as packet metadata or side data (they current don't).

>
> 2) Is there any reason not to make a valid timecode track (ala 
> AVMEDIA_TYPE_DATA AVStream) with timecode packets? Would that conflict 
> with the side data approach currently implemented?

I see no conflict, you might implement a timecode "track", but I don't see 
why that would make your life any easier.

>
> I have found that (much as you originally predicted) my original 
> approach relies on what could generously be called a race case. Since I 
> was using the decklink demuxer to feed the movenc format writer I was 
> relying on having set the timecode metadata on the video stream before 
> either mov_init or mov_write_header get called (since those two 
> functions create the timecode track for “free” if they detect the 
> metadata initially). This is not really a deterministic approach and 
> seems to be worse than making a valid timecode stream to begin with.

AVStream->timecode is still set if the first decklink frame has a 
timecode, so anything that worked with your initial patch should work now 
as well. Obvisously this approach has limitations, but it works well for 
some use cases.

>
> What I am trying to understand now is if there is any overlap in 
> responsibility between the video packet side data approach and creating 
> a dedicated timecode data stream. Please let me know what you think.

I don't think there is any overlap, but I am also not aware of any muxer 
which supports timecode tracks, as separate data streams. Not even movenc 
as far as I see.

Regards,
Marton