[FFmpeg-cvslog] avdevice/decklink_enc: don't take for granted that first frame to decklink output will be PTS 0

[Devin Heitmueller]

ffmpeg | branch: master | Devin Heitmueller <devin.heitmueller at ltnglobal.com> | Fri Mar  3 16:08:13 2023 -0500| [8fd345f0180c655d18071d0445d797f27bb1efe7] | committer: Marton Balint

avdevice/decklink_enc: don't take for granted that first frame to decklink output will be PTS 0

The existing code assumed that the first frame received by the decklink
output would always be PTS zero.  However if running in other timing
modes than the default of CBR, items such as frame dropping at the
beginning may result in starting at a non-zero PTS.

For example, in our setup because we discard probing data and run
with "-vsync 2" the first video frame scheduled to the decklink
output will have a PTS around 170.  Scheduling frames too far into
the future will either fail or cause a backlog of frames scheduled
far enough into the future that the entire pipeline will stall.

Issue can be reproduced with the following command-line:

./ffmpeg -copyts -i foo.ts -f decklink -vcodec v210 -ac 2  'DeckLink Duo (4)'

Keep track of the PTS of the first frame received, so that when
we enable start playback we can provide that value to the decklink
driver.

Thanks to Marton Balint for review and suggestion to use
AV_NOPTS_VALUE rather than zero for the initial value.

Signed-off-by: Devin Heitmueller <dheitmueller at ltnglobal.com>
Signed-off-by: Marton Balint <cus at passwd.hu>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=8fd345f0180c655d18071d0445d797f27bb1efe7
---

 libavdevice/decklink_common.h | 1 +
 libavdevice/decklink_enc.cpp  | 8 ++++++--
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h
index 79d6ac5b38..088e165ee7 100644
--- a/libavdevice/decklink_common.h
+++ b/libavdevice/decklink_common.h
@@ -118,6 +118,7 @@ struct decklink_ctx {
 
     /* Status */
     int playback_started;
+    int64_t first_pts;
     int64_t last_pts;
     unsigned long frameCount;
     unsigned int dropped;
diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp
index fb686b9032..5a435ddc78 100644
--- a/libavdevice/decklink_enc.cpp
+++ b/libavdevice/decklink_enc.cpp
@@ -486,6 +486,9 @@ static int decklink_write_video_packet(AVFormatContext *avctx, AVPacket *pkt)
     ctx->frames_buffer_available_spots--;
     pthread_mutex_unlock(&ctx->mutex);
 
+    if (ctx->first_pts == AV_NOPTS_VALUE)
+        ctx->first_pts = pkt->pts;
+
     /* Schedule frame for playback. */
     hr = ctx->dlo->ScheduleVideoFrame((class IDeckLinkVideoFrame *) frame,
                                       pkt->pts * ctx->bmd_tb_num,
@@ -505,14 +508,14 @@ static int decklink_write_video_packet(AVFormatContext *avctx, AVPacket *pkt)
                " Video may misbehave!\n");
 
     /* Preroll video frames. */
-    if (!ctx->playback_started && pkt->pts > ctx->frames_preroll) {
+    if (!ctx->playback_started && pkt->pts > (ctx->first_pts + ctx->frames_preroll)) {
         av_log(avctx, AV_LOG_DEBUG, "Ending audio preroll.\n");
         if (ctx->audio && ctx->dlo->EndAudioPreroll() != S_OK) {
             av_log(avctx, AV_LOG_ERROR, "Could not end audio preroll!\n");
             return AVERROR(EIO);
         }
         av_log(avctx, AV_LOG_DEBUG, "Starting scheduled playback.\n");
-        if (ctx->dlo->StartScheduledPlayback(0, ctx->bmd_tb_den, 1.0) != S_OK) {
+        if (ctx->dlo->StartScheduledPlayback(ctx->first_pts * ctx->bmd_tb_num, ctx->bmd_tb_den, 1.0) != S_OK) {
             av_log(avctx, AV_LOG_ERROR, "Could not start scheduled playback!\n");
             return AVERROR(EIO);
         }
@@ -559,6 +562,7 @@ av_cold int ff_decklink_write_header(AVFormatContext *avctx)
     ctx->list_formats = cctx->list_formats;
     ctx->preroll      = cctx->preroll;
     ctx->duplex_mode  = cctx->duplex_mode;
+    ctx->first_pts    = AV_NOPTS_VALUE;
     if (cctx->link > 0 && (unsigned int)cctx->link < FF_ARRAY_ELEMS(decklink_link_conf_map))
         ctx->link = decklink_link_conf_map[cctx->link];
     cctx->ctx = ctx;