[FFmpeg-devel] [PATCH] avdevice/decklink_dec: Extract 1080i and NTSC VANC

Tue Jan 23 05:20:19 EET 2018

Hi Ray,

Thanks for your patch.  A few questions:

Could you confirm whether you are capturing via SDI or analog (i.e. composite/component)?  Also what is the capturing device and SDK version you are using?  I’ve found various bugs in the numbering of VANC lines in some cards, particularly with interlaced formats, and it would be good to understand if perhaps this is a card-specific issue.

> On Jan 20, 2018, at 12:33 PM, Ray Tiley <raytiley at gmail.com> wrote:
> 
> This changes the vertical blanking lines extracted for NTSC and 1080i
> resolutions that in personal testing were required to extract closed
> caption data from the decklink video frames.
> 
> Additionally NTSC resolutions have the vanc data interleved between the uyvy
> and not just the luma as in high definition resolutions.
> 
> In my testing this allows a decklink card encoding valid NTSC and 1080i
> closed captions to pass the caption data to the x264 encoder.
> 
> Signed-off-by: Ray Tiley <raytiley at gmail.com>
> ---
> libavdevice/decklink_dec.cpp | 37 ++++++++++++++++++++++++++++++++-----
> 1 file changed, 32 insertions(+), 5 deletions(-)
> 
> diff --git a/libavdevice/decklink_dec.cpp b/libavdevice/decklink_dec.cpp
> index 94dae26..bceced5 100644
> --- a/libavdevice/decklink_dec.cpp
> +++ b/libavdevice/decklink_dec.cpp
> @@ -67,8 +67,7 @@ typedef struct VANCLineNumber {
>  * another source during switching*/
> static VANCLineNumber vanc_line_numbers[] = {
>     /* SD Modes */
> -
> -    {bmdModeNTSC, 11, 19, 274, 282},
> +    {bmdModeNTSC, 4, 21, 24, 284},

I hadn’t previously reviewed this table, but now that I look at it, I’m not confident your proposed patch is correct.  VANC data generally cannot appear until after the switching line (through the first line of active video), which is on line 10 for NTSC 480i video.  Could you elaborate on what equipment you have which is putting VBI data out on lines 4-10?  Agreed though that the upper limit for field one is incorrect - it should be 21 as your patch proposed.

Also, it’s highly unlikely that you really want to search line 24-284 for field 2 VBI data.  Perhaps you meant 274-284?

FYI:  SMPTE RP 168-2009 is the normative reference for the location of switching lines across various video resolutions/framerates.

>     {bmdModeNTSC2398, 11, 19, 274, 282},
>     {bmdModePAL, 7, 22, 320, 335},
>     {bmdModeNTSCp, 11, -1, -1, 39},
> @@ -82,7 +81,7 @@ static VANCLineNumber vanc_line_numbers[] = {
>     {bmdModeHD1080p2997, 8, -1, -1, 42},
>     {bmdModeHD1080p30, 8, -1, -1, 42},
>     {bmdModeHD1080i50, 8, 20, 570, 585},
> -    {bmdModeHD1080i5994, 8, 20, 570, 585},
> +    {bmdModeHD1080i5994, 6, 30, 568, 595},

For 1080i the switching line is line 7, hence VBI should appear on line 8+.  It’s possible for data to appear on line 7 if the transmitting equipment is misconfigured, and only some cards can actually capture that data (for example, the Decklink Duo cannot but the Decklink Duo2 can).

Again, what is the signal source?

>     {bmdModeHD1080i6000, 8, 20, 570, 585},
>     {bmdModeHD1080p50, 8, -1, -1, 42},
>     {bmdModeHD1080p5994, 8, -1, -1, 42},
> @@ -92,7 +91,7 @@ static VANCLineNumber vanc_line_numbers[] = {
> 
>     {bmdModeHD720p50, 8, -1, -1, 26},
>     {bmdModeHD720p5994, 8, -1, -1, 26},
> -    {bmdModeHD720p60, 8, -1, -1, 26},
> +    {bmdModeHD720p60, 7, -1, -1, 26},

Same questions as with the 1080i change above.  Also, switching line location is the same for 720p/5994 and 720p/60, so presumably we would want to adjust both if it is actually needed.

> 
>     /* For all other modes, for which we don't support VANC */
>     {bmdModeUnknown, 0, -1, -1, -1}
> @@ -149,6 +148,30 @@ static void extract_luma_from_v210(uint16_t *dst, const uint8_t *src, int width)
>     }
> }
> 
> +static void unpack_v210(uint16_t *dst, const uint8_t *src, int width)
> +{
> +    int i;
> +    for (i = 0; i < width / 6; i++) {
> +        *dst++ =  src[0]       + ((src[1] & 3)  << 8);
> +        *dst++ = (src[1] >> 2) + ((src[2] & 15) << 6);
> +        *dst++ = (src[2] >> 4) + ((src[3] & 63) << 4);
> +
> +        *dst++ =  src[4]       + ((src[5] & 3)  << 8);
> +        *dst++ = (src[5] >> 2) + ((src[6] & 15) << 6);
> +        *dst++ = (src[6] >> 4) + ((src[7] & 63) << 4);
> +
> +        *dst++ =  src[8]       + ((src[9] & 3)  << 8);
> +        *dst++ = (src[9] >> 2) + ((src[10] & 15) << 6);
> +        *dst++ = (src[10] >> 4) + ((src[11] & 63) << 4);
> +
> +        *dst++ =  src[12]       + ((src[13] & 3)  << 8);
> +        *dst++ = (src[13] >> 2) + ((src[14] & 15) << 6);
> +        *dst++ = (src[14] >> 4) + ((src[15] & 63) << 4);
> +
> +        src += 16;
> +    }
> +}
> +
> static uint8_t calc_parity_and_line_offset(int line)
> {
>     uint8_t ret = (line < 313) << 5;
> @@ -741,7 +764,11 @@ HRESULT decklink_input_callback::VideoInputFrameArrived(
>                         uint8_t *buf;
>                         if (vanc->GetBufferForVerticalBlankingLine(i, (void**)&buf) == S_OK) {
>                             uint16_t luma_vanc[MAX_WIDTH_VANC];
> -                            extract_luma_from_v210(luma_vanc, buf, videoFrame->GetWidth());
> +                            if (ctx->bmd_mode == bmdModeNTSC) {
> +                              unpack_v210(luma_vanc, buf, videoFrame->GetWidth());
> +                            } else {
> +                              extract_luma_from_v210(luma_vanc, buf, videoFrame->GetWidth());
> +                            }

So I have to admit I find this bit odd.  For composite video, the VBI data would always be in the luma region (in fact, most VBI decoders will strip out the chroma channel prior to processing).  For SDI, it’s possible that VANC could appear in the luma or chroma channel, but it wouldn’t be be interleaved across both.

I would be really interested to know what the source equipment is here, since I’m seeing a number of things that I believe would violate the various standards.  While I’m a big believer in Postel’s Law, I would really like to know whether this is just some really bad output implementation, and if it’s open source then bugs should be raised against it rather than putting a bunch of hacks into ffmpeg.  I guess it’s also possible the video went through a scaler which scaled the video but preserved the original VBI format (which would be bad but not terribly surprising).

Cheers,

Devin