[FFmpeg-devel] VP9 RTP encoder and WebRTC

[Jan Ekström]

Hi,

On Sat, Jun 1, 2019 at 8:35 PM Alex Protasenko <aprotasenko at bkmks.com> wrote:
>
> Hello,
>
>
> I'm trying to play some realtime video sources (web/IP cam) using WebRTC
> in a browser. I'm sending RTP stream via Janus gateway using VP9 codec,
> hardware transcoded using ffmpeg.
>
> Everything works fine except random frame corruption happening around
> moving objects, portions of frame "flowing off the screen" and such
> until next keyframe fixes it. This happens consistently especially at
> higher framerates.
>
> It turns out the issue  could be narrowed down to the VP9 RTP
> packetizer. The problem is it's not marking P frames vs I frames in the
> VP9 payload descriptor octet (the P bit). Gstreamer does that and
> doesn't experience any such corruption issues.
>
> I added this simple change and now WebRTC plays any stream 100% solid
> and corruption free for me.
>
>
> Could somebody implement this simple fix in the upstream. Basically
> in libavformat/rtpenc_vp9.c add something to the effect of the following
> two lines (to set the P bit for all but I frames):
>
>      /* mark the first fragment */
>      *rtp_ctx->buf_ptr++ = 0x08;
>
> +    if (!keyframe) {
> +        rtp_ctx->buf[0] |= 0x40;
>
>
> Where the "keyframe" is an additional boolean parameter to the
> ff_rtp_send_vp9 function which could be called as:
>
> ff_rtp_send_vp9(s1, pkt->data, size, pkt->flags & AV_PKT_FLAG_KEY);
>

Is https://tools.ietf.org/html/draft-ietf-payload-vp9-06#page-7 the
correct specification for VP9 in RTP?

Best regards,
Jan