[Libav-user] RTP Opus demuxer is missing extradata with Opus header

[Carl Eugen Hoyos]

Am Sa., 31. Okt. 2020 um 18:57 Uhr schrieb Jonathan Baudanza <jon at jonb.org>:
>
> On Sat, Oct 31, 2020, at 5:17 AM, Carl Eugen Hoyos wrote:
>
> > I am trying to write the contents of an RTP-OPUS stream to an OGG-OPUS file. The purpose is to send a WebRTC session to Google's Speech-to-text API, which accepts OGG-OPUS as input.
> >
> > The opus muxer requires that an opus header be present in extradata. It seems that the demuxer from my RTP stream doesn't include this.
> >
> > If I construct the opus header myself and insert it into extradata, everything works great. This seems like a hack though. Is this something that the RTP demuxer should be doing?
>
> At least channel count and sample rate are supposed to be read
> from the SDP file.

Correction:
Only the channel count should be read, the sample rate is special for opus.

> Yes, indeed the sample rate and channel count is read by the sdp demuxer, specifically in rtsp.c/sdp_parse_rtpmap(). This function assigns these values to codecpar->sample_rate and codecpar->channels. The problem is that the opus muxer expects codecpar->extradata to also be set with a valid opus header. See oggenc.c/ogg_init, lines 500-504.
>
> My question is, where is the appropriate place for extradata to be assigned?
>
> 1) Should it be done by the user? This is what I'm doing I'm done now. It feels like a hack and isn't possible unless you're using the C API.
>
> 2) Should it be done in the demuxer somewhere? ff_rtp_parse_open in rtpdec.c does some codec specific changes to codecpar. Maybe this is an appropriate place to build an opus header for extradata.
>
> 3) Should it be done in the muxer? Instead of throwing an error, maybe ogg_init() should create a header based on the sample_rate and channel count in codecpar. It can use some sensible defaults for the rest.
>
> 4) Carl, you had a patch about a year ago to libavcodev/opus.c that looks like it addressed a similar issue.
> https://ffmpeg.org/pipermail/ffmpeg-devel/2019-September/250177.html
> It looks like the patch never landed because of a regression with webm playback. I noticed that the patch doesn't seem to assign the sample rate in the opus header (at offset 12), only the channel count (at offset 9). Maybe thats why?

Opus is always 48kHz.

I don't remember why there was an issue but does the patch help in your case?

There is something wrong with your quoting, Carl Eugen