[FFmpeg-devel] rtpdec_h264.c RTP to annex-b patch

[Martin Storsjö]

On Wed, 1 Feb 2012, Matthew Szatmary wrote:

> using -vcodec copy I tried outputting as mp4, mkv, fly and mpegts (with 
> and without -bsf h264_mp4toannexb). None of them worked before this 
> patch. They all work after this patch is applied

Which errors did you get when they didn't work?

The h264_mp4toannexb bsf isn't exactly what's needed in this case - the 
data is in annex b format already, but with SPS/PPS as extradata instead 
of inline in the stream. (This is exactly the same format as libx264 
outputs if encoding for mp4 files, when the global header flag is set, 
FWIW.) Not sure if there currently is any bitstream filter to do exactly 
this conversion (removing extradata and embedding it into the stream) 
though.

In my tests, remuxing from rtp/h264 into mp4 works just fine with the 
current code, try e.g. this:

ffmpeg -allowed_media_types video -rtsp_transport tcp -i 
rtsp://albin.abo.fi:8554/sample_h264_100kbit.mp4 -vcodec copy -an -y 
test.mp4

The video produced with this command currently works better than what I 
get with your patch applied.

However in other cases, I do see the issue you're facing with mkv, but 
I'm not really sure it's a more correct solution even if some cases might 
work better.

Currently, the rtp depacketizer sets timestamps for all packets, but it 
seems that the h264 parser clears this from the first frame, so that it 
has dts=pts=N/A. With the current code, this frame has keyframe=1 set, and 
the mkv muxer will fail to write it with "Can't write packet with unknown 
timestamp". With your patch applied, this frame is marked as keyframe=0 by 
the parser, and ffmpeg.c won't start to try to write the packets to mkv 
until you run into the next keyframe, which has a timestamp. (If you try 
your patch with -copyinkf you'll get the same error again.)

So it seems to me that with SPS/PPS inline, the h264 parser behaves 
slightly differently in what it outputs (especially when it doesn't set 
the keyframe flag for the first packets), which happens to work better 
when muxing.

As for muxing into mpegts, you will probably need either your patch or a 
bsf to move the extradata into the stream though.

// Martin