[FFmpeg-devel] h264_annexbtomp4 filter ideas, was: Collection of patches

Baptiste Coudurier baptiste.coudurier
Thu Apr 24 14:54:54 CEST 2008


Thorsten Jordan wrote:
> Thorsten Jordan schrieb:
>> Michael Niedermayer schrieb:
>>> On Wed, Apr 23, 2008 at 09:15:40AM +0200, Thorsten Jordan wrote:
>>> [...]
>>>> work. Maybe it has been fixed differently meanwhile, but i can't
>>>> imaginge how.
>>> Your patch generates invalid/broken h264. The current code generates broken
>>> ones too but they dont work which prevents them from spreading. Thus your
>>> patch clearly makes it worse, the issue is not fixed it just appears that it
>>> is. The spliting API in ffmpeg has to be changed to allow spliting random
>>> subsets of the headers out. Please see the mailing list where i described
>>> this more precissely IIRC.
>> ah now i see your point. The splitter should thus extract the SPS and
>> PPS only and put it in extradata, and leave the rest of the stream as it
>> is (including the SEI NALs). If there are NALs other than SPS/PPS
>> inside, it would be incompatible to MP4 extradata format, as it expects
>> only SPS and PPS units.
>> So if there is "SPS - SEI - PPS - other/rest..." in the stream, extract
>> "SPS - PPS" to extra data, and keep "SEI - other/rest..." in the stream
>> In the discussion about that problem on this list, you said that the two
>> SEI NALs in my example shouldnt be part of global headers. I checked the
>> h264 spec and there are 22 types of SEIs. None of them sound like they
>> are SPS/PPS related.
>> Which brings up the question if there should be never SEI units in
>> global headers. I found nothing near the SPS/PPS description that
>> mentions SEI, so they are not used as header extension?

IIRC libx264 wrapper puts SEI in extradata.

> If one would solve this by writing a h264_annexbtomp4 bitstream filter,
> there are some issues:
> The global headers (extradata) of MP4 should contain SPS and PPS only,
> but it can contain multiple SPS/PPS afaics.
> This means the bitstream filter has to scan the whole h264 elementary
> stream, rip out every SPS+PPS inside, and collect them in one global
> block. For every SPS/PPS in the stream it must compare it to the last
> found ones to avoid storing the same SPS/PPS twice. I suppose there is a
> limit on the number of SPS/PPS, as they need to be given to the decoder
> all at once on decoding - but if there are more than one SPS/PPS each
> the reverse filter mp4toannexb wouldn't work...
> This means the extradata (global headers) are not ready before the full
> stream has been filtered. I don't know if that is a problem for the mp4
> muxer.

No, mp4 muxer use a "trailer" and therefore computes atom when all
frames have been muxed. However, this might be a problem when stream
copying and using fragmented muxing, but we will see when somebody will
try to implement it.

> Where should the collected headers get stored? Into the
> avcodeccontext.extradata pointer of the given avcodeccontext?


IMHO just be sure to remove duplicate sps/pps when adding them to extradata.

> Should such a filter change the Stream-Mode (with startcodes) to MP4
> package mode also (would be incompatible to current mp4 muxer i guess)?

Yes, and muxer can be changed later. I think you can implement it right
now without breaking muxing.

Baptiste COUDURIER                              GnuPG Key Id: 0x5C1ABAAA
SMARTJOG SAS                                     http://www.smartjog.com
Key fingerprint                 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA
Phone: +33 1 49966312

More information about the ffmpeg-devel mailing list