[Libav-user] convert MP4 frames to Annex B

[Selmeci, Tamas]

Hi all!

I'm using ffmpeg in our set-top-box to demultiplex MPEG-TS input streams
into single frames. The resulting frames are then decoded by the
processor's (Freescale iMX6 Solo) decoder. By this time I have been
dealing with MPEG-TS only, with MPEG2 or H264-encoded frames inside.

MP4 container support is now also required. The problem with MP4 is that
it doesn't conform to "Annex B" when it comes to storing the H264
frames, so that frames returned by av_read_frame(...) can't be decoded.
The decoder in the processor is very picky and is only capable of
handling Annex B compatible frames.

I've figured out that this can be solved by the command line ffmpeg
util. Let's assume we have a non-conformant MP4, this command fixes it:

ffmpeg -i input.mov \
        -codec:v copy \
        -codec:a copy \
        -bsf:v h264_mp4toannexb \
        -f mp4 \
        -y output.mp4

And the decoder in the processor happily decodes the frames.
--------------------------------------------
I want to implement bitstream filtering in the player in the STB. I've
made some attempts to perform bitstream filtering on-the-fly in the
program, with not too much success. Is there any way to have an AVFrame
bitstream-filtered into Annex B format without too much effort?

If av_bitstream_filter_filter(...) is called with avctx = NULL it
crashes (however the doc says it should work).

Doing something like this simply doesn't filter the input buffer:

int out_size;
uint8_t *out;
AVPacket packet;
AVCodec *codec;
AVCodecContext *codec_ctx;
AVBitStreamFilterContext *annexb;

annexb = av_bitstream_filter_init("h264_mp4toannexb");

av_read_frame(context, &packet);
av_read_frame(context, &packet);
...
codec = avcodec_find_decoder(AV_CODEC_ID_H264);
codec_ctx = avcodec_alloc_context3(codec);
avcodec_open2(codec_ctx, codec, NULL);
av_bitstream_filter_filter(annexb,
    codec_ctx,
    NULL,
    &out,
    &out_size,
    packet.data,
    packet.size,
    packet.flags & AV_PKT_FLAG_KEY);
avcodec_close(codec_ctx);
av_free(codec_ctx);
av_bitstream_filter_close(annexb);

out == packet.data and out_size == packet.size, no filtering is done.

What am I doing wrong? It's mandatory to use the processor's decoder and
ffmpeg's sw decoding must not be used due to speed considerations. All
other stuff (mx/dmx/filtering/etc.) are also allowed to be used.

Thanks, regards,
-- 
Selmeci, Tamás
http://www.open-st.eu/