[Libav-user] Decoding single NALUs from x264 programmatically (no luck even with parser), works fine with concatenated NALUs

[Faraz Khan]

Are you using x264? x264 has this awesome option called intra_refresh - set
that to one and use the keyint_max stuff to specify how quickly you want a
intra refresh. It can be as quick as 2-3 times the framerate. With that,
the decoder would magically recover even if you have 100% packet loss :)



On Sat, Jul 6, 2013 at 9:42 AM, Attila Sukosd <attila.sukosd at gmail.com>wrote:

> Hi Faraz,
>
> Thanks a lot for your fast reply! I guess I will try to do it as you say,
> reconstruct the frame out of the arrived NALUs and feed that to the decoder
> at once.
> Did you run some tests on what happens in case some of the NALUs get lost
> due to packet loss?
>
> Best,
>
> Attila
>
>
> -----------------------------------------
> DTU Computing Center - www.cc.dtu.dk
> attila at cc.dtu.dk, gbaras at student.dtu.dk, s070600 at student.dtu.dk
>
>
>
> On Sat, Jul 6, 2013 at 6:35 PM, Faraz Khan <faraz at screenhero.com> wrote:
>
>> Atilla,
>> We tried this and came to the conclusion that ffmpeg is probably not
>> designed to handle a single NALU at a time. You can get the parser to work
>> in a sort of unreliable fashion if you turn on Annex-b headers in your
>> encoder (we were using x264). However the parser may still be unable to
>> read frames. The only reliable way is to not use the parser and feed the
>> entire frame (directly) to ffmpeg decode. For this to work you would need
>> some rtp magic / a jitterbuffer to accumulate these NALUs back into the
>> entire frame. This works just fine with annex-b headers/repeat headers
>> using x264.
>>
>>
>>
>>
>>
>> On Sat, Jul 6, 2013 at 6:49 AM, Attila Sukosd <attila.sukosd at gmail.com>wrote:
>>
>>> Hi guys,
>>>
>>> There was this thread about decoding NALUs one-by-one with
>>> parser2/decode2 and I'm having exactly the same issue.
>>>
>>> I have x264 creating a bunch of NALUs, in annex_b and repeate_headers,
>>> and trying to decode them one by one on the client side. When I concatenate
>>> all the NALUs per frame into one large packet, and feed that directly into
>>> avcodec_decode_video2, everything is fine, the decode is successful.
>>> However, when I try to feed single NALs directly to
>>> avcodec_decode_video2, it spits out a bunch of errors. After searching for
>>> a while on the net, I read some place that certain types of NALs affect the
>>> rest of the data stream, and they need to be group together.
>>> I've tried to run the NALs through av_parser_parse2 and tried to decode
>>> the output buffer when the output size was larger than zero, however, after
>>> looking at the output buffer contents, it seems like while the output size
>>> seems reasonable (around the size of the NALs or a combination of a number
>>> of previous NALs), the output buffer content contains 3-4 bytes, and the
>>> rest is zero.
>>>
>>> The decoding looks like this:
>>>
>>>  ret = av_parser_parse2(decoder->pParserCtx, decoder->pCodecCtx,
>>> decoder->tmp_data, &outsize, buff, size, 0, 0, AV_NOPTS_VALUE);
>>>  printf("%p buff, %p outdata, consumed %d, %d bufsize, %d outsize\n",
>>> buff, decoder->tmp_data,ret, size, outsize);
>>>  disp_buff(decoder->tmp_data, 50);
>>>
>>>  if (outsize <= 0) return 0;
>>>
>>>  decoder->avpkt.flags = AV_PKT_FLAG_KEY;
>>>  decoder->avpkt.data = decoder->tmp_data;
>>>  decoder->avpkt.size = outsize;
>>>
>>>   got_picture = 0;
>>>   if ((ret = avcodec_decode_video2(decoder->pCodecCtx, decoder->pFrame,
>>> &got_picture, &decoder->avpkt)) < 0) {
>>>                 LOGE(__FUNCTION__, "H264 decoding failed!\n");
>>>                 return -1;
>>>   }
>>>
>>>
>>> Could someone shed some light on if I'm really off the right track? or
>>> am I just missing something very trivial?
>>>
>>>
>>> Best Regards,
>>>
>>> Attila
>>>
>>> _______________________________________________
>>> Libav-user mailing list
>>> Libav-user at ffmpeg.org
>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>
>>>
>>
>>
>> --
>>
>> --
>> Faraz Khan
>> Simple Collaboration Screensharing
>> www.screenhero.com
>>
>> _______________________________________________
>> Libav-user mailing list
>> Libav-user at ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/libav-user
>>
>>
>
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user
>
>


-- 

--
Faraz Khan
Simple Collaboration Screensharing
www.screenhero.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20130706/edf6f9d7/attachment.html>