[Libav-user] Unable to decode H.264 stream received via RTMP

[Григорий]

Alex Cohn <alexcohn at netvision.net.il> писал(а) в своём письме Wed, 25 Apr  
2012 21:59:34 +0400:

> On Apr 25, 2012 8:27 PM, "Григорий" <grishka93 at gmail.com> wrote:
>>
>> Alex Cohn <alexcohn at netvision.net.il> писал(а) в своём письме Wed, 25  
>> Apr
> 2012 17:54:44 +0400:
>>
>>> On Wed, Apr 25, 2012 at 16:45, Григорий <grishka93 at gmail.com> wrote:
>>>>
>>>> Andrey Utkin <andrey.krieger.utkin at gmail.com> писал(а) в своём письме
> Wed,
>>>> 25 Apr 2012 17:29:09 +0400:
>>>>
>>>>
>>>>> 25 апреля 2012 г. 15:52 пользователь Григорий <grishka93 at gmail.com>
>>>>> написал:
>>>>>>
>>>>>>
>>>>>> Hi all.
>>>>>> I've built libffmpeg for Android and am trying to decode H.264  
>>>>>> stream
>>>>>> using
>>>>>> it. I'm receiving that stream via RTMP connection and passing all
> video
>>>>>> packets (excluding first byte that contains flags related to RTMP)  
>>>>>> to
> the
>>>>>> avcodec_decode_video2 function. With H.263 it works perfectly, but
> when I
>>>>>> try to use the same code (only with codec ID changed) for H.264,  
>>>>>> every
>>>>>> call
>>>>>> to avcodec_decode_video2 returns -1 and nothing gets decoded.
>>>>>> Here is the output:
>>>>>
>>>>>
>>>>>
>>>>> Do you know libavformat can read and demux RTMP stream by itself?
>>>>> Seems like problem in your demuxer, because decoding the frames it
>>>>> produces fail.
>>>>>
>>>>
>>>> Yes I know, but I need more than simply receiving and playing, this
> will be
>>>> a kind of VoIP service. My demuxer seems to work fine - I am able to
>>>> successfully decode audio using Speex and video using libavcodec when
> it is
>>>> encoded into H.263. Only problem is with H.264.
>>>
>>>
>>> Packing of h264 into a stream is not trivial. You may find some
>>> curious information at
>>>
> http://livertmpjavapublisher.blogspot.com/2011/06/h264-avcvideopacket.html
>>>
>>> Alex
>>> _______________________________________________
>>> Libav-user mailing list
>>> Libav-user at ffmpeg.org
>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>
>>
>> Thank you for this link. I've made it to parse the decoder configuration
> record, extract SPS, PPS and some other things. I've tried to pass PPS  
> and
> SPS to avcodec_decode_video2 but it still returns -1, prints "no frame!"
> and does not decode anything else. Prepending 0x00000001 before them or
> making AVCDCR as described here
> http://stackoverflow.com/questions/3493742/problem-to-decode-h264-video-over-rtp-with-ffmpeg-libavcodecgive
> the same result.
>
> H264 decoder expects SPS and PPS to arrive together with the first (IDR)
> frame.
>
> Удачи,
> Alex

So I'm doing the following:
- Create a buffer
- Get a video RTMP packet from the queue
- Read 3 fields at the beginning of it - FLV flags (1 byte), AVC packet  
type (1 byte), and CompositionTime (24-bit integer)
- If AVC packet type is 0, parse the rest of the packet as a decoder  
configuration record. Build AVCDCR with PPS, SPS, AVCProfileIndication,  
profile_compatibility and AVCLevelIndication and append it to the buffer.
- If AVC packet type is 1 and there is something in the buffer, append  
packet (usually starts with 0x00000002) to the buffer, pass the buffer  
contents to avcodec_decode_video2 and clear the buffer.
- If the buffer is empty, simply pass the packet to avcodec_decode_video2.
At the moment when I call avcodec_decode_video2 with buffer contents (that  
consist of 2 AVCDCRs and one encoded frame), before lots of "no frame!" it  
prints "Current profile doesn't provide more RBSP data in PPS, skipping".  
I think I've made some progress but am still doing something wrong.