[Libav-user] avcodec_decode_video2 strange behaviour on lost frame

[serdel]

Actually I yet I have no idea why do the frames get corrupted (at least
this is what I suspect..). When I saw the packet loss it was obvious
because it was enough for the whole frame to be dropped by live555. But why
there is no packet loss and still the frame does not get decoded... that is
still a riddle. However reinitialising the coded at this unusual situation
seems to help, but still I need more testing here.

On Wed, 28 Nov 2012 21:15:07 +0200, Alex Cohn <alexcohn at netvision.net.il>
wrote:
> On 28 Nov 2012 20:21, "serdel" <serdel at toya.net.pl> wrote:
>>
>> I implemented a simple algorithm which kept track of previous frame and
> the
>> current one and checked if the sequence is correct. If it wasn't I
didn't
>> put the frame to the avcodec_decode_video2 function and stayed that way
>> until next sequence has started. It brought the desired results until
one
>> unlucky moment where there was no missing packets in live555, but the
>> main
>> frame still got decoded incorrectly (positive return value and 0 as
>> got_picture_ptr) and the same situation all over again. Why doesn't
>> flushing the buffer help here? I am now thinking about reinitializing
the
>> whole ffmpeg on that occurrence, but it seems brutal...
>>
>>
>> On Wed, 28 Nov 2012 18:48:45 +0200, Alex Cohn
<alexcohn at netvision.net.il>
>> wrote:
>> > On Wed, Nov 28, 2012 at 2:27 PM, serdel <serdel at toya.net.pl> wrote:
>> >
>> >> Hi,
>> >>
>> >> I am using ffmpeg to decode an rtps (live555) streaming. I decode
each
>> >> frame separately in a standard sequence. The normal sequence is SPS
>> >> and
>> >> PPS, then NAL_IDR_SLICE and finaly NAL_SLICE until next SPS&PPS. On
>> >> normal
>> >> work, avcodec_decode_video2 returns positive value of bytes used and
>> sets
>> >> the got_picture_ptr to the positive value. On SPS&PPS the return
value
>> is
>> >> -1 and got_picture_ptr is 0. This is a normal sequence and works
>> >> perfectly
>> >> without any errors or lags. The problem appears when some packet is
> lost
>> >> in
>> >> frame transmission (WiFi transmission with some noise around).
Live555
>> >> rejects the rest of the packets for this frame so the entire frame is
>> >> lost.
>> >> In almost all of the cases it is the  NAL_IDR_SLICE frame since it is
>> >> usually about 20 times bigger than NAL_SLICE frames. So when the next
>> >> NAL_SLICE frame arrives complete the avcodec_decode_video2 returns
>> >> positive
>> >> value of used bytes, but the got_picture_ptr is 0 - not a normal
>> >> behaviour.
>> >> The worse thing is that this continues to happen to every next
> NAL_SLICE
>> >> frame and to the next NAL_IDR_SLICE with it's following NAL_SLICE
>> frames.
>> >> This keeps on going a couple of 'full seqences' (SPS&PPS,NAL_IDR_SLIC
>> and
>> >> following NAL_SLICE) and finally on some of the frames jumps bakc to
>> >> normal
>> >> work (avcodec_decode_video2 return is positive and got_picture_ptr is
>> >> also
>> >> non-zero positive). On this occur the screen picture freezes for 2-3
>> >> seconds and continuous running with no jump_over! You don't see any
> jump
>> >> in
>> >> the screen which would indicate some lost frames - no, the picture
> keeps
>> >> on
>> >> from where it froze and is just delayed to the sender stream by those
>> 2-3
>> >> seconds.
>> >>
>> >> I tried detecting this situation and when it occurs I run
>> >> avcodec_flush_buffers() followed by  avcodec_decode_video2 with a
>> >> dummy
>> >> empty AVPacket to somehow reset the coded state but it didn't help.
> Does
>> >> any one have any other suggestions?
>> >>
>> >
>> > It looks as if the decoder expects an I-frame after SPS&PPS, so it
>> refuses
>> > to accept P-frames. This does not readily explain your finding that it
>> > takes a couple of "full sequences", but this is at least some start.
>> >
>> > The remedy then would be to delay feeding SPS&PPS to decoder until you
>> have
>> > a complete I-frame from RTPS. If LIVE555 throwed this frame away, then
>> > throw away SPS&PPS and proceed with the following frames.
>> >
>> > Note that H264 stream is not error-resilient, and you can see all
kinds
>> of
>> > artifacts when some frames are missing. Things can become especially
>> nasty
>> > if the stream uses B-frames (not in 'Ultrafast' profile). But at least
>> this
>> > way you will hopefully not have the movie freeze for few seconds.
>> >
>> > Sincerely,
>> > Alex
>>
> 
> So, you sometimes receive frames that fail to decode. How often does it
> happen? And what is your GOP length and FPS?
> 
> BR,
> Alex