[Libav-user] avcodec_decode_video2 strange behaviour on lost frame

[Alex Cohn]

On 28 Nov 2012 20:21, "serdel" <serdel at toya.net.pl> wrote:
>
> I implemented a simple algorithm which kept track of previous frame and
the
> current one and checked if the sequence is correct. If it wasn't I didn't
> put the frame to the avcodec_decode_video2 function and stayed that way
> until next sequence has started. It brought the desired results until one
> unlucky moment where there was no missing packets in live555, but the main
> frame still got decoded incorrectly (positive return value and 0 as
> got_picture_ptr) and the same situation all over again. Why doesn't
> flushing the buffer help here? I am now thinking about reinitializing the
> whole ffmpeg on that occurrence, but it seems brutal...
>
>
> On Wed, 28 Nov 2012 18:48:45 +0200, Alex Cohn <alexcohn at netvision.net.il>
> wrote:
> > On Wed, Nov 28, 2012 at 2:27 PM, serdel <serdel at toya.net.pl> wrote:
> >
> >> Hi,
> >>
> >> I am using ffmpeg to decode an rtps (live555) streaming. I decode each
> >> frame separately in a standard sequence. The normal sequence is SPS and
> >> PPS, then NAL_IDR_SLICE and finaly NAL_SLICE until next SPS&PPS. On
> >> normal
> >> work, avcodec_decode_video2 returns positive value of bytes used and
> sets
> >> the got_picture_ptr to the positive value. On SPS&PPS the return value
> is
> >> -1 and got_picture_ptr is 0. This is a normal sequence and works
> >> perfectly
> >> without any errors or lags. The problem appears when some packet is
lost
> >> in
> >> frame transmission (WiFi transmission with some noise around). Live555
> >> rejects the rest of the packets for this frame so the entire frame is
> >> lost.
> >> In almost all of the cases it is the  NAL_IDR_SLICE frame since it is
> >> usually about 20 times bigger than NAL_SLICE frames. So when the next
> >> NAL_SLICE frame arrives complete the avcodec_decode_video2 returns
> >> positive
> >> value of used bytes, but the got_picture_ptr is 0 - not a normal
> >> behaviour.
> >> The worse thing is that this continues to happen to every next
NAL_SLICE
> >> frame and to the next NAL_IDR_SLICE with it's following NAL_SLICE
> frames.
> >> This keeps on going a couple of 'full seqences' (SPS&PPS,NAL_IDR_SLIC
> and
> >> following NAL_SLICE) and finally on some of the frames jumps bakc to
> >> normal
> >> work (avcodec_decode_video2 return is positive and got_picture_ptr is
> >> also
> >> non-zero positive). On this occur the screen picture freezes for 2-3
> >> seconds and continuous running with no jump_over! You don't see any
jump
> >> in
> >> the screen which would indicate some lost frames - no, the picture
keeps
> >> on
> >> from where it froze and is just delayed to the sender stream by those
> 2-3
> >> seconds.
> >>
> >> I tried detecting this situation and when it occurs I run
> >> avcodec_flush_buffers() followed by  avcodec_decode_video2 with a dummy
> >> empty AVPacket to somehow reset the coded state but it didn't help.
Does
> >> any one have any other suggestions?
> >>
> >
> > It looks as if the decoder expects an I-frame after SPS&PPS, so it
> refuses
> > to accept P-frames. This does not readily explain your finding that it
> > takes a couple of "full sequences", but this is at least some start.
> >
> > The remedy then would be to delay feeding SPS&PPS to decoder until you
> have
> > a complete I-frame from RTPS. If LIVE555 throwed this frame away, then
> > throw away SPS&PPS and proceed with the following frames.
> >
> > Note that H264 stream is not error-resilient, and you can see all kinds
> of
> > artifacts when some frames are missing. Things can become especially
> nasty
> > if the stream uses B-frames (not in 'Ultrafast' profile). But at least
> this
> > way you will hopefully not have the movie freeze for few seconds.
> >
> > Sincerely,
> > Alex
>

So, you sometimes receive frames that fail to decode. How often does it
happen? And what is your GOP length and FPS?

BR,
Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20121128/703c82c4/attachment.html>