[Libav-user] avcodec_decode_video2 strange behaviour on lost frame

[Alex Cohn]

On Wed, Nov 28, 2012 at 2:27 PM, serdel <serdel at toya.net.pl> wrote:

> Hi,
>
> I am using ffmpeg to decode an rtps (live555) streaming. I decode each
> frame separately in a standard sequence. The normal sequence is SPS and
> PPS, then NAL_IDR_SLICE and finaly NAL_SLICE until next SPS&PPS. On normal
> work, avcodec_decode_video2 returns positive value of bytes used and sets
> the got_picture_ptr to the positive value. On SPS&PPS the return value is
> -1 and got_picture_ptr is 0. This is a normal sequence and works perfectly
> without any errors or lags. The problem appears when some packet is lost in
> frame transmission (WiFi transmission with some noise around). Live555
> rejects the rest of the packets for this frame so the entire frame is lost.
> In almost all of the cases it is the  NAL_IDR_SLICE frame since it is
> usually about 20 times bigger than NAL_SLICE frames. So when the next
> NAL_SLICE frame arrives complete the avcodec_decode_video2 returns positive
> value of used bytes, but the got_picture_ptr is 0 - not a normal behaviour.
> The worse thing is that this continues to happen to every next NAL_SLICE
> frame and to the next NAL_IDR_SLICE with it's following NAL_SLICE frames.
> This keeps on going a couple of 'full seqences' (SPS&PPS,NAL_IDR_SLIC and
> following NAL_SLICE) and finally on some of the frames jumps bakc to normal
> work (avcodec_decode_video2 return is positive and got_picture_ptr is also
> non-zero positive). On this occur the screen picture freezes for 2-3
> seconds and continuous running with no jump_over! You don't see any jump in
> the screen which would indicate some lost frames - no, the picture keeps on
> from where it froze and is just delayed to the sender stream by those 2-3
> seconds.
>
> I tried detecting this situation and when it occurs I run
> avcodec_flush_buffers() followed by  avcodec_decode_video2 with a dummy
> empty AVPacket to somehow reset the coded state but it didn't help. Does
> any one have any other suggestions?
>

It looks as if the decoder expects an I-frame after SPS&PPS, so it refuses
to accept P-frames. This does not readily explain your finding that it
takes a couple of "full sequences", but this is at least some start.

The remedy then would be to delay feeding SPS&PPS to decoder until you have
a complete I-frame from RTPS. If LIVE555 throwed this frame away, then
throw away SPS&PPS and proceed with the following frames.

Note that H264 stream is not error-resilient, and you can see all kinds of
artifacts when some frames are missing. Things can become especially nasty
if the stream uses B-frames (not in 'Ultrafast' profile). But at least this
way you will hopefully not have the movie freeze for few seconds.

Sincerely,
Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20121128/f379e717/attachment.html>