[Ffmpeg-devel] lavc decoding of network transmitted frames - packet loss

[leon zadorin]

Hi all

In short, I am using lavc to
encode video frames (e.g.  avcodec_encode_video)
and then transmitting the encoded data over
network (evenly dividing the data stream into UDP
packets). Transmitted data is then read by the
receiving code and frames are reconstructed prior
to being passed to avcodec_decode_video... Of
course, the problem is when some UDP packets are
lost on their way to the receiver.

Ideally I would like to be able to know which
parts of the partially decoded frame (i.e. which
MACROBLOCKS) were decoded OK and then use those in
combination with OTHER macroblocks from OTHER
FRAMES to reconstruct an image without any bad
areas (i.e. this final image might have
macroblocks from different times/frames, but all
of such macroblocks represent a correctly decoded
data at some point in time).

My next few questions are as follows:

1) is there a way to register an error_callback
with lavc during a decoding process so that lavc
will call it when it detects an error on a given
macroblock. For example, when decoding various
H263 or mpeg2 streams there are numerous printouts
of text similar to: "Error on MB 123"... it would
be nice to be able to place a hook into this
functionality (this way client app can build a
vector of invalid macroblocks and then do not use
corresponding image areas from the decoded image).
In other words, a function  like:
"void error_callback (int mvNumber)"...  this
could be matched by an accessor to also get the
macroblock dimensions (width and height)...

2) I know that we can use rtp callbacks during
encoding to determine the explicit number of
macroblocks of a given payload packet prior to
sending it over the network. But during the
decoding - is there a way to tell the decoder
which macroblocks it should be decoding (as
opposed to decoding the whole, possibly corrupt,
frame with avcodec_decode_video)... For example,
if I had received only the macroblocks 1, 2, 3 and
5 (thus macroblock 4 has been lost) - is there a
way for me to present the decoder with only the
data for macroblocks 1, 2, 3, and 5 and tell it to
decode those? In other words, if I am able to tell
which macroblocks were lost during the
transmission of the network, how could I
communicate this information to lavc decoding
processes?

Lastly, if there are already some alternative and
well established mechanisms in lavc which
effectively deal with packet loss of video data
trasmitted over the network (i.e. they maintain
the minimum of erroneous macroblocks being
contained by the decoded, finally reconstructed
images) - then I would certainly appreciate
hearing about those :-)

Thanks for your time in this matter.

Kind regards,
Leon.