[Ffmpeg-devel] [PATCH] VC1 init fail and crash

[Ivan Kalvachev]

2006/6/29, Kostya <kostya.shishkov at gmail.com>:
> On Tue, Jun 27, 2006 at 06:34:37PM +0200, Reimar D?ffinger wrote:
> > Hi,
> > attached patches fix two minor problems:
> > 1) vc1 (not wmv3) decoder init fails due to a missing case in h263dec
> > 2) decoder crashes since EMU_EDGE is set for the AVCodecContext but not
> > for MpegEncContext. Feel free to move this initialization someplace
> > else...
>
> Applied (I suspect flag EMU_EDGE does not change anything though).

There are 2 ways to handle motion vectors that are getting outside of
the picture.
1. check if the motion vector is in range and if it is not, fill the
MB with pixels taken from the edge.

2. Make the picture 1 block bigger on all sides. Fill these extra
"frame" blocks with pixels from the edge. When MV is outside the
picture, you copy these extra edged "frame" blocks.

#1 is usually considered slower, because it requires at least 2
additional checks (4 without the unsigned substract trick). On the
other side there is nothing that guarantees that MV in #2 case would
be within the "extended" by MB range (vector could be done larger to
get more optimal code). So #2 still needs checks.

#1 reads 8 pixels and writes 64 one.
#2 reads 64 and writes 64 pixels. (worse bandwidth). Also #2 needs to
fill these extra MB before using Motion Compensation.

So
#1 is EMU_EDGE and is more beneficial for decoding.
#2 is usually preferred by some encoders that perform a lot of random
compares needed by Motion Estamination.