[MPlayer-dev-eng] [PATCH] fix artifacts from spp on qp 1 MPEG2

[Diego Biurrun]

On Sat, Sep 08, 2007 at 07:19:44PM +0200, Michael Niedermayer wrote:
> On Sat, Sep 08, 2007 at 01:54:14AM -0400, Alexander Strange wrote:
> > MPEG2 blocks at qp 1 get overfiltered by spp, apparently because "qp>>1" 
> > turns it into 0
> > which causes an integer overflow later. Attached patch clips qp at 1.
> > (see straight lines in http://img513.imageshack.us/img513/6021/qp1aq7.png, 
> > sample
> > available after I figure out how to cut out the middle of an MPEG stream)

dd is your friend.

> > I don't know if it's really worth running the filter on qp 1 macroblocks, 
> > anyway...
> >
> > I also tried the attached spp-mpeg2quant.diff, which uses non_linear_qscale 
> > from libavcodec
> > instead of just >>1, but the effect on very high qp is so much stronger it 
> > looks worse than without it.
> > I'm not even sure if that's the theoretically right way to do it.
> > (Plus it causes the table to be duplicated in the binary.)
> >
> > --- libmpcodecs/vf_spp.c	(revision 24361)
> > +++ libmpcodecs/vf_spp.c	(working copy)
> > @@ -402,7 +402,7 @@
> >  				qp= p->qp;
> >  			else{
> >  				qp= qp_store[ (XMIN(x, width-1)>>qps) + (XMIN(y, height-1)>>qps) * qp_stride];
> > -				if(p->mpeg2) qp>>=1;
> > +				if(p->mpeg2) qp = FFMAX(1, qp>>1);
> >  			}
> >  			for(i=0; i<count; i++){
> >  				const int x1= x + offset[i+count-1][0];
> 
> ok

Patch applied.

Diego