[MPlayer-dev-eng] [PATCH] fix artifacts from spp on qp 1 MPEG2

[Michael Niedermayer]

Hi

On Sat, Sep 08, 2007 at 01:54:14AM -0400, Alexander Strange wrote:
> MPEG2 blocks at qp 1 get overfiltered by spp, apparently because "qp>>1" 
> turns it into 0
> which causes an integer overflow later. Attached patch clips qp at 1.
> (see straight lines in http://img513.imageshack.us/img513/6021/qp1aq7.png, 
> sample
> available after I figure out how to cut out the middle of an MPEG stream)
>
> I don't know if it's really worth running the filter on qp 1 macroblocks, 
> anyway...
>
> I also tried the attached spp-mpeg2quant.diff, which uses non_linear_qscale 
> from libavcodec
> instead of just >>1, but the effect on very high qp is so much stronger it 
> looks worse than without it.
> I'm not even sure if that's the theoretically right way to do it.
> (Plus it causes the table to be duplicated in the binary.)
>

> Index: libmpcodecs/vf_spp.c
> ===================================================================
> --- libmpcodecs/vf_spp.c	(revision 24361)
> +++ libmpcodecs/vf_spp.c	(working copy)
> @@ -402,7 +402,7 @@
>  				qp= p->qp;
>  			else{
>  				qp= qp_store[ (XMIN(x, width-1)>>qps) + (XMIN(y, height-1)>>qps) * qp_stride];
> -				if(p->mpeg2) qp>>=1;
> +				if(p->mpeg2) qp = FFMAX(1, qp>>1);
>  			}
>  			for(i=0; i<count; i++){
>  				const int x1= x + offset[i+count-1][0];

ok

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Freedom in capitalist society always remains about the same as it was in
ancient Greek republics: Freedom for slave owners. -- Vladimir Lenin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/attachments/20070908/50fce341/attachment.pgp>