[MEncoder-users] mcdeint=2 segfault

[Reimar Döffinger]

On Sun, Mar 29, 2009 at 01:24:55PM +0200, Christian Ebert wrote:
> * Christian Ebert on Saturday, March 28, 2009 at 23:03:41 +0100
> > * Reimar Döffinger on Saturday, March 28, 2009 at 19:41:31 +0100
> >> Well, it works without issues for me using latest SVN MPlayer on x86_64
> >> Linux. Not even valgrind complains.
> > 
> > $ mplayer -vf yadif=3,mcdeint=2:1:10,framestep=2,softskip,scale,harddup test.mov
> > MPlayer SVN-r29079-4.0.1 (C) 2000-2009 MPlayer Team
> 
> <snip>
>  
> > MPlayer interrupted by signal 11 in module: decode video
> > - MPlayer crashed by bad usage of CPU/FPU/RAM.
> >  Recompile MPlayer with --enable-debug and make a 'gdb' backtrace and
> >  disassembly. Details in DOCS/HTML/en/bugreports_what.html#bugreports_crash.
> > - MPlayer crashed. This shouldn't happen.
> >  It can be a bug in the MPlayer code _or_ in your drivers _or_ in your
> >  gcc version. If you think it's MPlayer's fault, please read
> >  DOCS/HTML/en/bugreports.html and follow the instructions there. We can't and
> >  won't help unless you provide this information when reporting a possible bug.
> > 
> > 
> > Will compile with --enable-debug when I have time to investigate.
> 
> Done with --enable-debug=3 and following the instructions from
> http://www.mplayerhq.hu/DOCS/HTML/en/bugreports_what.html#bugreports_crash
> 
> The backtrace is here:
> 
> http://bugzilla.mplayerhq.hu/show_bug.cgi?id=1426#c1

The bug is in the snow encoder, given the bad state it is in overall I
am not sure if using mccdeint is a good idea.
You can try attached patch, though I have not verified it is correct.
Ideally you should try reproducing this issue with FFmpeg when encoding
to snow (-vcodec snow) and then file a bug report on their bug tracker.
-------------- next part --------------
Index: libavcodec/snow.c
===================================================================
--- libavcodec/snow.c	(revision 18214)
+++ libavcodec/snow.c	(working copy)
@@ -2859,8 +2859,14 @@
             }
         }
     }else{
+        uint8_t *srctmp = cur;
+        uint8_t *dsttmp = cur + block_w*2*block_w*2;
         assert(block_w==8);
-        distortion = s->dsp.me_cmp[0](&s->m, src + sx + sy*ref_stride, dst + sx + sy*ref_stride, ref_stride, block_w*2);
+        for (y = 0; y < block_w*2; y++) {
+            memcpy(srctmp + y * block_w*2, src + sx + (sy+y)*ref_stride, block_w*2);
+            memcpy(dsttmp + y * block_w*2, dst + sx + (sy+y)*ref_stride, block_w*2);
+        }
+        distortion = s->dsp.me_cmp[0](&s->m, srctmp, dsttmp, block_w*2, block_w*2);
     }
 
     if(plane_index==0){