[FFmpeg-user] Segfault in mpdecimate filter when using ssse3/mmx extensions

[Ed Torbett]

> (This doesn't help because your valgrind version
> is too old.)

Compiled latest valgrind from source; here's the output:

[etorbett at ssl26 samples]$ /usr/local/bin/valgrind /ldisk/ffmpeg/ffmpeg_sources/ffmpeg/ffmpeg_g -i combined.mp4 -vf mpdecimate -f null -
==29272== Memcheck, a memory error detector
==29272== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==29272== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info
==29272== Command: /ldisk/ffmpeg/ffmpeg_sources/ffmpeg/ffmpeg_g -i combined.mp4 -vf mpdecimate -f null -
==29272==
ffmpeg version N-41485-gf5d0398 Copyright (c) 2000-2013 the FFmpeg developers
  built on Dec 19 2013 10:39:35 with gcc 4.1.2 (GCC) 20080704 (Red Hat 4.1.2-54)
  configuration: --enable-gpl --enable-libx264
  libavutil      52. 58.101 / 52. 58.101
  libavcodec     55. 45.102 / 55. 45.102
  libavformat    55. 22.100 / 55. 22.100
  libavdevice    55.  5.102 / 55.  5.102
  libavfilter     4.  0.100 /  4.  0.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'combined.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.9.100
  Duration: 00:00:50.08, start: 0.000000, bitrate: 451 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x576 [SAR 16:15 DAR 4:3], 450 kb/s, 12.50 fps, 12.50 tbr, 12800 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.22.100
    Stream #0:0(und): Video: rawvideo (I420 / 0x30323449), yuv420p, 720x576 [SAR 16:15 DAR 4:3], q=2-31, 200 kb/s, 90k tbn, 12.50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> rawvideo)
Press [q] to stop, [?] for help
[null @ 0x44df360] Encoder did not produce proper pts, making some up.
==29272==
==29272== Process terminating with default action of signal 11 (SIGSEGV)
==29272==  General Protection Fault
==29272==    at 0x87175CC: sum_abs_dctelem_ssse3 (dsputilenc_mmx.c:833)
==29272==    by 0x80AF75A: filter_frame (vf_mpdecimate.c:90)
==29272==    by 0x80787B1: ff_filter_frame_framed (avfilter.c:1072)
==29272==    by 0x8078A3F: ff_filter_frame (avfilter.c:1147)
==29272==    by 0x807D38F: request_frame (buffersrc.c:491)
==29272==    by 0x807D975: av_buffersrc_add_frame_internal (buffersrc.c:170)
==29272==    by 0x807DAD9: av_buffersrc_add_frame_flags (buffersrc.c:107)
==29272==    by 0x8063C91: output_packet (ffmpeg.c:1777)
==29272==    by 0x8066DB7: transcode_step (ffmpeg.c:3215)
==29272==    by 0x8069A9B: transcode (ffmpeg.c:3363)
==29272==    by 0x806A994: main (ffmpeg.c:3543)
==29272==
==29272== HEAP SUMMARY:
==29272==     in use at exit: 8,884,572 bytes in 552 blocks
==29272==   total heap usage: 2,283 allocs, 1,731 frees, 12,526,816 bytes allocated
==29272==
==29272== LEAK SUMMARY:
==29272==    definitely lost: 0 bytes in 0 blocks
==29272==    indirectly lost: 0 bytes in 0 blocks
==29272==      possibly lost: 816 bytes in 6 blocks
==29272==    still reachable: 8,883,756 bytes in 546 blocks
==29272==         suppressed: 0 bytes in 0 blocks
==29272== Rerun with --leak-check=full to see details of leaked memory
==29272==
==29272== For counts of detected and suppressed errors, rerun with: -v
==29272== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 31 from 8)
Killed

> I suspect this is a compiler problem, can you easily
> test another version?

On other machines, a windows zeranoe build from 8th December works without crash and the build I used originally (with the 'unusual' configure) works when compiled and run on another architecture (8-core Xeon server, which is the target architecture).

I don't think I can get another version on my PC without compiling it myself, which of course defeats the point of seeing if it's my compiler or my machine. I'm running CentOS 5.7 32bit.