[FFmpeg-user] Segfault in mpdecimate filter when using ssse3/mmx extensions
Ed Torbett
ed.torbett at simulation-systems.co.uk
Thu Dec 19 11:43:00 CET 2013
> I cannot reproduce your crash, valgrind does not report
> anything suspicious.
Here's relevant output of valgrind for me:
vex x86->IR: unhandled instruction bytes: 0xF3 0xF 0xBC 0xF6
==9526== valgrind: Unrecognised instruction at address 0x8a4728e.
==9526== Your program just tried to execute an instruction that Valgrind
==9526== did not recognise. There are two possible reasons for this.
==9526== 1. Your program has a bug and erroneously jumped to a non-code
==9526== location. If you are running Memcheck and you just saw a
==9526== warning about a bad jump, it's probably your program's fault.
==9526== 2. The instruction is legitimate but Valgrind doesn't handle it,
==9526== i.e. it's Valgrind's fault. If you think this is the case or
==9526== you are not sure, please let us know and we'll try to fix it.
==9526== Either way, Valgrind will now raise a SIGILL signal which will
==9526== probably kill your program.
==9526==
==9526== Process terminating with default action of signal 4 (SIGILL)
==9526== Illegal opcode at address 0x8A4728E
==9526== at 0x8A4728E: ??? (in /ldisk/ffmpeg/ffmpeg_sources/ffmpeg/ffmpeg_g)
Full output available at http://edward.torbett.co.uk/valgrind.log
> Could you confirm the md5sum of
> 56f371bc6e11583275d76288dc0ad9c7 for your file?
This is the correct checksum.
> This is at least unusual, is the crash reproducible with:
> $ ffmpeg -i combined.mp4 -vf mpdecimate -f null -
Yes, still crashes
> $ ffmpeg -i combined.mp4 -vf mpdecimate -vcodec mpeg4 out.avi
Still crashes.
> or does a shorter configure line help?
> $ ./configure --enable-gpl --enable-libx264 && make
Still crashes:
$ /ldisk/ffmpeg/ffmpeg_sources/ffmpeg/ffmpeg -v debug -i combined.mp4 -vf mpdecimate decimated.mp4
ffmpeg version N-41485-gf5d0398 Copyright (c) 2000-2013 the FFmpeg developers
built on Dec 19 2013 10:39:35 with gcc 4.1.2 (GCC) 20080704 (Red Hat 4.1.2-54)
configuration: --enable-gpl --enable-libx264
libavutil 52. 58.101 / 52. 58.101
libavcodec 55. 45.102 / 55. 45.102
libavformat 55. 22.100 / 55. 22.100
libavdevice 55. 5.102 / 55. 5.102
libavfilter 4. 0.100 / 4. 0.100
libswscale 2. 5.101 / 2. 5.101
libswresample 0. 17.104 / 0. 17.104
libpostproc 52. 3.100 / 52. 3.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument 'debug'.
Reading option '-i' ... matched as input file with argument 'combined.mp4'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'mpdecimate'.
Reading option 'decimated.mp4' ... matched as output file.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input file combined.mp4.
Successfully parsed a group of options.
Opening an input file: combined.mp4.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xa0d1d80] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xa0d1d80] ISO: File Type Major Brand: isom
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xa0d1d80] Before avformat_find_stream_info() pos: 2826966 bytes read:41094 seeks:1
[h264 @ 0xa0d2760] no picture
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xa0d1d80] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xa0d1d80] After avformat_find_stream_info() pos: 64663 bytes read:105709 seeks:2 frames:1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'combined.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf55.9.100
Duration: 00:00:50.08, start: 0.000000, bitrate: 451 kb/s
Stream #0:0(und), 1, 1/12800: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x576 [SAR 16:15 DAR 4:3], 1/25, 450 kb/s, 12.50 fps, 12.50 tbr, 12800 tbn, 25 tbc (default)
Metadata:
handler_name : VideoHandler
Successfully opened the file.
Parsing a group of options: output file decimated.mp4.
Applying option vf (set video filters) with argument mpdecimate.
Successfully parsed a group of options.
Opening an output file: decimated.mp4.
Successfully opened the file.
detected 2 logical cores
[Parsed_mpdecimate_0 @ 0xa0f2280] max_drop_count:0 hi:768 lo:320 frac:0.330000
[graph 0 input from stream 0:0 @ 0xa0da0a0] Setting 'video_size' to value '720x576'
[graph 0 input from stream 0:0 @ 0xa0da0a0] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0xa0da0a0] Setting 'time_base' to value '1/12800'
[graph 0 input from stream 0:0 @ 0xa0da0a0] Setting 'pixel_aspect' to value '16/15'
[graph 0 input from stream 0:0 @ 0xa0da0a0] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0xa0da0a0] Setting 'frame_rate' to value '25/2'
[graph 0 input from stream 0:0 @ 0xa0da0a0] w:720 h:576 pixfmt:yuv420p tb:1/12800 fr:25/2 sar:16/15 sws_param:flags=2
[format @ 0xa0d9d00] compat: called with args=[yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|nv12|nv16]
[format @ 0xa0d9d00] Setting 'pix_fmts' to value 'yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|nv12|nv16'
[AVFilterGraph @ 0xa0f2980] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
[libx264 @ 0xa0d8ca0] using mv_range_thread = 88
[libx264 @ 0xa0d8ca0] using SAR=16/15
[libx264 @ 0xa0d8ca0] using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
[libx264 @ 0xa0d8ca0] profile High, level 2.2
[libx264 @ 0xa0d8ca0] 264 - core 133 r2334 a3ac64b - H.264/MPEG-4 AVC codec - Copyleft 2003-2013 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=12 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'decimated.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf55.22.100
Stream #0:0(und), 0, 1/12800: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 720x576 [SAR 16:15 DAR 4:3], 2/25, q=-1--1, 12800 tbn, 12.50 tbc (default)
Metadata:
handler_name : VideoHandler
Stream mapping:
Stream #0:0 -> #0:0 (h264 -> libx264)
Press [q] to stop, [?] for help
[h264 @ 0xa841260] no picture
[h264 @ 0xa886100] no picture
[Parsed_mpdecimate_0 @ 0xa0f2280] keep pts:0 pts_time:0 drop_count:-1
Segmentation fault
Here's some more info from the previous compilation (I didn't include debug symbols this time)
(gdb) disass $pc-32,$pc+32
Dump of assembler code for function sum_abs_dctelem_sse2:
0x08714d60 <sum_abs_dctelem_sse2+0>: mov 0x4(%esp),%eax
0x08714d64 <sum_abs_dctelem_sse2+4>: pxor %xmm0,%xmm0
0x08714d68 <sum_abs_dctelem_sse2+8>: pxor %xmm1,%xmm1
0x08714d6c <sum_abs_dctelem_sse2+12>: movdqa (%eax),%xmm2
0x08714d70 <sum_abs_dctelem_sse2+16>: movdqa 0x10(%eax),%xmm3
0x08714d75 <sum_abs_dctelem_sse2+21>: movdqa 0x20(%eax),%xmm4
0x08714d7a <sum_abs_dctelem_sse2+26>: movdqa 0x30(%eax),%xmm5
0x08714d7f <sum_abs_dctelem_sse2+31>: pxor %xmm6,%xmm6
0x08714d83 <sum_abs_dctelem_sse2+35>: psubw %xmm2,%xmm6
0x08714d87 <sum_abs_dctelem_sse2+39>: pmaxsw %xmm6,%xmm2
0x08714d8b <sum_abs_dctelem_sse2+43>: paddusw %xmm2,%xmm0
0x08714d8f <sum_abs_dctelem_sse2+47>: pxor %xmm7,%xmm7
0x08714d93 <sum_abs_dctelem_sse2+51>: psubw %xmm3,%xmm7
0x08714d97 <sum_abs_dctelem_sse2+55>: pmaxsw %xmm7,%xmm3
0x08714d9b <sum_abs_dctelem_sse2+59>: paddusw %xmm3,%xmm1
0x08714d9f <sum_abs_dctelem_sse2+63>: pxor %xmm6,%xmm6
0x08714da3 <sum_abs_dctelem_sse2+67>: psubw %xmm4,%xmm6
0x08714da7 <sum_abs_dctelem_sse2+71>: pmaxsw %xmm6,%xmm4
0x08714dab <sum_abs_dctelem_sse2+75>: paddusw %xmm4,%xmm0
0x08714daf <sum_abs_dctelem_sse2+79>: pxor %xmm7,%xmm7
0x08714db3 <sum_abs_dctelem_sse2+83>: psubw %xmm5,%xmm7
0x08714db7 <sum_abs_dctelem_sse2+87>: pmaxsw %xmm7,%xmm5
0x08714dbb <sum_abs_dctelem_sse2+91>: paddusw %xmm5,%xmm1
0x08714dbf <sum_abs_dctelem_sse2+95>: movdqa 0x40(%eax),%xmm2
0x08714dc4 <sum_abs_dctelem_sse2+100>: movdqa 0x50(%eax),%xmm3
0x08714dc9 <sum_abs_dctelem_sse2+105>: movdqa 0x60(%eax),%xmm4
0x08714dce <sum_abs_dctelem_sse2+110>: movdqa 0x70(%eax),%xmm5
0x08714dd3 <sum_abs_dctelem_sse2+115>: pxor %xmm6,%xmm6
0x08714dd7 <sum_abs_dctelem_sse2+119>: psubw %xmm2,%xmm6
0x08714ddb <sum_abs_dctelem_sse2+123>: pmaxsw %xmm6,%xmm2
0x08714ddf <sum_abs_dctelem_sse2+127>: paddusw %xmm2,%xmm0
0x08714de3 <sum_abs_dctelem_sse2+131>: pxor %xmm7,%xmm7
0x08714de7 <sum_abs_dctelem_sse2+135>: psubw %xmm3,%xmm7
0x08714deb <sum_abs_dctelem_sse2+139>: pmaxsw %xmm7,%xmm3
0x08714def <sum_abs_dctelem_sse2+143>: paddusw %xmm3,%xmm1
0x08714df3 <sum_abs_dctelem_sse2+147>: pxor %xmm6,%xmm6
0x08714df7 <sum_abs_dctelem_sse2+151>: psubw %xmm4,%xmm6
0x08714dfb <sum_abs_dctelem_sse2+155>: pmaxsw %xmm6,%xmm4
0x08714dff <sum_abs_dctelem_sse2+159>: paddusw %xmm4,%xmm0
0x08714e03 <sum_abs_dctelem_sse2+163>: pxor %xmm7,%xmm7
0x08714e07 <sum_abs_dctelem_sse2+167>: psubw %xmm5,%xmm7
0x08714e0b <sum_abs_dctelem_sse2+171>: pmaxsw %xmm7,%xmm5
0x08714e0f <sum_abs_dctelem_sse2+175>: paddusw %xmm5,%xmm1
0x08714e13 <sum_abs_dctelem_sse2+179>: paddusw %xmm1,%xmm0
0x08714e17 <sum_abs_dctelem_sse2+183>: movhlps %xmm0,%xmm1
0x08714e1a <sum_abs_dctelem_sse2+186>: paddusw %xmm1,%xmm0
0x08714e1e <sum_abs_dctelem_sse2+190>: pshuflw $0xe,%xmm0,%xmm1
0x08714e23 <sum_abs_dctelem_sse2+195>: paddusw %xmm1,%xmm0
0x08714e27 <sum_abs_dctelem_sse2+199>: pshuflw $0x1,%xmm0,%xmm1
0x08714e2c <sum_abs_dctelem_sse2+204>: paddusw %xmm1,%xmm0
0x08714e30 <sum_abs_dctelem_sse2+208>: movd %xmm0,%eax
0x08714e34 <sum_abs_dctelem_sse2+212>: and $0xffff,%eax
0x08714e39 <sum_abs_dctelem_sse2+217>: ret
End of assembler dump.
(gdb) info all-registers
eax 0xbfff8fac -1073770580
ecx 0x9e65038 166088760
edx 0x94ca404 156017668
ebx 0x9b669d8 162949592
esp 0xbfff8f2c 0xbfff8f2c
ebp 0xbfff8fac 0xbfff8fac
esi 0x9e63838 166082616
edi 0x8 8
eip 0x8714d6c 0x8714d6c <sum_abs_dctelem_sse2+12>
eflags 0x10247 [ CF PF ZF IF RF ]
cs 0x73 115
ss 0x7b 123
ds 0x7b 123
es 0x7b 123
fs 0x0 0
gs 0x33 51
st0 -nan(0x200040003) (raw 0xffff0000000200040003)
st1 -nan(0x100030003) (raw 0xffff0000000100030003)
st2 -nan(0x9300c000d800bf) (raw 0xffff009300c000d800bf)
st3 -nan(0x48003f0052006d) (raw 0xffff0048003f0052006d)
st4 -nan(0x1111111111111111) (raw 0xffff1111111111111111)
st5 1000000 (raw 0x4012f424000000000000)
st6 240000 (raw 0x4010ea60000000000000)
st7 -inf (raw 0xffff0000000000000000)
fctrl 0x37f 895
fstat 0x120 288
ftag 0x82aa 33450
fiseg 0x73 115
fioff 0x80ad18f 134926735
foseg 0x7b 123
fooff 0xbfff8f7c -1073770628
fop 0x35c 860
xmm0 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm1 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm2 {v4_float = {0x0, 0xffa8c4d3, 0x0, 0x0}, v2_double = {0x8000000000000000, 0x0}, v16_int8 = {0x18, 0x18, 0x18, 0x18, 0x5a, 0x76, 0xae, 0xca,
0xcb, 0xcc, 0xb7, 0xa2, 0x75, 0x5c, 0x40, 0x3d}, v8_int16 = {0x1818, 0x1818, 0x765a, 0xcaae, 0xcccb, 0xa2b7, 0x5c75, 0x3d40}, v4_int32 = {0x18181818,
0xcaae765a, 0xa2b7cccb, 0x3d405c75}, v2_int64 = {0xcaae765a18181818, 0x3d405c75a2b7cccb}, uint128 = 0x3d405c75a2b7cccbcaae765a18181818}
xmm3 {v4_float = {0x0, 0x0, 0x32121, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x18, 0x18, 0x68, 0xbc, 0xcc, 0xad, 0x69, 0x3f, 0x40, 0x48, 0x48,
0x48, 0x43, 0x43, 0x3b, 0x38}, v8_int16 = {0x1818, 0xbc68, 0xadcc, 0x3f69, 0x4840, 0x4848, 0x4343, 0x383b}, v4_int32 = {0xbc681818, 0x3f69adcc,
0x48484840, 0x383b4343}, v2_int64 = {0x3f69adccbc681818, 0x383b434348484840}, uint128 = 0x383b4343484848403f69adccbc681818}
xmm4 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x45, 0x0, 0x48, 0x0, 0x48, 0x0, 0x46, 0x0, 0x43, 0x0, 0x3c, 0x0,
0x40, 0x0, 0x30, 0x0}, v8_int16 = {0x45, 0x48, 0x48, 0x46, 0x43, 0x3c, 0x40, 0x30}, v4_int32 = {0x480045, 0x460048, 0x3c0043, 0x300040}, v2_int64 = {
0x46004800480045, 0x300040003c0043}, uint128 = 0x00300040003c00430046004800480045}
xmm5 {v4_float = {0xfedb8dd0, 0x0, 0x3212, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x18, 0x39, 0x92, 0xcb, 0xc2, 0x8c, 0x4e, 0x3e, 0x45, 0x48,
0x48, 0x46, 0x43, 0x3c, 0x40, 0x30}, v8_int16 = {0x3918, 0xcb92, 0x8cc2, 0x3e4e, 0x4845, 0x4648, 0x3c43, 0x3040}, v4_int32 = {0xcb923918, 0x3e4e8cc2,
0x46484845, 0x30403c43}, v2_int64 = {0x3e4e8cc2cb923918, 0x30403c4346484845}, uint128 = 0x30403c43464848453e4e8cc2cb923918}
xmm6 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x8000000000000000, 0x0}, v16_int8 = {0xd2, 0x7, 0x15, 0x0, 0xd2, 0x7, 0x3, 0xdf, 0x5c,
0xc1, 0xb, 0x0, 0x36, 0xff, 0xa, 0x0}, v8_int16 = {0x7d2, 0x15, 0x7d2, 0xdf03, 0xc15c, 0xb, 0xff36, 0xa}, v4_int32 = {0x1507d2, 0xdf0307d2, 0xbc15c,
0xaff36}, v2_int64 = {0xdf0307d2001507d2, 0xaff36000bc15c}, uint128 = 0x000aff36000bc15cdf0307d2001507d2}
xmm7 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0xff, 0x0, 0xff, 0x0, 0xff, 0x0, 0xff, 0x0, 0xff, 0x0, 0xff, 0x0,
0xff, 0x0, 0xff, 0x0}, v8_int16 = {0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}, v4_int32 = {0xff00ff, 0xff00ff, 0xff00ff, 0xff00ff}, v2_int64 = {
0xff00ff00ff00ff, 0xff00ff00ff00ff}, uint128 = 0x00ff00ff00ff00ff00ff00ff00ff00ff}
mxcsr 0x1fa0 [ PE IM DM ZM OM UM PM ]
mm0 {uint64 = 0x200040003, v2_int32 = {0x40003, 0x2}, v4_int16 = {0x3, 0x4, 0x2, 0x0}, v8_int8 = {0x3, 0x0, 0x4, 0x0, 0x2, 0x0, 0x0, 0x0}}
mm1 {uint64 = 0x100030003, v2_int32 = {0x30003, 0x1}, v4_int16 = {0x3, 0x3, 0x1, 0x0}, v8_int8 = {0x3, 0x0, 0x3, 0x0, 0x1, 0x0, 0x0, 0x0}}
---Type <return> to continue, or q <return> to quit---
mm2 {uint64 = 0x9300c000d800bf, v2_int32 = {0xd800bf, 0x9300c0}, v4_int16 = {0xbf, 0xd8, 0xc0, 0x93}, v8_int8 = {0xbf, 0x0, 0xd8, 0x0, 0xc0,
0x0, 0x93, 0x0}}
mm3 {uint64 = 0x48003f0052006d, v2_int32 = {0x52006d, 0x48003f}, v4_int16 = {0x6d, 0x52, 0x3f, 0x48}, v8_int8 = {0x6d, 0x0, 0x52, 0x0, 0x3f,
0x0, 0x48, 0x0}}
mm4 {uint64 = 0x1111111111111111, v2_int32 = {0x11111111, 0x11111111}, v4_int16 = {0x1111, 0x1111, 0x1111, 0x1111}, v8_int8 = {0x11, 0x11,
0x11, 0x11, 0x11, 0x11, 0x11, 0x11}}
mm5 {uint64 = 0xf424000000000000, v2_int32 = {0x0, 0xf4240000}, v4_int16 = {0x0, 0x0, 0x0, 0xf424}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x24, 0xf4}}
mm6 {uint64 = 0xea60000000000000, v2_int32 = {0x0, 0xea600000}, v4_int16 = {0x0, 0x0, 0x0, 0xea60}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x60, 0xea}}
mm7 {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0, 0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
More information about the ffmpeg-user
mailing list