I have encountered a H264 video file, which crashes mplayer if I try to decode it with the [ffh264vdpau] codec. Decoding the same file with the [ffh264] codec works properly, even with the [vdpau] vo device, These are results of my experiments: codec: [cch264vdpau] vo: [vdpau] - segfault codec: [cch264vdpau] vo: [xv] - segfault codec: [cch264vdpau] vo: [null] - segfault codec: [cch264] vo: [vdpau] - plays fine codec: [cch264] vo: [xv] - plays fine codec: [cch264] vo: [null] - plays fine (but of course no picture) So, the problem seems to be with the [ffh264vdpau] codec. System config: ----------------- Distribution: debian stable/testing/unstable hybrid Kernel: 3.2.0-1-amd64 #1 SMP Wed Feb 1 08:56:58 UTC 2012 x86_64 GNU/Linux (debian stock kernel AMD64 kernel, version 3.2.2-1) Libc6: 2.13 gcc version 4.6.2 (Debian 4.6.2-12) GNU ld (GNU Binutils for Debian) 2.22 GNU assembler (GNU Binutils for Debian) 2.22 CPU: Mobile AMD Sempron(tm) Processor 2100+ VGA: Nvidia Quadro NVS 300 X driver: Nvidia binary driver, version 290.10-1 ----------------- Mplayer version: SVN-r34338-snapshot-4.6 (source snapshot from 2012-02-02) Mplayer configured with: ./configure --enable-menu --enable-xvmc --enable-debug=3 MPlayer config file: ------------------------- vo=vdpau:deint=3,xv vc=ffmpeg12vdpau,ffh264vdpau,ffodivxvdpau,ffwmv3vdpau,ffvc1vdpau, ao=alsa framedrop=1 subfont-osd-scale=3 subfont-text-scale=2 cache=100000 spuaa=4 ------------------------- Here is the log of a debug log with gdb, as described in bugreports_what.html, --------------------- GNU gdb (GDB) 7.3-debian Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /home/kristof/vdpau_crash/mplayer...run -v done. (gdb) run -v vdpau_h264_decode_crash.mkv Starting program: /home/kristof/vdpau_crash/mplayer -v vdpau_h264_decode_crash.mkv [Thread debugging using libthread_db enabled] MPlayer SVN-r34646-snapshot-4.6 (C) 2000-2012 MPlayer Team CPU vendor name: AuthenticAMD max cpuid level: 1 CPU: Mobile AMD Sempron(tm) Processor 2100+ (Family: 15, Model: 108, Stepping: 2) extended cpuid-level: 24 extended cache-info: 16810304 Detected cache-line size is 64 bytes CPUflags: MMX: 1 MMX2: 1 3DNow: 1 3DNowExt: 1 SSE: 1 SSE2: 1 SSSE3: 0 Compiled for x86 CPU with extensions: MMX MMX2 3DNow 3DNowExt SSE SSE2 CMOV get_path('codecs.conf') -> '/home/localuser/.mplayer/codecs.conf' Reading optional codecs config file /home/localuser/.mplayer/codecs.conf: No such file or directory Reading optional codecs config file /usr/local/etc/mplayer/codecs.conf: No such file or directory Using built-in default codecs.conf. init_freetype Using MMX (with tiny bit MMX2) Optimized OnScreenDisplay get_path('fonts') -> '/home/localuser/.mplayer/fonts' Configuration: --enable-menu --enable-xvmc --enable-debug=3 CommandLine: '-v' 'vdpau_h264_decode_crash.mkv' Using nanosleep() timing get_path('input.conf') -> '/home/localuser/.mplayer/input.conf' Reading optional input config file /home/localuser/.mplayer/input.conf: No such file or directory Reading optional input config file /usr/local/etc/mplayer/input.conf: No such file or directory Falling back on default (hardcoded) input config get_path('vdpau_h264_decode_crash.mkv.conf') -> '/home/localuser/.mplayer/vdpau_h264_decode_crash.mkv.conf' Playing vdpau_h264_decode_crash.mkv. get_path('sub/') -> '/home/localuser/.mplayer/sub/' [file] File size is 1409110 bytes STREAM: [file] vdpau_h264_decode_crash.mkv STREAM: Description: File STREAM: Author: Albeu STREAM: Comment: based on the code from ??? (probably Arpi) CACHE_PRE_INIT: 0 [0] 0 pre:20480000 eof:0 Cache fill: 1.38% (1409110 bytes) libavformat version 54.0.100 (internal) Configuration: --enable-gpl --enable-postproc LAVF_check: Matroska/WebM file format libavformat file format detected. st:0 removing common factor 1000000 from timebase [h264 @ 0xd58240]no picture [h264 @ 0xd58240]no picture [matroska,webm @ 0xcde480]All info found rfps: 23.750000 0.011872 rfps: 23.750000 0.011872 rfps: 23.833333 0.004757 rfps: 23.916667 0.000861 rfps: 24.000000 0.000184 rfps: 24.083333 0.002727 rfps: 24.166667 0.008490 rfps: 24.166667 0.008490 rfps: 24.250000 0.017472 rfps: 24.250000 0.017472 rfps: 47.666667 0.019027 rfps: 47.750000 0.009625 rfps: 47.833333 0.003443 rfps: 47.916667 0.000480 rfps: 48.000000 0.000737 rfps: 48.083333 0.004213 rfps: 48.166667 0.010909 rfps: 23.976024 0.000049 ==> Found video stream: 0 ======= VIDEO Format ====== biSize 90 biWidth 1280 biHeight 720 biPlanes 0 biBitCount 0 biCompression 875967048='H264' biSizeImage 0 Unknown extra header dump: [1] [6e] [0] [32] [ff] [e1] [0] [1c] [67] [6e] [0] [32] [a6] [c7] [28] [44] [5] [0] [5b] [a6] [a0] [40] [40] [28] [0] [0] [1f] [48] [0] [5] [dc] [4] [78] [c1] [8c] [23] [1] [0] [7] [68] [e8] [43] [82] [72] [c8] [b0] [fd] [fa] [fa] [0] =========================== [lavf] stream 0: video (h264), -vid 0, [sage] Lupin III: Blood Seal ~Eternal Mermaid~ LAVF: 0 audio and 1 video streams found LAVF: build 3539044 VIDEO: [H264] 1280x720 0bpp 23.976 fps 0.0 kbps ( 0.0 kbyte/s) [V] filefmt:44 fourcc:0x34363248 size:1280x720 fps:23.976 ftime:=0.0417 Load subtitles in ./ get_path('sub/') -> '/home/localuser/.mplayer/sub/' [file] File size is 591 bytes STREAM: [file] ./vdpau_h264_decode_crash.txt STREAM: Description: File STREAM: Author: Albeu STREAM: Comment: based on the code from ??? (probably Arpi) SUB: Could not determine file format Cannot load subtitles './vdpau_h264_decode_crash.txt'. X11 opening display: :7 vo: X11 color mask: FFFFFF (R:FF0000 G:FF00 B:FF) vo: X11 running at 3520x1200 with depth 24 and 32 bpp (":7" => local display) [x11] Detected wm supports layers. [x11] Using workaround for Metacity bugs. [x11] Detected wm supports NetWM. [x11] Detected wm supports ABOVE state. [x11] Detected wm supports BELOW state. [x11] Detected wm supports FULLSCREEN state. [x11] Current fstype setting honours FULLSCREEN ABOVE BELOW X atoms ========================================================================== Forced video codec: ffmpeg12vdpau Forced video codec: ffh264vdpau Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family libavcodec version 54.0.102 (internal) Configuration: --enable-gpl --enable-postproc INFO: libavcodec init OK! Selected video codec: [ffh264vdpau] vfm: ffmpeg (FFmpeg H.264 (VDPAU)) ========================================================================== Audio: no sound Freeing 0 unused audio chunks. Starting playback... [h264_vdpau @ 0xd58240]no picture Program received signal SIGSEGV, Segmentation fault. ff_vdpau_add_data_chunk (s=, buf=0xd2806a "", buf_size=3) at libavcodec/vdpau.c:125 125 render->bitstream_buffers[render->bitstream_buffers_used].bitstream = buf; (gdb) bt #0 ff_vdpau_add_data_chunk (s=, buf=0xd2806a "", buf_size=3) at libavcodec/vdpau.c:125 #1 0x00000000008aa1df in decode_nal_units (h=0x18e7bd0, buf=0x1896450 "", buf_size=39699) at libavcodec/h264.c:3941 #2 0x00000000008aa5a9 in decode_frame (avctx=0x188fec0, data=0x188fcd0, data_size=0x7fffffffd3ec, avpkt=) at libavcodec/h264.c:4109 #3 0x00000000009eb742 in avcodec_decode_video2 (avctx=0x188fec0, picture=0x188fcd0, got_picture_ptr=0x7fffffffd3ec, avpkt=0x7fffffffd360) at libavcodec/utils.c:1255 #4 0x0000000000618eb7 in decode (sh=0x1895050, data=, len=39699, flags=) at libmpcodecs/vd_ffmpeg.c:785 #5 0x00000000005525d1 in decode_video (sh_video=0x1895050, start=, in_size=, drop_frame=0, pts=, full_frame=0x0) at libmpcodecs/dec_video.c:393 #6 0x00000000004ce7a5 in generate_video_frame (d_video=0x17718a0, sh_video=0x1895050) at mplayer.c:1807 #7 update_video (blit_frame=0x7fffffffe598) at mplayer.c:2461 #8 0x00000000004c249f in main (argc=, argv=) at mplayer.c:3731 (gdb) disass $pc-32 $pc+32 A syntax error in expression, near `$pc+32'. (gdb) disass $pc-32,$pc+32 Dump of assembler code from 0xa0c109 to 0xa0c149: 0x0000000000a0c109 : std 0x0000000000a0c10a : decl -0x76b7f3b5(%rbx) 0x0000000000a0c110 : rex.XB adc %cl,0x63(%r8) 0x0000000000a0c114 : sar %edi 0x0000000000a0c116 : rorl $0x34,-0x73(%rax) 0x0000000000a0c11a : (bad) 0x0000000000a0c11b : add %al,(%rax) 0x0000000000a0c11d : add %al,(%rax) 0x0000000000a0c11f : shl $0x5,%rdx 0x0000000000a0c123 : sub %rsi,%rdx 0x0000000000a0c126 : add %rdx,%rax => 0x0000000000a0c129 : mov %r12,0x8(%rax) 0x0000000000a0c12d : mov %ebp,0x10(%rax) 0x0000000000a0c130 : movl $0x0,(%rax) 0x0000000000a0c136 : mov 0x8(%rsp),%rbp 0x0000000000a0c13b : mov %ecx,0xc(%rbx) 0x0000000000a0c13e : mov 0x10(%rsp),%r12 0x0000000000a0c143 : mov (%rsp),%rbx 0x0000000000a0c147 : add $0x18,%rsp End of assembler dump. (gdb) info all-registers rax 0xfffffff40c0c0c00 -51337491456 rbx 0x1a0bdf0 27311600 rcx 0x80808081 2155905153 rdx 0xfffffff40c0c0c00 -51337491456 rsi 0xfffffffc04040400 -17112497152 rdi 0x8080808080808080 -9187201950435737472 rbp 0x3 0x3 rsp 0x7fffffffd150 0x7fffffffd150 r8 0x7ffff7fa37c0 140737353758656 r9 0x0 0 r10 0x0 0 r11 0x246 582 r12 0xd2806a 13795434 r13 0x0 0 r14 0x0 0 r15 0x4c1f7 311799 rip 0xa0c129 0xa0c129 eflags 0x10286 [ PF SF IF RF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 st0 -inf (raw 0xffff0000000000000000) st1 -inf (raw 0xffff0000000000000000) st2 -inf (raw 0xffff0000000000000000) st3 -inf (raw 0xffff0000000000000000) st4 -inf (raw 0xffff0000000000000000) st5 -nan(0x606060606060606) (raw 0xffff0606060606060606) st6 -nan(0x303030303030303) (raw 0xffff0303030303030303) st7 -nan(0x101010101010101) (raw 0xffff0101010101010101) fctrl 0x37f 895 fstat 0x0 0 ftag 0xffff 65535 fiseg 0x0 0 fioff 0x0 0 foseg 0x0 0 fooff 0x0 0 fop 0x0 0 xmm0 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x8000000000000000}, v16_int8 = {0x0 , 0xff, 0xff, 0xff, 0xff}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xffff, 0xffff}, v4_int32 = {0x0, 0x0, 0x0, 0xffffffff}, v2_int64 = {0x0, 0xffffffff00000000}, uint128 = 0xffffffff000000000000000000000000} xmm1 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x8000000000000000, 0x0}, v16_int8 = {0x6e, 0x6f, 0x20, 0x70, 0x69, 0x63, 0x74, 0x75, 0x72, 0x65, 0x20, 0xa, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x6f6e, 0x7020, 0x6369, 0x7574, 0x6572, 0xa20, 0x0, 0x0}, v4_int32 = {0x70206f6e, 0x75746369, 0xa206572, 0x0}, v2_int64 = { 0x7574636970206f6e, 0xa206572}, uint128 = 0x000000000a2065727574636970206f6e} xmm2 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x55, 0x54, 0x46, 0x2d, 0x38, 0x0 }, v8_int16 = {0x5455, 0x2d46, 0x38, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x2d465455, 0x38, 0x0, 0x0}, v2_int64 = {0x382d465455, 0x0}, uint128 = 0x0000000000000000000000382d465455} xmm3 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 , 0xff, 0x0}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff}, v4_int32 = {0x0, 0x0, 0x0, 0xff0000}, v2_int64 = {0x0, 0xff000000000000}, uint128 = 0x00ff0000000000000000000000000000} xmm4 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x8000000000000000, 0x8000000000000000}, v16_int8 = {0x65, 0x6c, 0x65, 0x63, 0x74, 0x20, 0x6d, 0x75, 0x6c, 0x74, 0x69, 0x74, 0x68, 0x72, 0x65, 0x61}, v8_int16 = {0x6c65, 0x6365, 0x2074, 0x756d, 0x746c, 0x7469, 0x7268, 0x6165}, v4_int32 = {0x63656c65, 0x756d2074, 0x7469746c, 0x61657268}, v2_int64 = {0x756d207463656c65, 0x616572687469746c}, uint128 = 0x616572687469746c756d207463656c65} xmm5 {v4_float = {0x3, 0x3, 0x3, 0x3}, v2_double = {0x20, 0x20}, v16_int8 = {0x40 }, v8_int16 = {0x4040, 0x4040, 0x4040, 0x4040, 0x4040, 0x4040, 0x4040, 0x4040}, v4_int32 = {0x40404040, 0x40404040, 0x40404040, 0x40404040}, v2_int64 = {0x4040404040404040, 0x4040404040404040}, uint128 = 0x40404040404040404040404040404040} xmm6 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x8000000000000000, 0x8000000000000000}, v16_int8 = {0x5b }, v8_int16 = {0x5b5b, 0x5b5b, 0x5b5b, 0x5b5b, 0x5b5b, 0x5b5b, 0x5b5b, 0x5b5b}, v4_int32 = {0x5b5b5b5b, 0x5b5b5b5b, 0x5b5b5b5b, 0x5b5b5b5b}, v2_int64 = {0x5b5b5b5b5b5b5b5b, 0x5b5b5b5b5b5b5b5b}, uint128 = 0x5b5b5b5b5b5b5b5b5b5b5b5b5b5b5b5b} xmm7 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x20 }, v8_int16 = {0x2020, 0x2020, 0x2020, 0x2020, 0x2020, 0x2020, 0x2020, 0x2020}, v4_int32 = {0x20202020, 0x20202020, 0x20202020, 0x20202020}, v2_int64 = {0x2020202020202020, 0x2020202020202020}, uint128 = 0x20202020202020202020202020202020} xmm8 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x20, 0x20, 0x20, 0x0 }, v8_int16 = {0x2020, 0x20, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x202020, 0x0, 0x0, 0x0}, v2_int64 = {0x202020, 0x0}, uint128 = 0x00000000000000000000000000202020} xmm9 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x8000000000000000, 0x8000000000000000}, v16_int8 = {0xff }, v8_int16 = {0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff}, v4_int32 = {0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff}, v2_int64 = {0xffffffffffffffff, 0xffffffffffffffff}, uint128 = 0xffffffffffffffffffffffffffffffff} xmm10 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x0, 0x20}, v8_int16 = {0x0, 0x0, 0x0, 0x2000, 0x2020, 0x2020, 0x2020, 0x2000}, v4_int32 = {0x0, 0x20000000, 0x20202020, 0x20002020}, v2_int64 = {0x2000000000000000, 0x2000202020202020}, uint128 = 0x20002020202020202000000000000000} xmm11 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x8000000000000000, 0x8000000000000000}, v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x0, 0xff}, v8_int16 = {0x0, 0x0, 0x0, 0xffff, 0xffff, 0xffff, 0xffff, 0xff00}, v4_int32 = {0x0, 0xffff0000, 0xffffffff, 0xff00ffff}, v2_int64 = {0xffff000000000000, 0xff00ffffffffffff}, uint128 = 0xff00ffffffffffffffff000000000000} xmm12 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x29, 0xf2, 0x88, 0x6c, 0xa6, 0x49, 0xde, 0x3e, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0xf229, 0x6c88, 0x49a6, 0x3ede, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x6c88f229, 0x3ede49a6, 0x0, 0x0}, v2_int64 = {0x3ede49a66c88f229, 0x0}, uint128 = 0x00000000000000003ede49a66c88f229} ---Type to continue, or q to quit--- xmm13 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0xb3, 0x12, 0x58, 0x17, 0x64, 0x46, 0xe6, 0x3b, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x12b3, 0x1758, 0x4664, 0x3be6, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x175812b3, 0x3be64664, 0x0, 0x0}, v2_int64 = {0x3be64664175812b3, 0x0}, uint128 = 0x00000000000000003be64664175812b3} xmm14 {v4_float = {0x0, 0x3, 0x0, 0x0}, v2_double = {0x2d, 0x0}, v16_int8 = {0xc0, 0x9, 0xf2, 0x16, 0xb5, 0xdf, 0x46, 0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x9c0, 0x16f2, 0xdfb5, 0x4046, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x16f209c0, 0x4046dfb5, 0x0, 0x0}, v2_int64 = {0x4046dfb516f209c0, 0x0}, uint128 = 0x00000000000000004046dfb516f209c0} xmm15 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0xac, 0x3, 0xac, 0x3, 0xac, 0x3, 0xac, 0x3, 0xac, 0x3, 0xac, 0x3, 0xac, 0x3, 0xac, 0x3}, v8_int16 = {0x3ac, 0x3ac, 0x3ac, 0x3ac, 0x3ac, 0x3ac, 0x3ac, 0x3ac}, v4_int32 = {0x3ac03ac, 0x3ac03ac, 0x3ac03ac, 0x3ac03ac}, v2_int64 = {0x3ac03ac03ac03ac, 0x3ac03ac03ac03ac}, uint128 = 0x03ac03ac03ac03ac03ac03ac03ac03ac} mxcsr 0x1fa0 [ PE IM DM ZM OM UM PM ] ---------------