[Ffmpeg-devel] Segfault when encoding MPEG with MMX support

[Tobias Grimm]

Hi!

I have an application that encodes still images as MPEG2 to be viewed by
a DVB card. With the update to Debian Sarge the application is broken,
so I tried out the latest ffmpeg CVS snapshot (2005-07-10).
Unfortunately, this does not solve the problem. I always get a segfault,
when avcodec_encode_video() is invoked (see backtrace at the end). Only
if I compile ffmpeg with --disable-mmx, it works.

I'm using gcc 3.3.5 / libc6 2.3.2.

Any ideas, what may be going wrong? What can I do to find the source of
the problem?

bye,

Tobias

Here's the backtrace:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 180236 (LWP 5984)]
0x4082dc6e in dct_quantize_SSE2 (s=0x8f078f0, block=0x8f48200, n=0,
qscale=1083196800, overflow=0xbe3f7140)
    at mpegvideo_mmx_template.c:156
156     mpegvideo_mmx_template.c: Datei oder Verzeichnis nicht gefunden.
        in mpegvideo_mmx_template.c
(gdb) bt
#0  0x4082dc6e in dct_quantize_SSE2 (s=0x8f078f0, block=0x8f48200, n=0,
qscale=1083196800, overflow=0xbe3f7140)
    at mpegvideo_mmx_template.c:156
#1  0x405ea604 in encode_mb (s=0x8f078f0, motion_x=0, motion_y=0) at
mpegvideo.c:4217
#2  0x405e72fa in encode_thread (c=0x8f05ac0, arg=0x8f078f0) at
mpegvideo.c:5111
#3  0x405cfee3 in avcodec_default_execute (c=0x8f05ac0, func=0x405e4ff0
<encode_thread>, arg=0x8f079b4, ret=0x0, count=1)
    at utils.c:404
#4  0x405da780 in encode_picture (s=0x8f078f0, picture_number=0) at
mpegvideo.c:5414
#5  0x405d7386 in MPV_encode_picture (avctx=0x8f05ac0, buf=0x4144e008
"", buf_size=1244160, data=0x8f1b8d0)
    at mpegvideo.c:2321
#6  0x405d0394 in avcodec_encode_video (avctx=0x8f05ac0, buf=0x8f1b8d0
"\006", buf_size=1095032840, pict=0x8f05980)
    at utils.c:564
#7  0x405cc9f5 in cEncode::EncodeFrames (this=0x8f0523c,
context=0x8f05ac0, frame=0x8f05980) at encode.c:194
#8  0x405cc789 in cEncode::Encode (this=0x8f0523c) at encode.c:122
#9  0x405cd35b in cStillImage::Action (this=0x8f05210) at stillimage.c:89
#10 0x08122666 in cThread::StartThread (Thread=0x8f05210) at thread.c:233
#11 0x40041e51 in pthread_start_thread () from /lib/libpthread.so.0
#12 0x40041ecf in pthread_start_thread_event () from /lib/libpthread.so.0
#13 0x4025192a in clone () from /lib/libc.so.6