[FFmpeg-cvslog] pngdec: frame multithreading support

Michael Niedermayer michaelni at gmx.at
Fri Aug 23 20:01:10 CEST 2013 

On Thu, Aug 22, 2013 at 04:45:58PM +0200, Paul B Mahol wrote:
> ffmpeg | branch: master | Paul B Mahol <onemda at gmail.com> | Mon Aug 12 10:56:20 2013 +0000| [6e643239d9956ce32888d87451706ee7c8757afa] | committer: Paul B Mahol
> 
> pngdec: frame multithreading support
> 
> Signed-off-by: Paul B Mahol <onemda at gmail.com>
> 
> > http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=6e643239d9956ce32888d87451706ee7c8757afa
> ---
> 
>  libavcodec/pngdec.c |   69 +++++++++++++++++++++++++++++++++++++--------------
>  1 file changed, 51 insertions(+), 18 deletions(-)

It seems this still deadlocks and crashes occasionally but not
always (same file as the last one)

Deadlock: (duplicate backtraces ommited)

Thread 27 (Thread 0x7fffe017f700 (LWP 10742)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1  0x00000000008e16a7 in frame_worker_thread (arg=0x1743520) at libavcodec/pthread.c:330
#2  0x00007ffff0467e9a in start_thread (arg=0x7fffe017f700) at pthread_create.c:308
#3  0x00007ffff0194ccd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#4  0x0000000000000000 in ?? ()


Thread 21 (Thread 0x7fffe3185700 (LWP 10736)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1  0x00000000008e26d7 in ff_thread_await_progress (f=0x173add0, n=2147483647, field=0) at libavcodec/pthread.c:692
#2  0x00000000008d3d42 in decode_frame (avctx=0x173a920, data=0x1741dd0, got_frame=0x1742040, avpkt=0x1741d60) at libavcodec/pngdec.c:843
#3  0x00000000008e176f in frame_worker_thread (arg=0x1741c60) at libavcodec/pthread.c:339
#4  0x00007ffff0467e9a in start_thread (arg=0x7fffe3185700) at pthread_create.c:308
#5  0x00007ffff0194ccd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#6  0x0000000000000000 in ?? ()

Thread 1 (Thread 0x7ffff7faa780 (LWP 10714)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1  0x00000000008e23d7 in ff_thread_decode_frame (avctx=0x174fec0, picture=0x174b1e0, got_picture_ptr=0x7fffffffdca8, avpkt=0x7fffffffd9d0) at libavcodec/pthread.c:631
#2  0x0000000000982228 in avcodec_decode_video2 (avctx=0x174fec0, picture=0x174b1e0, got_picture_ptr=0x7fffffffdca8, avpkt=0x7fffffffdbf0) at libavcodec/utils.c:1979
#3  0x0000000000424306 in decode_video (ist=0x1752780, pkt=0x7fffffffdbf0, got_output=0x7fffffffdca8) at ffmpeg.c:1668
#4  0x0000000000425193 in output_packet (ist=0x1752780, pkt=0x7fffffffdd70) at ffmpeg.c:1866
#5  0x000000000042ad63 in process_input (file_index=0) at ffmpeg.c:3085
#6  0x000000000042b0e2 in transcode_step () at ffmpeg.c:3181
#7  0x000000000042b1ef in transcode () at ffmpeg.c:3233
#8  0x000000000042b71a in main (argc=11, argv=0x7fffffffe398) at ffmpeg.c:3411

------------
segfault (again same file, doesnt happe under valgrind)

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe2d26700 (LWP 19664)]
malloc_consolidate (av=0x7fffc0000020) at malloc.c:4251
4251    malloc.c: No such file or directory.
(gdb) bt
#0  malloc_consolidate (av=0x7fffc0000020) at malloc.c:4251
#1  0x00007fffef4c1446 in malloc_consolidate (av=0x7fffc0000020) at malloc.c:4226
#2  _int_malloc (av=0x7fffc0000020, bytes=7232) at malloc.c:3543
#3  0x00007fffef4c2811 in _int_memalign (av=0x7fffc0000020, alignment=32, bytes=<optimized out>) at malloc.c:4503
#4  0x00007fffef4c4aa6 in __GI___libc_memalign (alignment=32, bytes=7152) at malloc.c:3104
#5  0x00007fffef4c5d19 in __posix_memalign (memptr=0x7fffe2d25cb8, alignment=<optimized out>, size=<optimized out>) at malloc.c:5049
#6  0x0000000000bd2bda in av_malloc (size=7152) at libavutil/mem.c:93
#7  av_mallocz (size=7152) at libavutil/mem.c:223
#8  0x00007fffefe445b7 in inflateInit2_ () from /lib/x86_64-linux-gnu/libz.so.1
#9  0x00000000008fd864 in decode_frame (avctx=0x16d2c40, data=0x16db150, got_frame=0x16db3c0, avpkt=0x16db0e0) at libavcodec/pngdec.c:539
#10 0x000000000090a755 in frame_worker_thread (arg=0x16dafe0) at libavcodec/pthread.c:339
#11 0x00007fffef807e9a in start_thread (arg=0x7fffe2d26700) at pthread_create.c:308
#12 0x00007fffef534ccd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#13 0x0000000000000000 in ?? ()

asan shows these:

=================================================================
==29015==ERROR: AddressSanitizer: heap-use-after-free on address 0x60700001f74c at pc 0x13757b0 bp 0x7f8a041959a0 sp 0x7f8a04195998
WRITE of size 4 at 0x60700001f74c thread T20
    #0 0x13757af in atomic_int_add_and_fetch_gcc ffmpeg/libavutil/atomic_gcc.h:45
    #1 0x1375887 in av_buffer_unref ffmpeg/libavutil/buffer.c:114
    #2 0x13819d8 in av_frame_unref ffmpeg/libavutil/frame.c:347
    #3 0xdbca46 in decode_frame ffmpeg/libavcodec/pngdec.c:519
    #4 0xde0d25 in frame_worker_thread ffmpeg/libavcodec/pthread.c:339
    #5 0x466e73 in _ZN6__asan10AsanThread11ThreadStartEm ??:0
    #6 0x7f8a175c0e99 in start_thread /build/buildd/eglibc-2.15/nptl/pthread_create.c:308
    #7 0x7f8a16ed3ccc in ?? /build/buildd/eglibc-2.15/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:112
0x60700001f74c is located 12 bytes inside of 40-byte region [0x60700001f740,0x60700001f768)
freed by thread T19 here:
    #0 0x4607f4 in free ??:0
    #1 0x138d9f1 in av_freep ffmpeg/libavutil/mem.c:217
    #2 0x13758f1 in av_buffer_unref ffmpeg/libavutil/buffer.c:116
    #3 0x13819d8 in av_frame_unref ffmpeg/libavutil/frame.c:347
    #4 0xdc018d in decode_frame ffmpeg/libavcodec/pngdec.c:872
    #5 0xde0d25 in frame_worker_thread ffmpeg/libavcodec/pthread.c:339
    #6 0x466e73 in _ZN6__asan10AsanThread11ThreadStartEm ??:0
previously allocated by thread T19 here:
    #0 0x460b6c in posix_memalign ??:0
    #1 0x138d6e0 in av_malloc ffmpeg/libavutil/mem.c:93
    #2 0x138da2e in av_mallocz ffmpeg/libavutil/mem.c:223
    #3 0x137523a in av_buffer_create ffmpeg/libavutil/buffer.c:34
    #4 0x1375584 in av_buffer_alloc ffmpeg/libavutil/buffer.c:74
    #5 0x13755dd in av_buffer_allocz ffmpeg/libavutil/buffer.c:83
    #6 0x1376817 in pool_alloc_buffer ffmpeg/libavutil/buffer.c:305
    #7 0xf07ca8 in video_get_buffer ffmpeg/libavcodec/utils.c:574
    #8 0xf094a4 in get_buffer_internal ffmpeg/libavcodec/utils.c:864
    #9 0xf08e1e in ff_get_buffer ffmpeg/libavcodec/utils.c:876
    #10 0xddeb6f in thread_get_buffer_internal ffmpeg/libavcodec/pthread.c:962
    #11 0xdde6ae in ff_thread_get_buffer ffmpeg/libavcodec/pthread.c:1017
    #12 0xdbe5be in decode_frame ffmpeg/libavcodec/pngdec.c:646
    #13 0xde0d25 in frame_worker_thread ffmpeg/libavcodec/pthread.c:339
    #14 0x466e73 in _ZN6__asan10AsanThread11ThreadStartEm ??:0
Thread T20 created by T0 here:
    #0 0x45c9d8 in __interceptor_pthread_create ??:0
    #1 0xde0075 in frame_thread_init ffmpeg/libavcodec/pthread.c:872
    #2 0xf0b73c in avcodec_open2 ffmpeg/libavcodec/utils.c:1222
    #3 0x4a6354 in init_input_stream ffmpeg/ffmpeg.c:1983
    #4 0x49074a in transcode_init ffmpeg/ffmpeg.c:2486
    #5 0x48bff2 in transcode ffmpeg/ffmpeg.c:3204
    #6 0x48b462 in main ffmpeg/ffmpeg.c:3411
    #7 0x7f8a16e0176c in __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226
Thread T19 created by T0 here:
    #0 0x45c9d8 in __interceptor_pthread_create ??:0
    #1 0xde0075 in frame_thread_init ffmpeg/libavcodec/pthread.c:872
    #2 0xf0b73c in avcodec_open2 ffmpeg/libavcodec/utils.c:1222
    #3 0x4a6354 in init_input_stream ffmpeg/ffmpeg.c:1983
    #4 0x49074a in transcode_init ffmpeg/ffmpeg.c:2486
    #5 0x48bff2 in transcode ffmpeg/ffmpeg.c:3204
    #6 0x48b462 in main ffmpeg/ffmpeg.c:3411
    #7 0x7f8a16e0176c in __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226

---------
a different one:
==29194==ERROR: AddressSanitizer: heap-use-after-free on address 0x605000002780 at pc 0x1375763 bp 0x7fffeb33d210 sp 0x7fffeb33d208
READ of size 24 at 0x605000002780 thread T0
    #0 0x1375762 in av_buffer_ref ffmpeg/libavutil/buffer.c:98
    #1 0xf167da in ff_thread_ref_frame ffmpeg/libavcodec/utils.c:3151
    #2 0xddc447 in submit_packet ffmpeg/libavcodec/pthread.c:519
    #3 0xddb48b in ff_thread_decode_frame ffmpeg/libavcodec/pthread.c:602
    #4 0xf10358 in avcodec_decode_video2 ffmpeg/libavcodec/utils.c:1979
    #5 0x499bc5 in decode_video ffmpeg/ffmpeg.c:1668
    #6 0x49542c in output_packet ffmpeg/ffmpeg.c:1866
    #7 0x4a198b in process_input ffmpeg/ffmpeg.c:3085
    #8 0x49332f in transcode_step ffmpeg/ffmpeg.c:3181
    #9 0x48c083 in transcode ffmpeg/ffmpeg.c:3233
    #10 0x48b462 in main ffmpeg/ffmpeg.c:3411
    #11 0x7f8cba42476c in __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226
    #12 0x46e93c in _start ??:0
0x605000002780 is located 0 bytes inside of 24-byte region [0x605000002780,0x605000002798)
freed by thread T19 here:
    #0 0x4607f4 in free ??:0
    #1 0x138d9f1 in av_freep ffmpeg/libavutil/mem.c:217
    #2 0x1375867 in av_buffer_unref ffmpeg/libavutil/buffer.c:112
    #3 0xddeee1 in ff_thread_release_buffer ffmpeg/libavcodec/pthread.c:1044
    #4 0xdc018d in decode_frame ffmpeg/libavcodec/pngdec.c:872
    #5 0xde0d25 in frame_worker_thread ffmpeg/libavcodec/pthread.c:339
    #6 0x466e73 in _ZN6__asan10AsanThread11ThreadStartEm ??:0
previously allocated by thread T19 here:
    #0 0x460b6c in posix_memalign ??:0
    #1 0x138d6e0 in av_malloc ffmpeg/libavutil/mem.c:93
    #2 0x138da2e in av_mallocz ffmpeg/libavutil/mem.c:223
    #3 0x13753ae in av_buffer_create ffmpeg/libavutil/buffer.c:47
    #4 0x1375584 in av_buffer_alloc ffmpeg/libavutil/buffer.c:74
    #5 0xdde8cf in thread_get_buffer_internal ffmpeg/libavcodec/pthread.c:944
    #6 0xdde6ae in ff_thread_get_buffer ffmpeg/libavcodec/pthread.c:1017
    #7 0xdbe5be in decode_frame ffmpeg/libavcodec/pngdec.c:646
    #8 0xde0d25 in frame_worker_thread ffmpeg/libavcodec/pthread.c:339
    #9 0x466e73 in _ZN6__asan10AsanThread11ThreadStartEm ??:0
Thread T19 created by T0 here:
    #0 0x45c9d8 in __interceptor_pthread_create ??:0
    #1 0xde0075 in frame_thread_init ffmpeg/libavcodec/pthread.c:872
    #2 0xf0b73c in avcodec_open2 ffmpeg/libavcodec/utils.c:1222
    #3 0x4a6354 in init_input_stream ffmpeg/ffmpeg.c:1983
    #4 0x49074a in transcode_init ffmpeg/ffmpeg.c:2486
    #5 0x48bff2 in transcode ffmpeg/ffmpeg.c:3204
    #6 0x48b462 in main ffmpeg/ffmpeg.c:3411
    #7 0x7f8cba42476c in __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

During times of universal deceit, telling the truth becomes a
revolutionary act. -- George Orwell
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-cvslog/attachments/20130823/0b259b19/attachment.asc>

More information about the ffmpeg-cvslog mailing list