[FFmpeg-devel] Possible incomplete commit "avcodec/nvenc: support RGB input"

[Sven C. Dack]

On 08/09/16 08:46, Timo Rothenpieler wrote:
> Am 08.09.2016 um 02:29 schrieb Sven C. Dack:
>> On 08/09/16 00:57, Hendrik Leppkes wrote:
>>> The image copying code was refactored in an earlier patch to be
>>> generic and not rely on hard-coding format info, hence the second part
>>> is not needed anymore.
>>>
>> This is not quite accurate. It doesn't explain the seg. fault. This
>> didn't happen in my patch and I am currently using my own version of
>> nvenc.c where it's working fine and without the re-factoring. I will not
>> make a second patch, but see Timo being in charge of this as he is the
>> one who signed it off. I am going to "do the Pope" and have a little faith.
>>
>> Sven
> Can you send a full backtrace of your segfault?

I've updated my git clone and I did a fully compilition just to be sure, but it 
still occurs. Funny things is it's working when run from within gdb, but core 
dumps when run from the shell.

Sven

$ ffmpeg_g -f x11grab -framerate 200 -s hd1080 -i :0.0 -c:v hevc_nvenc -y test.mkv
...
(gdb) bt
#0  0x00007fd69639ba3e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x0000000001018591 in av_image_copy_plane (height=8, bytewidth=7680, 
src_linesize=<optimized out>, src=<optimized out>, dst_linesize=<optimized out>, 
dst=<optimized out>) at src/libavutil/imgutils.c:296
#2  av_image_copy (dst_data=dst_data at entry=0x7ffe17cf07c0, 
dst_linesizes=dst_linesizes at entry=0x7ffe17cf07b0, 
src_data=src_data at entry=0x454e3c0, src_linesizes=src_linesizes at entry=0x454e400, 
pix_fmt=AV_PIX_FMT_BGR0, width=1920,
     height=1088) at src/libavutil/imgutils.c:334
#3  0x0000000000e7ab09 in nvenc_copy_frame (avctx=0x3e7ce80, 
nv_surface=0x43215a0, nv_surface=0x43215a0, lock_buffer_params=0x7ffe17cf07e0, 
lock_buffer_params=0x7ffe17cf07e0, frame=0x454e3c0) at src/libavcodec/nvenc.c:1312
#4  nvenc_upload_frame (nvenc_frame=0x43215a0, frame=0x454e3c0, avctx=0x3e7ce80) 
at src/libavcodec/nvenc.c:1430
#5  ff_nvenc_encode_frame (avctx=0x3e7ce80, pkt=0x7ffe17cf1d20, frame=0x454e3c0, 
got_packet=0x7ffe17cf1bdc) at src/libavcodec/nvenc.c:1660
#6  0x0000000000acb003 in avcodec_encode_video2 (avctx=avctx at entry=0x3e7ce80, 
avpkt=avpkt at entry=0x7ffe17cf1d20, frame=frame at entry=0x454e3c0, 
got_packet_ptr=got_packet_ptr at entry=0x7ffe17cf1bdc) at src/libavcodec/utils.c:1961
#7  0x0000000000496cb5 in do_video_out (s=0x3e7b2a0, ost=ost at entry=0x3e7cc80, 
next_picture=next_picture at entry=0x454e3c0, sync_ipts=<optimized out>, 
sync_ipts at entry=-7.62939453125e-06) at src/ffmpeg.c:1175
#8  0x0000000000497f6d in reap_filters (flush=flush at entry=0) at src/ffmpeg.c:1366
#9  0x000000000049ab1c in transcode_step () at src/ffmpeg.c:4117
#10 transcode () at src/ffmpeg.c:4161
#11 0x000000000047d0ac in main (argc=<optimized out>, argv=0x7ffe17cf2668) at 
src/ffmpeg.c:4356

Another one:

(gdb) bt
#0  0x00007ff6ffe7ba3e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x0000000001018591 in av_image_copy_plane (height=8, bytewidth=7680, 
src_linesize=<optimized out>, src=<optimized out>, dst_linesize=<optimized out>, 
dst=<optimized out>) at src/libavutil/imgutils.c:296
#2  av_image_copy (dst_data=dst_data at entry=0x7ffd063a4f80, 
dst_linesizes=dst_linesizes at entry=0x7ffd063a4f70, 
src_data=src_data at entry=0x40bf3e0, src_linesizes=src_linesizes at entry=0x40bf420, 
pix_fmt=AV_PIX_FMT_BGR0, width=1920,
     height=1088) at src/libavutil/imgutils.c:334
#3  0x0000000000e7ab09 in nvenc_copy_frame (avctx=0x39ede80, 
nv_surface=0x3e92620, nv_surface=0x3e92620, lock_buffer_params=0x7ffd063a4fa0, 
lock_buffer_params=0x7ffd063a4fa0, frame=0x40bf3e0) at src/libavcodec/nvenc.c:1312
#4  nvenc_upload_frame (nvenc_frame=0x3e92620, frame=0x40bf3e0, avctx=0x39ede80) 
at src/libavcodec/nvenc.c:1430
#5  ff_nvenc_encode_frame (avctx=0x39ede80, pkt=0x7ffd063a64e0, frame=0x40bf3e0, 
got_packet=0x7ffd063a639c) at src/libavcodec/nvenc.c:1660
#6  0x0000000000acb003 in avcodec_encode_video2 (avctx=avctx at entry=0x39ede80, 
avpkt=avpkt at entry=0x7ffd063a64e0, frame=frame at entry=0x40bf3e0, 
got_packet_ptr=got_packet_ptr at entry=0x7ffd063a639c) at src/libavcodec/utils.c:1961
#7  0x0000000000496cb5 in do_video_out (s=0x39ec2a0, ost=ost at entry=0x39edc80, 
next_picture=next_picture at entry=0x40bf3e0, sync_ipts=<optimized out>, 
sync_ipts at entry=-7.62939453125e-06) at src/ffmpeg.c:1175
#8  0x0000000000497f6d in reap_filters (flush=flush at entry=0) at src/ffmpeg.c:1366
#9  0x000000000049ab1c in transcode_step () at src/ffmpeg.c:4117
#10 transcode () at src/ffmpeg.c:4161
#11 0x000000000047d0ac in main (argc=<optimized out>, argv=0x7ffd063a6e28) at 
src/ffmpeg.c:4356