[FFmpeg-devel] [PATCH v2] lavfi: add nlmeans CUDA filter

[Timo Rothenpieler]

On 05.09.2021 20:02, Dylan Fernando wrote:
> 
> Impossible to convert between the formats supported by the filter 'graph 0
> input from stream 0:0' and the filter 'auto_scale_0'
> 
> Thread 1 "ffmpeg" received signal SIGSEGV, Segmentation fault.
> 0x00005555556726eb in uninit (ctx=0x55555817e800) at
> libavfilter/vf_nlmeans_cuda.c:704
> 704         CudaFunctions *cu = s->hwctx->internal->cuda_dl;

Seems like some error state can cause uninit to be called before the 
hwctx is properly set up.
So it needs a guard against that case.

> (gdb) backtrace
> #0  0x00005555556726eb in uninit (ctx=0x55555817e800) at
> libavfilter/vf_nlmeans_cuda.c:704
> #1  0x0000555555742e65 in avfilter_free (filter=0x55555817e800) at
> libavfilter/avfilter.c:769
> #2  0x0000555555744cac in avfilter_graph_free (graph=0x55555716ded0) at
> libavfilter/avfiltergraph.c:126
> #3  0x0000555555707ae0 in cleanup_filtergraph (fg=0x55555716dec0) at
> fftools/ffmpeg_filter.c:952
> #4  configure_filtergraph (fg=fg at entry=0x55555716dec0) at
> fftools/ffmpeg_filter.c:1130
> #5  0x000055555571b060 in ifilter_send_frame (frame=0x555557a72d00,
> ifilter=0x55555716db40) at fftools/ffmpeg.c:2242
> #6  send_frame_to_filters (ist=ist at entry=0x55555716c5c0,
> decoded_frame=decoded_frame at entry=0x555557a72d00) at fftools/ffmpeg.c:2323
> #7  0x000055555571c204 in decode_video (decode_failed=<optimized out>,
> eof=<optimized out>, duration_pts=<optimized out>, got_output=<optimized
> out>, pkt=<optimized out>, ist=<optimized out>)
>      at fftools/ffmpeg.c:2520
> #8  process_input_packet (ist=ist at entry=0x55555716c5c0, pkt=0x55555716c7c0,
> no_eof=no_eof at entry=0) at fftools/ffmpeg.c:2682
> #9  0x000055555571daee in process_input (file_index=<optimized out>) at
> fftools/ffmpeg.c:4636
> #10 transcode_step () at fftools/ffmpeg.c:4776
> #11 transcode () at fftools/ffmpeg.c:4830
> #12 0x00005555556f84a7 in main (argc=<optimized out>, argv=<optimized out>)
> at fftools/ffmpeg.c:5035

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4494 bytes
Desc: S/MIME Cryptographic Signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20210905/7645ab6e/attachment.bin>