[FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened

[Shane Warren]

I have created a bug and linked a sample video that changes resolutions on the fly:

https://trac.ffmpeg.org/ticket/10762

Get back to me if you need anything else I can help with.
________________________________
From: ffmpeg-user <ffmpeg-user-bounces at ffmpeg.org> on behalf of Dennis Mungai <dmngaie at gmail.com>
Sent: Wednesday, December 27, 2023 11:38 AM
To: FFmpeg user questions <ffmpeg-user at ffmpeg.org>
Cc: Haihao <haihao.xiang at intel.com>; artem.galin at gmail.com <artem.galin at gmail.com>; fei.w.wang at intel.com <fei.w.wang at intel.com>
Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened

[You don't often get email from dmngaie at gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

This email originated from outside Innovative Systems. Do not click links or open attachments unless you recognize the sender and know the content is safe.


On Wed, 27 Dec 2023 at 18:40, Shane Warren <shanew at innovsys.com> wrote:

>
> I'm testing out an Intel Flex 140 card and I am attempting to transcode 4
> inputs to 1 output using the xstack_qsv filter.  My command takes in 4 udp
> multicast ts files (h264 encoded) and outputs one h264_qsv encoded udp
> transport stream. My command looks like so:
>
> /tmp/ffmpeg_g  -nostats -loglevel verbose -init_hw_device
> qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1
> -fflags discardcorrupt  -fflags +genpts -fflags discardcorrupt  \
> -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> -async_depth 1 -i "udp://@
> 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1"
> \
> -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> -async_depth 1 -i "udp://@
> 225.105.0.14:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1"
> \
> -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> -async_depth 1 -i "udp://@
> 225.105.0.5:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1"
> \
> -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> -async_depth 1 -i "udp://@
> 225.105.0.4:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1"
> \
> -noautoscale -filter_complex "\
> [0:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v1]; \
> [1:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v2]; \
> [2:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v3]; \
> [3:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v4]; \
> [v1][v2][v3][v4] xstack_qsv=inputs=4:layout=0_0|0_h0|w0_0|w0_h0[mosiac]; \
> [mosiac]vpp_qsv=w=1920:h=1080,hwdownload,format=nv12,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,hwupload=extra_hw_frames=64,format=qsv[v]"
> \
> -map "[v]" -map 0:a:0 -map 1:a:0 -map 2:a:0 -map 3:a:0 \
> -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v
> 30000/1001 -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -bufsize:v 12000k
> -threads 1 -profile:v high -bf:v 0 -g:v 15 \
> -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k
> -fps_mode auto \
> -f mpegts -muxrate 7450599 -pes_payload_size 1528 "udp://@
> 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=7450599&burst_bits=10528
> "
>
> The issue is eventually stream 1 changes resolution from 1280x720
> (progressive) to 1920x1080 (interlaced). The stream dies with the following
> logs coming out:
>
> [h264_qsv @ 0x55a240e68000] Error submitting the frame for encoding.
> [vost#0:0/h264_qsv @ 0x55a240e30780] Error submitting video frame to the
> encoder
> Error while filtering: Internal bug, should not have happened
>
> I can transcode the single stream that changes resolution on its own and
> it can survive a resolution change using this command:
>
> /tmp/ffmpeg_g  -nostats -loglevel verbose -init_hw_device
> qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1
> -fflags discardcorrupt  -fflags +genpts -fflags discardcorrupt  -hwaccel
> qsv -hwaccel_output_format qsv -async_depth 1 -i "udp://@
> 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun_nonfatal=1"
> -vf "vpp_qsv=deinterlace=2:w=1920:h=1080:framerate=30000/1001" -map "0:v"
> -map 0:a:0 -c:v h264_qsv -noautoscale  -async_depth 1 -scenario
> livestreaming -r:v 30000/1001 -b:v 5000k -minrate:v 5000k -maxrate:v 5000k
> -bufsize:v 10000k -a53cc 1 -forced_idr 1 -threads 1 -profile:v high -bf:v 0
> -g:v 15 -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a
> 192k -fps_mode auto -f mpegts -muxrate 6450599 -pes_payload_size 1528
> "udp://@
> 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=6450599&burst_bits=10528
> "
>
> When that stream changes resolution I see these come out in the log, and
> the stream carries on without dying:
>
> [h264_qsv @ 0x561bc472f8c0] Video parameter change
> [AVHWDeviceContext @ 0x7f313000d1c0] VAAPI driver: Intel iHD driver for
> Intel(R) Gen Graphics - 23.2.4 ().
> [AVHWDeviceContext @ 0x7f313000d1c0] Driver not found in known nonstandard
> list, using standard behaviour.
> [h264_qsv @ 0x561bc472f8c0] Decoder: output is video memory surface
>
> My quesiton is why does the stream die when using xstack_qsv but works
> fine on its own.
>

This is definitely a bug, something to do with frame pool setting(s) in
that filter chain. You may want to open a ticket for this on ffmpeg trac.
The xstack_qsv may need a dynamic frame pool workaround to address this
anomaly on video parameter changes.
Adding in the Intel guys on this one.
_______________________________________________
ffmpeg-user mailing list
ffmpeg-user at ffmpeg.org
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fffmpeg.org%2Fmailman%2Flistinfo%2Fffmpeg-user&data=05%7C02%7Cshanew%40innovsys.com%7C845c76c6fb5043f1d7b508dc0702bfbb%7C7a48ce45ee974a95ac183390878a179b%7C0%7C0%7C638392955596088321%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zOOg0JRDiaJ5MpLbWQBjapaeFEyXsr32NsL2DLFNlUw%3D&reserved=0<https://ffmpeg.org/mailman/listinfo/ffmpeg-user>

To unsubscribe, visit link above, or email
ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".