[FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened

Xiang, Haihao haihao.xiang at intel.com
Thu Dec 28 04:14:59 EET 2023 

> I have created a bug and linked a sample video that changes resolutions on the
> fly:
> 
> https://trac.ffmpeg.org/ticket/10762
> 
> Get back to me if you need anything else I can help with.
> ________________________________
> From: ffmpeg-user <ffmpeg-user-bounces at ffmpeg.org> on behalf of Dennis Mungai
> <dmngaie at gmail.com>
> Sent: Wednesday, December 27, 2023 11:38 AM
> To: FFmpeg user questions <ffmpeg-user at ffmpeg.org>
> Cc: Haihao <haihao.xiang at intel.com>;
> artem.galin at gmail.com <artem.galin at gmail.com>;
> fei.w.wang at intel.com <fei.w.wang at intel.com>
> Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering:
> Internal bug, should not have happened
> 
> [You don't often get email from dmngaie at gmail.com. Learn why this is important
> at https://aka.ms/LearnAboutSenderIdentification ]
> 
> This email originated from outside Innovative Systems. Do not click links or
> open attachments unless you recognize the sender and know the content is safe.
> 
> 
> On Wed, 27 Dec 2023 at 18:40, Shane Warren <shanew at innovsys.com> wrote:
> 
> > 
> > I'm testing out an Intel Flex 140 card and I am attempting to transcode 4
> > inputs to 1 output using the xstack_qsv filter.  My command takes in 4 udp
> > multicast ts files (h264 encoded) and outputs one h264_qsv encoded udp
> > transport stream. My command looks like so:
> > 
> > /tmp/ffmpeg_g  -nostats -loglevel verbose -init_hw_device
> > qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1

Please use a render node instead, and you should use the key of child_device if
you want to use a non-default render node:

qsv=card1,child_device=/dev/dri/renderD129 -hwaccel_device card1 -
filter_hw_device card1


> > -fflags discardcorrupt  -fflags +genpts -fflags discardcorrupt  \
> > -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> > -async_depth 1 -i "udp://@
> > 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
> > _nonfatal=1"
> > \
> > -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> > -async_depth 1 -i "udp://@
> > 225.105.0.14:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overru
> > n_nonfatal=1"
> > \
> > -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> > -async_depth 1 -i "udp://@
> > 225.105.0.5:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
> > _nonfatal=1"
> > \
> > -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> > -async_depth 1 -i "udp://@
> > 225.105.0.4:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
> > _nonfatal=1"
> > \
> > -noautoscale -filter_complex "\
> > [0:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v1]; \
> > [1:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v2]; \
> > [2:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v3]; \
> > [3:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v4]; \
> > [v1][v2][v3][v4] xstack_qsv=inputs=4:layout=0_0|0_h0|w0_0|w0_h0[mosiac]; \
> > [mosiac]vpp_qsv=w=1920:h=1080,hwdownload,format=nv12,pad=1920:1080:(ow-
> > iw)/2:(oh-ih)/2,hwupload=extra_hw_frames=64,format=qsv[v]"

-c:v h264_qsv encoder can accept data in system memory, you may remove
'hwupload=extra_hw_frames=64,format=qsv' from the filtergraph. 

If you want to use hwupload filter to upload data from system memory to video
memory before -c:v h264_qsv encoder, could you try with
https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318 which added the
support for dynamic frame pool in qsv (Note a newer version of oneVPL-intel-gpu
is required, you may download oneVPL-intel-gpu from
https://github.com/oneapi-src/oneVPL-intel-gpu/tags , and should not use
extra_hw_frame=64 anymore )

Thanks
Haihao


> > \
> > -map "[v]" -map 0:a:0 -map 1:a:0 -map 2:a:0 -map 3:a:0 \
> > -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v
> > 30000/1001 -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -bufsize:v 12000k
> > -threads 1 -profile:v high -bf:v 0 -g:v 15 \
> > -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k
> > -fps_mode auto \
> > -f mpegts -muxrate 7450599 -pes_payload_size 1528 "udp://@
> > 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=7450599&burst_bit
> > s=10528
> > "
> > 
> > The issue is eventually stream 1 changes resolution from 1280x720
> > (progressive) to 1920x1080 (interlaced). The stream dies with the following
> > logs coming out:
> > 
> > [h264_qsv @ 0x55a240e68000] Error submitting the frame for encoding.
> > [vost#0:0/h264_qsv @ 0x55a240e30780] Error submitting video frame to the
> > encoder
> > Error while filtering: Internal bug, should not have happened
> > 
> > I can transcode the single stream that changes resolution on its own and
> > it can survive a resolution change using this command:
> > 
> > /tmp/ffmpeg_g  -nostats -loglevel verbose -init_hw_device
> > qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1
> > -fflags discardcorrupt  -fflags +genpts -fflags discardcorrupt  -hwaccel
> > qsv -hwaccel_output_format qsv -async_depth 1 -i "udp://@
> > 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
> > _nonfatal=1"
> > -vf "vpp_qsv=deinterlace=2:w=1920:h=1080:framerate=30000/1001" -map "0:v"
> > -map 0:a:0 -c:v h264_qsv -noautoscale  -async_depth 1 -scenario
> > livestreaming -r:v 30000/1001 -b:v 5000k -minrate:v 5000k -maxrate:v 5000k
> > -bufsize:v 10000k -a53cc 1 -forced_idr 1 -threads 1 -profile:v high -bf:v 0
> > -g:v 15 -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a
> > 192k -fps_mode auto -f mpegts -muxrate 6450599 -pes_payload_size 1528
> > "udp://@
> > 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=6450599&burst_bit
> > s=10528
> > "
> > 
> > When that stream changes resolution I see these come out in the log, and
> > the stream carries on without dying:
> > 
> > [h264_qsv @ 0x561bc472f8c0] Video parameter change
> > [AVHWDeviceContext @ 0x7f313000d1c0] VAAPI driver: Intel iHD driver for
> > Intel(R) Gen Graphics - 23.2.4 ().
> > [AVHWDeviceContext @ 0x7f313000d1c0] Driver not found in known nonstandard
> > list, using standard behaviour.
> > [h264_qsv @ 0x561bc472f8c0] Decoder: output is video memory surface
> > 
> > My quesiton is why does the stream die when using xstack_qsv but works
> > fine on its own.
> > 
> 
> This is definitely a bug, something to do with frame pool setting(s) in
> that filter chain. You may want to open a ticket for this on ffmpeg trac.
> The xstack_qsv may need a dynamic frame pool workaround to address this
> anomaly on video parameter changes.
> Adding in the Intel guys on this one.
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user<
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user>
> 
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> 
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".

More information about the ffmpeg-user mailing list