[FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened

Thu Dec 28 09:23:13 EET 2023

On Thu, 28 Dec 2023, 05:15 Xiang, Haihao, <
haihao.xiang-at-intel.com at ffmpeg.org> wrote:

>
> > I have created a bug and linked a sample video that changes resolutions
> on the
> > fly:
> >
> > https://trac.ffmpeg.org/ticket/10762
> >
> > Get back to me if you need anything else I can help with.
> > ________________________________
> > From: ffmpeg-user <ffmpeg-user-bounces at ffmpeg.org> on behalf of Dennis
> Mungai
> > <dmngaie at gmail.com>
> > Sent: Wednesday, December 27, 2023 11:38 AM
> > To: FFmpeg user questions <ffmpeg-user at ffmpeg.org>
> > Cc: Haihao <haihao.xiang at intel.com>;
> > artem.galin at gmail.com <artem.galin at gmail.com>;
> > fei.w.wang at intel.com <fei.w.wang at intel.com>
> > Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while
> filtering:
> > Internal bug, should not have happened
> >
> > [You don't often get email from dmngaie at gmail.com. Learn why this is
> important
> > at https://aka.ms/LearnAboutSenderIdentification ]
> >
> > This email originated from outside Innovative Systems. Do not click
> links or
> > open attachments unless you recognize the sender and know the content is
> safe.
> >
> >
> > On Wed, 27 Dec 2023 at 18:40, Shane Warren <shanew at innovsys.com> wrote:
> >
> > >
> > > I'm testing out an Intel Flex 140 card and I am attempting to
> transcode 4
> > > inputs to 1 output using the xstack_qsv filter.  My command takes in 4
> udp
> > > multicast ts files (h264 encoded) and outputs one h264_qsv encoded udp
> > > transport stream. My command looks like so:
> > >
> > > /tmp/ffmpeg_g  -nostats -loglevel verbose -init_hw_device
> > > qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1
>
> Please use a render node instead, and you should use the key of
> child_device if
> you want to use a non-default render node:
>
> qsv=card1,child_device=/dev/dri/renderD129 -hwaccel_device card1 -
> filter_hw_device card1
>
>
> > > -fflags discardcorrupt  -fflags +genpts -fflags discardcorrupt  \
> > > -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> > > -async_depth 1 -i "udp://@
> > >
> 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
> > > _nonfatal=1"
> > > \
> > > -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> > > -async_depth 1 -i "udp://@
> > >
> 225.105.0.14:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overru
> > > n_nonfatal=1"
> > > \
> > > -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> > > -async_depth 1 -i "udp://@
> > >
> 225.105.0.5:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
> > > _nonfatal=1"
> > > \
> > > -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> > > -async_depth 1 -i "udp://@
> > >
> 225.105.0.4:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
> > > _nonfatal=1"
> > > \
> > > -noautoscale -filter_complex "\
> > > [0:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v1]; \
> > > [1:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v2]; \
> > > [2:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v3]; \
> > > [3:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v4]; \
> > > [v1][v2][v3][v4]
> xstack_qsv=inputs=4:layout=0_0|0_h0|w0_0|w0_h0[mosiac]; \
> > > [mosiac]vpp_qsv=w=1920:h=1080,hwdownload,format=nv12,pad=1920:1080:(ow-
> > > iw)/2:(oh-ih)/2,hwupload=extra_hw_frames=64,format=qsv[v]"
>
> -c:v h264_qsv encoder can accept data in system memory, you may remove
> 'hwupload=extra_hw_frames=64,format=qsv' from the filtergraph.
>
> If you want to use hwupload filter to upload data from system memory to
> video
> memory before -c:v h264_qsv encoder, could you try with
> https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318 which
> added the
> support for dynamic frame pool in qsv (Note a newer version of
> oneVPL-intel-gpu
> is required, you may download oneVPL-intel-gpu from
> https://github.com/oneapi-src/oneVPL-intel-gpu/tags , and should not use
> extra_hw_frame=64 anymore )
>
> Thanks
> Haihao
>
>
> > > \
> > > -map "[v]" -map 0:a:0 -map 1:a:0 -map 2:a:0 -map 3:a:0 \
> > > -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v
> > > 30000/1001 -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -bufsize:v
> 12000k
> > > -threads 1 -profile:v high -bf:v 0 -g:v 15 \
> > > -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k
> > > -fps_mode auto \
> > > -f mpegts -muxrate 7450599 -pes_payload_size 1528 "udp://@
> > >
> 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=7450599&burst_bit
> > > s=10528
> > > "
> > >
> > > The issue is eventually stream 1 changes resolution from 1280x720
> > > (progressive) to 1920x1080 (interlaced). The stream dies with the
> following
> > > logs coming out:
> > >
> > > [h264_qsv @ 0x55a240e68000] Error submitting the frame for encoding.
> > > [vost#0:0/h264_qsv @ 0x55a240e30780] Error submitting video frame to
> the
> > > encoder
> > > Error while filtering: Internal bug, should not have happened
> > >
> > > I can transcode the single stream that changes resolution on its own
> and
> > > it can survive a resolution change using this command:
> > >
> > > /tmp/ffmpeg_g  -nostats -loglevel verbose -init_hw_device
> > > qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1
> > > -fflags discardcorrupt  -fflags +genpts -fflags discardcorrupt
> -hwaccel
> > > qsv -hwaccel_output_format qsv -async_depth 1 -i "udp://@
> > >
> 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
> > > _nonfatal=1"
> > > -vf "vpp_qsv=deinterlace=2:w=1920:h=1080:framerate=30000/1001" -map
> "0:v"
> > > -map 0:a:0 -c:v h264_qsv -noautoscale  -async_depth 1 -scenario
> > > livestreaming -r:v 30000/1001 -b:v 5000k -minrate:v 5000k -maxrate:v
> 5000k
> > > -bufsize:v 10000k -a53cc 1 -forced_idr 1 -threads 1 -profile:v high
> -bf:v 0
> > > -g:v 15 -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a
> > > 192k -fps_mode auto -f mpegts -muxrate 6450599 -pes_payload_size 1528
> > > "udp://@
> > >
> 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=6450599&burst_bit
> > > s=10528
> > > "
> > >
> > > When that stream changes resolution I see these come out in the log,
> and
> > > the stream carries on without dying:
> > >
> > > [h264_qsv @ 0x561bc472f8c0] Video parameter change
> > > [AVHWDeviceContext @ 0x7f313000d1c0] VAAPI driver: Intel iHD driver for
> > > Intel(R) Gen Graphics - 23.2.4 ().
> > > [AVHWDeviceContext @ 0x7f313000d1c0] Driver not found in known
> nonstandard
> > > list, using standard behaviour.
> > > [h264_qsv @ 0x561bc472f8c0] Decoder: output is video memory surface
> > >
> > > My quesiton is why does the stream die when using xstack_qsv but works
> > > fine on its own.
> > >
> >
> > This is definitely a bug, something to do with frame pool setting(s) in
> > that filter chain. You may want to open a ticket for this on ffmpeg trac.
> > The xstack_qsv may need a dynamic frame pool workaround to address this
> > anomaly on video parameter changes.
> > Adding in the Intel guys on this one.
>

Is this patchwork
https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318 already
present in ffmpeg-cartwheel?

>