[FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened

Thu Dec 28 16:46:10 EET 2023

It must be in the latest ffmpeg because I took all the hwupload and hwdownload out of my command and everything still works.

One question I have is:

Why is hwdownload not needed before the pad filter and hwupload needed after the pad filter. Before trying out intel's hw transcoding I used nvidia's cards and as far as I remember one always had to hwdownload when doing cpu filters and hwupload when going back to gpu filters or gpu encoding.

Anyway, thanks for the help, updated command survives resolution changes.

________________________________
From: ffmpeg-user <ffmpeg-user-bounces at ffmpeg.org> on behalf of Dennis Mungai <dmngaie at gmail.com>
Sent: Thursday, December 28, 2023 1:23 AM
To: FFmpeg user questions <ffmpeg-user at ffmpeg.org>
Cc: artem.galin at gmail.com <artem.galin at gmail.com>; Wang, Fei W <fei.w.wang at intel.com>
Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while filtering: Internal bug, should not have happened

On Thu, 28 Dec 2023, 05:15 Xiang, Haihao, <
haihao.xiang-at-intel.com at ffmpeg.org> wrote:

>
> > I have created a bug and linked a sample video that changes resolutions
> on the
> > fly:
> >
> > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftrac.ffmpeg.org%2Fticket%2F10762&data=05%7C02%7Cshanew%40innovsys.com%7C16af3691f50e441b105e08dc0775eba2%7C7a48ce45ee974a95ac183390878a179b%7C0%7C0%7C638393450253894506%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=oxcuG2TePe4NNYrzXnCJkJsw5X5BDIJP6nIUzUvfSaM%3D&reserved=0<https://trac.ffmpeg.org/ticket/10762>
> >
> > Get back to me if you need anything else I can help with.
> > ________________________________
> > From: ffmpeg-user <ffmpeg-user-bounces at ffmpeg.org> on behalf of Dennis
> Mungai
> > <dmngaie at gmail.com>
> > Sent: Wednesday, December 27, 2023 11:38 AM
> > To: FFmpeg user questions <ffmpeg-user at ffmpeg.org>
> > Cc: Haihao <haihao.xiang at intel.com>;
> > artem.galin at gmail.com <artem.galin at gmail.com>;
> > fei.w.wang at intel.com <fei.w.wang at intel.com>
> > Subject: Re: [FFmpeg-user] Intel QSV and xstack_qsv: Error while
> filtering:
> > Internal bug, should not have happened
> >
> > [You don't often get email from dmngaie at gmail.com. Learn why this is
> important
> > at https://aka.ms/LearnAboutSenderIdentification ]
> >
> > This email originated from outside Innovative Systems. Do not click
> links or
> > open attachments unless you recognize the sender and know the content is
> safe.
> >
> >
> > On Wed, 27 Dec 2023 at 18:40, Shane Warren <shanew at innovsys.com> wrote:
> >
> > >
> > > I'm testing out an Intel Flex 140 card and I am attempting to
> transcode 4
> > > inputs to 1 output using the xstack_qsv filter.  My command takes in 4
> udp
> > > multicast ts files (h264 encoded) and outputs one h264_qsv encoded udp
> > > transport stream. My command looks like so:
> > >
> > > /tmp/ffmpeg_g  -nostats -loglevel verbose -init_hw_device
> > > qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1
>
> Please use a render node instead, and you should use the key of
> child_device if
> you want to use a non-default render node:
>
> qsv=card1,child_device=/dev/dri/renderD129 -hwaccel_device card1 -
> filter_hw_device card1
>
>
> > > -fflags discardcorrupt  -fflags +genpts -fflags discardcorrupt  \
> > > -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> > > -async_depth 1 -i "udp://@
> > >
> 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
> > > _nonfatal=1"
> > > \
> > > -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> > > -async_depth 1 -i "udp://@
> > >
> 225.105.0.14:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overru
> > > n_nonfatal=1"
> > > \
> > > -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> > > -async_depth 1 -i "udp://@
> > >
> 225.105.0.5:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
> > > _nonfatal=1"
> > > \
> > > -hwaccel qsv -hwaccel_output_format qsv -thread_queue_size 4096
> > > -async_depth 1 -i "udp://@
> > >
> 225.105.0.4:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
> > > _nonfatal=1"
> > > \
> > > -noautoscale -filter_complex "\
> > > [0:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v1]; \
> > > [1:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v2]; \
> > > [2:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v3]; \
> > > [3:v]vpp_qsv=deinterlace=2:w=720:h=480:framerate=30000/1001[v4]; \
> > > [v1][v2][v3][v4]
> xstack_qsv=inputs=4:layout=0_0|0_h0|w0_0|w0_h0[mosiac]; \
> > > [mosiac]vpp_qsv=w=1920:h=1080,hwdownload,format=nv12,pad=1920:1080:(ow-
> > > iw)/2:(oh-ih)/2,hwupload=extra_hw_frames=64,format=qsv[v]"
>
> -c:v h264_qsv encoder can accept data in system memory, you may remove
> 'hwupload=extra_hw_frames=64,format=qsv' from the filtergraph.
>
> If you want to use hwupload filter to upload data from system memory to
> video
> memory before -c:v h264_qsv encoder, could you try with
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.ffmpeg.org%2Fproject%2Fffmpeg%2Flist%2F%3Fseries%3D10318&data=05%7C02%7Cshanew%40innovsys.com%7C16af3691f50e441b105e08dc0775eba2%7C7a48ce45ee974a95ac183390878a179b%7C0%7C0%7C638393450253894506%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZlVkTZJm%2BEZLHToo5dKMfK70Pc1AgixiiH5UQiuf4aI%3D&reserved=0<https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318> which
> added the
> support for dynamic frame pool in qsv (Note a newer version of
> oneVPL-intel-gpu
> is required, you may download oneVPL-intel-gpu from
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Foneapi-src%2FoneVPL-intel-gpu%2Ftags&data=05%7C02%7Cshanew%40innovsys.com%7C16af3691f50e441b105e08dc0775eba2%7C7a48ce45ee974a95ac183390878a179b%7C0%7C0%7C638393450253894506%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DzItmcdB1xbz%2BrnChvp7%2B%2FG%2F8vmNzD9bbB8boVvNW10%3D&reserved=0<https://github.com/oneapi-src/oneVPL-intel-gpu/tags> , and should not use
> extra_hw_frame=64 anymore )
>
> Thanks
> Haihao
>
>
> > > \
> > > -map "[v]" -map 0:a:0 -map 1:a:0 -map 2:a:0 -map 3:a:0 \
> > > -c:v h264_qsv -noautoscale -async_depth 1 -scenario livestreaming -r:v
> > > 30000/1001 -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -bufsize:v
> 12000k
> > > -threads 1 -profile:v high -bf:v 0 -g:v 15 \
> > > -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a 192k
> > > -fps_mode auto \
> > > -f mpegts -muxrate 7450599 -pes_payload_size 1528 "udp://@
> > >
> 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=7450599&burst_bit
> > > s=10528
> > > "
> > >
> > > The issue is eventually stream 1 changes resolution from 1280x720
> > > (progressive) to 1920x1080 (interlaced). The stream dies with the
> following
> > > logs coming out:
> > >
> > > [h264_qsv @ 0x55a240e68000] Error submitting the frame for encoding.
> > > [vost#0:0/h264_qsv @ 0x55a240e30780] Error submitting video frame to
> the
> > > encoder
> > > Error while filtering: Internal bug, should not have happened
> > >
> > > I can transcode the single stream that changes resolution on its own
> and
> > > it can survive a resolution change using this command:
> > >
> > > /tmp/ffmpeg_g  -nostats -loglevel verbose -init_hw_device
> > > qsv=card1:/dev/dri/card1 -hwaccel_device card1 -filter_hw_device card1
> > > -fflags discardcorrupt  -fflags +genpts -fflags discardcorrupt
> -hwaccel
> > > qsv -hwaccel_output_format qsv -async_depth 1 -i "udp://@
> > >
> 225.105.0.1:10102?fifo_size=214688&buffer_size=851968&timeout=800000&overrun
> > > _nonfatal=1"
> > > -vf "vpp_qsv=deinterlace=2:w=1920:h=1080:framerate=30000/1001" -map
> "0:v"
> > > -map 0:a:0 -c:v h264_qsv -noautoscale  -async_depth 1 -scenario
> > > livestreaming -r:v 30000/1001 -b:v 5000k -minrate:v 5000k -maxrate:v
> 5000k
> > > -bufsize:v 10000k -a53cc 1 -forced_idr 1 -threads 1 -profile:v high
> -bf:v 0
> > > -g:v 15 -filter:a "aresample=async=1" -c:a aac -ac:a 2 -ar:a 48000 -b:a
> > > 192k -fps_mode auto -f mpegts -muxrate 6450599 -pes_payload_size 1528
> > > "udp://@
> > >
> 229.100.100.44:10102?pkt_size=1316&fifo_size=90000&bitrate=6450599&burst_bit
> > > s=10528
> > > "
> > >
> > > When that stream changes resolution I see these come out in the log,
> and
> > > the stream carries on without dying:
> > >
> > > [h264_qsv @ 0x561bc472f8c0] Video parameter change
> > > [AVHWDeviceContext @ 0x7f313000d1c0] VAAPI driver: Intel iHD driver for
> > > Intel(R) Gen Graphics - 23.2.4 ().
> > > [AVHWDeviceContext @ 0x7f313000d1c0] Driver not found in known
> nonstandard
> > > list, using standard behaviour.
> > > [h264_qsv @ 0x561bc472f8c0] Decoder: output is video memory surface
> > >
> > > My quesiton is why does the stream die when using xstack_qsv but works
> > > fine on its own.
> > >
> >
> > This is definitely a bug, something to do with frame pool setting(s) in
> > that filter chain. You may want to open a ticket for this on ffmpeg trac.
> > The xstack_qsv may need a dynamic frame pool workaround to address this
> > anomaly on video parameter changes.
> > Adding in the Intel guys on this one.
>

Is this patchwork
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.ffmpeg.org%2Fproject%2Fffmpeg%2Flist%2F%3Fseries%3D10318&data=05%7C02%7Cshanew%40innovsys.com%7C16af3691f50e441b105e08dc0775eba2%7C7a48ce45ee974a95ac183390878a179b%7C0%7C0%7C638393450253894506%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZlVkTZJm%2BEZLHToo5dKMfK70Pc1AgixiiH5UQiuf4aI%3D&reserved=0<https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10318> already
present in ffmpeg-cartwheel?

>
_______________________________________________
ffmpeg-user mailing list
ffmpeg-user at ffmpeg.org
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fffmpeg.org%2Fmailman%2Flistinfo%2Fffmpeg-user&data=05%7C02%7Cshanew%40innovsys.com%7C16af3691f50e441b105e08dc0775eba2%7C7a48ce45ee974a95ac183390878a179b%7C0%7C0%7C638393450253894506%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dQ914ybkb%2FTpul8SqZ0KMm3jOUejwz1tSTUpRt%2BRwU8%3D&reserved=0<https://ffmpeg.org/mailman/listinfo/ffmpeg-user>

To unsubscribe, visit link above, or email
ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".