[FFmpeg-user] FFmpeg single threaded bottleneck
Gabriel Balaich
roderrooder at gmail.com
Thu May 14 22:49:41 EEST 2020
So just read a question on stack exchange in-which someone was experiencing
something similar to me, it looked like the issue may have been the audio
codec being used was limited to one thread for processing, here is the
question:
https://video.stackexchange.com/questions/15996/ffmpeg-encoding-and-core-usage
So I decided to remove every audio device / stream from a command for
testing:
ffmpeg -y `
-thread_queue_size 9999 -indexmem 9999 -f dshow -rtbufsize 2147.48M
-video_size 3440x1440 -framerate 100 `
-pixel_format nv12 -i video="Video (00 Pro Capture HDMI 4K+)" `
-map 0 -c:v h264_nvenc -preset: hp -r 100 -rc-lookahead 200 -pix_fmt nv12
-b:v 288M -minrate 288M `
-maxrate 288M -bufsize 288M -an `
-vsync 1 -max_muxing_queue_size 9999 `
C:\Users\gabri\Videos\MW1.ts `
-map 0 -c:v h264_nvenc -preset: hp -r 100 -rc-lookahead 200 -pix_fmt nv12
-b:v 288M -minrate 288M `
-maxrate 288M -bufsize 288M -an `
-vsync 1 -max_muxing_queue_size 9999 `
C:\Users\gabri\Videos\MW2.ts `
-map 0 -c:v h264_nvenc -preset: hp -r 100 -rc-lookahead 200 -pix_fmt nv12
-b:v 288M -minrate 288M `
-maxrate 288M -bufsize 288M -an `
-vsync 1 -max_muxing_queue_size 9999 `
C:\Users\gabri\Videos\MW3.ts `
-map 0 -c:v h264_nvenc -preset: hp -r 100 -rc-lookahead 200 -pix_fmt nv12
-b:v 288M -minrate 288M `
-maxrate 288M -bufsize 288M -an `
-vsync 1 -max_muxing_queue_size 9999 `
C:\Users\gabri\Videos\MW4.ts
3440x1440 at 100FPS is essentially 4K60 in terms of bandwidth, so with the
above command, I am essentially encoding 16K60 in real-time (see attached
log for full output). For a second I thought I may have found the issue,
that being that the audio processing is being limited to one thread...
Alas, that same thread on my processor that seems to be getting most of the
processing load on the commands with audio streams included is still
getting most of the processing load without the audio streams included,
albeit to a lesser extent:
https://i.postimg.cc/2j9LLSRc/image.png
[image: image.png]
Though it's interesting nonetheless that when encoding that much more video
in a single command the thread in question is under a less stressful load
than when encoding much less video but several audio streams. This seems to
just further confirm to me that some aspect of every transcode in a single
FFmpeg instance that is being run through a single processing thread, which
is where my bottleneck resides.
ISTR a thread here not that long ago about threading and hardware
> acceleration; may want to search that out.
>
Thanks for the heads up, but I can't seem to find it in the last 100
threads, I'll keep looking.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 97742 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-user/attachments/20200514/117c06e6/attachment.png>
-------------- next part --------------
PS C:\Users\gabri> ffmpeg -y `
>> -thread_queue_size 9999 -indexmem 9999 -f dshow -rtbufsize 2147.48M -video_size 3440x1440 -framerate 100 `
>> -pixel_format nv12 -i video="Video (00 Pro Capture HDMI 4K+)" `
>> -map 0 -c:v h264_nvenc -preset: hp -r 100 -rc-lookahead 200 -pix_fmt nv12 -b:v 288M -minrate 288M `
>> -maxrate 288M -bufsize 288M -an `
>> -vsync 1 -max_muxing_queue_size 9999 `
>> C:\Users\gabri\Videos\MW1.ts `
>> -map 0 -c:v h264_nvenc -preset: hp -r 100 -rc-lookahead 200 -pix_fmt nv12 -b:v 288M -minrate 288M `
>> -maxrate 288M -bufsize 288M -an `
>> -vsync 1 -max_muxing_queue_size 9999 `
>> C:\Users\gabri\Videos\MW2.ts `
>> -map 0 -c:v h264_nvenc -preset: hp -r 100 -rc-lookahead 200 -pix_fmt nv12 -b:v 288M -minrate 288M `
>> -maxrate 288M -bufsize 288M -an `
>> -vsync 1 -max_muxing_queue_size 9999 `
>> C:\Users\gabri\Videos\MW3.ts `
>> -map 0 -c:v h264_nvenc -preset: hp -r 100 -rc-lookahead 200 -pix_fmt nv12 -b:v 288M -minrate 288M `
>> -maxrate 288M -bufsize 288M -an `
>> -vsync 1 -max_muxing_queue_size 9999 `
>> C:\Users\gabri\Videos\MW4.ts
ffmpeg version git-2020-04-03-52523b6 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 9.3.1 (GCC) 20200328
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libsrt --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
libavutil 56. 42.102 / 56. 42.102
libavcodec 58. 77.101 / 58. 77.101
libavformat 58. 42.100 / 58. 42.100
libavdevice 58. 9.103 / 58. 9.103
libavfilter 7. 77.101 / 7. 77.101
libswscale 5. 6.101 / 5. 6.101
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
Input #0, dshow, from 'video=Video (00 Pro Capture HDMI 4K+)':
Duration: N/A, start: 6654.776000, bitrate: N/A
Stream #0:0: Video: rawvideo (NV12 / 0x3231564E), nv12, 3440x1440, 100 fps, 100 tbr, 10000k tbn, 10000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_nvenc))
Stream #0:0 -> #1:0 (rawvideo (native) -> h264 (h264_nvenc))
Stream #0:0 -> #2:0 (rawvideo (native) -> h264 (h264_nvenc))
Stream #0:0 -> #3:0 (rawvideo (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
Output #0, mpegts, to 'C:\Users\gabri\Videos\MW1.ts':
Metadata:
encoder : Lavf58.42.100
Stream #0:0: Video: h264 (h264_nvenc) (Main), nv12, 3440x1440, q=-1--1, 288000 kb/s, 100 fps, 90k tbn, 100 tbc
Metadata:
encoder : Lavc58.77.101 h264_nvenc
Side data:
cpb: bitrate max/min/avg: 288000000/0/288000000 buffer size: 288000000 vbv_delay: N/A
Output #1, mpegts, to 'C:\Users\gabri\Videos\MW2.ts':
Metadata:
encoder : Lavf58.42.100
Stream #1:0: Video: h264 (h264_nvenc) (Main), nv12, 3440x1440, q=-1--1, 288000 kb/s, 100 fps, 90k tbn, 100 tbc
Metadata:
encoder : Lavc58.77.101 h264_nvenc
Side data:
cpb: bitrate max/min/avg: 288000000/0/288000000 buffer size: 288000000 vbv_delay: N/A
Output #2, mpegts, to 'C:\Users\gabri\Videos\MW3.ts':
Metadata:
encoder : Lavf58.42.100
Stream #2:0: Video: h264 (h264_nvenc) (Main), nv12, 3440x1440, q=-1--1, 288000 kb/s, 100 fps, 90k tbn, 100 tbc
Metadata:
encoder : Lavc58.77.101 h264_nvenc
Side data:
cpb: bitrate max/min/avg: 288000000/0/288000000 buffer size: 288000000 vbv_delay: N/A
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (62% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (64% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (65% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (66% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (67% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (68% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (69% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (70% of size: 2147480000 [rtbufsize parameter])! frame dropped!
Output #3, mpegts, to 'C:\Users\gabri\Videos\MW4.ts':
Metadata:
encoder : Lavf58.42.100
Stream #3:0: Video: h264 (h264_nvenc) (Main), nv12, 3440x1440, q=-1--1, 288000 kb/s, 100 fps, 90k tbn, 100 tbc
Metadata:
encoder : Lavc58.77.101 h264_nvenc
Side data:
cpb: bitrate max/min/avg: 288000000/0/288000000 buffer size: 288000000 vbv_delay: N/A
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (70% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (69% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (68% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (67% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (66% of size: 2147480000 [rtbufsize parameter])! frame dropped!
Last message repeated 1 times
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (65% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (64% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (63% of size: 2147480000 [rtbufsize parameter])! frame dropped!
[dshow @ 00000118c7d43600] real-time buffer [Video (00 Pro Capture HDMI 4K+)] [video input] too full or near too full (62% of size: 2147480000 [rtbufsize parameter])! frame dropped!
Last message repeated 1 times
frame= 6263 fps= 99 q=9.0 Lq=9.0 q=9.0 q=9.0 size= 133021kB time=00:01:02.62 bitrate=17401.9kbits/s dup=80 drop=0 speed=0.995x
video:517043kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
More information about the ffmpeg-user
mailing list