[FFmpeg-user] [EXTERNAL] Re: Very large memory foot print using join or adelay+amix
Paul B Mahol
onemda at gmail.com
Wed Mar 2 20:38:28 EET 2022
On Wed, Mar 2, 2022 at 6:14 PM Bryce Newman
<bryce.newman at gettyimages.com> wrote:
>
> Thank you.
> The steaming https is now working as well.
> Though the only thing that appears to be suboptimal is using -filter_complex -i http\://localhost/amix-audio.mov and movie=http\://localhost/amix-audio.mov.
> Isn’t the amix-audio.mov now being streamed twice?
If both have same unused streams then yes. Point is to have audio and
video split from each other, otherwise one of them will be buffered in
memory, once timeline manipulation is started.
> adelay=delays=41s:all=1[adelay3];[0:a:4]adelay=delays=56s:all=1[adelay4];[adelay0][adelay1][adelay2][adelay3][adelay4]amix=5,volumedetect,asplit=2[adp_audio_1][adp_audio_2];[640_adpp]scale='640x360'[640_adpp];[640_ADPP]scale='640x360'[640_ADPP]'
> > \
> > -map '[640_adpp]' -map '[adp_audio_1]' -map_metadata -1 -f mp4 -vcodec
> > libx264 -pix_fmt yuv420p -profile:v main -level 3.1 -crf 18 -b:v 2000K
> > -movflags faststart -refs 4 -color_primaries bt709 -color_trc bt709
> > -colorspace bt709 -preset slower /tmp/123456_1.mp4 \
> > -map '[640_ADPP]' -map '[adp_audio_2]' -map_metadata -1 -f mp4 -vcodec
> > libx264 -pix_fmt yuv420p -profile:v main -level 3.1 -crf 18 -b:v 2000K
> > -movflags faststart -refs 4 -color_primaries bt709 -color_trc bt709
> > -colorspace bt709 -preset slower /tmp/123456_2.mp4
> >
> > ….
> > Stream mapping:
> > Stream #0:1 (aac) -> adelay:default
> > Stream #0:2 (aac) -> adelay:default
> > Stream #0:3 (aac) -> adelay:default
> > Stream #0:4 (aac) -> adelay:default
> > Stream #0:5 (aac) -> adelay:default
> > scale:default -> Stream #0:0 (libx264)
> > asplit -> Stream #0:1 (aac)
> > scale:default -> Stream #1:0 (libx264)
> > asplit -> Stream #1:1 (aac)
> > …
> >
> > Also, in some workflows the source file is not local and we stream it. For
> > example we use https to get the file but apperantly move does not stream
> > “[Parsed_movie_0 @ 0x561d883e3800] Failed to avformat_open_input 'https'”.
> > Is there another way to do this command to support streaming the movie or
> > amovie – if the latter is applicable?
> >
>
> It is only matter of properly escaping ':' with \\\\ on shell, in windows
> it is similar. You can scale before split, it will be faster, when scaling
> to same size.
>
>
> >
> > Thank you again.
> > Bryce
> >
> > From: ffmpeg-user <ffmpeg-user-bounces at ffmpeg.org> on behalf of Paul B
> > Mahol <onemda at gmail.com>
> > Date: Tuesday, March 1, 2022 at 3:36 PM
> > To: FFmpeg user questions <ffmpeg-user at ffmpeg.org>
> > Subject: [EXTERNAL] Re: [FFmpeg-user] Very large memory foot print using
> > join or adelay+amix
> > On Tue, Mar 1, 2022 at 11:26 PM Bryce Newman <bryce.newman at gettyimages.com
> > >
> > wrote:
> >
> > > Hello,
> > >
> > > Using FFmpeg 4.4.1_3 or 5.0 the command below causes FFmpeg to use 6.5GB
> > > of RAM.
> > > The input file amix-audio.mov is only 348MB. I have also seen the same
> > > large memory footprint using join.
> > > Using either join or adelay+amix both merge the audio correctly into both
> > > outputs, but don’t expect FFmpeg to allocate 6.5GB of RAM.
> > > Am I doing something incorrectly?
> > > Am I not optimizing something that I should?
> > >
> > > ffmpeg -loglevel info -y -hide_banner -i amix-audio.mov -sws_flags
> > bicubic
> > > -filter_complex
> > >
> > '[0:v]split=2[640_ADPP][640_adpp];[0:a:0]adelay=delays=0s:all=1[adelay0];[0:a:1]adelay=delays=14s:all=1[adelay1];[0:a:2]adelay=delays=29s:all=1[adelay2];[0:a:3]adelay=delays=41s:all=1[adelay3];[0:a:4]adelay=delays=56s:all=1[adelay4];[adelay0][adelay1][adelay2][adelay3][adelay4]amix=5,volumedetect,asplit=2[adp_audio_1][adp_audio_2];[640_adpp]scale='640x360'[640_adpp];[640_ADPP]scale='640x360'[640_ADPP]'
> > > \
> > > -map '[640_adpp]' -map '[adp_audio_1]' -map_metadata -1 -f mp4 -vcodec
> > > libx264 -pix_fmt yuv420p -profile:v main -level 3.1 -crf 18 -b:v 2000K
> > > -movflags faststart -refs 4 -color_primaries bt709 -color_trc bt709
> > > -colorspace bt709 -preset slower /tmp/123456_1.mp4 \
> > > -map '[640_ADPP]' -map '[adp_audio_2]' -map_metadata -1 -f mp4 -vcodec
> > > libx264 -pix_fmt yuv420p -profile:v main -level 3.1 -crf 18 -b:v 2000K
> > > -movflags faststart -refs 4 -color_primaries bt709 -color_trc bt709
> > > -colorspace bt709 -preset slower /tmp/123456_2.mp4
> > >
> > >
> > Split video and audio by using (a)movie filters, otherwise buffering may
> > occur if you change timestamps or insert samples.
> >
> >
> > >
> > > Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'amix-audio.mov':
> > > Metadata:
> > > major_brand : qt
> > > minor_version : 537199360
> > > compatible_brands: qt
> > > creation_time : 2015-09-23T22:51:18.000000Z
> > > com.apple.quicktime.player.version: 7.7.6 (Dev)
> > > com.apple.quicktime.version: 7.7.6 (1680.95.31)
> > > com.apple.quicktime.player.movie.audio.gain: 1.000000
> > > com.apple.quicktime.player.movie.audio.treble: 0.000000
> > > com.apple.quicktime.player.movie.audio.bass: 0.000000
> > > com.apple.quicktime.player.movie.audio.balance: 0.000000
> > > com.apple.quicktime.player.movie.audio.pitchshift: 0.000000
> > > com.apple.quicktime.player.movie.audio.mute:
> > > com.apple.quicktime.player.movie.visual.brightness: 0.000000
> > > com.apple.quicktime.player.movie.visual.color: 1.000000
> > > com.apple.quicktime.player.movie.visual.tint: 0.000000
> > > com.apple.quicktime.player.movie.visual.contrast: 1.000000
> > > Duration: 00:01:13.54, start: 0.000000, bitrate: 39681 kb/s
> > > Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv,
> > > bt709), 1920x1080, 39358 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc
> > > (default)
> > > Metadata:
> > > rotate : 0
> > > creation_time : 2015-09-23T22:51:18.000000Z
> > > handler_name : Apple Video Media Handler
> > > vendor_id : [0][0][0][0]
> > > encoder : AVC Coding
> > > Side data:
> > > displaymatrix: rotation of -0.00 degrees
> > > Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz,
> > stereo,
> > > fltp, 317 kb/s (default)
> > > Metadata:
> > > rotate : 0
> > > creation_time : 2015-09-23T22:51:18.000000Z
> > > handler_name : Apple Sound Media Handler
> > > vendor_id : [0][0][0][0]
> > > Stream #0:2(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo,
> > > fltp, 317 kb/s (default)
> > > Metadata:
> > > rotate : 0
> > > creation_time : 2015-09-23T22:51:18.000000Z
> > > handler_name : Apple Sound Media Handler
> > > vendor_id : [0][0][0][0]
> > > Stream #0:3(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo,
> > > fltp, 317 kb/s (default)
> > > Metadata:
> > > rotate : 0
> > > creation_time : 2015-09-23T22:51:18.000000Z
> > > handler_name : Apple Sound Media Handler
> > > vendor_id : [0][0][0][0]
> > > Stream #0:4(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo,
> > > fltp, 317 kb/s (default)
> > > Metadata:
> > > rotate : 0
> > > creation_time : 2015-09-23T22:51:18.000000Z
> > > handler_name : Apple Sound Media Handler
> > > vendor_id : [0][0][0][0]
> > > Stream #0:5(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo,
> > > fltp, 317 kb/s (default)
> > > Metadata:
> > > rotate : 0
> > > creation_time : 2015-09-23T22:51:18.000000Z
> > > handler_name : Apple Sound Media Handler
> > > vendor_id : [0][0][0][0]
> > > [Parsed_volumedetect_7 @ 0x7fb984e0f540] n_samples: 0
> > > Stream mapping:
> > > Stream #0:0 (h264) -> split
> > > Stream #0:1 (aac) -> adelay
> > > Stream #0:2 (aac) -> adelay
> > > Stream #0:3 (aac) -> adelay
> > > Stream #0:4 (aac) -> adelay
> > > Stream #0:5 (aac) -> adelay
> > > scale -> Stream #0:0 (libx264)
> > > asplit:output0 -> Stream #0:1 (aac)
> > > scale -> Stream #1:0 (libx264)
> > > asplit:output1 -> Stream #1:1 (aac)
> > > Press [q] to stop, [?] for help
> > > [out_1_0 @ 0x7fb98888a6c0] 100 buffers queued in out_1_0, something may
> > be
> > > wrong.
> > > [out_1_0 @ 0x7fb98888a6c0] 1000 buffers queued in out_1_0, something may
> > > be wrong.
> > > [out_0_0 @ 0x7fb988889cc0] 100 buffers queued in out_0_0, something may
> > be
> > > wrong.
> > > [out_0_0 @ 0x7fb988889cc0] 1000 buffers queued in out_0_0, something may
> > > be wrong.
> > > [libx264 @ 0x7fb98500c000] using cpu capabilities: MMX2 SSE2Fast SSSE3
> > > SSE4.2 AVX FMA3 BMI2 AVX2
> > > [libx264 @ 0x7fb98500c000] profile Main, level 3.1, 4:2:0, 8-bit
> > > [libx264 @ 0x7fb98500c000] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4
> > AVC
> > > codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html -
> > options:
> > > cabac=1 ref=4 deblock=1:0:0 analyse=0x1:0x131 me=umh subme=9 psy=1
> > > psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=0
> > > cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=11
> > > lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0
> > > bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=2
> > > b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25
> > > scenecut=40 intra_refresh=0 rc_lookahead=60 rc=crf mbtree=1 crf=18.0
> > > qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
> > > Output #0, mp4, to '/tmp/123456_1.mp4':
> > > Metadata:
> > > encoder : Lavf58.76.100
> > > Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709,
> > > progressive), 640x360, q=2-31, 2000 kb/s, 29.97 fps, 30k tbn (default)
> > > Metadata:
> > > encoder : Lavc58.134.100 libx264
> > > Side data:
> > > cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 0 vbv_delay: N/A
> > > Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo,
> > > fltp, 128 kb/s (default)
> > > Metadata:
> > > encoder : Lavc58.134.100 aac
> > > [libx264 @ 0x7fb985018c00] using cpu capabilities: MMX2 SSE2Fast SSSE3
> > > SSE4.2 AVX FMA3 BMI2 AVX2
> > > [libx264 @ 0x7fb985018c00] profile Main, level 3.1, 4:2:0, 8-bit
> > > [libx264 @ 0x7fb985018c00] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4
> > AVC
> > > codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html -
> > options:
> > > cabac=1 ref=4 deblock=1:0:0 analyse=0x1:0x131 me=umh subme=9 psy=1
> > > psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=0
> > > cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=11
> > > lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0
> > > bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=2
> > > b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25
> > > scenecut=40 intra_refresh=0 rc_lookahead=60 rc=crf mbtree=1 crf=18.0
> > > qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
> > > Output #1, mp4, to '/tmp/123456_2.mp4':
> > > Metadata:
> > > encoder : Lavf58.76.100
> > > Stream #1:0: Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709,
> > > progressive), 640x360, q=2-31, 2000 kb/s, 29.97 fps, 30k tbn (default)
> > > Metadata:
> > > encoder : Lavc58.134.100 libx264
> > > Side data:
> > > cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 0 vbv_delay: N/A
> > > Stream #1:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo,
> > > fltp, 128 kb/s (default)
> > > Metadata:
> > > encoder : Lavc58.134.100 aac
> > > [mp4 @ 0x7fb985016800] Starting second pass: moving the moov atom to the
> > > beginning of the file=1.01x
> > > [mp4 @ 0x7fb985017a00] Starting second pass: moving the moov atom to the
> > > beginning of the file
> > > frame= 2204 fps= 31 q=-1.0 Lq=-1.0 size= 23120kB time=00:01:13.44
> > > bitrate=2579.0kbits/s speed=1.02x
> > > video:43855kB audio:2292kB subtitle:0kB other streams:0kB global
> > > headers:0kB muxing overhead: unknown
> > > [Parsed_volumedetect_7 @ 0x7fb98888a5c0] n_samples: 7016448
> > > [Parsed_volumedetect_7 @ 0x7fb98888a5c0] mean_volume: -28.8 dB
> > > [Parsed_volumedetect_7 @ 0x7fb98888a5c0] max_volume: -10.3 dB
> > > [Parsed_volumedetect_7 @ 0x7fb98888a5c0] histogram_10db: 26
> > > [Parsed_volumedetect_7 @ 0x7fb98888a5c0] histogram_11db: 39
> > > [Parsed_volumedetect_7 @ 0x7fb98888a5c0] histogram_12db: 243
> > > [Parsed_volumedetect_7 @ 0x7fb98888a5c0] histogram_13db: 887
> > > [Parsed_volumedetect_7 @ 0x7fb98888a5c0] histogram_14db: 2794
> > > [Parsed_volumedetect_7 @ 0x7fb98888a5c0] histogram_15db: 7046
> > > [libx264 @ 0x7fb98500c000] frame I:11 Avg QP:16.16 size: 95713
> > > [libx264 @ 0x7fb98500c000] frame P:640 Avg QP:19.02 size: 29545
> > > [libx264 @ 0x7fb98500c000] frame B:1553 Avg QP:25.39 size: 1604
> > > [libx264 @ 0x7fb98500c000] consecutive B-frames: 3.9% 3.1% 10.5% 82.6%
> > > [libx264 @ 0x7fb98500c000] mb I I16..4: 1.1% 0.0% 98.9%
> > > [libx264 @ 0x7fb98500c000] mb P I16..4: 0.1% 0.0% 0.3% P16..4: 51.1%
> > > 18.5% 21.9% 3.7% 2.6% skip: 1.8%
> > > [libx264 @ 0x7fb98500c000] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 23.2%
> > > 2.8% 1.5% direct:10.6% skip:61.9% L0:36.3% L1:49.2% BI:14.5%
> > > [libx264 @ 0x7fb98500c000] direct mvs spatial:99.6% temporal:0.4%
> > > [libx264 @ 0x7fb98500c000] coded y,uvDC,uvAC intra: 95.1% 92.0% 79.7%
> > > inter: 29.3% 18.4% 6.3%
> > > [libx264 @ 0x7fb98500c000] i16 v,h,dc,p: 32% 14% 16% 38%
> > > [libx264 @ 0x7fb98500c000] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 12% 9% 7%
> > > 8% 8% 10% 8% 15%
> > > [libx264 @ 0x7fb98500c000] i8c dc,h,v,p: 40% 29% 24% 8%
> > > [libx264 @ 0x7fb98500c000] Weighted P-Frames: Y:2.3% UV:0.3%
> > > [libx264 @ 0x7fb98500c000] ref P L0: 58.6% 14.2% 12.2% 14.0% 1.0% 0.0%
> > > [libx264 @ 0x7fb98500c000] ref B L0: 69.1% 19.8% 11.1%
> > > [libx264 @ 0x7fb98500c000] ref B L1: 89.6% 10.4%
> > > [libx264 @ 0x7fb98500c000] kb/s:2442.56
> > > [aac @ 0x7fb985017400] Qavg: 638.288
> > > [libx264 @ 0x7fb985018c00] frame I:11 Avg QP:16.16 size: 95713
> > > [libx264 @ 0x7fb985018c00] frame P:640 Avg QP:19.02 size: 29545
> > > [libx264 @ 0x7fb985018c00] frame B:1553 Avg QP:25.39 size: 1604
> > > [libx264 @ 0x7fb985018c00] consecutive B-frames: 3.9% 3.1% 10.5% 82.6%
> > > [libx264 @ 0x7fb985018c00] mb I I16..4: 1.1% 0.0% 98.9%
> > > [libx264 @ 0x7fb985018c00] mb P I16..4: 0.1% 0.0% 0.3% P16..4: 51.1%
> > > 18.5% 21.9% 3.7% 2.6% skip: 1.8%
> > > [libx264 @ 0x7fb985018c00] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 23.2%
> > > 2.8% 1.5% direct:10.6% skip:61.9% L0:36.3% L1:49.2% BI:14.5%
> > > [libx264 @ 0x7fb985018c00] direct mvs spatial:99.6% temporal:0.4%
> > > [libx264 @ 0x7fb985018c00] coded y,uvDC,uvAC intra: 95.1% 92.0% 79.7%
> > > inter: 29.3% 18.4% 6.3%
> > > [libx264 @ 0x7fb985018c00] i16 v,h,dc,p: 32% 14% 16% 38%
> > > [libx264 @ 0x7fb985018c00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 12% 9% 7%
> > > 8% 8% 10% 8% 15%
> > > [libx264 @ 0x7fb985018c00] i8c dc,h,v,p: 40% 29% 24% 8%
> > > [libx264 @ 0x7fb985018c00] Weighted P-Frames: Y:2.3% UV:0.3%
> > > [libx264 @ 0x7fb985018c00] ref P L0: 58.6% 14.2% 12.2% 14.0% 1.0% 0.0%
> > > [libx264 @ 0x7fb985018c00] ref B L0: 69.1% 19.8% 11.1%
> > > [libx264 @ 0x7fb985018c00] ref B L1: 89.6% 10.4%
> > > [libx264 @ 0x7fb985018c00] kb/s:2442.56
> > > [aac @ 0x7fb98501a600] Qavg: 638.288
> > >
> > >
> > >
> > > Bryce Chester Newman | Principal Developer
> > >
> > > p: +12069255045 |
> > >
> > >
> > >
> > > _______________________________________________
> > > ffmpeg-user mailing list
> > > ffmpeg-user at ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > >
> > _______________________________________________
> > ffmpeg-user mailing list
> > ffmpeg-user at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> >
> >
> >
> > Bryce Chester Newman | Principal Developer
> >
> > p: +12069255045 |
> >
> >
> >
> > _______________________________________________
> > ffmpeg-user mailing list
> > ffmpeg-user at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> >
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>
>
>
> Bryce Chester Newman | Principal Developer
>
> p: +12069255045 |
>
>
>
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
More information about the ffmpeg-user
mailing list