[FFmpeg-devel] [PATCH 01/35] fftools/ffmpeg_mux: add private muxer context

[Anton Khirnov]

Quoting Michael Niedermayer (2022-07-08 18:58:11)
> On Fri, Jun 17, 2022 at 12:27:18PM +0200, Anton Khirnov wrote:
> > The current version of this set can also be found in my tree
> > git://git.khirnov.net/libav
> > branch ffmpeg_mt/mux
> 
> There are really many files changing, its hard to say for sure that all are the
> same issue, but basically it all seems more or less frames in some streams
> including cases where there are hugely more or 0
> 
> Here are some examples:
> 
> 
> ffmpeg -i matrixbench_mpeg2.mpg -vcodec rawvideo -pix_fmt rgb555 -allow_raw_vfw 1 -vframes 1 -bitexact file-rgb555.mkv
> 
> the new file is much bigger (due to the audio track)
> 
> -rw-r----- 1 michael michael 2765813 Jul  8 16:57 file-rgb555.mkv
> -rw-r----- 1 michael michael  834643 Jul  8 17:02 file-rgb555-ref.mkv

Where can I find this file?

> another one:
> ./ffmpeg -y -i vlcticket/8344/DVR_NVR_IP\ Camera01_20130321162325_20130321162358_576877.mp4 -vframes 1 -aframes 1 -bitexact -f framecrc -
> 
> This appears to loose the video stream
> 
>  #channel_layout_name 1: mono
> -0,          0,          0,        1,  2880000, 0x4136bc92
>  1,        112,        112,      320,      640, 0x2cd73b36
> 
> sample in https://samples.ffmpeg.org/camera-dvr/hikvision/

The sync queue got confused by stale frame durations, which are now
overwritten in the updated version of 22/35.

But do note that there may be valid situations where a stream will
"disappear" when you specify -frames constraints on multiple streams,
because -frames is supposed to cut the whole file once the constraint is
reached, not just the stream it applies to. If you want a specific
number of frames in each stream, you should rather use the trim/atrim
filters.

See also my discussion with Andreas under 24/35 in this thread.

> This one fails a bit worse than before (ffmpeg succeeds before besides producing errors as well)
> my notes say this worked better only before 04aa09c4bcf2d5a634a35da3a3ae3fc1abe30ef8
> the file is a little big and i havnt found it anywhere online, i will try to
> send it privately to you
>  
> ffmpeg -i 2014-10-17\ 11.31\ i95Dev\ -\ Carlo\ Pazolini\ _\ KWI\ -\ Meeting.g2m -bitexact -max_muxing_queue_size 8000 -vframes 2 file-g2m5.avi
> 
>   Metadata:
>     DeviceConformanceTemplate: L2
>     WMFSDKNeeded    : 0.0.0.0000
>     WMFSDKVersion   : 12.0.7601.17514
>     IsVBR           : 1
>     WM/ToolVersion  : 6.4.3 Build 1767
>     WM/ToolName     : GoToMeeting
>     BitRateFrom the writer: 97087
>     Audio samples   : 34341
>     Video samples   : 3740
>     recording time  : Fri, 17 Oct 2014 12:28:16 Eastern Daylight Time
>   Duration: 00:57:13.86, start: 0.000000, bitrate: 100 kb/s
>   Stream #0:0: Audio: wmav2 (a[1][0][0] / 0x0161), 44100 Hz, mono, fltp, 48 kb/s
>   Stream #0:1: Data: none, 2 kb/s
>   Stream #0:2: Video: g2m (G2M5 / 0x354D3247), rgb24, 1440x900, 49 kb/s, 1k tbr, 1k tbn
> Stream mapping:
>   Stream #0:2 -> #0:0 (g2m (native) -> mpeg4 (native))
>   Stream #0:0 -> #0:1 (wmav2 (native) -> mp3 (libmp3lame))
> Press [q] to stop, [?] for help
> [libmp3lame @ 0x55c17cf03140] Queue input is backward in time
> Output #0, avi, to 'file-g2m5.avi':
>   Metadata:
>     DeviceConformanceTemplate: L2
>     WMFSDKNeeded    : 0.0.0.0000
>     WMFSDKVersion   : 12.0.7601.17514
>     IsVBR           : 1
>     WM/ToolVersion  : 6.4.3 Build 1767
>     WM/ToolName     : GoToMeeting
>     BitRateFrom the writer: 97087
>     Audio samples   : 34341
>     Video samples   : 3740
>     recording time  : Fri, 17 Oct 2014 12:28:16 Eastern Daylight Time
>   Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p(tv, progressive), 1440x900, q=2-31, 200 kb/s, 1k fps, 1k tbn
>     Metadata:
>       encoder         : Lavc mpeg4
>     Side data:
>       cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
>   Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 44100 Hz, mono, fltp
>     Metadata:
>       encoder         : Lavc libmp3lame
> [avi @ 0x55c17cf31340] Too large number of skipped frames 194184 > 600000kbits/s speed= 140x    
> av_interleaved_write_frame(): Invalid argument
> Error muxing a packet for output file #0
> [avi @ 0x55c17cf31340] Too large number of skipped frames 194085 > 60000
> frame=    2 fps=1.3 q=2.0 Lsize=    1855kB time=00:03:14.18 bitrate=  78.3kbits/s speed= 129x    
> video:149kB audio:1517kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 11.381945%
> Conversion failed!

The new code ends up sending a few more audio packets to the muxer, so
av_interleaved_write_frame() returns an error, which results in a
non-zero exit status.

In the current code, the same error appears during flushing the muxing
queue in av_write_trailer() and is actually returned from
av_write_trailer(), but ffmpeg for some reason does not treat
av_write_trailer() errors the same way. I do not know why that is, seems
like a bug to me.

In any case, conversion is quite broken both before and after my patches,
so I would say it is not a problem with my code.

> ./ffmpeg -i tickets/1666/avc-intra-panasonic-AG-HPX301E.mov     -vframes 3 -aframes 2 -bitexact -f framecrc -
> duplicate behavior of the a/vframe issue above, one stream disappears
> 
> sample in https://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket1666/

Again, should be resolved by the updated patch.

-- 
Anton Khirnov