[FFmpeg-devel] Bizarre runaway recursion in split filter, depending on the order the outputs are used

[Ray Simard]

I'm having some weird problems with the split filter.  The following trivial tests show what's going on.

With -vf "[in]split[in2][in1];[in1][in2]overlay[out]"

where the output pads of split are used in the order shown,
it works properly, if not very usefully.  However, with the outputs in the other order, it goes into
an infinite recursion.   start_frame in vf_split.c is called, and the first call there to
avfilter_start_frame takes place, but it never gets past that line to the second call.

I made a test binary with a quick hack vf_split.c to monitor the recursion level and report the function name,
line number and recursion count if it exceeded 30, and 
abort if it got to 100, testing on entry, between the two calls to avfilter_start-frame, and
on exit. A test run is shown below. Note that it never gets past line 30.

In vf_split.c (shown here without the test hacks):
static void start_frame(AVFilterLink *inlink, AVFilterBufferRef *picref)
{
(line 30)    avfilter_start_frame(inlink->dst->outputs[0],
                          avfilter_ref_buffer(picref, ~AV_PERM_WRITE));
    avfilter_start_frame(inlink->dst->outputs[1],
                          avfilter_ref_buffer(picref, ~AV_PERM_WRITE));
}
 
I haven't had the chance yet to dig into this, but perhaps it might ring a bell with someone.

Ray Simard
rhs.ffmpeg at sylvan-glade.com

rsimard at higgins:/commons1/npg> /usr/local_test01/bin/ffmpeg -i test-c.mpg -vf "[in]split[in1][in2];[in1][in2]overlay[out]" -target ntsc-dvd -y test.mpg >temp 2>&1 
Aborted

Contents of temp after that:

ffmpeg version N-36958-g475bf0c Copyright (c) 2000-2012 the FFmpeg developers
  built on Jan 20 2012 18:27:03 with gcc 4.5.1 20101208 [gcc-4_5-branch revision 167585]
  configuration: --prefix=/usr/local_test02 --enable-debug=3 --disable-stripping --disable-optimizations
  libavutil      51. 34.101 / 51. 34.101
  libavcodec     53. 57.105 / 53. 57.105
  libavformat    53. 30.100 / 53. 30.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 59.101 /  2. 59.101
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  6.100 /  0.  6.100
[mpeg @ 0x15773e0] max_analyze_duration 5000000 reached at 5005000
Input #0, mpeg, from 'test-c.mpg':
  Duration: 00:03:53.03, start: 1.000000, bitrate: 4216 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p, 720x240 [SAR 1:1 DAR 3:1], 4000 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Stream #0:1[0x1c0]: Audio: mp2, 48000 Hz, stereo, s16, 128 kb/s
[buffer @ 0x157e820] w:720 h:240 pixfmt:yuv420p tb:1/1000000 sar:1/1 sws_param:
[overlay @ 0x18b2680] auto-inserting filter 'auto-inserted scale 0' between the filter 'Parsed_split_0' and the filter 'Parsed_overlay_1'
[scale @ 0x157e540] w:720 h:240 fmt:yuv420p -> w:720 h:480 fmt:yuv420p flags:0x4
[scale @ 0x1578d80] w:720 h:480 fmt:yuv420p -> w:720 h:480 fmt:yuva420p flags:0x4
[overlay @ 0x18b2680] main w:720 h:480 fmt:yuv420p overlay x:0 y:0 w:720 h:480 fmt:yuva420p
[overlay @ 0x18b2680] main_tb:1/1000000 overlay_tb:1/1000000 -> tb:1/1000000 exact:1
Incompatible sample format 's16' for codec 'ac3', auto-selecting format 'flt'
Output #0, dvd, to 'test.mpg':
  Metadata:
    encoder         : Lavf53.30.100
    Stream #0:0: Video: mpeg2video, yuv420p, 720x480 [SAR 2:1 DAR 3:1], q=2-31, 6000 kb/s, 90k tbn, 29.97 tbc
    Stream #0:1: Audio: ac3, 48000 Hz, stereo, flt, 448 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video -> mpeg2video)
  Stream #0:1 -> #0:1 (mp2 -> ac3)
Press [q] to stop, [?] for help
start_frame 30: recursion = 31
start_frame 30: recursion = 32
start_frame 30: recursion = 33
...
start_frame 30: recursion = 97
start_frame 30: recursion = 98
start_frame 30: recursion = 99
start_frame 30: recursion = 100
start_frame 30: recursion = 101
Abort at libavfilter/vf_split.c:30