[FFmpeg-devel] [PATCH] lavfi: detect merge failure for unknown layouts.

[Nicolas George]

Detect when filtering known layouts from an explicit list
results in an empty list.

Fix erratic behavior.

Signed-off-by: Nicolas George <nicolas.george at normalesup.org>
---
 libavfilter/formats.c |    4 ++++
 1 file changed, 4 insertions(+)


The problem can be observed with:

./ffmpeg_g -guess_layout_max 0 -f s16le -ac 1 -i /dev/zero -af atempo -c mp2 -f null -

(atempo does not support unknown layouts yet)


diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index 43718e4..ea24627 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -184,6 +184,10 @@ AVFilterChannelLayouts *ff_merge_channel_layouts(AVFilterChannelLayouts *a,
             for (i = j = 0; i < b->nb_channel_layouts; i++)
                 if (KNOWN(b->channel_layouts[i]))
                     b->channel_layouts[j++] = b->channel_layouts[i];
+            /* Not optimal: the unknown layouts of b may become known after
+               another merge. */
+            if (!j)
+                return NULL;
             b->nb_channel_layouts = j;
         }
         MERGE_REF(b, a, channel_layouts, AVFilterChannelLayouts, fail);
-- 
1.7.10.4