[FFmpeg-devel] [PATCH] check sample_fmt in avcodec_open

[Reimar Döffinger]

Hello,
this fixes an inconsistency (most codecs do not check the sample_fmt)
and also allows to remove the check in flacenc.c.
I am however not sure if such a check is correct: IIRC sample_fmt field
was an API addition so this would actually break applications like
mencoder that did not set this at all (they worked fone up to now for
most codecs - the flac encoder was an exception).
One possibility would be to detect avctx->sample_fmt ==
AV_SAMPLE_FMT_NONE and override it to S16, at least until the next major
bump.
Either way, this one at least prints an error, flacenc just fails
silently currently for that case.
Index: libavcodec/utils.c
===================================================================
--- libavcodec/utils.c  (revision 26178)
+++ libavcodec/utils.c  (working copy)
@@ -532,6 +532,16 @@
                avctx->codec->max_lowres);
         goto free_and_end;
     }
+    if (avctx->codec->sample_fmts) {
+        int i;
+        for (i = 0; avctx->codec->sample_fmts[i] != AV_SAMPLE_FMT_NONE; i++)
+            if (avctx->sample_fmt == avctx->codec->sample_fmts[i])
+                break;
+        if (avctx->codec->sample_fmts[i] == AV_SAMPLE_FMT_NONE) {
+            av_log(avctx, AV_LOG_ERROR, "Specified sample_fmt is not supported.\n");
+            goto free_and_end;
+        }
+    }
 
     if(avctx->codec->init){
         ret = avctx->codec->init(avctx);