[FFmpeg-cvslog] avcodec_find_best_pix_fmt_of_2: fix handling or PIX_FMT_NONE

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michaelni at gmx.at> | Sat Oct 13 23:41:10 2012 +0200| [8cda27b7537d15ef159cff94f3fe3dd7ab1330d4] | committer: Michael Niedermayer

avcodec_find_best_pix_fmt_of_2: fix handling or PIX_FMT_NONE

Fixes CID733775
Signed-off-by: Michael Niedermayer <michaelni at gmx.at>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=8cda27b7537d15ef159cff94f3fe3dd7ab1330d4
---

 libavcodec/imgconvert.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/imgconvert.c b/libavcodec/imgconvert.c
index 4846172..202fbf5 100644
--- a/libavcodec/imgconvert.c
+++ b/libavcodec/imgconvert.c
@@ -564,7 +564,7 @@ enum AVPixelFormat avcodec_find_best_pix_fmt_of_2(enum AVPixelFormat dst_pix_fmt
         loss_order1 = loss1 & loss_mask_order[i];
         loss_order2 = loss2 & loss_mask_order[i];
 
-        if (loss_order1 == 0 && loss_order2 == 0){ /* use format with smallest depth */
+        if (loss_order1 == 0 && loss_order2 == 0 && dst_pix_fmt2 != AV_PIX_FMT_NONE && dst_pix_fmt1 != AV_PIX_FMT_NONE){ /* use format with smallest depth */
             dst_pix_fmt = avg_bits_per_pixel(dst_pix_fmt2) < avg_bits_per_pixel(dst_pix_fmt1) ? dst_pix_fmt2 : dst_pix_fmt1;
         } else if (loss_order1 == 0 || loss_order2 == 0) { /* use format with no loss */
             dst_pix_fmt = loss_order2 ? dst_pix_fmt1 : dst_pix_fmt2;