[FFmpeg-cvslog] prores: get correct size for coded V plane if alpha is present

[Mans Rullgard]

ffmpeg | branch: master | Mans Rullgard <mans at mansr.com> | Sat Oct 15 14:35:49 2011 +0100| [91038cdbd160310174aad6833d1d08c65d850e78] | committer: Mans Rullgard

prores: get correct size for coded V plane if alpha is present

The size check must be updated to take into account both manners
in which v_data_size might be set.

Signed-off-by: Mans Rullgard <mans at mansr.com>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=91038cdbd160310174aad6833d1d08c65d850e78
---

 libavcodec/proresdec.c      |    6 ++++--
 tests/ref/fate/prores-alpha |    4 ++--
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/libavcodec/proresdec.c b/libavcodec/proresdec.c
index ca465f4..b4c7945 100644
--- a/libavcodec/proresdec.c
+++ b/libavcodec/proresdec.c
@@ -546,9 +546,11 @@ static int decode_slice(AVCodecContext *avctx, ProresThreadData *td)
     hdr_size    = buf[0] >> 3;
     y_data_size = AV_RB16(buf + 2);
     u_data_size = AV_RB16(buf + 4);
-    v_data_size = slice_data_size - y_data_size - u_data_size - hdr_size;
+    v_data_size = hdr_size > 7 ? AV_RB16(buf + 6) :
+        slice_data_size - y_data_size - u_data_size - hdr_size;
 
-    if (v_data_size < 0 || hdr_size < 6) {
+    if (hdr_size + y_data_size + u_data_size + v_data_size > slice_data_size ||
+        v_data_size < 0 || hdr_size < 6) {
         av_log(avctx, AV_LOG_ERROR, "invalid data size\n");
         return AVERROR_INVALIDDATA;
     }
diff --git a/tests/ref/fate/prores-alpha b/tests/ref/fate/prores-alpha
index 8ad611d..4592652 100644
--- a/tests/ref/fate/prores-alpha
+++ b/tests/ref/fate/prores-alpha
@@ -1,2 +1,2 @@
-0, 0, 12441600, 0x79c18863
-0, 3003, 12441600, 0x79c18863
+0, 0, 12441600, 0x9d3dc525
+0, 3003, 12441600, 0x9d3dc525