[Ffmpeg-devel] patch for libswscale: error handling added to rgb2rgb_template.c:1489

[Ulrich Habel]

Hello list,
having played with kdenlive I produced several times an exception in

  libswscale --> ffmpeg --> lmt/lmt++ --> kdenlive


repository:
------------
svn://svn.mplayerhq.hu/mplayer/trunk/libswscale/rgb2rgb_template.c Line 1489    
(Rev. 22862)


in function:
------------
static inline void RENAME(yuvPlanartoyuy2)(const uint8_t *ysrc, const uint8_t 
*usrc, const uint8_t *vsrc, uint8_t *dst,
	long width, long height,
	long lumStride, long chromStride, long dstStride, long vertLumPerChroma)


due to the fact:
------------------
the parameters ysrc, usrc and/or vsrc are null pointers because sometimes the 
videostream seems to be in a not sane state.


my modifications:
--------------------
added sanity check and a fallback handling for the destination buffer.


my suggested patch for more robustness:
---------------------------------------------------
-- 8< --
Index: rgb2rgb_template.c
===================================================================
--- rgb2rgb_template.c  (Revision 22862)
+++ rgb2rgb_template.c  (Arbeitskopie)
@@ -1482,8 +1482,23 @@
 {
        long y;
        const long chromWidth= width>>1;
+
+       // sanity check
+       char scBadSource = 0;
+        int sci, *scidst = (int32_t *) dst;
+        if (!ysrc || !usrc || !vsrc) {
+              scBadSource = 1;
+        }
+
        for(y=0; y<height; y++)
        {
+              if (scBadSource) {
+                       for(sci = 0; sci < chromWidth; ++sci) {
+                               *scidst++ = 0;
+                       }
+                       continue;
+              }
+
 #ifdef HAVE_MMX
 //FIXME handle 2 lines a once (fewer prefetch, reuse some chrom, but very 
likely limited by mem anyway)
                asm volatile(
-- 8< --

Thank you for listening...

Greetings
   Uli Habel