[Ffmpeg-devel] ffmpeg hanging when transcoding mpeg4

[Aaron Williams]

I think this patch might be a little faster and should do the same
thing.  This will fix a corner case where lambda is less than qscale. 
In this case, it may never increment and there will be an endless
loop.    With this fix, if this condition is hit it will still increase
lambda.

Patch for ffmpeg:

--- mpegvideo.c~        2006-10-17 02:31:20.000000000 -0700
+++ mpegvideo.c 2006-10-21 14:18:33.000000000 -0700
@@ -2536,7 +2536,8 @@
             int max_size= rcc->buffer_index/3;

             if(put_bits_count(&s->pb) > max_size && s->qscale <
s->avctx->qmax){
-                s->next_lambda= s->lambda*(s->qscale+1) / s->qscale;
+                s->next_lambda= (s->lambda >= s->qscale) ?
+                        (s->lambda*(s->qscale+1) / s->qscale) :
s->lambda+1;
                 s->mb_skipped = 0;        //done in MPV_frame_start()
                 if(s->pict_type==P_TYPE){ //done in encode_picture() so
we must undo it
                     if(s->flipflop_rounding || s->codec_id ==
CODEC_ID_H263P || s->codec_id == CODEC_ID_MPEG4)


-Aaron