[Ffmpeg-devel] Intra-frame rate control

[RLopez]

Hi,

>> If avcodec.lib can only change Q values at picture boundaries for MPEG2
>> files that's a huge limitation.  The advantage of MPEG2 over MJPEG is the
>> ability to change the Q in the middle of an image.

>The advantage of MPEG2 over MJPEG is P- and B-frames. I'd call adaptive 
>quant rather minor in comparison.

You're right.  I should have said the advantage of MPEG I-frames over MJPEG
is the ability to change the Q in the middle of the image.

>> A common clip people use to show this off is a field of daisies under 
>> a blue sky.  The blue sky can  be encoded well with a high Q value while 
>> the more complex field of daisies portion of the image can be encoded 
>> with a lower Q value.

>But the blue sky also uses much less bitrate at the same QP. So I'm not so 
>sure that the optimal encoding mode is to raise the QP of the sky.
>On the contrary, x264's adaptive quant uses _lower_ QP for smooth 
>gradients like a sky, since any tiny deviation from the gradient will be 
>blatantly visible, whereas small errors in the daisies would be hidden by 
>the masses of detail. I don't think H.264 is so different from MPEG2 as to 
>reverse the direction of that decision...

>> Another problem I'm seeing caused by avcodec.lib only changing Q values
at
>> picture boundaries is that the instantaneous bitrate of an encoded clip
>> jumps around too much.  For example when a complex image follows several
>> black frames the bitrate skyrockets because the rate controller lowered
the
>> Q during the black frames and doesn't readjust it until the end of the
>> complex frame.  This problem is especially bad when working with HD
images.

>Just because it only changes QP between frames doesn't mean it can only 
>react. RC examines the current frame before choosing a QP. The QP chosen 
>for one frame will be independent of the complexity of the previous 
>frames, except insofar as RC reacts to the VBV state or otherwise 
>introduces an explicit dependency (qblur, etc).

I did a test where I encoded 10 flat gray frames(1920x1080) followed by 10
frames of random data using avcodec.lib and again using Pexeltool MPEG2
encoder.  I set the bitrate to 20 Mbs.  Here are the results:

                    avcodec               Pixeltools
Frame #   Frames size   Q values    Frames size   Q values
=======   =======================   ======================
   1         30987    Fixed at 2      83245       1 - 3
   2         30987    Fixed at 2      83242       1 - 3
   3         30987    Fixed at 2      83239     Fixed at 1
   4         30987    Fixed at 2      83239     Fixed at 1
   5         30987    Fixed at 2      83239     Fixed at 1
   6         30987    Fixed at 2      83239     Fixed at 1
   7         30987    Fixed at 2      83239     Fixed at 1
   8         30987    Fixed at 2      83239     Fixed at 1
   9         30987    Fixed at 2      83239     Fixed at 1
  10         30987    Fixed at 2      83239     Fixed at 1
  11       3102876    Fixed at 2      86344       1 - 27
  12       3103653    Fixed at 2      85680      12 - 31
  13       1846080    Fixed at 5      83110      18 - 31
  14       1498372    Fixed at 7      83141      27 - 31
  15       1178371    Fixed at 10     83210      25 - 30
  16        972263    Fixed at 13     83174      24 - 30
  17        819350    Fixed at 16     83189      25 - 29
  18        701635    Fixed at 19     83210      26 - 29
  19        611076    Fixed at 22     83135      28 - 31
  20        535178    Fixed at 25     83227      27 - 31

The average frame size should be 83416.  As you can see avcodec only changes
the Q at picture boundaries and when it hits frame 11 (the first random
frame) the frame rate skyrockets and it only adjusts the q value by about 3
for each subsequent frame as it attempts to bring the bitrate down.  The
Pixeltools encoder varies the Q throughout the frame to keep the bitrate
around 20 Mbs.  At frame 11 the Q values vary widely throughout the frame as
it tries to control the bitrate eventually settling at a point where all the
slices end up with Q values between 27 and 31.  I attached the code I used
for the avcodec test.

Rudy
 
>--Loren Merritt

#include "avformat.h"

int picturebuf[1920*1080/2];
unsigned char outbuf[4*1024*1024];
AVCodecContext ffenc;
AVFrame        picture;

int main(int argc, char* argv[])
{
    int i,j,k,random,size1,size2;
    FILE *fid;
    AVCodec        *codec;

    int bitrate  = 20000000; 
    int width    = 1920; 
    int height   = 1080; 
    int gopsize  = 1;
    int bframes  = 0;
    int ratenum  = 1001;
    int rateden  = 30000;

    av_register_all();
    av_log_set_level(-1);

    avcodec_get_frame_defaults(&picture);
    avpicture_fill((AVPicture *)&picture, 0, PIX_FMT_YUV420P, width,
height);
    size1 = picture.data[1] - picture.data[0];
    size2 = picture.data[2] - picture.data[1];
    avcodec_get_context_defaults(&ffenc);

    ffenc.codec_id                = CODEC_ID_MPEG2VIDEO;
    ffenc.codec_type              = CODEC_TYPE_VIDEO;

    ffenc.sample_aspect_ratio.num = 1;
    ffenc.sample_aspect_ratio.den = 1;
    ffenc.bit_rate                = bitrate;
    ffenc.width                   = width;  
    ffenc.height                  = height;
    ffenc.profile                 = 4;
    ffenc.level                   = 4;
    ffenc.time_base.num           = ratenum;
    ffenc.time_base.den           = rateden;
    ffenc.gop_size                = gopsize;
    ffenc.max_b_frames            = bframes;

    ffenc.scenechange_threshold   = 1000000000;
    ffenc.bit_rate_tolerance      = 4000*1000;
    ffenc.qblur                   = 0.5;
    ffenc.rc_buffer_aggressivity  = 1.0;
    ffenc.pix_fmt                 = PIX_FMT_YUV420P;
    codec = avcodec_find_encoder(ffenc.codec_id);
    avcodec_open(&ffenc, codec);
    
    fid = fopen("c:\\test.mpg","wb");
    for (random=0;random<2;random++) {
        if (!random) for (j=0;j<(1920*1080/2);j++) picturebuf[j] =
0x80808080;
        for (i=0;i<10;i++) {
            if (random) for (j=0;j<(1920*1080/2);j++) picturebuf[j] =
rand();
            picture.data[0] = (unsigned char*) picturebuf;
            picture.data[1] = picture.data[0] + size1;
            picture.data[2] = picture.data[1] + size2;
            k = avcodec_encode_video(&ffenc, outbuf, 16*1024*1024,
&picture);
            fwrite(outbuf,1,k,fid);
        }
    }
    fclose(fid);
    avcodec_close(&ffenc);
    return 0;
}