[Libav-user] Problem Generating B-Frames Using libavcodec/libx264

[Robert Schmidt]

I finally tracked down the source of my issue and thought I should
post the answer in case someone else has similar problems.

The problem ended up not being my stream and encoder set up, as I
originally thought, but how I was passing pictures to the encoder.
Since my test utility simply transcodes video data, I was passing
pictures emitted by the H.264 encoder to the libx264 encoder.
However, the decoder sets the pict_type field of AVFrame structures it
populates to AV_PICTURE_TYPE_P.  In addition, the libx264 encoder uses
the pict_type field of incoming pictures to determine what kind of
frame to output.  As a result, the encoder only produced P-frames
since all of the incoming pictures specified that picture type.

I updated the code to reset the pict_type field to
AV_PICTURE_TYPE_NONE prior to passing the picture to the encoder.
This allows the encoder to select the frame type it feels is best.
Once I made this change, the encoder produced both B-frames and
P-frames and my output was identical to what ffmpeg produced.

This step is not done explicitly in any of the code samples since they
normally allocate a new AVFrame structure for each frame they encode
causing the pict_type field to already be set to AV_PICTURE_TYPE_NONE.
 I also missed the line that was performing this step in the ffmpeg
code when I was comparing it to my own.

Hopefully this will save some grief for someone else should they
encounter the same issue in the future.

Thanks.

Rob