[Libav-user] Frames are not visually lining up between my code and ffmpeg.exe

[Simon Daniels]

Hey guys,

I'm doing some simply video analysis in which I walk through frames of a
video and do some calculations on the pixel data. I started noticing some
issues with frame timing so I tried to compare my results with a simple
thumbnail extraction script using ffmpeg.exe.

*The thumbnails from my code did not visually line up with the thumbnails
generated by ffmpeg.exe.*

For example, without frame skipping, frame #225 in my code visually lines
up to #250 from ffmpeg.exe. When skipping every 10 frames (in my "real"
video analysis"), #280 visually lines up to #250 from ffmpeg.exe! I'd like
to think I'm just calculating my frame number wrong, but even the total
frames generated by both are different.

I recently upgraded ffmpeg from 0.8.3 to 0.11.1 and I don't remember these
issues before. I'm going to check 0.8.3 now.

*More info*
- Link to video: https://dl.dropbox.com/u/28441949/sample%20video.MP4
- Total frames reported using avStream->nb_frames: *480*
- Total frames by iterating through av_read_frame(): *480*
- Total frames if I save each frame as a jpg (ffmpeg.exe -i input.mp4
-vframes 1 -f image2 –sameq -vcodec mjpeg frame-%05d.jpg): *510*
- If I turn AV_LOG_DEBUG on, I see several messages like this: [h264 @
06e420c0] Frame num gap

int SimplifiedScan(const char* inputfile)
{

AVFormatContext *pFormatCtx;
AVStream *pStream;
AVFrame         *pFrameYUV;
AVCodecContext  *pCodecCtx;
AVCodec *pCodec;
AVPacket packet;

int             numBytes;
uint8_t         *buffer;
int width;
int height;
int av_errno;
 int isFrameFinished = 0;
 int frameLocation = 0;

av_register_all();
 // Open video file
pFormatCtx = avformat_alloc_context();
av_errno = avformat_open_input(&pFormatCtx, inputfile, NULL, NULL);

if (av_errno < 0)
{
return ErrorOpeningFile;
}
 // Retrieve stream information
av_errno = avformat_find_stream_info(pFormatCtx, NULL);
if (av_errno < 0)
{
return ErrorFindingStreamInformation;
}

// Find the first video stream
int videoStreamIndex = -1;
for(unsigned int i=0; i<pFormatCtx->nb_streams; i++)
{
if(pFormatCtx->streams[i]->codec->codec_type==AVMEDIA_TYPE_VIDEO)
{
videoStreamIndex = i;
break;
}
}
if(videoStreamIndex == -1)
{
pLog->Write("AVHelper::GetAVStream(): Couldn't find a video stream");
return ErrorFindingVideoStream;
}

pStream = pFormatCtx->streams[videoStreamIndex];


// Get a pointer to the codec context for the video stream
pCodecCtx = pStream->codec;

// Find the decoder for the video stream
pCodec=avcodec_find_decoder(pCodecCtx->codec_id);
if(pCodec==NULL)
{
return UnsupportedCodec;
}

// Inform the codec that we can handle truncated bitstreams -- i.e.,
// bitstreams where frame boundaries can fall in the middle of packets
if(pCodec->capabilities & CODEC_CAP_TRUNCATED)
pCodecCtx->flags|=CODEC_FLAG_TRUNCATED;

// Open codec
av_errno = avcodec_open2(pCodecCtx, pCodec, NULL);
if (av_errno<0)
{
return ErrorOpeningCodec;
}


width = pCodecCtx->width;
height = pCodecCtx->height;


// Allocate an AVFrame structure
pFrameYUV=avcodec_alloc_frame();
if(pFrameYUV==NULL)
{
return NULL;
}



// Determine required buffer size and allocate buffer
numBytes=avpicture_get_size(PIX_FMT_YUV420P, width, height);
buffer=(uint8_t *)av_malloc(numBytes*sizeof(uint8_t));

// Assign appropriate parts of buffer to image planes in pFrameYUV
avpicture_fill((AVPicture *)pFrameYUV, buffer, PIX_FMT_YUV420P, width,
height);
 int frameSkip = 1;
 while (av_read_frame(pFormatCtx, &packet) >= 0)
{

// Is this a packet from the video stream?
if (packet.stream_index == videoStreamIndex && frameLocation++ % frameSkip
== 0)
{
 // Decode video frame
avcodec_decode_video2(pCodecCtx, pFrameYUV, &isFrameFinished, &packet);

#if DEBUG
WriteJPEG(pCodecCtx, pFrameYUV, frameLocation);
#endif

// Did we get a video frame?
if (isFrameFinished) {
 // done scanning frame
#if DEBUG
pLog->Write("Frame finished %d", frameLocation);
#endif
}
else
pLog->Write("Frame *NOT* finished %d", frameLocation);


} // whether we were supposed to decode the frame
 av_free_packet(&packet);

} // for each available frame

//delete [] buffer;
av_free(buffer);

// Free the YUV frame
av_free(pFrameYUV);

// Close the codec
avcodec_close(pCodecCtx);

// Close the video file
av_close_input_file(pFormatCtx);

return 0;

}




*Ffmpeg version 0.11.1: *
ffmpeg version 0.11.1 Copyright (c) 2000-2012 the FFmpeg developers
  built on Jun 12 2012 21:10:06 with gcc 4.6.3
  configuration: --enable-version3 --disable-w32threads
--enable-runtime-cpudete
ct --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass
--enable-lib
celt --enable-libopencore-amrnb --enable-libopencore-amrwb
--enable-libfreetype
--enable-libgsm --enable-libmp3lame --enable-libnut --enable-libopenjpeg
--enabl
e-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora
--enable
-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx
--enabl
e-zlib
  libavutil      51. 54.100 / 51. 54.100
  libavcodec     54. 23.100 / 54. 23.100
  libavformat    54.  6.100 / 54.  6.100
  libavdevice    54.  0.100 / 54.  0.100
  libavfilter     2. 77.100 /  2. 77.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100


Any thoughts? Thanks guys!

Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20120626/6af280c1/attachment.html>