[Libav-user] How can we extract frames from a video stream in MPEG-4 AVC format

[Spencer Graffe]

As far as random frame extraction for MPEG's is concerned, it gets a little
tricky. The FFmpeg libraries do not have a consistent interface to seeking
into files of all different formats and extracting their frames. The reason
why is because different video formats treat time differently from others,
which means that the way they record time is also different.

After doing some experiments with MPEG's, I found that the best way to
extract frames based on a specific time is the following:

1) Convert the time you want (in seconds) to a PTS value for your stream.
This is the value you are trying to seek to. To do so, use the video
stream's (AVStream) time_base and multiply your time by it.
2) Initialize your frame counter to 0.
3) Go into a reading loop just like in the tutorials.
4) After decoding and getting a finished frame, add 1 to your frame counter
and calculate the current frame's PTS using the following:
    calcPts = (frame->coded_picture_number * (1.0 / fps) *
formatContext->streams[reader->videoStream]->time_base.den) /
formatContext->streams[reader->videoStream]->time_base.num;
5) Compare your frame's PTS with your seek PTS and see if it is the same or
greater.

It all boils down to converting a video frame number into an accurate PTS
and comparing them. I had to do slightly different hacks to get accurate PTS
values for other formats.

On Wed, Apr 6, 2011 at 10:26 PM, Carson Harper <carsonharper at gmail.com>wrote:

> Hey Amir,
>
> Could you tell us a little bit more about what you want to do with the
> "raw" YUV or RGB data? For instance, do you want to save those individual
> frames as still images (BMP, PNG...) or are you wanting to transcode them
> for use in another video stream?
>
> If you want to just save to disk try-
>
> ffmpeg -f image2 </path/to/your/video.mp4> frame_%06d.png
>
> See here<http://www.catswhocode.com/blog/19-ffmpeg-commands-for-all-needs> for
> some other useful CLI stuff.
>
> I recently went through the tutorials listed above and came up with a C++
> routine that convert's the first video stream from an input file to a vector
> of ImageMagick objects, but it would be trivial to adapt it to something
> else.
>
> You mentioned having random access to frames by selecting frame number,
> which is something I'll need to write eventually anyways, but this may work
> for you as a lazy solution. I mostly deal with small/short video clips which
> is why I can get aways with loading the whole video into memory.
>
> I tried to tweak a few things to make it easier to understand, but you
> will definitely still need to change things to get it to work for you.
> Obviously, I'm throwing my own custom exceptions etc... You could return an
> array of (int8_t *) if you don't want to use the ImageMagick stuff, and then
> just play with the raw data from there.
>
> Hope this helps,
>
> CH
>
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user
>
>


-- 
Spencer Graffe
- A Guy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20110407/5ea2df43/attachment.html>