[FFmpeg-devel] determining the reference frame for motion_val

[Martin Luessi]

On 8/22/07, Michael Niedermayer <michaelni at gmx.at> wrote:
> Hi
>
> On Wed, Aug 22, 2007 at 09:21:25AM -0500, Martin Luessi wrote:
> > Hi,
> >
> > For my work I need to extract motion vectors from h264 video. By
> > looking at the function ff_print_debug_info in mpegvideo.c and
> > previous posts on this list, I figured out how to do that. Right now
> > my motion vector extraction function accesses the motion_val table of
> > the returned AVFrame and creates a table with one motion vector for
> > each 8x8 block. However, I still have the following questions:
> >
> > 1)
> > How can I determine which reference frame is used for a given motion
> > vector? I use the USES_LIST macro to find out if there is a forward or
> > backward motion vector assigned to a a given block. Let's say we have
> > a forward motion vector, how can I find out which previous frame  is
> > used as reference? If the current frame is a P-frame and the previous
> > one was an I-frame, obviously the previous frame is used. But what
> > about if the current frame is a B-frame and the previous frame is B as
> > well. As you know h264 has an option to support using B frames as
> > reference, so the reference frame could be the previous B-frame or a
> > I/P frame further back in the past. I hope you see what I'm getting
> > at.
>
> AVFrame.ref_index

Ok, thanks. I looked at the h264 code and wrote a function to copy the
ref_index from the current frame. However, I cannot make much sense of
the values in ref_index. My function looks something like this:

void extract_ref_index(MpegEncContext * mpeg_ctx, int list, AVFrame *
pict, uint8_t arr)
{
  for (int mb8_y = 0; mb8_y < mpeg_ctx->mb_height * 2; mb8_y++){
     for (int mb8_x = 0; mb8_x < mpeg_ctx->mb_width * 2; mb8_x++){
       int b8_xy = mb8_x + mb8_y * mpeg_ctx->b8_stride;
       arr[mb8_x + mb8_y * 2 * mpeg_ctx->mb_width] =
pict->ref_index[list][b8_xy];
    }
  }
}

>From the way the h264 code uses ref_index I figured that the ref_index
has an entry for every 8x8 block and the array size is
"mpeg_ctx->b8_stride * 2 * mpeg_ctx->mb_height * sizeof(uint8_t)",
which means for a QVGA frame 41x30 bytes, is that correct? However, as
I said before the values I get are kind of weird, they are mostly 0
and sometimes 255, can anyone explain to me what this means?

Also, does anybody know if the contents of ref_index are still valid
when the frame is returned by avcodec_decode_vide(..)? I'm working on
a python wrapper library for ffmpeg that can be used for video
processing research. It consists of a low level C wrapper library and
high level Python class interface. For the user it is very easy to
read frames from video files extract motion vectors etc, for example:

video = videofile(filename)

for frame in video:
    motion_vectors = frame.get_forward_motion_vectors()

..

Where motion_vectors is a numpy array that holds the motion vectors
for the current frame. My goal is to write the wrapper library without
having to modify the ffmpeg source. Right now most things are already
working quite well but there are still some problems, like the
ref_index. When the code is a little more mature I'm planing to
release the library under the GPL. This may be a bit off topic but I
just wanted to let you know what I'm working at.

> >
> > 2)
> > Is there an easy way to find out if a block uses 4x4 partitions in
> > h264, if yes how can I access the motion vectors?
>
> the vectors should be in AVFrame.motion_val

Ok, I will look into this again

>
> [...]
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Good people do not need laws to tell them to act responsibly, while bad
> people will find a way around the laws. -- Plato
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
>
> iD8DBQFGzMoMYR7HhwQLD6sRAq91AJ4r9615VD5ypzPzC7pjPnY6w+2NtACfVE7I
> TTpyMP0MvTjaOTicUTTLPh8=
> =V93o
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at mplayerhq.hu
> http://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-devel
>