[FFmpeg-devel] [PATCH] [RFC] avcodec: export MB information in frame side data

[Michael Niedermayer]

On Tue, Aug 05, 2014 at 04:27:53PM +0200, wm4 wrote:
> On Tue, 22 Jul 2014 01:51:46 +0200
> Michael Niedermayer <michaelni at gmx.at> wrote:
> 
> > On Wed, Jul 16, 2014 at 05:06:27PM +0200, Clément Bœsch wrote:
> > > Also add an example exporting the MB information as a CSV stream.
> > > 
> > > ---
> > > 
> > > A bunch of questions stand:
> > > 
> > > - Ideally the "source" for a given macroblock should be a specific
> > >   reference to a future or paste frame (something like -4 or +2).
> > >   Currently it's just ±1 depending on the direction. I don't see how I
> > >   can extract that information.
> > > 
> > > - how the MB "type" should be exported? Like, what "generic" type we
> > >   need to expose for the users
> > > 
> > > - Who is motivated to port -vismv & various other vis_* debug to a video
> > >   filter? (The hard part will probably be writing its documentation...)
> > > 
> > > (TODO: avcodec version bump & APIChanges entry at least)
> > > ---
> > [...]
> > > @@ -2109,6 +2129,93 @@ void ff_print_debug_info2(AVCodecContext *avctx, AVFrame *pict, uint8_t *mbskip_
> > >                           int *low_delay,
> > >                           int mb_width, int mb_height, int mb_stride, int quarter_sample)
> > >  {
> > > +    if ((avctx->flags2 & CODEC_FLAG2_SD_MB_INFO) && mbtype_table && motion_val[0]) {
> > > +        const int shift = 1 + quarter_sample;
> > > +        const int mv_sample_log2 = avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id == AV_CODEC_ID_SVQ3 ? 2 : 1;
> > > +        const int mv_stride      = (mb_width << mv_sample_log2) +
> > > +                                   (avctx->codec->id == AV_CODEC_ID_H264 ? 0 : 1);
> > > +        int mb_x, mb_y, mbcount = 0;
> > > +
> > > +        /* width * height * directions * 4MB (4MB for IS_8x8) */
> > > +        AVMBInfo_MB *mbs = av_malloc_array(mb_width * mb_height, 2 * 4 * sizeof(AVMBInfo_MB));
> > > +        if (!mbs)
> > > +            return;
> > > +
> > > +        // TODO: refactor with the following code
> > > +        for (mb_y = 0; mb_y < mb_height; mb_y++) {
> > > +            for (mb_x = 0; mb_x < mb_width; mb_x++) {
> > > +                int i, direction, mb_type = mbtype_table[mb_x + mb_y * mb_stride];
> > > +                for (direction = 0; direction < 2; direction++) {
> > 
> > 
> > > +                    if (direction == 0 &&
> > > +                        pict->pict_type != AV_PICTURE_TYPE_B &&
> > > +                        pict->pict_type != AV_PICTURE_TYPE_P)
> > > +                        continue;
> > > +                    if (direction == 1 &&
> > > +                        pict->pict_type != AV_PICTURE_TYPE_B)
> > > +                        continue;
> > > +                    if (!USES_LIST(mb_type, direction))
> > > +                        continue;
> > 
> > these checks seems a bit redundant
> > 
> > 
> > [...]
> > > diff --git a/libavutil/mbinfo.h b/libavutil/mbinfo.h
> > > new file mode 100644
> > > index 0000000..89538ba
> > > --- /dev/null
> > > +++ b/libavutil/mbinfo.h
> > > @@ -0,0 +1,32 @@
> > > +/*
> > > + * This file is part of FFmpeg.
> > > + *
> > > + * FFmpeg is free software; you can redistribute it and/or
> > > + * modify it under the terms of the GNU Lesser General Public
> > > + * License as published by the Free Software Foundation; either
> > > + * version 2.1 of the License, or (at your option) any later version.
> > > + *
> > > + * FFmpeg is distributed in the hope that it will be useful,
> > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > > + * Lesser General Public License for more details.
> > > + *
> > > + * You should have received a copy of the GNU Lesser General Public
> > > + * License along with FFmpeg; if not, write to the Free Software
> > > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> > > + */
> > > +
> > > +#ifndef AVUTIL_MBINFO_H
> > > +#define AVUTIL_MBINFO_H
> > > +
> > > +#include <stdint.h>
> > > +
> > > +typedef struct AVMBInfo_MB {
> > 
> >     > +    int8_t source; /* -1/+1 XXX: set exact relative ref frame instead of "direction" */
> > 
> > i suspect this is too small for long term references
> > 
> > 
> > > +    uint32_t type; /* how much codec specific can this be? */
> > > +    uint8_t w, h;
> > > +    uint16_t src_x, src_y;
> > > +    uint16_t dst_x, dst_y;
> > > +} AVMBInfo_MB;
> > 
> > as side data this is not optimal as it depends on native endianness
> > making exchange of side data over the network or serialization
> > into file storage more difficult
> 
> Note that there's a lot of side data now that uses native structs.
> 
> And it is certainly less awkward to use than an "in memory bitstream",
> even if that should make serializing harder. (So what, you just need to
> write a serialization function once -if that is even a valid use
> case- but normal access is going to be much more common by users.)

storing multimedia related data in files and transmitting over the
net isnt a valid use case ?

but on the main point of what you wrote i agree, native structs are
more convenient, iam not objecting to one being used here 

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Frequently ignored answer#1 FFmpeg bugs should be sent to our bugtracker. User
questions about the command line tools should be sent to the ffmpeg-user ML.
And questions about how to use libav* should be sent to the libav-user ML.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20140805/d5b59d6e/attachment.asc>