[FFmpeg-devel] [PATCH] [RFC] avcodec: export MB information in frame side data

[Michael Niedermayer]

On Wed, Jul 16, 2014 at 05:06:27PM +0200, Clément Bœsch wrote:
> Also add an example exporting the MB information as a CSV stream.
> 
> ---
> 
> A bunch of questions stand:
> 
> - Ideally the "source" for a given macroblock should be a specific
>   reference to a future or paste frame (something like -4 or +2).
>   Currently it's just ±1 depending on the direction. I don't see how I
>   can extract that information.
> 
> - how the MB "type" should be exported? Like, what "generic" type we
>   need to expose for the users
> 
> - Who is motivated to port -vismv & various other vis_* debug to a video
>   filter? (The hard part will probably be writing its documentation...)
> 
> (TODO: avcodec version bump & APIChanges entry at least)
> ---
[...]
> @@ -2109,6 +2129,93 @@ void ff_print_debug_info2(AVCodecContext *avctx, AVFrame *pict, uint8_t *mbskip_
>                           int *low_delay,
>                           int mb_width, int mb_height, int mb_stride, int quarter_sample)
>  {
> +    if ((avctx->flags2 & CODEC_FLAG2_SD_MB_INFO) && mbtype_table && motion_val[0]) {
> +        const int shift = 1 + quarter_sample;
> +        const int mv_sample_log2 = avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id == AV_CODEC_ID_SVQ3 ? 2 : 1;
> +        const int mv_stride      = (mb_width << mv_sample_log2) +
> +                                   (avctx->codec->id == AV_CODEC_ID_H264 ? 0 : 1);
> +        int mb_x, mb_y, mbcount = 0;
> +
> +        /* width * height * directions * 4MB (4MB for IS_8x8) */
> +        AVMBInfo_MB *mbs = av_malloc_array(mb_width * mb_height, 2 * 4 * sizeof(AVMBInfo_MB));
> +        if (!mbs)
> +            return;
> +
> +        // TODO: refactor with the following code
> +        for (mb_y = 0; mb_y < mb_height; mb_y++) {
> +            for (mb_x = 0; mb_x < mb_width; mb_x++) {
> +                int i, direction, mb_type = mbtype_table[mb_x + mb_y * mb_stride];
> +                for (direction = 0; direction < 2; direction++) {


> +                    if (direction == 0 &&
> +                        pict->pict_type != AV_PICTURE_TYPE_B &&
> +                        pict->pict_type != AV_PICTURE_TYPE_P)
> +                        continue;
> +                    if (direction == 1 &&
> +                        pict->pict_type != AV_PICTURE_TYPE_B)
> +                        continue;
> +                    if (!USES_LIST(mb_type, direction))
> +                        continue;

these checks seems a bit redundant


[...]
> diff --git a/libavutil/mbinfo.h b/libavutil/mbinfo.h
> new file mode 100644
> index 0000000..89538ba
> --- /dev/null
> +++ b/libavutil/mbinfo.h
> @@ -0,0 +1,32 @@
> +/*
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#ifndef AVUTIL_MBINFO_H
> +#define AVUTIL_MBINFO_H
> +
> +#include <stdint.h>
> +
> +typedef struct AVMBInfo_MB {

    > +    int8_t source; /* -1/+1 XXX: set exact relative ref frame instead of "direction" */

i suspect this is too small for long term references


> +    uint32_t type; /* how much codec specific can this be? */
> +    uint8_t w, h;
> +    uint16_t src_x, src_y;
> +    uint16_t dst_x, dst_y;
> +} AVMBInfo_MB;

as side data this is not optimal as it depends on native endianness
making exchange of side data over the network or serialization
into file storage more difficult

also QP, intra prediction mode, ... is missing 


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

During times of universal deceit, telling the truth becomes a
revolutionary act. -- George Orwell
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20140722/d159b027/attachment.asc>