[FFmpeg-devel] [PATCH, v3] lavf/qsvvpp:allocate continuous memory

[Fu, Linjie]

> -----Original Message-----
> From: Li, Zhong
> Sent: Friday, June 14, 2019 11:34
> To: FFmpeg development discussions and patches <ffmpeg-
> devel at ffmpeg.org>
> Cc: Fu, Linjie <linjie.fu at intel.com>
> Subject: RE: [FFmpeg-devel] [PATCH,v3] lavf/qsvvpp:allocate continuous
> memory
> 
> > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On
> Behalf
> > Of Linjie Fu
> > Sent: Tuesday, June 4, 2019 11:59 PM
> > To: ffmpeg-devel at ffmpeg.org
> > Cc: Fu, Linjie <linjie.fu at intel.com>
> > Subject: [FFmpeg-devel] [PATCH,v3] lavf/qsvvpp:allocate continuous
> > memory
> >
> > Mediasdk calls CMRT to copy from video to system memory and requires
> > memory to be continuously allocated across Y and UV.
> >
> > Add a new path to allocate continuous memory when using system out.
> > Use av_frame_get_buffer to arrange data according to pixfmt, and remove
> > the introduced plane_padding.
> >
> > Signed-off-by: Linjie Fu <linjie.fu at intel.com>
> > ---
> >  libavfilter/qsvvpp.c | 26 ++++++++++++++++++++------
> >  1 file changed, 20 insertions(+), 6 deletions(-)
> >
> > diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index
> > 5cd1d5d345..c06171444f 100644
> > --- a/libavfilter/qsvvpp.c
> > +++ b/libavfilter/qsvvpp.c
> > @@ -350,7 +350,7 @@ static QSVFrame *query_frame(QSVVPPContext *s,
> > AVFilterLink *outlink)  {
> >      AVFilterContext *ctx = outlink->src;
> >      QSVFrame        *out_frame;
> > -    int              ret;
> > +    int              i, ret;
> >
> >      clear_unused_frames(s->out_frame_list);
> >
> > @@ -374,12 +374,26 @@ static QSVFrame *query_frame(QSVVPPContext
> > *s, AVFilterLink *outlink)
> >          out_frame->surface = (mfxFrameSurface1
> > *)out_frame->frame->data[3];
> >      } else {
> >          /* Get a frame with aligned dimensions.
> > -         * Libmfx need system memory being 128x64 aligned */
> > -        out_frame->frame = ff_get_video_buffer(outlink,
> > -
> > FFALIGN(outlink->w, 128),
> > -
> > FFALIGN(outlink->h, 64));
> > -        if (!out_frame->frame)
> > +         * Libmfx need system memory being 128x64 aligned
> > +         * and continuously allocated across Y and UV */
> > +        out_frame->frame = av_frame_alloc();
> > +        if (!out_frame->frame) {
> >              return NULL;
> > +        }
> > +
> > +        out_frame->frame->width  = FFALIGN(outlink->w, 128);
> > +        out_frame->frame->height = FFALIGN(outlink->h, 64);
> > +        out_frame->frame->format = outlink->format;
> > +
> > +        ret = av_frame_get_buffer(out_frame->frame, 128);
> > +        if (ret < 0)
> > +            return NULL;
> > +
> > +        /* remove plane_padding introduced by av_frame_get_buffer */
> Should be well explained in the comments why need to reomove.

Tried to explain that continuous memory is need for  libmfx in commit message
and before av_frame_get_buffer was called.

If that's not enough, would you please offer some advices on how to explain better?

> 
> > +        for (i = 1; i < 4; i++) {
> > +            if (out_frame->frame->data[i])
> > +                out_frame->frame->data[i] -= i * 128;
> > +        }
> 
> Looks like tricky workaround. If padding bytes size changed in
> av_frame_get_buffer, will break qsv vpp.

Calls av_frame_get_buffer-> get_video_buffer could allocate memory at one time and
make the code clearer, but will introduce plane_padding:

plane_padding = FFMAX(16 + 16/*STRIDE_ALIGN*/, align);

The padding bytes was initialized by variable "align", and "align" was set to 128 as a requirement.
So the padding bytes will always be set to align and I think it won't break vpp pipeline.

In fact, it's a reverse action of padding_bytes being introduced in get_video_buffer():
for (i = 1; i < 4; i++) {
        if (frame->data[i])
            frame->data[i] += i * plane_padding;
    }

To make it more clear and less tricky, I thought 128 can be replaced by variable or Macro like:
Int align = plane_padding = 128;
or
#define PLANE_PADDING 128

Thanks,
Linjie