[FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack filters

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Xiang, Haihao
> Sent: Thursday, 5 August 2021 04:33
> To: ffmpeg-devel at ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the
> stack filters
> 
> On Wed, 2021-08-04 at 09:17 +0000, Soft Works wrote:
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > > Haihao Xiang
> > > Sent: Wednesday, 4 August 2021 10:33
> > > To: ffmpeg-devel at ffmpeg.org
> > > Cc: Haihao Xiang <haihao.xiang at intel.com>
> > > Subject: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the
> > > stack filters
> > >
> > > Include hstack_qsv, vstack_qsv and xstack_qsv, some code is copy and
> > > pasted from other filters
> > >
> > > Example:
> > > $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -filter_complex
> > > "[0:v][0:v]hstack_qsv" -f null -
> > > ---
> >
> > [...]
> >
> > > +
> > > +/*
> > > + * Callback for qsvvpp
> > > + * @Note: qsvvpp composition does not generate PTS for result frame.
> > > + *        so we assign the PTS from framesync to the output frame.
> > > + */
> > > +
> > > +static int filter_callback(AVFilterLink *outlink, AVFrame *frame) {
> > > +    QSVStackContext *sctx = outlink->src->priv;
> > > +
> > > +    frame->pts = av_rescale_q(sctx->fs.pts,
> > > +                              sctx->fs.time_base, outlink->time_base);
> > > +    return ff_filter_frame(outlink, frame); }
> >
> > If the surface.Data.TimeStamp gets overwritten by libMFX, why not copy
> > the PTS from the input frame in ff_qsvvpp_filter_frame ?
> >
> > That would apply the timestamp from the last input, though. Preferably
> > would it be taken from the first input instead. For 2-n, you could
> > perhaps clone the frames and assign the pts from the first input's
> > frame?
> 
> Thanks for the comment and suggestion. This callback function was copy-
> and- pasted from overlay_qsv filter because MSDK composition is also used
> by this filter, I'd like to use the same way to generate pts for these filters, but
> I'll try your suggestion for these filters in the future.

Yea I see - the overlay_qsv filter does it the same way. This has probably been
ok earlier because the callback happened synchronously. This is no longer the
case since the async_depth patch which introduced the fifo processing. Now it
can happen that the calback is performed for an earlier frame than the one
that is currently gated by framesync.

I agree that this should be addressed in another patch.

Thanks,
sw