[FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the stack filters

[Xiang, Haihao]

On Thu, 2021-08-05 at 15:53 +0000, Soft Works wrote:
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > Xiang, Haihao
> > Sent: Thursday, 5 August 2021 04:33
> > To: ffmpeg-devel at ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the
> > stack filters
> > 
> > On Wed, 2021-08-04 at 09:17 +0000, Soft Works wrote:
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > > > Haihao Xiang
> > > > Sent: Wednesday, 4 August 2021 10:33
> > > > To: ffmpeg-devel at ffmpeg.org
> > > > Cc: Haihao Xiang <haihao.xiang at intel.com>
> > > > Subject: [FFmpeg-devel] [PATCH v2] avfilter: add QSV variants of the
> > > > stack filters
> > > > 
> > > > Include hstack_qsv, vstack_qsv and xstack_qsv, some code is copy and
> > > > pasted from other filters
> > > > 
> > > > Example:
> > > > $> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.h265 -filter_complex
> > > > "[0:v][0:v]hstack_qsv" -f null -
> > > > ---
> > > 
> > > [...]
> > > 
> > > > +
> > > > +/*
> > > > + * Callback for qsvvpp
> > > > + * @Note: qsvvpp composition does not generate PTS for result frame.
> > > > + *        so we assign the PTS from framesync to the output frame.
> > > > + */
> > > > +
> > > > +static int filter_callback(AVFilterLink *outlink, AVFrame *frame) {
> > > > +    QSVStackContext *sctx = outlink->src->priv;
> > > > +
> > > > +    frame->pts = av_rescale_q(sctx->fs.pts,
> > > > +                              sctx->fs.time_base, outlink->time_base);
> > > > +    return ff_filter_frame(outlink, frame); }
> > > 
> > > If the surface.Data.TimeStamp gets overwritten by libMFX, why not copy
> > > the PTS from the input frame in ff_qsvvpp_filter_frame ?
> > > 
> > > That would apply the timestamp from the last input, though. Preferably
> > > would it be taken from the first input instead. For 2-n, you could
> > > perhaps clone the frames and assign the pts from the first input's
> > > frame?
> > 
> > Thanks for the comment and suggestion. This callback function was copy-
> > and- pasted from overlay_qsv filter because MSDK composition is also used
> > by this filter, I'd like to use the same way to generate pts for these
> > filters, but
> > I'll try your suggestion for these filters in the future.
> 
> Yea I see - the overlay_qsv filter does it the same way. This has probably
> been
> ok earlier because the callback happened synchronously. This is no longer the
> case since the async_depth patch which introduced the fifo processing. Now it
> can happen that the calback is performed for an earlier frame than the one
> that is currently gated by framesync.

async_depth is not enabled for overlay_qsv and stack qsv filters, s->async_depth 
is 0, so the callback is still performed synchronously for these filters. 

Thanks
Haihao