[FFmpeg-devel] [EXT] Re: [PATCH v3 1/3] avcodec/v4l2_context: don't reinit output queue when dynamic resolution change
Ming Qian
ming.qian at nxp.com
Tue Aug 17 04:22:45 EEST 2021
> -----Original Message-----
> From: Andriy Gelman [mailto:andriy.gelman at gmail.com]
> Sent: Tuesday, August 17, 2021 1:34 AM
> To: Ming Qian <ming.qian at nxp.com>
> Cc: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
> Subject: Re: [EXT] Re: [FFmpeg-devel] [PATCH v3 1/3] avcodec/v4l2_context:
> don't reinit output queue when dynamic resolution change
>
> Caution: EXT Email
>
> On Mon, 02. Aug 02:32, Ming Qian wrote:
> > > -----Original Message-----
> > > From: Andriy Gelman [mailto:andriy.gelman at gmail.com]
> > > Sent: Monday, August 2, 2021 7:36 AM
> > > To: FFmpeg development discussions and patches
> > > <ffmpeg-devel at ffmpeg.org>
> > > Cc: Ming Qian <ming.qian at nxp.com>
> > > Subject: [EXT] Re: [FFmpeg-devel] [PATCH v3 1/3]
> > > avcodec/v4l2_context: don't reinit output queue when dynamic
> > > resolution change
> > >
> > > Caution: EXT Email
> > >
> > > Hi,
> > >
> > > On Thu, 29. Jul 14:00, Ming Qian wrote:
> > > > in the v4l2 stateful video document, we can see the following
> > > > description:
> > > > During the resolution change sequence, the OUTPUT queue must
> > > remain
> > > > streaming. Calling VIDIOC_STREAMOFF() on the OUTPUT queue
> would
> > > > abort the sequence and initiate a seek.
> > > >
> > > > In principle, the OUTPUT queue operates separately from the
> CAPTURE
> > > > queue and this remains true for the duration of the entire
> > > > resolution change sequence as well.
> > > >
> > > > so don't reinit the output queue when handling the resolution
> > > > change event
> > > >
> > > > Signed-off-by: Ming Qian <ming.qian at nxp.com>
> > > > ---
> > > > libavcodec/v4l2_context.c | 27 ++-------------------------
> > > > 1 file changed, 2 insertions(+), 25 deletions(-)
> > > >
> > > > diff --git a/libavcodec/v4l2_context.c b/libavcodec/v4l2_context.c
> > > > index ff1ea8e57b08..dda5157698c3 100644
> > > > --- a/libavcodec/v4l2_context.c
> > > > +++ b/libavcodec/v4l2_context.c
> > > > @@ -162,9 +162,8 @@ static int v4l2_handle_event(V4L2Context *ctx)
> {
> > > > V4L2m2mContext *s = ctx_to_m2mctx(ctx);
> > > > struct v4l2_format cap_fmt = s->capture.format;
> > > > - struct v4l2_format out_fmt = s->output.format;
> > > > struct v4l2_event evt = { 0 };
> > > > - int full_reinit, reinit, ret;
> > > > + int reinit, ret;
> > > >
> > > > ret = ioctl(s->fd, VIDIOC_DQEVENT, &evt);
> > > > if (ret < 0) {
> > > > @@ -180,25 +179,12 @@ static int v4l2_handle_event(V4L2Context
> *ctx)
> > > > if (evt.type != V4L2_EVENT_SOURCE_CHANGE)
> > > > return 0;
> > > >
> > > > - ret = ioctl(s->fd, VIDIOC_G_FMT, &out_fmt);
> > > > - if (ret) {
> > > > - av_log(logger(ctx), AV_LOG_ERROR, "%s VIDIOC_G_FMT\n",
> > > s->output.name);
> > > > - return 0;
> > > > - }
> > > > -
> > > > ret = ioctl(s->fd, VIDIOC_G_FMT, &cap_fmt);
> > > > if (ret) {
> > > > av_log(logger(ctx), AV_LOG_ERROR, "%s VIDIOC_G_FMT\n",
> > > s->capture.name);
> > > > return 0;
> > > > }
> > > >
> > > > - full_reinit = v4l2_resolution_changed(&s->output, &out_fmt);
> > > > - if (full_reinit) {
> > > > - s->output.height = v4l2_get_height(&out_fmt);
> > > > - s->output.width = v4l2_get_width(&out_fmt);
> > > > - s->output.sample_aspect_ratio = v4l2_get_sar(&s->output);
> > > > - }
> > > > -
> > > > reinit = v4l2_resolution_changed(&s->capture, &cap_fmt);
> > > > if (reinit) {
> > > > s->capture.height = v4l2_get_height(&cap_fmt); @@ -206,18
> > > > +192,9 @@ static int v4l2_handle_event(V4L2Context *ctx)
> > > > s->capture.sample_aspect_ratio = v4l2_get_sar(&s->capture);
> > > > }
> > > >
> > > > - if (full_reinit || reinit)
> > > > + if (reinit)
> > > > s->reinit = 1;
> > > >
> > > > - if (full_reinit) {
> > > > - ret = ff_v4l2_m2m_codec_full_reinit(s);
> > > > - if (ret) {
> > > > - av_log(logger(ctx), AV_LOG_ERROR,
> > > "v4l2_m2m_codec_full_reinit\n");
> > > > - return AVERROR(EINVAL);
> > > > - }
> > > > - goto reinit_run;
> > > > - }
> > > > -
> > > > if (reinit) {
> > > > if (s->avctx)
> > > > ret = ff_set_dimensions(s->avctx, s->capture.width,
> > > s->capture.height);
> > > > --
> > > > 2.32.0
> > > >
> > >
> > > You probably missed my question in the v2 thread. I asked:
> > >
> > > Could you send me a sample for testing, please.
> > > What boards did you test on?
> > >
> > > Thanks,
> > > --
> > > Andriy
> >
> > Hi Andriy,
> > I tested on IMX8QXP and IMX8MP, and they are based on different codec
> IP.
> > On imx8qxp, the vpu doesn't handle the buffer of v4l2 output queue, the
> driver need to copy the stream data to a ring buffer.
> > But on imx8mp, the vpu can decode the buffer of v4l2 output queue
> directly.
> >
> > My test script is as below:
> >
> > ffmpeg -vsync passthrough -vcodec h264_v4l2m2m -i
> > /mnt/test_352_288.h264 -f rawvideo -y output_nv12.yuv
> >
> > the test_352_288.h264 is attached, but it doesn't really matter which stream
> file is tested.
>
> Sorry for the delay replying.
>
> I tested on RPi4 with an example where there is a resolution change
> midstream, and for me this doesn't work at all (with or without your patch).
> The code gets stuck trying to wait for the buffers to get released before reinit is
> done:
>
> 3833e70] capture changed (1280x720) -> (640x368) [h264_v4l2m2m @
> 0x3833e70] reinit context [h264_v4l2m2m @ 0x3833e70] waiting for user to
> release AVBufferRefs
>
> I don't know how much the resolution change part was tested in the past.
Yes, I found it too, it will stall at waiting user to return the capture buffer.
Actually we test resolution change flow with gstreamer, and the driver flow
can pass the gstreamer.
And consider the specification of v4l2 stateful decoder, I think ffmpeg should also follow the flow.
>
> I'll share the sample with you by email.
> I tried to upload the sample to
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstreams
> .videolan.org%2Fupload%2F&data=04%7C01%7Cming.qian%40nxp.com
> %7C295466e646684938def108d960dc1d3a%7C686ea1d3bc2b4c6fa92cd99c
> 5c301635%7C0%7C1%7C637647320762155592%7CUnknown%7CTWFpbGZs
> b3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0
> %3D%7C2000&sdata=VrXdX3fQA6GAOhTAtqA14h65F%2FgSfe8cx9N2m%
> 2FHh2BI%3D&reserved=0, but it doesn't say what's the url of the file after
> uploading. It doesn't exist in ffmpeg directory..
> Anyone used this before?
>
> Thanks,
> --
> Andriy
More information about the ffmpeg-devel
mailing list