[FFmpeg-devel] [PATCH] avfilter/vf_xcam: add xcam video filter

[赵娟]

The xcam filter is providing high quality stitching, accepting data from
the current 360 camera, one example is:
https://www.insta360.com/product/insta360-pro2/.
Now the current 360 camera output doesn't provide a broad pixel format,
only YUV420P and NV12. (For other formats, we can use current FFmpeg
filters to handle with a little performance and quality drop.)
Beside this, this xcam filter can work with FFmpeg's v360 filter together,
one example is transforming the stitching result from ERP (res:7680x3840)
to CubeMap (res:5760x3840), this can save 50% bitrate.

Paul B Mahol <onemda at gmail.com> 于2020年7月31日周五 下午4:39写道：

> On 7/31/20, Zong, Wei <wei.zong at intel.com> wrote:
> >
> >
> >> -----Original Message-----
> >> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Paul
> B
> >> Mahol
> >> Sent: Friday, July 31, 2020 3:57 PM
> >> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org
> >
> >> Cc: Liu, YinhangX <yinhangx.liu at intel.com>
> >> Subject: Re: [FFmpeg-devel] [PATCH] avfilter/vf_xcam: add xcam video
> >> filter
> >>
> >> On 7/31/20, Zong, Wei <wei.zong at intel.com> wrote:
> >> > Thank you for the comments on this patch, I want to make some
> >> > explanation on your comments.
> >> >
> >> > 1. the purpose of this patch
> >> >      We got some requirements from users who are trying libxcam
> >> > stitching or developers who showing interests. Many of them want an
> >> > easy way to construct an end 2 end pipeline of high quality immersive
> >> > media
> >> application.
> >> >     The key points are high resolution and low latency, FFMpeg
> >> > provides high quality video codec framework, if we implement video
> >> > stitching as a video filter in FFMpeg, they will get benefits.
> >> >
> >> > 2. 3rd party dependency
> >> >     The functions exposed by xcam filter only depends on essential
> >> > build tool-chain.
> >> > According to hardware configuration, user can get performance
> >> > improvement by install OCL, GLES or Vulkan.
> >> > These libraries are not necessary for video stitching, user can build
> >> > a CPU version without these libraries.
> >> >     Beside these, stitching and DVS function depend on feature match
> >> > algorithm from OpenCV library to get better quality, I posted demo
> >> > link in commit message.
> >> > User can run stitching without install OpenCV, the side effect is the
> >> > quality will decrease.
> >> >
> >> > 3. Immersive media standard
> >> >     Normally the camera manufactures provide video stitching software,
> >> > they defined private camera calibration parameters, these are very
> >> > critical to stitching quality. I know MPEG is making proposal on
> >> > transfer these meta-data in video stream, this will help compose
> >> > immersive media on cloud or edge server. I think it's a good idea to
> >> > create such a video filter in FFMpeg pipeline.
> >> >
> >> > 4. Why not implement stitching inside FFMpeg video filter
> >> >     The algorithms are complexity and the performance are also
> >> > important, the stitching function manage computing resource for
> >> > example thread handling, image data block dispatching. Personally I
> >> > think we don't need to construct a wheel from the beginning.
> >>
> >> Number of supported pixel formats are rather very low.
> >
> >    Do you mean the pixel format (NV12 & YUV420) that xcam filter
> supported
> > is not enough?
> >  xcam filter works as a post processing component between video decoder
> and
> > encoder, we suppose the data comes from camera or stored video clips, so
> it
> > handles YUV format.
> > libxcam algorithm optimization highly depends on pixel layout, to support
> > YUV422 or YUV444 will cost much effort, we do not meet the requirement
> till
> > now.
>
> Ok then, I thought library provided support for it already.
>
> Supporting only few pixel formats is very limited functionality IMHO.
>
> >
> >
> >> _______________________________________________
> >> ffmpeg-devel mailing list
> >> ffmpeg-devel at ffmpeg.org
> >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >>
> >> To unsubscribe, visit link above, or email
> >> ffmpeg-devel-request at ffmpeg.org
> >> with subject "unsubscribe".
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".



-- 
---------------------------
            *^_^*
Many Thanks & Best Regards!
Amy Zhao 赵娟