[FFmpeg-devel] [PATCH] avfilter: add panorama filter

James Almer jamrial at gmail.com
Fri Mar 9 17:17:09 EET 2018

On 3/9/2018 12:12 PM, James Almer wrote:
> On 3/9/2018 12:04 PM, Paul B Mahol wrote:
>> On 3/9/18, James Almer <jamrial at gmail.com> wrote:
>>> On 3/9/2018 7:05 AM, Paul B Mahol wrote:
>>>> On 3/9/18, Paul B Mahol <onemda at gmail.com> wrote:
>>>>> On 3/9/18, wm4 <nfxjfg at googlemail.com> wrote:
>>>>>> On Fri, 9 Mar 2018 09:15:13 +0100
>>>>>> Paul B Mahol <onemda at gmail.com> wrote:
>>>>>>> On 3/9/18, wm4 <nfxjfg at googlemail.com> wrote:
>>>>>>>> On Thu, 8 Mar 2018 21:53:48 -0300
>>>>>>>> James Almer <jamrial at gmail.com> wrote:
>>>>>>>>> On 3/8/2018 9:50 PM, Hazem Ashmawy wrote:
>>>>>>>>>> [PATCH] avfilter: add panorama filter
>>>>>>>>>> Sorry about that! I removed them now.
>>>>>>>>>> For the future, any recommendation for a tool for linting /
>>>>>>>>>> checking
>>>>>>>>>> formating
>>>>>>>>>> rules?
>>>>>>>>> There's tools/patcheck. Feed it a git format-patch style of patch to
>>>>>>>>> find common issues, but keep in mind it can generate a lot of false
>>>>>>>>> positives.
>>>>>>>>> I don't know if we have documentation about actual formatting rules
>>>>>>>>> anywhere.
>>>>>>>> Also:
>>>>>>>> <_jamrial> shouldn't that panorama filter sent to the ml use the
>>>>>>>> spherical
>>>>>>>> frame side data?
>>>>>>>> I think so.
>>>>>>> Are there actual files that have such data?
>>>>>> Is that a trick question? I only know the non-standard, Google specific
>>>>>> metadata in mkv and mp4 that lavf can read (was any of this
>>>>>> standardized yet?).
>>>>>> But that doesn't change that we can tag AVFrames with this info, and
>>>>>> for files which don't have the metadata, it makes sense to me to set it
>>>>>> with a new vf_format argument or some sort of vf_setinfo (if we don't
>>>>>> have anything like this yet).
>>>>>> The part that is annoying is that vf_panorama still seems to require
>>>>>> setting an output projection, which would make the whole thing more
>>>>>> annoying instead of less, but even then I'd argue it should default to
>>>>>> taking the AVFrame configuration (AV_FRAME_DATA_SPHERICAL) as input by
>>>>>> default, even if the filter arguments can override it.
>>>>> That frame side data is very specific and thus considered barely useful
>>>>> here.
>>>>> Is it at all updated to the latest improvements, like new equi-angular
>>>>> cubemap projection?
>>>> I guess not at all, I get this:
>>>> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x21c1740] Unknown projection type
>>> Sample? Also, where is this new projection defined?
>> https://www.youtube.com/watch?v=qhLExhpXX0E
>> It is defined by Google?
> Vittorio and I used
> https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md
> to write the current Matroska and mov implementations, and by extension
> the AVSphericalMapping API. Specifically the Equirectangular and Cubemap
> projections.
> This sample seems to have a "ytmp" projection box, but it's not defined
> in the above document. I guess the name hints at it being a very early
> an internal draft? We can't really do much without a spec...

The h264/mp4 version has this unknown "ytmp" box, but the vp9/webm one
has a ProjectionType element with a value of 3, plus some binary data in
the ProjectionPrivate element, which according to the spec means a Mesh
projection. We don't currently support that one.

Weird, seeing mp4 supposedly has the "mshp" box for this.

> In any case, I insist much like wm4 that a filter like this should use
> the metadata stored in the AVFrame if available.

More information about the ffmpeg-devel mailing list