[FFmpeg-devel] [PATCH 3/6] Add AV_PIX_FMT_NV12T.

[Alexis Ballier]

On Fri, 21 Nov 2014 11:23:49 +0100
wm4 <nfxjfg at googlemail.com> wrote:
> > I didn't know such "opaque" things existed; it probably makes more
> > sense with it indeed.
> > 
> > How should I do it ? add .flags = AV_PIX_FMT_FLAG_HWACCEL and be
> > done ? I would like to keep 'av_pix_fmt_count_planes()' working for
> > it at least
> 
> No, this would imply that the pointer to the opaque data is in
> AVFrame.data[3], and all other pointers are ignored. So you have only
> 1 pointer. AVFrame.linesize has no meaning either in this case.
> 
> > [...]
> > > It also means that _every_ software which wants to use this new
> > > decocer has to know how to insert the filter etc. - what's the
> > > point?
> > 
> > This could be "fixed" by adding support in swscale for it, but I see
> > little to no point to it.
> > 
> > I think you have to specifically ask for v4l2m2m codecs if you want
> > to use them do hw (de/en)code; so why not do everything in hw and
> > also setup the filter ?
> 
> Well, if the filter copies to system RAM anyway (if that chip
> distinguishes between GPU and system RAM at all),

Speaking only for that chip (this will likely differ for others), there
is no such thing as "gpu ram", but there is cma allocated & reserved
memory in system ram.
Default settings ask the driver to mmap this memory and use this as a
buffer; this indeed introduces some copies to feed the filter or codec
with data: if we get random input buffers, we have to memcpy() them
at the mmap adress given by the driver. mfc codec refuses to handle
anything but mmaped io.
You can feed the filter with userptr memory (basically a userspace
pointer that the driver will translate himself) but this memory must be
_physically_ contiguous, which I don't think we have any way to ensure
at userspace level.
The solution is to take the data from the decoder, feed the frame with
zero copy (thanks to refcounted frames) to the filter: Since the
buffers come from the decoder who has mmaped them in its cma-allocated
memory it is guaranteed to be physically contiguous.

> then why do you need
> such a filter as user-visible thing at all?
> Where exactly is the problem in putting this code into libavcodec?

There is a limited # of hw filters; you may also want to scale the
video to fit your display, which the filter can do together with the
fmt conversion in one pass. I don't think up/down scaling the
decoded video should be done in lavc.

> > (note: I shall write some doc about all this once this has settled)
> > 
> > > But if it just works without requiring the API user to jump
> > > through hoops, I see at least some value in it.
> > 
> > 
> > Would AV_PIX_FMT_FLAG_HWACCEL be enough ?
> 
> I can't speak for others whether this approach is acceptable.

I don't understand:

> No, this would imply that the pointer to the opaque data is in
> AVFrame.data[3], and all other pointers are ignored. So you have only
> 1 pointer. AVFrame.linesize has no meaning either in this case.

this won't work with NV12(T) since the format has 2 planes