[Libav-user] How to be a considerate thief (holding onto hardware frames without choking the decoder)

[Dan Egnor]

Hello lovelies,

For my Linux application, I want to run a decoder which uses V4L2
(such as h264_v4l2m2m, or hevc with the appropriate context) to
generate AV_PIX_FMT_DRM_PRIME frames, which will be shown directly
with KMS/DRM (zero copy), and I want to *hold onto a number of those
frames* for various nonlinear effects.

For a small number of frames this works fine, I can simply hang onto
the AVFrame and not unref it, which means that frame's underlying
V4L2Buffer won't get released or recycled.

HOWEVER if I hold on to too many frames, there aren't enough buffers
left for the decoder to work, and I get errors. (Precise errors
available on request, though I don't think anyone would be surprised.)

IDEALLY I would be able to hold onto these frames and allocate
replacement buffers for the decoder to fill. But this doesn't seem to
be possible with the way the existing libav V4L2 code works? Maybe I
am missing something? The options I can see:

- Set the num_capture_buffers option to a high value, so there are
still some buffers left even if I hold onto some. This requires
predicting how many I'll need, also even if I don't need them it will
end up using memory for them all (buffer management seems to be simple
FIFO).

- Write my own decoder (or adapt libav code) using V4L2, probably
using V4L2_MEMORY_DMABUF so I can swap the actual buffers being
returned, letting me allocate new buffers to replace ones that are
stolen. This doesn't seem like an easy adaptation, especially since
there's no direct way to allocate a single DMA buffer with V4L2 alone
(though one can with a DRM driver).

- Give up on zero-copy and just copy the frame data. I think that's
what I'll be doing for now...

Am I missing anything?

-- egnor