[FFmpeg-user] Stereoscopic hwaccel

[Ted Park]

> On Sep 17, 2019, at 12:33 PM, JackDesBwa <jackdesbwa at desbwa.org> wrote:
> 
> Removing the full chain of filter_complex to keep the simplest example,
> computing using CPU would give:
> ffmpeg -i aligned/l_01.mp4 -i aligned/r_01.mp4 -filter_complex
> [0][1]framepack=sbs -c:v libx264 out_libx264.mp4
> It takes 22s wall time and produces a 18Mio output.
> 
> Accelerating with vaapi:
> ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi
> -hwaccel_output_format vaapi -i aligned/l_01.mp4 -i aligned/r_01.mp4
> -filter_complex '[0][1]framepack=sbs,format=nv12|vaapi,hwupload' -c:v
> h264_vaapi -qp 22 -profile:v high out_vaapi.mp4
> It takes 5.6s wall time and produces a 32Mio output.
> 
> Accelerating with nvenc:
> ffmpeg -i aligned/l_01.mp4 -i aligned/r_01.mp4 -filter_complex
> [0][1]framepack=sbs -c:v h264_nvenc -rc:v vbr_hq -qp:v 22 -profile:v high
> out_nvenc.mp4
> It takes 2.7s wall time and produces a 2.3Mio output.
> 
> The numbers are only for one short 240-frames
> 3604x1064-unfriendly-dimentions clip here.
> The nvenc version is fast and create small file, but has very bad quality.
> Perhaps I do not use the right parameters.
> The vaapi version is quite fast too, but create a bigger file. Perhaps I
> should tweak parameters here too.
> Advices on parameters are welcomed, but the more import thing is the
> following.
> 
> *The two accelerated versions do not add the SBS frame-packing information
> in the stream*. The documentation of framepack filter says that it would
> set proper metadata on supported codecs. Does it mean that I am stuck with
> CPU codecs to get the proper output stream with stereo format information
> in it?
> 
> JackDesBwa

So I feel like the encoding part is the biggest bottleneck and it wouldn’t make much of a difference if the decoder and filter part are software. I don’t think the filter supports hw formats anyway, and for the second test only one of the decoders are vaapi. Setting the profile on the encoder might be what gets rid of the stereo frame packing settings, though I don’t know if the hw accelerated encoders configure those settings in the first place.