[FFmpeg-user] Transcoding h264 to mjpeg using vaapi

[Erik Slagter]

Hi all,

I am trying to transcoding h264 to mjpeg (for easy editing, that's all). 
I can do it with ffmpeg, no problem there. But I want to use vaapi for 
it, works so much faster.

I have it all already working, using this command line:

ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/card0 
-hwaccel_output_format vaapi -i testfile.ts -codec:v mjpeg_vaapi 
-global_quality 90 -map 0:v -codec:a flac -ac 2 -ar 48000 -map 0:2 -sn 
-dn -vsync cfr -shortest -f testfile.mkv

There is only one thing left. Decoded h264 yields yuv420p format (i.e. 
16-235 range) while mjpeg uses yuvj420p (full range). using the software 
encoder (and still the vaapi h264 decoder) this fact is accounted for 
and the range is transformed automatically.

When using complete vaapi h264 to mjpeg transcoding like this, there is 
no step in the chain to perform the transformation and the result is too 
high brightness, too low contrast.

I can workaround it like this, add this to the command line: -vf 
"hwdownload,format=nv12 scale=in_range=mpeg:out_range=jpeg". Works like 
a charm, but also slows the transcoding process by a factor 10 which 
makes it slower than a pure software transcode.

I've been searching for options to force the vaapi h264 decoder output 
yuvj420p (full range) or to force the vaapi mjpeg encoder to use yuv420p 
(limited range) as input, but none found. All of them only want to use 
pix_fmt = "vaapi_vld" or "nv12".

Is there some way I am overlooking here? Like an option like above, or 
e.g. a scaling option that makes the vaapi engine do the transformation 
itself?

Thanks,
Erik.

BTW there is an ugly workaround and that's to do the actual editing in 
the wrong range and then have ffmpeg do the transformation in the next 
step, when the media is encoded to h264 or h265. But I don't like it 
very much that way, it's ugly.

PS please CC me directly.