[FFmpeg-user] CPU and GPU

[Moritz Barsnick]

On Mon, Mar 02, 2020 at 04:57:23 -0500, Mark Filipak wrote:
> I used to work at Intel as a product line architect. I sat on 3
> divisional planning councils, including graphics processors, and
> co-chaired one.

[OT, sorry]

BTW, I have worked as Intel Corp. as well, incidentally both in
graphics processors ("Petascale Development" ;-)) and in FPGA
technology. Both in the same project, and before I was all too avid in
video technology.

> I outlined what appeared to me to be the direction of open source video
> development: That what is developed using open source code is migrated
> to FPGAs and hence to GPUs. I asked if I have it right (correct).

I'd like to pitch in to what Carl Eugen doubted: You hardly want to
"convert" software to hardware. Simplified (not sure about all the
terms): In software, execution is linear, procedular, while in
hardware, everything is parallel by design (while data flow again can
be "linear"). You wouldn't want to give up this advantage of hardware
over software. You want to mix the best of both worlds.

(BTW, for those who aren't aware: hardware, such as chips, ASICs, and
also FGPA content, is described in something you could call software.
But it's not procedural in the same way.)

What hardware would do, is to build accelerated "calculators" for
certain operations. This might be a complete codec (raw in, encoded
out, like GPUs do, and probably like Amazon AWS does), or just bits of
hardware capable of computations which standard CPU components aren't
capable of at the same speed. Think MMX, SSE, but specialized for
graphics or e.g. vector accelerations. Either of these would be hooked
into a software flow, as ffmpeg does with hardware ("GPU") encoding.

Many smaller video related operations could already be shifted to GPU
hardware, but I believe ffmpeg's GPU accelerations don't take advantage
of this. Perhaps its OpenCL and OpenGL uses do, but I don't believe
there are many. (Or is there any ffmpeg software encoder assisted by
hardware, except for the MMX/SSE/AVX type CPU accelerations?)

> Of course, the ffmpeg crew works to exploit the capabilities that are
> thereby created. This is all to be celebrated. What is it that you
> object to? What in what I wrote is wrong or that industry chip providers
> would find offensive?

If someone took the ffmpeg sources to "reimplement" them in hardware,
selling that service without open-sourcing their efforts. I believe
that is what is meant.

Sorry if I drifted off, used some incorrect terms, or have a
misunderstanding of this. It's just that my understanding of "FPGA
based" seems to be different than yours.

Cheers,
Moritz