[Ffmpeg-devel] Parallelizing the h.264 Decoder

[Loren Merritt]

On Mon, 13 Nov 2006, Philip Peter wrote:

> i'm currently trying to make a port of the h.264 decoder to multicore systems 
> with special emphasis on the cell processor.
> Since the cells cores are limited in local memory, i want to parallelize the 
> decoding of an actual frame, not just decode multiple frames at once.
> To make the port easier i decided to go with a fixed distribution of the 
> decoding steps to the different cores, rather than with an dynamic model.
> Therefore i would have one or more core do the cabac decoding and one or more 
> cores to the transformation, quantization and filtering.
>
> As i understand the h264.c the best place to do the partition would be in the 
> decode_slice function. In that function decode_mb_cabac and hl_decode_mb are 
> called.
> However at the moment, those functions are called directly after each other 
> for each macroblock. This would be suitable for a pipelined approach, but i'm 
> afraid that one core won't suffice to do the cabac decoding.

It is not possible to split cabac into multiple cores, except by decoding 
multiple frames at once. (Ok, so multiple slices also works, but I'd 
rather not depend on the bitstream using multiple slices, especially since 
I'm removing that requirement from x264's smp.) Each and every cabac bit 
(within one slice) must be processed in the bitstream order, and the other 
stuff in decode_mb_cabac is needed in order to decide how to decode the 
next cabac bit.

> With the current sources would it be possible to first decode the cabac for 
> all macroblocks (preferably split over a few cores) and then do the rest of 
> the decoding process?

The simple way to do this with the current sources would be to duplicate 
the dct, mvd, and other arrays which currently only store 1 macroblock at 
a time. Which would take about 3 times as much memory as a frame itself.
The only way I can see to reduce that memory is to synchronize the threads 
more often so you don't have to store as many macroblocks.

--Loren Merritt