[FFmpeg-devel] mutlithreading in ffmpeg?

[Thomas Worth]

On Sat, Mar 19, 2011 at 5:08 AM, Reimar Döffinger
<Reimar.Doeffinger at gmx.de> wrote:
> On Fri, Mar 18, 2011 at 05:33:04PM -0700, Thomas Worth wrote:
>> On Fri, Mar 18, 2011 at 7:31 AM, Michael Niedermayer <michaelni at gmx.at> wrote:
>> > On Thu, Mar 17, 2011 at 04:10:58PM -0700, Thomas Worth wrote:
>> >> >> There is ffmpeg-mt for frame level multithreaded decoding.
>> >> >> http://gitorious.org/ffmpeg/ffmpeg-mt
>> >> >> It would be nice to see that integrated into mainline ffmpeg.
>> >> >> Its quite effective, scales nicely to whatever number of cores you've got
>> >> >> and not sensitive to what to tool was used to create the content.
>> >> >
>> >> > It will be integrated soon.
>> >>
>> >> Hopefully you guys can fix the seeking problem with the H.264 decoder
>> >> before then... hint, hint ;-)
>> >
>> > which issue number on the bug tracker is that?
>>
>> I didn't file a bug report in case I was overlooking something.
>> However, I've spent some time with this and I am pretty sure it's a
>> problem with multi-threaded decoding. Anyway, I put together a simple
>> page that explains it in case it's a problem with my code:
>>
>> http://rarevision.com/ffmpeg-mt/
>
> Well, it is in sofar a problem with your code as it does not show any _bug_,
> just some conspiciously inconsistent behaviour.
> And even that is unsure since you do not show the full code, e.g. whether
> you correctly flush after a seek.
> But either way not immediately returning a decoded frame is _not_ a bug,
> if your code in any way assumes this it is _broken_ and it will not
> work correctly with some current and future (also hw-accelerated) codecs.

I have assumed this entire time that the problem was with my code, but
after trying several variations to no avail and the fact that it works
on regular ffmpeg and not ffmpeg-mt should be the first clue that
something is wrong. Besides, why would my code influence things like
whether or not the frame immediately after a keyframe (consistently,
as shown by the output) gets returned? And why is it returning the
same frame twice, even though the timestamps are one frame apart? This
works fine on all frames _after_ the one following the keyframe. And
why is avcodec_decode_video2() returning the decompressed size of the
frame but not actually putting it into an AVFrame? I fully agree that
this is conspicuously inconsistent behavior.

Oh and to just be thorough, I do call avcodec_flush_buffers(). I left
that part of the code out, but indeed I have to do that otherwise
seeking doesn't work properly as you pointed out.

Anyway, this problem is very easy to reproduce and 100% consistent if
anyone is interested in having a look. It would be nice to have it
working, as ffmpeg-mt's decoding is much faster than standard ffmpeg.