[FFmpeg-cvslog] random thoughts about refactoring

[Michael Niedermayer]

On Sun, Jan 10, 2010 at 02:48:00PM -0500, compn wrote:
> On Sun, 10 Jan 2010 17:42:24 +0100, Michael Niedermayer wrote:
> >On Sun, Jan 10, 2010 at 12:44:24PM +0100, Diego Biurrun wrote:
> >> On Fri, Jan 08, 2010 at 02:28:16AM +0100, Michael Niedermayer wrote:
> >> > On Fri, Jan 08, 2010 at 01:20:03AM +0100, Diego Biurrun wrote:
> >> > > On Wed, Jan 06, 2010 at 06:38:14PM +0100, Michael Niedermayer wrote:
> >> > > > On Tue, Jan 05, 2010 at 11:08:22PM +0100, Diego Biurrun wrote:
> >> > > > > On Tue, Jan 05, 2010 at 08:43:51PM +0100, Michael Niedermayer wrote:
> >> > > > > > On Tue, Jan 05, 2010 at 11:29:46AM +0100, Diego Biurrun wrote:
> >> > > > > > > On Tue, Jan 05, 2010 at 03:20:47AM +0100, Michael Niedermayer wrote:
> >> > > > > > > > On Mon, Jan 04, 2010 at 08:23:23PM +0100, Diego Biurrun wrote:
> >> > > > > There are people here who know H.264 well enough to lend
> >> > > > > a helping hand in theory, but in practice the implementation in lavc is
> >> > > > > just too impenetrable.  h264.c is around 10.000 lines if you count svq3.c
> >> > > > > which it directly includes!
> >> > > > 
> >> > > > > 
> >> > > > > Also, our H.264 decoder is not a speed demon.
> >> > > > 
> >> > > > That seems to depend on CPU and situation, gcc making poor inlining choices
> >> > > > and overflowing the available caches is not exactly the codes fault ...
> >> > > 
> >> > > Carl Eugen was the first person I have ever heard claim that our
> >> > > H.264 decoder was faster than anything, ever...
> >> > 
> >> > Ive never tried CoreAVC, but i can assure you the reference implementation
> >> > is much much slower
> >> 
> >> You are referring to the H.264 reference implementation?
> >> 
> >> Jason mentioned two new H.264 decoders on IRC that are even faster than
> >> CoreAVC...
> >
> >Why is this information not posted on the mailing list ?
> 
> i'll paste some relevent irc chat:

thank you!
while not as usefull as i thought, it is usefull.

some comments below


> 
> [17:17] <Dark_Shikari> I would say the single most important thing for h264 decoding
> [17:17] <Dark_Shikari> is to get ffmpeg-mt mergedc
> [17:17] <Dark_Shikari> that will give more speed boost than any other optimizations

yes, thats true of course but it seems we lack a volunteer to work on
this.


[...]
> [17:49] <DonDiego> Dark_Shikari: we have no students who reach the patch review phase, so this is not a problem

ehm? there are plenty who did reach patch review, theres even a patch laying
around that i should review. You make it look quite a bit worse than it is


> [17:49] <Dark_Shikari> lol
> [17:49] <DonDiego> however, we have students who seem to take the money and do hardly anything in return..
> 
> [17:50] <Dark_Shikari> 2) students should be on IRC every single day
> [17:50] <Dark_Shikari> and talking about their project, every single day
> [17:50] <Dark_Shikari> 3) students should be on IRC before applying
> [17:50] <Dark_Shikari> 4) students should be on IRC after the end date
> [17:50] <DonDiego> what requirements do you have for x264?
> [17:50] <Dark_Shikari> requirements for x264:
> [17:50] <mru> jai is exempt from that rule ;-)
> [17:51] <Dark_Shikari> of course, if you prove yourself otherwise, you can break the rules.

> [17:51] <Dark_Shikari> 1) You must complete a qualification task that represents a portion of the actual project you intend to do.

This is not a bad idea where that is possible


> [17:51] <Dark_Shikari> For example, if you were writing an h264 decoder, you would have to write a header + bitstream parser
> [17:51] <Dark_Shikari> that printed dct coefficients for cavlc streams
> [17:51] <DonDiego> yes, i notice that many people from there apply and i suspect that money could be a motivation, there's $4500 to earn, i can live off that kind of money for 1/2 year in germany..
> [17:51] <Dark_Shikari> that kind of thing is _useful for your actual project_
> [17:51] <kierank> the x264 qualifications are meatier than ffmpeg ones
> [17:51] <DonDiego> mru: jai has just taken money and done nothing in return :-/
> [17:52] <Dark_Shikari> 2) You must be on IRC often, and get to know the community
> [17:52] <Dark_Shikari> you must be involved
> [17:52] <Dark_Shikari> you must respond to pings
> [17:52] <Dark_Shikari> If I ping you and say "there's a bug in your code here", you should be able to respond
> [17:52] <Dark_Shikari> obviously, I can't control timezones/sleep schedules/etc
> [17:52] <mru> DonDiego: jai has done stuff though
> [17:52] <Dark_Shikari> but in general, you shouold be available.
> [17:52] <kierank> 22:42] <@Dark_Shikari> you must respond to pings --> like janihan ;)
> [17:52] <Dark_Shikari> lol
> [17:52] <DonDiego> mru: that was last year, this year it was money for nothing..
> [17:53] * mru takes the chicks for free...

> [17:53] <Dark_Shikari> I don't think the x264 requirements are harder than ffmpeg
> [17:53] <Dark_Shikari> the ffmpeg qual tasks are really hard
> [17:53] <Dark_Shikari> and don't even relate to the main project often
> [17:53] <Dark_Shikari> I think the main thing is that we create a community and force people to be involved
> [17:53] <Dark_Shikari> and if they can't stay online and involved, they don't get in
> [17:53] <mru> that's how communities work
> [17:53] <mru> _saying_ you're in doesn't make it so
> [17:54] <Dark_Shikari> yup
> [17:54] <Dark_Shikari> having a qual task that leads into the main project is the best idea of mine IMO
> [17:54] <Dark_Shikari> it gives people a feeling that they're actually part done with the project
> [17:54] <Dark_Shikari> before SOC even starts
> [17:54] <Dark_Shikari> and people are less likely to quit in the middle
> [17:55] <Dark_Shikari> as opposed to simply bullshitting and quitting after getting nothing done
> [17:55] <DonDiego> we need to work on our soc, it's in a sad state and not improving

> [17:55] <DonDiego> also, our tasks are *much* too hard
> [17:55] <Dark_Shikari> well, here's my advice for improving ffmpeg SOC

> [17:55] <DonDiego> basically the only student that ever finished a decoder is kostya

complete nonsense


> [17:55] <DylanZA> just saw this conversation - I rate that ffmpeg makes you jump through too many hoops to get qualified (from personal experience). you never want to do a patch review just to qualify to maybe do something

> [17:55] <Dark_Shikari> 1) Adopt the x264 qualification task method: qualification tasks consist of small, relatively easy subsets of the main project the student wants to apply for.
> [17:55] <Dark_Shikari> No patch review is required for qualification tasks.
> [17:55] <Dark_Shikari> Well, the mentor should review of course
> [17:55] <Dark_Shikari> in order to judge the student's ability
> [17:55] <Dark_Shikari> but it shouldn't be committed yet

Theres a part you seem to be missing, our qualification tasks by themselfs
improve ffmpeg as the code is reviewed and commited. This is not a
insignifiant contribution.



> [17:56] <Dark_Shikari> 2) Encourage interaction on IRC.  People who can't keep up to speed on a daily basis shouldn't be students.

> [17:56] <DonDiego> DylanZA: i disagree, we still don't get enough capable students..

yes, thats the primary problem


> [17:56] <Dark_Shikari> DonDiego: yes, because capable ones don't like the hoops

i disagree the arogant and lazy ones dont like the hoops. They may or may not
be capable at the same time.


> [17:56] <Dark_Shikari> DylanZA was one of our successes last year
> [17:56] <Dark_Shikari> he picked x264 over ffmpeg
> [17:56] <DonDiego> hmm
> [17:57] <DonDiego> qual tasks were started after the first year
> [17:57] <DylanZA> most reasonable students will pick a couple of projects anyway
> [17:57] <DonDiego> to weed out incompetent students
> [17:57] <Dark_Shikari> DonDiego: IMO there are two requirements
> [17:57] <Dark_Shikari> 1) competent/self-motivated
> [17:57] <Dark_Shikari> 2) involved
> [17:57] <DylanZA> but if one is too difficult then they choose another. but the poor students might just stick to one project and try their luck
> [17:57] <Dark_Shikari> unless you have both, you will have a failure
> [17:58] <DylanZA> Dark_Shikari: yuo also need luck

> [17:58] <Dark_Shikari> this is why I'm biased against indian/chinese students, they do not tend to be very involved

if iam not mistaken our mxf muxer was written by a chinese student together
with his mentor baptiste
also jai is indian if iam not mistaken and he wrote the alac stuff
jai failed on jpeg2k but kamil who i think was from poland failed on that
before.


> [17:58] <DonDiego> you probably have a point there..
> [17:58] <DylanZA> obviously any student can just pull out at some point
> [17:58] <Dark_Shikari> DylanZA: of course, but it's far less likely that a good one will
> [17:58] <Dark_Shikari> especially after completing a large part of their stuff
> [17:58] <peloverde> You need somebody with "Excellent English skills"
> [17:58] <DonDiego> after looking at our soc results, i share that bias

> [17:58] <DonDiego> i guess the money is just too tempting

Everyone needs money, especially students tend to be short on money as they
dont have a job with income yet ...


> 
> [18:01] <Dark_Shikari> a student should be willing to come on IRC and talk about what he's doing
> [18:01] <Dark_Shikari> and not just willing, but he should do it without asking
> [18:01] <Dark_Shikari> if he's stumped, he should come on and discuss what the best way to do X is
> [18:01] <Dark_Shikari> and others should be there to help
> [18:01] <DonDiego> i think the mentors are failing in that regard as well
> [18:02] <Dark_Shikari> that's a problem created due to lack of use of IRC
> [18:02] <DonDiego> unless everybody is talking to their students in private
> [18:02] <DonDiego> which would kind of be a waste

I do agree that communication beteen students and mentors probably was
not sufficient in some cases. a requirement for students to be on IRC might
indeed help here i dont know


> [18:02] <Dark_Shikari> our students got a lot of help on IRC even when mentors weren't available
> [18:02] <Dark_Shikari> e.g. holger wasn't always there to mentor Yuvi
> [18:02] <DonDiego> that could happen on the ml as well, it's not tied to irc
> [18:02] <Dark_Shikari> I don't think an ML is a suitable replacement for IRC.
> [18:02] <Dark_Shikari> for some types of discussion, realtime is much preferable
> 
> [18:05] <Dark_Shikari> one thing I notice about ffmpeg
> [18:05] <Dark_Shikari> at least half the comments are utterly trivial shit

yes iam aware of this and id love to see this be reduced.
If some people absolutely need to have english spelling and K&R correct in
code & comments then they should be doing the cleanup themselfs not scare
potentially great developers away.


> [18:05] <DonDiego> i think michael pulled it from some javadoc style guide

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The greatest way to live with honor in this world is to be what we pretend
to be. -- Socrates
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-cvslog/attachments/20100110/8fb6127a/attachment-0001.pgp>