[MPlayer-DOCS] [PATCH] How to do regression testing using CVS

[The Wanderer]

On 11/26/2005 04:08 PM, Guillaume POIRIER wrote:

> Hi there, I've ripped off the doc on wine website
> (http://www.winehq.org/site/docs/winedev-guide/x1344) to add
> information about how to do regression tests using binary search to
> our docs.

I've read that doc, for further advice after having done a partial
regression on my own for a Wine bug which as far as I know is still
outstanding... it's good, but not perfect, although I don't offhand
remember the one thing I think I want to add to it. (I'll have to go
back and read the discussion of that bug report.)

> Comments?

> +If any non-programmer reads this, the fastest method to get at the point
> +where the problem occurred is to use a binary search, that is, search the
> +date of the breackage by repeatedly dividing the search interval in half.

I think it would be better to use an emdash here, after "search", rather
than a comma.

> +For example, if the problem occurred in 2003, start at mid-year, then is
> +the problem is already here, back to 1st April, if not, to 1st October,
> +and so on.

Bad grouping in that sentence. Try

==
then ask "Is the problem already here?". If yes, go back to the first of
April; if not, go to the first of October, and so on.
==

> +If you have lot of hard disk free space (a full compile currently takes
> +100 Mb), copy the oldest known working version before updating it, it will
> +save time if you need to go back.

"it; this will save time".

Personally I keep two separate Wine trees, one for "tracking current
CVS" and one for peforming regression tests; it's just easier that way.
Not that I've had occasion to do many Wine regression tests so far, it's
just that I created one for the first regression test and haven't
bothered to delete it.

It would be a good idea to make sure that the "100 Mb" number is correct
for MPlayer before committing this - actually I think it's incorrect; a
quick 'du -hs' on my primary, fully-compiled copy of main/ gives me
309MB. While we're at it, are we using the "Mb"/"MB" distinction or not?

> +(It's better to make distclean before going back in time, so you have to
> +make everything if you don't backup the older version).

"to run 'make distclean'"

"back up" (because "backup" is a noun)

> +When you have found the day where the problem happened, continue the search
> +using the mplayer-cvslog archive (sorted by date) and a more precise cvs
> +update including hour, minute, second:

"and second" - the preceding comma is optional but not incorrect.

> +<screen>
> +cvs update -PAd -D "2004-08-23 15:17:25 CDT"
> +</screen>
> +This will allow you to find easily the exact patch that did it.

"to easily find", if not a more extensive rephrasing.

Since I've snipped the other place where this appears: do we really want
to specify the examples in Central Daylight Time? I believe that the
reason Wine does so is because that's where their CVS server is located,
and so that's the time zone for which their CVS log will be
synchronized; since our server isn't located in the same time zone, we
may want to use a different TLA. (Using none is not advisable, for
reasons I'm not entirely clear on.)

> +If you find the patch that is the cause of the problem, you have almost won;
> +report about it to
> +<ulink url="http://bugzilla.mplayerhq.hu/">MPlayer Bugzilla</ulink> or

"the MPlayer Bugzilla" (the definite article doesn't necessarily have to
be part of the link name)

> +subscribe to 
> +<ulink url="http://mplayerhq.hu/mailman/listinfo/mplayer-users">MPlayer-users</ulink>
> +and post it there.
> +There is a chance that the author will jump in to suggest a fix; or there
> +is always the possibility to look hard at the patch until it is coerced to
> +reveal where is the bug :-).

There is something grammatically wrong with this last sentence (after
the semicolon, since it's really a separate sentence there despite the
ungrammatical "or"), but I'm not quite sure how to fix it without
introducing other problems.

(Hmm. Apparently I'm in a productive mood today.)

-- 
       The Wanderer

Warning: Simply because I argue an issue does not mean I agree with any
side of it.

Secrecy is the beginning of tyranny.