[MPlayer-DOCS] [PATCH] How to do regression testing using CVS

[Guillaume POIRIER]

Hi,

On 11/26/05, The Wanderer <inverseparadox at comcast.net> wrote:
> On 11/26/2005 04:08 PM, Guillaume POIRIER wrote:
[..]
> > +If any non-programmer reads this, the fastest method to get at the point
> > +where the problem occurred is to use a binary search, that is, search the
> > +date of the breackage by repeatedly dividing the search interval in half.
>
> I think it would be better to use an emdash here, after "search", rather
> than a comma.

done.


> > +For example, if the problem occurred in 2003, start at mid-year, then is
> > +the problem is already here, back to 1st April, if not, to 1st October,
> > +and so on.
>
> Bad grouping in that sentence. Try
>
> ==
> then ask "Is the problem already here?". If yes, go back to the first of
> April; if not, go to the first of October, and so on.
> ==

fixed your way.


> > +If you have lot of hard disk free space (a full compile currently takes
> > +100 Mb), copy the oldest known working version before updating it, it will
> > +save time if you need to go back.
>
> "it; this will save time".
>
> Personally I keep two separate Wine trees, one for "tracking current
> CVS" and one for peforming regression tests; it's just easier that way.
> Not that I've had occasion to do many Wine regression tests so far, it's
> just that I created one for the first regression test and haven't
> bothered to delete it.

I did not want to get into further details in advising do do the
binary search in a separate tree. I figure the user either never used
cvs and has no tree or has already a tree and will think about
backuping it if needed. If the doc is too verbose, ppl tend to skim
trough it.


> It would be a good idea to make sure that the "100 Mb" number is correct
> for MPlayer before committing this - actually I think it's incorrect; a
> quick 'du -hs' on my primary, fully-compiled copy of main/ gives me
> 309MB. While we're at it, are we using the "Mb"/"MB" distinction or not?

My tree is 103Mb but it's compiled without debug symbols, and without
all supported features of MPlayer. I guess this figure varies from ppl
to ppl. It's probably safer to say 300MB (which is the standard figure
for a MPlayer source tree compiled with debug symbols).


> > +(It's better to make distclean before going back in time, so you have to
> > +make everything if you don't backup the older version).
>
> "to run 'make distclean'"
>
> "back up" (because "backup" is a noun)

done.


> > +When you have found the day where the problem happened, continue the search
> > +using the mplayer-cvslog archive (sorted by date) and a more precise cvs
> > +update including hour, minute, second:
>
> "and second" - the preceding comma is optional but not incorrect.

done.


> > +<screen>
> > +cvs update -PAd -D "2004-08-23 15:17:25 CDT"
> > +</screen>
> > +This will allow you to find easily the exact patch that did it.
>
> "to easily find", if not a more extensive rephrasing.
>
> Since I've snipped the other place where this appears: do we really want
> to specify the examples in Central Daylight Time? I believe that the
> reason Wine does so is because that's where their CVS server is located,
> and so that's the time zone for which their CVS log will be
> synchronized; since our server isn't located in the same time zone, we
> may want to use a different TLA. (Using none is not advisable, for
> reasons I'm not entirely clear on.)

I've checked: the zone indicated is "-00" so it either means GMT, or
"local". In any case, I don't think it matters that much, so I removed
it


> > +If you find the patch that is the cause of the problem, you have almost won;
> > +report about it to
> > +<ulink url="http://bugzilla.mplayerhq.hu/">MPlayer Bugzilla</ulink> or
>
> "the MPlayer Bugzilla" (the definite article doesn't necessarily have to
> be part of the link name)
>
> > +subscribe to
> > +<ulink url="http://mplayerhq.hu/mailman/listinfo/mplayer-users">MPlayer-users</ulink>
> > +and post it there.
> > +There is a chance that the author will jump in to suggest a fix; or there
> > +is always the possibility to look hard at the patch until it is coerced to
> > +reveal where is the bug :-).
>
> There is something grammatically wrong with this last sentence (after
> the semicolon, since it's really a separate sentence there despite the
> ungrammatical "or"), but I'm not quite sure how to fix it without
> introducing other problems.

I came up with smth different. Please tell me if it's okay.

Guillaume
--
Remember, if you ever need a helping hand, it's
at the end of your arm, as you get older, remember
you have another hand:
The first is to help yourself,
the second is to help others.
-- Audrey Hepburn
-------------- next part --------------
Index: en/bugreports.xml
===================================================================
RCS file: /cvsroot/mplayer/main/DOCS/xml/en/bugreports.xml,v
retrieving revision 1.11
diff -u -r1.11 bugreports.xml
--- en/bugreports.xml	31 Mar 2005 22:28:24 -0000	1.11
+++ en/bugreports.xml	26 Nov 2005 23:17:36 -0000
@@ -22,6 +22,77 @@
 mailing list will assist you if you have questions.
 </para>
 </sect1>
+<sect1 id="bugreports_regression_test">
+<title>How to do regression testing using CVS</title>
+<para>
+A problem that can happen sometimes is 'it used to work before, now it
+doesn't anymore...'.
+Here is a step by step procedure to try to pinpoint when the problem
+occurred. This is <emphasis role="bold">not</emphasis> for casual users.
+</para>
+<para>
+First, you'd need to fetch MPlayer's source tree from CVS.
+Instructions can be found at the bottom of
+<ulink url="http://www.mplayerhq.hu/homepage/dload.html">this page</ulink>.
+</para>
+<para>
+You will have now in the main/ directory an image of the CVS tree, on the
+client side.
+Now update this image to the date you want:
+<screen>
+cd main/
+cvs update -PAd -D "2004-08-23"
+</screen>
+The date format is YYYY-MM-DD HH:MM:SS.
+Using the CDT date format ensure that you will be able to extract patches
+in a way that will be compatible with the
+<ulink url="http://mplayerhq.hu/pipermail/mplayer-cvslog/">MPlayer-cvslog archive</ulink>.
+</para>
+<para>
+Now proceed as for a normal update:
+<screen>
+./configure
+make
+</screen>
+</para>
+<para>
+If any non-programmer reads this, the fastest method to get at the point
+where the problem occurred is to use a binary search, that is &ndash;
+search the date of the breackage by repeatedly dividing the search
+interval in half.
+For example, if the problem occurred in 2003, start at mid-year, then ask
+"Is the problem already here?".
+If yes, go back to the first of April; if not, go to the first of October,
+and so on.
+</para>
+<para>
+If you have lot of hard disk free space (a full compile currently takes
+100 Mb), copy the oldest known working version before updating it; it will
+save time if you need to go back.
+(It's better to run 'make distclean' before going back in time, so you
+have to make everything if you don't back up the older version).
+</para>
+<para>
+When you have found the day where the problem happened, continue the search
+using the mplayer-cvslog archive (sorted by date) and a more precise cvs
+update including hour, minute and second:
+<screen>
+cvs update -PAd -D "2004-08-23 15:17:25"
+</screen>
+This will allow you to easily find the exact patch that did it.
+</para>
+<para>
+If you find the patch that is the cause of the problem, you have almost won;
+report about it to the
+<ulink url="http://bugzilla.mplayerhq.hu/">MPlayer Bugzilla</ulink> or
+subscribe to 
+<ulink url="http://mplayerhq.hu/mailman/listinfo/mplayer-users">MPlayer-users</ulink>
+and post it there.
+There is a chance that the author will jump in to suggest a fix.
+You may also look hard at the patch until it is coerced to reveal where
+the bug is :-).
+</para>
+</sect1>
 <sect1 id="bugreports_report">
 <title>How to report bugs</title>
 <para>