A.3. How to do regression testing using Subversion

A problem that can happen sometimes is 'it used to work before, now it doesn't anymore...'. Here is a step by step procedure to try to pinpoint when the problem occurred. This is not for casual users.

First, you'd need to fetch MPlayer's source tree from Subversion. Instructions can be found in the Subversion section of the download page.

You will have now in the mplayer/ directory an image of the Subversion tree, on the client side. Now update this image to the date you want:

cd mplayer/
svn update -r {"2004-08-23"}

The date format is YYYY-MM-DD HH:MM:SS. Using this date format ensure that you will be able to extract patches according to the date at which they were committed, as in the MPlayer-cvslog archive.

Now proceed as for a normal update:


If any non-programmer reads this, the fastest method to get at the point where the problem occurred is to use a binary search — that is, search the date of the breakage by repeatedly dividing the search interval in half. For example, if the problem occurred in 2003, start at mid-year, then ask "Is the problem already here?". If yes, go back to the first of April; if not, go to the first of October, and so on.

If you have lot of free hard disk space (a full compile currently takes 100 MB, and around 300-350 MB if debugging symbols are enabled), copy the oldest known working version before updating it; this will save time if you need to go back. (It is usually necessary to run 'make distclean' before recompiling an earlier version, so if you do not make a backup copy of your original source tree, you will have to recompile everything in it when you come back to the present.) Alternatively you may use ccache to speed up compilation.

When you have found the day where the problem happened, continue the search using the mplayer-cvslog archive (sorted by date) and a more precise svn update including hour, minute and second:

svn update -r {"2004-08-23 15:17:25"}

This will allow you to easily find the exact patch that did it.

If you find the patch that is the cause of the problem, you have almost won; report about it on MPlayer's issue tracker or subscribe to MPlayer-users and post it there. There is a chance that the author will jump in to suggest a fix. You may also look hard at the patch until it is coerced to reveal where the bug is :-).