[Pachi] Regression test suite

Petr Baudis pasky at ucw.cz
Mon Nov 7 19:11:19 CET 2011


On Thu, Jul 28, 2011 at 11:28:35PM +0200, Petr Baudis wrote:
> On Thu, Jul 28, 2011 at 11:22:46PM +0200, Jean-loup Gailly wrote:
> > * http://files.gokgs.com/games/2011/7/28/pachi2-Novicer.sgf
> >   Pachi makes many stupid moves because it thinks the top left
> >   corner is unsettled although it is clearly alive. This shape
> >   (4 in a row) is so simple that it should get it right.
> >   W 252 S11 bad
> >   W 256 T12 bad
> >   W 264 E19 doesn't make sense, but still 69%
> >   W 266 H19 doesn't make sense, but still 61%
> >   W 268 T9 should be T8
> >   W 292 A17 50%, W 294 R6 37% shows that top left was not
> >     evaluated correctly.
> > 
> > Jean-loup
> > 
> > PS: I didn't get any feedback on all the reports in this thread but I'm
> > still vaguely
> >   hoping that one day they may be useful... Fixing these bugs should gain at
> > least one stone.
> I still do hope to be able to convert them to a semi-automated
> regression suite and testbed for new features. But it may take
> another month or two. :-(

  The infrastructure is now +- in place in the t-regress/ subdirectory.
The test script itself is very proof of concept for now, it needs to be
tweaked for batch testing - ideally, one would provide a simple testspec
that says "check 5 times with -t =5000, check 2 times with -t =20000",
the result would be checked automatically and the standard output would
contain only the results, with all the normal output written to some
temporary file sideways...

  Also, just a single testcase has been imported so far. Importing
others based on t-regress/TODO should not be too hard (at least in most
of the cases).

  Any help is welcome with both tasks. :-)

  Happy hacking,

				Petr "Pasky" Baudis
	The goal of Computer Science is to build something that will
	last at least until we've finished building it.

More information about the Pachi mailing list