[Pachi] Regression test suite
pasky at ucw.cz
Mon Nov 7 19:11:19 CET 2011
On Thu, Jul 28, 2011 at 11:28:35PM +0200, Petr Baudis wrote:
> On Thu, Jul 28, 2011 at 11:22:46PM +0200, Jean-loup Gailly wrote:
> > * http://files.gokgs.com/games/2011/7/28/pachi2-Novicer.sgf
> > Pachi makes many stupid moves because it thinks the top left
> > corner is unsettled although it is clearly alive. This shape
> > (4 in a row) is so simple that it should get it right.
> > W 252 S11 bad
> > W 256 T12 bad
> > W 264 E19 doesn't make sense, but still 69%
> > W 266 H19 doesn't make sense, but still 61%
> > W 268 T9 should be T8
> > W 292 A17 50%, W 294 R6 37% shows that top left was not
> > evaluated correctly.
> > Jean-loup
> > PS: I didn't get any feedback on all the reports in this thread but I'm
> > still vaguely
> > hoping that one day they may be useful... Fixing these bugs should gain at
> > least one stone.
> I still do hope to be able to convert them to a semi-automated
> regression suite and testbed for new features. But it may take
> another month or two. :-(
The infrastructure is now +- in place in the t-regress/ subdirectory.
The test script itself is very proof of concept for now, it needs to be
tweaked for batch testing - ideally, one would provide a simple testspec
that says "check 5 times with -t =5000, check 2 times with -t =20000",
the result would be checked automatically and the standard output would
contain only the results, with all the normal output written to some
temporary file sideways...
Also, just a single testcase has been imported so far. Importing
others based on t-regress/TODO should not be too hard (at least in most
of the cases).
Any help is welcome with both tasks. :-)
Petr "Pasky" Baudis
The goal of Computer Science is to build something that will
last at least until we've finished building it.
More information about the Pachi