[Pachi] Regression test suite

Petr Baudis pasky at ucw.cz
Mon Nov 7 19:26:26 CET 2011


On Sun, Jun 19, 2011 at 01:36:23PM +0100, Matthew Woodcraft wrote:
> Petr Baudis wrote:
> > Anyone willing to figure out the details? Or at least write a brief
> > summary of the approach taken by others (probably mainly GNUGo and
> > Fuego)?
> I've had a look at this.

  Thanks a lot for the analysis!

> Gnugo
> -----
> Gnugo keeps the test descriptions separate from the SGF files containing
> the test positions.
> Each test description specifies the position (by SGF filename and move
> number), a move-generation command, and expected output.

  I'm not really fond of this way, however. First, it violates Pachi's
philosophy of a UNIXy tool that tries to avoid things like implementing
a SGF parser etc. - in this regard, keeping the test description within
the data (SGF) seems much more convenient and also more maintainable to
me. Second, just matching the output is quick'n'dirty, but I miss a lot
of flexibility there.

> Fuego
> -----
> Fuego uses the gogui-regress command to run its test suite. This uses
> the same .tst file format as gnugo.
> Documentation:
> http://gogui.sourceforge.net/doc/reference-regress.html

  However, I'm wondering that maybe the quickest way to get a regression
suite verifier doing would be (i) a wrapper converting SGF to .tst; and
(ii) a wrapper substituting loadsgf with sgf2gtp output; (iii) using
gogui-regress. ;-) I know, I'm a duct-tape man.

> They also use special-purpose GTP extension commands to test particular
> subsystems.

  BTW, we have an infrastructure for this too in t-unit/. But from my
POV, this seems like quite a different task than game-based regression

				Petr "Pasky" Baudis
	The goal of Computer Science is to build something that will
	last at least until we've finished building it.

More information about the Pachi mailing list