[Pachi] Regression test suite

Petr Baudis pasky at ucw.cz
Mon Nov 7 19:26:26 CET 2011


  Hi!

On Sun, Jun 19, 2011 at 01:36:23PM +0100, Matthew Woodcraft wrote:
> Petr Baudis wrote:
> > Anyone willing to figure out the details? Or at least write a brief
> > summary of the approach taken by others (probably mainly GNUGo and
> > Fuego)?
> 
> I've had a look at this.

  Thanks a lot for the analysis!

> Gnugo
> -----
> 
> Gnugo keeps the test descriptions separate from the SGF files containing
> the test positions.
> Each test description specifies the position (by SGF filename and move
> number), a move-generation command, and expected output.

  I'm not really fond of this way, however. First, it violates Pachi's
philosophy of a UNIXy tool that tries to avoid things like implementing
a SGF parser etc. - in this regard, keeping the test description within
the data (SGF) seems much more convenient and also more maintainable to
me. Second, just matching the output is quick'n'dirty, but I miss a lot
of flexibility there.

> Fuego
> -----
> 
> Fuego uses the gogui-regress command to run its test suite. This uses
> the same .tst file format as gnugo.
> 
> Documentation:
> http://gogui.sourceforge.net/doc/reference-regress.html

  However, I'm wondering that maybe the quickest way to get a regression
suite verifier doing would be (i) a wrapper converting SGF to .tst; and
(ii) a wrapper substituting loadsgf with sgf2gtp output; (iii) using
gogui-regress. ;-) I know, I'm a duct-tape man.

> They also use special-purpose GTP extension commands to test particular
> subsystems.

  BTW, we have an infrastructure for this too in t-unit/. But from my
POV, this seems like quite a different task than game-based regression
testing.

-- 
				Petr "Pasky" Baudis
	The goal of Computer Science is to build something that will
	last at least until we've finished building it.


More information about the Pachi mailing list