[Pachi] Regression test suite
pasky at ucw.cz
Mon Nov 7 19:26:26 CET 2011
On Sun, Jun 19, 2011 at 01:36:23PM +0100, Matthew Woodcraft wrote:
> Petr Baudis wrote:
> > Anyone willing to figure out the details? Or at least write a brief
> > summary of the approach taken by others (probably mainly GNUGo and
> > Fuego)?
> I've had a look at this.
Thanks a lot for the analysis!
> Gnugo keeps the test descriptions separate from the SGF files containing
> the test positions.
> Each test description specifies the position (by SGF filename and move
> number), a move-generation command, and expected output.
I'm not really fond of this way, however. First, it violates Pachi's
philosophy of a UNIXy tool that tries to avoid things like implementing
a SGF parser etc. - in this regard, keeping the test description within
the data (SGF) seems much more convenient and also more maintainable to
me. Second, just matching the output is quick'n'dirty, but I miss a lot
of flexibility there.
> Fuego uses the gogui-regress command to run its test suite. This uses
> the same .tst file format as gnugo.
However, I'm wondering that maybe the quickest way to get a regression
suite verifier doing would be (i) a wrapper converting SGF to .tst; and
(ii) a wrapper substituting loadsgf with sgf2gtp output; (iii) using
gogui-regress. ;-) I know, I'm a duct-tape man.
> They also use special-purpose GTP extension commands to test particular
BTW, we have an infrastructure for this too in t-unit/. But from my
POV, this seems like quite a different task than game-based regression
Petr "Pasky" Baudis
The goal of Computer Science is to build something that will
last at least until we've finished building it.
More information about the Pachi