[Pachi] Regression test suite

Matthew Woodcraft matthew at woodcraft.me.uk
Sun Jun 19 14:36:23 CEST 2011


Petr Baudis wrote:
> Anyone willing to figure out the details? Or at least write a brief
> summary of the approach taken by others (probably mainly GNUGo and
> Fuego)?

I've had a look at this.


Gnugo
-----

Gnugo keeps the test descriptions separate from the SGF files containing
the test positions.

Each test description specifies the position (by SGF filename and move
number), a move-generation command, and expected output.


The test description files are text files with entries like the
following:

# E3 and H4 are not blunders, however, H2 is.
loadsgf games/nngs/gnugo-3.4-viking4-200308191053.sgf 283
32 reg_genmove white
#? [E3|H4]

These directly specify GTP input, using the `loadsgf` GTP command to
describe the position.

The '#?' line is the expected output; the bit between [] is a regular
expression.


A final * means the test is expected to fail:

# White D4 kills at least six stones.
loadsgf games/blunder10.sgf 2
12 reg_genmove white
#? [D4]*


There's a syntax to mean "accept only non-matches":

# Both A3 and A5 are blunders due to a spectacular atari-atari sequence.
loadsgf games/blunder7.sgf
8 reg_genmove black
#? [!(A3|A5)]


Tests sometimes use a GTP extension command `gg_genmove`, which is like
`reg_genmove` except that you can optionally provide a seed for the
random number generator. There are other special GTP commands like
`owl_defend` used for testing particular subsystems.


Documentation is at
http://www.gnu.org/software/gnugo/gnugo_20.html

The test runner is at
http://git.savannah.gnu.org/cgit/gnugo.git/tree/regression/regress.pike


Fuego
-----

Fuego uses the gogui-regress command to run its test suite. This uses
the same .tst file format as gnugo.

Documentation:
http://gogui.sourceforge.net/doc/reference-regress.html

They also use special-purpose GTP extension commands to test particular
subsystems.


-M-


More information about the Pachi mailing list