[Pachi] Regression test suite

Matthew Woodcraft matthew at woodcraft.me.uk
Sun Jun 19 14:36:23 CEST 2011

Petr Baudis wrote:
> Anyone willing to figure out the details? Or at least write a brief
> summary of the approach taken by others (probably mainly GNUGo and
> Fuego)?

I've had a look at this.


Gnugo keeps the test descriptions separate from the SGF files containing
the test positions.

Each test description specifies the position (by SGF filename and move
number), a move-generation command, and expected output.

The test description files are text files with entries like the

# E3 and H4 are not blunders, however, H2 is.
loadsgf games/nngs/gnugo-3.4-viking4-200308191053.sgf 283
32 reg_genmove white
#? [E3|H4]

These directly specify GTP input, using the `loadsgf` GTP command to
describe the position.

The '#?' line is the expected output; the bit between [] is a regular

A final * means the test is expected to fail:

# White D4 kills at least six stones.
loadsgf games/blunder10.sgf 2
12 reg_genmove white
#? [D4]*

There's a syntax to mean "accept only non-matches":

# Both A3 and A5 are blunders due to a spectacular atari-atari sequence.
loadsgf games/blunder7.sgf
8 reg_genmove black
#? [!(A3|A5)]

Tests sometimes use a GTP extension command `gg_genmove`, which is like
`reg_genmove` except that you can optionally provide a seed for the
random number generator. There are other special GTP commands like
`owl_defend` used for testing particular subsystems.

Documentation is at

The test runner is at


Fuego uses the gogui-regress command to run its test suite. This uses
the same .tst file format as gnugo.


They also use special-purpose GTP extension commands to test particular


More information about the Pachi mailing list