[Pachi] Regression test suite

Jean-loup Gailly jloup at gailly.net
Wed Jun 8 00:45:13 CEST 2011


Here is my current list of good candidates to put in a regression suite.

* http://files.gokgs.com/games/2011/6/7/fidibus-pachi30s.sgf
   move 133 P1 forces opponent to live, should be R1 to kill
   move 135 S1 plays the ko threat before the ko at L12
     (this occurs frequently)

* http://files.gokgs.com/games/2011/6/6/tyzef-pachi30s.sgf
  pachi tries to run away from a losing ladder against a 7k in even
  game. The 7k, disgusted, passed then resigned.

* http://files.gokgs.com/games/2011/6/5/pachi2-SimpleBot.sgf
  pachi shouldn't play the broken ladders up to move 53

* http://files.gokgs.com/games/2011/6/5/Zen19-pachi2.sgf should have won
 . move 185 S15 very dubious

 . move 211 T18 pachi still thinks the semeai is undecided.
   but white has a lot more liberties. How can the playouts
   not see this?

 . move 227 S3: here are suggestions by Aja
    If a false will become a real eye by capturing a diagonal group,
    then capture the group instead of filling the false eye. So if
    pachi wants to fill s3 it will capture at T1 instead. Then S3
    will become an illegal move so pachi will evaluate this group
    100% alive and won't play at T2 such stupid move.

* http://files.gokgs.com/games/2011/6/5/tazaki-pachi30s.sgf
  pachi doesn't see even one move ahead in a semeai, it expects
  an answer outside the semeai. The score drops from 0.625 to 0.337:

  move 194:
    [880000] best 0.624794 komi 0.4 | seq L19 G12 F11 P11 | can L19(0.625)
O19(0.623) N19(0.469) P11(0.523)
    FINEST: Got successful response to command "genmove b": = L19
    FINEST: Got successful response to command "play w j16": =
  move 196:
    FINEST: Queued command sent to engine: genmove b
    [1026644] best 0.336825 komi 0.2 | seq P11  T8  R3  M9 | can P11(0.337)
O11(0.336) O19(0.334) N19(0.340)
    FINEST: Got successful response to command "genmove b": = P11

  The game was lost already but pachi should have known this before.

  move 244: T13 doesn't make sense. This is due to wrong estimate of
  white's right group:
    [1922010] best 0.492722 | seq T13  T4  D5  M6 | can T13(0.493)
S18(0.493)  L4(0.493)  E6(0.491)
    FINEST: Got successful response to command "genmove b": = T13

  At move 282 pachi still thinks white's group is undecided.
  We must find a way of not filling the S7 eye in the playouts.

* http://files.gokgs.com/games/2011/5/29/snowmen-pachi2.sgf
    169 B5 seems to be the losing move. This move doesn't make sense.
  Pachi has been wrongly evaluating both bottom group around S4 (it's just
  hairy for simulations) and top group around A19 - that one is a bit more
  interesting, the problem is that black is double likely to play E19
  (since it's C19 selfatari cousin), but after E19, white will not notice
  the need to connect at C19. Not sure how to solve this yet, perhaps some
  way to add cousins to last_move logic...

* http://files.gokgs.com/games/2011/5/28/pachi2-Rubel.sgf
    won game lost: 212 H13 should be at J12.

* http://files.gokgs.com/games/2011/5/27/ManyFaces1-pachi2.sgf
   had won game, move 135 F1 very bad. at end doesn't see ko at t7. (b t10
kills above)

* http://files.gokgs.com/games/2011/5/23/pachi2-ManyFaces1.sgf
  leading up to 90% then lost.
  bad evaluation of 3 areas according to pasky.

* http://files.gokgs.com/games/2011/5/25/pachi2-Zen19S.sgf
  lost but was leading according to ajahuang and Zen itself

* http://files.gokgs.com/games/2011/2/20/ramesses2-pachi2.sgf
  good example of losing a won game.

* http://files.gokgs.com/games/2011/1/11/llopl-pachi2.sgf
  terrible ladder bug

* http://files.gokgs.com/games/2010/12/5/pachi-CzechBot.sgf
  big mistake at F9 didn't see snapback. Pasky comments:
    The "does not seen snapback" bug is nasty.  The problem is that
    there is also a "global capture heuristic" if there're no fun
    local moves, it looks for groups to capture and if white tenukis
    and does not capture immediately, this heuristic kicks in.  But
    there is no "global atari heuristic" that would drive black
    similarly to play at F1 and create the snapback if I test with
    playout=moggy:capturerate=0, it reliably destroys the snapback
    i.e. with capturerate=90, white is much more likely to come back
    to the situation later even when playing tenuki first.

* http://files.gokgs.com/games/2011/1/15/pachi2-rollingon.sgf
  big semeai with easy counting of liberties misjuged
  move 169 0.7322 -> move 171  0.2600

* http://files.gokgs.com/games/2011/2/18/pachi2-tf-3.sgf
  lost because of unrecognized seki

Jean-loup
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rover.ms.mff.cuni.cz/pipermail/pachi/attachments/20110608/0f2be24f/attachment.html>


More information about the Pachi mailing list