Testing¶
Perft Testing¶
Perft (performance test) validates move generation by counting all leaf nodes at a given depth:
position startpos
go perft 6
Depth |
Nodes |
|---|---|
1 |
20 |
2 |
400 |
3 |
8,902 |
4 |
197,281 |
5 |
4,865,609 |
6 |
119,060,324 |
Perft results from starting position
Test Positions¶
The engine includes several built-in test positions:
#define startPosition "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"
#define tricky_position "r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1"
#define killer_position "rnbqkb1r/pp1p1pPp/8/2p1pP2/1P1P4/3P3P/P1P1P3/RNBQKBNR w KQkq e6 0 1"
#define cmk_position "r2q1rk1/ppp2ppp/2n1bn2/2b1p3/3pP3/3P1NPP/PPP1NPB1/R1BQ1RK1 b - - 0 9"
The automated suite¶
Perft and the fixed positions above are the engine’s correctness core, but they are a small part of what runs. The project’s standing rule is that every test must run without hardware, which is what makes the whole suite executable on a laptop and in CI:
ctest --test-dir build # full suite
ctest --test-dir build -R EngineTest # a single suite by name
Suites are split by the layer they exercise — engine validity and skill level, board interpreter, game controller, scanner, session orchestration, personality, handicap, LED framebuffer and assist policy, game recording and analysis, setup planning, desync recovery, networked play, PGN reading, broadcast, puzzles and telemetry, plus the GUI bridge suites in kirin-gui/tests/. The current pass counts are regenerated into kirin/tests/TEST_RESULTS.md:
cmake --build build --target test_report
Note
TEST_RESULTS.md is committed and should be regenerated in the same change that alters tests. CI regenerates it and publishes it to the run summary and as an artifact, but deliberately does not commit it — keeping the checked-in copy fresh is a local responsibility.
Hardware-dependent behaviour is covered by simulation backends rather than skipped: the scanner has a simulation mode driving synthetic occupancy transitions, the LED stack runs on a SimLedDriver that captures frames in memory, and the board backend selects a SimBackend when no gantry serial port is present. What genuinely cannot be tested this way — that a real dropped piece re-seats on retry, that a physical setup lands where intended — is called out as hardware-gated in the relevant chapters rather than pretended away.
Strength is a separate question¶
Nothing in this chapter measures how well the engine plays. Move generation can be perfect while the search is weak. Playing-strength validation has its own methodology, its own harness, and a hard rule that it precedes any strength-affecting merge — Ch. Measuring Strength.