Add GUT test framework and seed determinism tests #15

Merged

liamjd merged 3 commits from 12-add-gut-test-framework into main

2026-08-16 09:07:08 +01:00

liamjd commented

2026-08-16 08:56:23 +01:00

Owner

Adds GUT as the test framework and the first permanent assertions, replacing the throwaway SceneTree harnesses that were written and deleted for each change. Part of #12.

What landed

GUT 9.7.1 vendored in addons/gut/, plugin enabled in project.godot. Committed on its own so the rest of the diff stays readable.
.gutconfig.json holds the runner options, so the suite command is just -s addons/gut/gut_cmdln.gd.
test/unit/test_smoke.gd — proves the runner itself works. Both of the unknowns flagged in #12 fail as parse errors rather than test failures if they are wrong, which is miserable to diagnose from inside a real test, so they are isolated here.
test/unit/test_terrain_seed.gd — seed determinism for TerrainGrid.
CLAUDE.md — replaces the "no test framework exists, do not fabricate a command" note with a ## Tests section.

Verification

Scripts 2 | Tests 10 | Passing 10 | Asserts 13 | 0.94s

Run headless via Godot_..._console.exe --path . --headless -s addons/gut/gut_cmdln.gd. No orphans reported. -gselect=test_terrain_seed confirmed to work layered over the config file.

Answers to the open questions in #12

Autoloads resolve by bare name under GUT's runner. The root.get_node("GameState") workaround is only needed in raw -s harnesses; GUT boots a full SceneTree, which is the difference. Nothing needed in a test README.
class_name globals resolve, given an editor scan has been run. That scan remains a prerequisite after adding any new class_name — recorded in CLAUDE.md.
AssetLib ships 9.6.0, which declares godot_max 4.6.999. 9.7.x is the 4.7 line, so GUT must be installed from a release, not AssetLib. Also recorded.

Test design notes

Assertions go through the public level_at() rather than reaching into _levels, so the private array and the indexing arithmetic on top of it are both covered without widening the API.

Three of the seven terrain tests guard failure modes that a plain same-seed/different-seed pair would miss:

test_generated_terrain_has_relief — a flat plate is perfectly deterministic and would pass an equality check. Without this, "generation silently collapsed" reads as a pass.
test_regeneration_is_stable_without_a_seed_change — catches state leaking between generate() calls.
test_reroll_changes_both_seed_and_terrain — a reroll that moves the seed but not the map, or vice versa, both present as "the R key doesn't do much".

Grids are generated at 24x24 rather than the 64 default; determinism does not depend on grid size and every test generates at least twice.

Findings

The island preset nearly drowns at small grid sizes. Visible in the test output:

island, 24x24:  14 walkable (5 on main landmass)
plains, 24x24: 127 walkable (96 on main landmass)

_radial() normalises by the grid centre, so the shape is scale-independent, but falloff_start combined with the water threshold is not — shrink the grid and the island submerges. At 64 it looks fine, which is why this had not shown up. Not addressed here; it may be intended, since small grids are not a supported configuration. Filed as a note rather than a fix.

Caveats

.gutconfig.json was written by the GUT editor panel, so it serialises editor-only cosmetics (font, colours, opacity, gut_on_top, compact_mode) alongside the runner options. The panel rewrites the whole file whenever its settings are touched, so expect diff churn unrelated to test configuration.
Rendering is not under test. gl_compatibility draws nothing headless, so a green suite says nothing about how anything looks. Visual verification stays manual, headed and screenshot-based.

Not done

#12 also lists seed resolution in main_menu.gd (empty → random, valid int → literal, otherwise → hash(), with both entry paths agreeing) and the coordinate invariants carried from the POC. Neither is covered here, so this PR does not close the issue.

Adds GUT as the test framework and the first permanent assertions, replacing the throwaway `SceneTree` harnesses that were written and deleted for each change. Part of #12. ## What landed - **GUT 9.7.1** vendored in `addons/gut/`, plugin enabled in `project.godot`. Committed on its own so the rest of the diff stays readable. - **`.gutconfig.json`** holds the runner options, so the suite command is just `-s addons/gut/gut_cmdln.gd`. - **`test/unit/test_smoke.gd`** — proves the runner itself works. Both of the unknowns flagged in #12 fail as *parse* errors rather than test failures if they are wrong, which is miserable to diagnose from inside a real test, so they are isolated here. - **`test/unit/test_terrain_seed.gd`** — seed determinism for `TerrainGrid`. - **CLAUDE.md** — replaces the "no test framework exists, do not fabricate a command" note with a `## Tests` section. ## Verification ``` Scripts 2 | Tests 10 | Passing 10 | Asserts 13 | 0.94s ``` Run headless via `Godot_..._console.exe --path . --headless -s addons/gut/gut_cmdln.gd`. No orphans reported. `-gselect=test_terrain_seed` confirmed to work layered over the config file. ## Answers to the open questions in #12 - **Autoloads resolve by bare name** under GUT's runner. The `root.get_node("GameState")` workaround is only needed in raw `-s` harnesses; GUT boots a full `SceneTree`, which is the difference. Nothing needed in a test README. - **`class_name` globals resolve**, given an editor scan has been run. That scan remains a prerequisite after adding any new `class_name` — recorded in CLAUDE.md. - **AssetLib ships 9.6.0**, which declares `godot_max 4.6.999`. 9.7.x is the 4.7 line, so GUT must be installed from a release, not AssetLib. Also recorded. ## Test design notes Assertions go through the public `level_at()` rather than reaching into `_levels`, so the private array and the indexing arithmetic on top of it are both covered without widening the API. Three of the seven terrain tests guard failure modes that a plain same-seed/different-seed pair would miss: - `test_generated_terrain_has_relief` — a flat plate is perfectly deterministic and would pass an equality check. Without this, "generation silently collapsed" reads as a pass. - `test_regeneration_is_stable_without_a_seed_change` — catches state leaking between `generate()` calls. - `test_reroll_changes_both_seed_and_terrain` — a reroll that moves the seed but not the map, or vice versa, both present as "the R key doesn't do much". Grids are generated at 24x24 rather than the 64 default; determinism does not depend on grid size and every test generates at least twice. ## Findings **The island preset nearly drowns at small grid sizes.** Visible in the test output: ``` island, 24x24: 14 walkable (5 on main landmass) plains, 24x24: 127 walkable (96 on main landmass) ``` `_radial()` normalises by the grid centre, so the *shape* is scale-independent, but `falloff_start` combined with the water threshold is not — shrink the grid and the island submerges. At 64 it looks fine, which is why this had not shown up. Not addressed here; it may be intended, since small grids are not a supported configuration. Filed as a note rather than a fix. ## Caveats - **`.gutconfig.json` was written by the GUT editor panel**, so it serialises editor-only cosmetics (font, colours, opacity, `gut_on_top`, `compact_mode`) alongside the runner options. The panel rewrites the whole file whenever its settings are touched, so expect diff churn unrelated to test configuration. - **Rendering is not under test.** `gl_compatibility` draws nothing headless, so a green suite says nothing about how anything looks. Visual verification stays manual, headed and screenshot-based. ## Not done #12 also lists seed *resolution* in `main_menu.gd` (empty → random, valid int → literal, otherwise → `hash()`, with both entry paths agreeing) and the coordinate invariants carried from the POC. Neither is covered here, so this PR does not close the issue.

liamjd added 3 commits

2026-08-16 08:56:23 +01:00