Add a test framework (GUT) and cover seed determinism and coordinate invariants #12

New issue

Closed

opened 2026-08-15 15:58:11 +01:00 by liamjd · 1 comment

liamjd commented

2026-08-15 15:58:11 +01:00

Owner

There is no test framework. Verification so far has been throwaway SceneTree harnesses written, run, and deleted for each change. That worked for one-off checks, but several properties are cheap to break and silent when broken, and deserve permanent assertions.

Framework

GUT (Godot Unit Test) is the recommended choice — it is the established GDScript option, has a CLI runner suited to this workflow, and CLAUDE.md already names it as the intended framework. Install into addons/gut/ via the AssetLib or a git checkout, and enable the plugin.

GdUnit4 is the main alternative if GUT proves awkward; it has richer assertions and a scene-runner, at the cost of a heavier install. Not worth switching to speculatively.

Command line runner, roughly:

"$GODOT" --path . --headless -s addons/gut/gut_cmdln.gd -gdir=res://test -gexit

Logic tests run fine headless. Only visual verification needs a headed run, because gl_compatibility renders nothing headless — that distinction should be recorded so nobody concludes the test suite covers rendering.

What to test first

Ordered by "how quietly would this break".

Seed determinism. The same seed produces identical _levels; a different seed produces different ones. This underpins the whole terrain and threat-cycle design, and a break looks exactly like normal operation.
Seed resolution in main_menu.gd: empty → random, valid integer → literal, anything else → hash(). Both entry paths (button and Enter) must agree. A real bug shipped here — the resolved seed was echoed into the field before resolution, so typing a seed and clicking New Game silently used the previous one while pressing Enter honoured it. A test comparing the two paths would have caught it immediately.
Coordinate invariants, carried from the POC where they caught real bugs:
- surface_y(L) == map_to_local(Vector3i(0, L, 0)).y + level_height
- world_to_cell(cell_to_world(c)) == c for every cell
Shape presets are distinguishable — e.g. the crater's rim band sits above its centre, the canyon has a contiguous low channel. Loose assertions, but they catch a preset silently degenerating to plains.

Known wrinkles to check during setup

class_name registration needs an editor scan (--headless --editor --quit) before a plain headless run can resolve the type. TerrainGrid and TerrainPreset are both affected. If the runner is invoked without that, tests fail to parse rather than fail to pass — a confusing first experience.
Autoload globals may not resolve in script mode. In throwaway -s harnesses, referencing GameState directly failed to compile and needed root.get_node("GameState"). GUT's runner is itself invoked with -s, so confirm whether test scripts can reference autoloads by name; if not, tests touching GameState need the same workaround, and that belongs in the test README.

Follow-on

Once a real command exists, update CLAUDE.md — it currently says none exists and instructs not to fabricate one. CI is out of scope here; there is no pipeline yet.

There is no test framework. Verification so far has been throwaway `SceneTree` harnesses written, run, and deleted for each change. That worked for one-off checks, but several properties are cheap to break and silent when broken, and deserve permanent assertions. ## Framework **GUT** (Godot Unit Test) is the recommended choice — it is the established GDScript option, has a CLI runner suited to this workflow, and CLAUDE.md already names it as the intended framework. Install into `addons/gut/` via the AssetLib or a git checkout, and enable the plugin. GdUnit4 is the main alternative if GUT proves awkward; it has richer assertions and a scene-runner, at the cost of a heavier install. Not worth switching to speculatively. Command line runner, roughly: ```sh "$GODOT" --path . --headless -s addons/gut/gut_cmdln.gd -gdir=res://test -gexit ``` Logic tests run fine headless. Only *visual* verification needs a headed run, because `gl_compatibility` renders nothing headless — that distinction should be recorded so nobody concludes the test suite covers rendering. ## What to test first Ordered by "how quietly would this break". 1. **Seed determinism.** The same seed produces identical `_levels`; a different seed produces different ones. This underpins the whole terrain and threat-cycle design, and a break looks exactly like normal operation. 2. **Seed resolution** in `main_menu.gd`: empty → random, valid integer → literal, anything else → `hash()`. Both entry paths (button and Enter) must agree. A real bug shipped here — the resolved seed was echoed into the field *before* resolution, so typing a seed and clicking New Game silently used the previous one while pressing Enter honoured it. A test comparing the two paths would have caught it immediately. 3. **Coordinate invariants**, carried from the POC where they caught real bugs: - `surface_y(L) == map_to_local(Vector3i(0, L, 0)).y + level_height` - `world_to_cell(cell_to_world(c)) == c` for every cell 4. **Shape presets are distinguishable** — e.g. the crater's rim band sits above its centre, the canyon has a contiguous low channel. Loose assertions, but they catch a preset silently degenerating to plains. ## Known wrinkles to check during setup - **`class_name` registration needs an editor scan** (`--headless --editor --quit`) before a plain headless run can resolve the type. `TerrainGrid` and `TerrainPreset` are both affected. If the runner is invoked without that, tests fail to parse rather than fail to pass — a confusing first experience. - **Autoload globals may not resolve in script mode.** In throwaway `-s` harnesses, referencing `GameState` directly failed to compile and needed `root.get_node("GameState")`. GUT's runner is itself invoked with `-s`, so confirm whether test scripts can reference autoloads by name; if not, tests touching `GameState` need the same workaround, and that belongs in the test README. ## Follow-on Once a real command exists, update CLAUDE.md — it currently says none exists and instructs not to fabricate one. CI is out of scope here; there is no pipeline yet.

liamjd referenced this issue

2026-08-16 08:56:23 +01:00

Add GUT test framework and seed determinism tests #15

liamjd referenced this issue

2026-08-16 09:08:33 +01:00

Extend test coverage: seed resolution, coordinate invariants, preset shape characteristics #16

liamjd commented

2026-08-16 09:08:39 +01:00

Author

Owner

Framework is in place and seed determinism is covered as of #15. The remaining items from this issue - seed resolution in main_menu.gd, the coordinate invariants, and per-preset shape characteristics - are carried over to #16, along with the island-at-small-grid-size finding the new tests surfaced.

Closing: "add a test framework" is done; what is left is extending coverage, which is ongoing work rather than a setup task.

Framework is in place and seed determinism is covered as of #15. The remaining items from this issue - seed resolution in `main_menu.gd`, the coordinate invariants, and per-preset shape characteristics - are carried over to #16, along with the island-at-small-grid-size finding the new tests surfaced. Closing: "add a test framework" is done; what is left is extending coverage, which is ongoing work rather than a setup task.

liamjd closed this issue

2026-08-16 09:08:42 +01:00