Run the GUT suite from inside Rider #106

New issue

Open

opened 2026-09-17 08:35:34 +01:00 by liamjd · 1 comment

liamjd commented

2026-09-17 08:35:34 +01:00

Owner

Low priority — tooling, not gameplay. Parked deliberately so the findings aren't lost; nothing here blocks game work and it should not displace a milestone issue.

Running the GUT suite from inside Rider, rather than only from the CLI or the in-editor GUT panel.

What already works

A .NET Executable run configuration runs the full suite and streams GUT's output into Rider's Run panel. Despite the name it is just "launch this exe" — it is the right template here.

Executable: Godot_v4.7.2-stable_win64_console.exe (see caveat below)
Program arguments: --path . --headless -s addons/gut/gut_cmdln.gd
Working directory: project root

The arguments stay short because .gutconfig.json already carries the test dir, subdirectory recursion and exit-on-finish.

"GDScript Start and Debug" does not work for this. For GDScript, Rider does not launch Godot itself — it asks the Editor to launch and wires the debugger into that, which runs the project's main scene. -s addons/gut/gut_cmdln.gd is script mode, not a scene run, so that config type has nothing to hand off.

Known gotchas

Build errors on launch. The .NET Executable template adds a "Build Solution" before-launch task, and there is no solution in this repo. Delete the entry from the config's Before launch list. Related and equally harmless: .idea/UntitledColonyBuilder.iml declares a JAVA_MODULE with JDK 23, left by an IDE that did not recognise the project. Nothing here is a buildable IDE project and nothing needs to be — Godot does its own importing.

Use the _console.exe variant. Output appears either way because Rider captures stdout, but the plain .exe is a Windows GUI binary: the usual failure mode is that print() survives while script errors and stack traces are silently lost. That is exactly the output needed when a test fails.

.idea/ is gitignored wholesale, so a run configuration made on one machine will not reach the other. To share it, un-ignore .idea/runConfigurations/ specifically and mark the configuration "store as project file".

Red/green already works, without extra wiring

addons/gut/gui/GutRunner.gd:112-115 sets a non-zero exit code whenever the fail count is non-zero, and .gutconfig.json sets should_exit: true. So Rider's "Process finished with exit code 1" is a genuine pass/fail signal.

GutRunner.gd:119-120 additionally lets a post-run hook override the exit code — relevant to the test-tree idea below, and a reason the two do not conflict.

Upstream support: not coming

RIDER-118338 "Provide support for GUT/GDUnit test frameworks" — Open, filed 2024-10-07, last touched ~July 2026, 0 votes, no fix version. Voting is the cheapest possible action and is probably why it has never moved.

Its comment thread is more useful than its status. A JetBrains dev states that as of Rider 2025.3 with Godot 4.6+ a copied "Player GDScript" configuration can both run and debug GUT/GdUnit tests. We are on 4.7.2, so that gate is cleared. Untested here, and possibly a different template from the "GDScript Start and Debug" that failed above — worth one look only if breakpoints inside test code turn out to matter.

Possible follow-up: a real test tree

The IntelliJ platform parses TeamCity service messages (##teamcity[testStarted name='...'], testFailed, testFinished) on a run configuration's stdout and renders a test tree with timings and clickable navigation. If that holds for Rider run configurations — unverified, and the load-bearing assumption — most of RIDER-118338 is reproducible locally. Gutter run icons are not; those need the real plugin.

Two emission points, both already in GUT:

post_run_script in .gutconfig.json — a script extending addons/gut/hook_script.gd receives the gut instance and can walk the summary. Single process, fast loop.
-gjunit_xml_file=<path> (addons/gut/cli/gut_cli.gd:150) plus a wrapper that converts the XML. Leaves GUT untouched and leaves a JUnit file behind, which is what any future CI would want anyway.

Spike the assumption first: a throwaway run configuration that echoes four hand-written service messages. If Rider builds a tree from that, the rest is mechanical. If it does not, close this section and stop.

Acceptance criteria

Run configuration launches the suite from Rider with no build errors.
A second configuration with -gselect=<partial_filename> for single-script runs (~1s vs ~13s for the full suite — that difference decides whether it actually gets used).
Configuration reaches both machines, or the issue records the decision not to share it.
Service-message spike either proves the test tree is reachable, or is recorded here as a dead end.

**Low priority — tooling, not gameplay.** Parked deliberately so the findings aren't lost; nothing here blocks game work and it should not displace a milestone issue. Running the GUT suite from inside Rider, rather than only from the CLI or the in-editor GUT panel. ## What already works A **.NET Executable** run configuration runs the full suite and streams GUT's output into Rider's Run panel. Despite the name it is just "launch this exe" — it is the right template here. - Executable: `Godot_v4.7.2-stable_win64_console.exe` (see caveat below) - Program arguments: `--path . --headless -s addons/gut/gut_cmdln.gd` - Working directory: project root The arguments stay short because `.gutconfig.json` already carries the test dir, subdirectory recursion and exit-on-finish. **"GDScript Start and Debug" does not work for this.** For GDScript, Rider does not launch Godot itself — it asks the *Editor* to launch and wires the debugger into that, which runs the project's **main scene**. `-s addons/gut/gut_cmdln.gd` is script mode, not a scene run, so that config type has nothing to hand off. ## Known gotchas **Build errors on launch.** The .NET Executable template adds a **"Build Solution" before-launch task**, and there is no solution in this repo. Delete the entry from the config's *Before launch* list. Related and equally harmless: `.idea/UntitledColonyBuilder.iml` declares a `JAVA_MODULE` with JDK 23, left by an IDE that did not recognise the project. Nothing here is a buildable IDE project and nothing needs to be — Godot does its own importing. **Use the `_console.exe` variant.** Output appears either way because Rider captures stdout, but the plain `.exe` is a Windows GUI binary: the usual failure mode is that `print()` survives while script errors and stack traces are silently lost. That is exactly the output needed when a test fails. **`.idea/` is gitignored wholesale**, so a run configuration made on one machine will not reach the other. To share it, un-ignore `.idea/runConfigurations/` specifically and mark the configuration "store as project file". ## Red/green already works, without extra wiring `addons/gut/gui/GutRunner.gd:112-115` sets a non-zero exit code whenever the fail count is non-zero, and `.gutconfig.json` sets `should_exit: true`. So Rider's *"Process finished with exit code 1"* is a genuine pass/fail signal. `GutRunner.gd:119-120` additionally lets a **post-run hook override the exit code** — relevant to the test-tree idea below, and a reason the two do not conflict. ## Upstream support: not coming [RIDER-118338 "Provide support for GUT/GDUnit test frameworks"](https://youtrack.jetbrains.com/issue/RIDER-118338/Provide-support-for-GUT-GDUnit-test-frameworks) — **Open**, filed 2024-10-07, last touched ~July 2026, **0 votes**, no fix version. Voting is the cheapest possible action and is probably why it has never moved. Its comment thread is more useful than its status. A JetBrains dev states that as of **Rider 2025.3 with Godot 4.6+** a copied **"Player GDScript"** configuration can both run *and debug* GUT/GdUnit tests. We are on 4.7.2, so that gate is cleared. Untested here, and possibly a different template from the "GDScript Start and Debug" that failed above — worth one look **only if breakpoints inside test code turn out to matter**. ## Possible follow-up: a real test tree The IntelliJ platform parses **TeamCity service messages** (`##teamcity[testStarted name='...']`, `testFailed`, `testFinished`) on a run configuration's stdout and renders a test tree with timings and clickable navigation. If that holds for Rider run configurations — **unverified, and the load-bearing assumption** — most of RIDER-118338 is reproducible locally. Gutter run icons are not; those need the real plugin. Two emission points, both already in GUT: - **`post_run_script`** in `.gutconfig.json` — a script extending `addons/gut/hook_script.gd` receives the `gut` instance and can walk the summary. Single process, fast loop. - **`-gjunit_xml_file=<path>`** (`addons/gut/cli/gut_cli.gd:150`) plus a wrapper that converts the XML. Leaves GUT untouched and leaves a JUnit file behind, which is what any future CI would want anyway. Spike the assumption first: a throwaway run configuration that echoes four hand-written service messages. If Rider builds a tree from that, the rest is mechanical. If it does not, close this section and stop. ## Acceptance criteria - [ ] Run configuration launches the suite from Rider with no build errors. - [ ] A second configuration with `-gselect=<partial_filename>` for single-script runs (~1s vs ~13s for the full suite — that difference decides whether it actually gets used). - [ ] Configuration reaches both machines, or the issue records the decision not to share it. - [ ] Service-message spike either proves the test tree is reachable, or is recorded here as a dead end.

liamjd added the

testing

enhancement

labels

2026-09-17 08:35:34 +01:00

liamjd commented

2026-09-17 08:47:29 +01:00

Author

Owner