Skip to content

fix(tests): build the test tree from node ids, not from message order - #71

Open
roxblnfk wants to merge 1 commit into
j-plugins:mainfrom
roxblnfk:fix/id-based-test-tree
Open

fix(tests): build the test tree from node ids, not from message order#71
roxblnfk wants to merge 1 commit into
j-plugins:mainfrom
roxblnfk:fix/id-based-test-tree

Conversation

@roxblnfk

Copy link
Copy Markdown

Problem

Testo runs tests concurrently — fibers, an event loop — so two nodes are open at once and its service
messages interleave. The name-based events convertor takes the tree from the order messages arrive in:
it nests whatever opened last inside whatever sits on its stack.

Running a case with two #[DataSet] batches (e.g. Testo's own tests/Sandbox/Self/AsyncTest.php) shows
the second batch inside the first, and neither node ever closes. Data sets of different batches carry
the same display name (Dataset #0:0 [0]), so the result also reads as duplicated rows.

Fix

Testo puts nodeId/parentNodeId on every service message, so the tree is stated rather than inferred.
isIdBasedTestTree() switches the platform to GeneralIdBasedToSMTRunnerEventsConvertor
(SMTestRunnerConnectionUtil picks it off that flag), which reads those attributes and stops caring about
message order.

What Testo emits now:

testSuiteStarted name='AsyncTest [test]' nodeId='2' parentNodeId='1'
testSuiteStarted name='slowDataSets'     nodeId='6' parentNodeId='2' flowId='6'
testStarted      name='Dataset #0:0 [0]' nodeId='8' parentNodeId='6' flowId='6'
testSuiteStarted name='fastDataSets'     nodeId='7' parentNodeId='2' flowId='7'   <- sibling, not child
testStarted      name='Dataset #0:0 [0]' nodeId='9' parentNodeId='7' flowId='7'

Node ids come from the run: every node — suite, case, test, data set — has one, and every message about it
repeats it, so a finish or an output chunk lands on the right node no matter what else is in flight.

Heads-up before merging

This needs a Testo release that emits the ids. The convertor drops test messages without a nodeId, so
with an older Testo the tree would come out empty. The emitting side is on Testo's feat/test-identity
branch and is not in 1.x yet — please merge/release this only after that lands.

There is no version gate here on purpose: isIdBasedTestTree() is read when the console is created, and
TestoVersionDetector is currently wired only into the settings form, so the version is not known at that
point. Happy to add a gate if you want one — it needs the detected version threaded to the console properties.

Follow-up left out of this PR

ChannelOutputStore keys output by test name: locationByName maps name -> locationHint, and the two
batches above share a name, so the second testStarted overwrites the first's entry and both tests' output
collapses into one bucket. That is a separate defect, visible without concurrency too. The proper key is the
nodeId, but that also touches the lookup side (TestoChannelsUi, TestoChannelHistory go through
keyFor(proxy.name), and SMTestProxy does not expose the node id), so it is your design call rather than
something to guess at blind.

Checks

./gradlew compileKotlin and ./gradlew test pass locally (JBR 25, Gradle 9.6).

🤖 Generated with Claude Code

Testo runs tests concurrently — fibers, an event loop — so two nodes are open at the same time and its
service messages interleave. The name-based convertor nests whatever opened last inside whatever sits
on its stack, so a run of a case with two `#[DataSet]` batches shows the second batch *inside* the
first, and neither node ever closes. Data sets of different batches share a display name, which makes
the result read as duplicated rows.

Testo already puts `nodeId`/`parentNodeId` on every service message, so the tree is stated rather than
inferred: `isIdBasedTestTree()` switches the platform to `GeneralIdBasedToSMTRunnerEventsConvertor`,
which reads them and no longer cares in what order the messages arrive.

Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant