By NODVIEW editors

Sources

An agent says fixed: check the repair, not the message

A lesson from Jepsen's analysis: separate a developer's assertion, the exact software build, the observed result and the reliability of the test itself.

An agent's success message starts the acceptance check. Reproduce the failure on the earlier version, then show that the same scenario passes after the repair. Otherwise, the check may exercise different code, a different environment or an altered test.

In his October 31 ParticleDB analysis, Kyle Kingsbury describes fix assertions that did not survive testing. Orkes, the system's maker, funded the research. This is one researcher's experience with a particular product, not proof that AI use causes every defect.

Separate tested builds from subsequent statements. Orkes reports that version 1.1.19 addresses the identified issues. That statement alone does not mean Jepsen or NODVIEW independently retested every repair.

Keep four items with a change: a minimal failing example, the build identifier, the code diff and the unchanged check's result. If the test must change, explain why and retain its previous form for comparison. Another engineer should be able to assess the conclusion without trusting the agent's explanation.

Review the checker too. The Jepsen test repository documents known generator problems and possible false failures. A failing check needs investigation; a green indicator does not establish correctness across the whole system either. NODVIEW reviewed the sources but did not run these tests or ParticleDB.