Your CI runs the tests you wrote. Spinal writes the one you didn't.
A green pipeline means your known tests passed. It does not prove this PR's newly suspected risk is safe.
CI is necessary. It is also not enough for agentic software delivery.
Your CI pipeline runs the tests your team already wrote. Spinal validation starts from a different question: this PR introduced a suspected production risk — can we prove or reject that risk before merge?
CI asks: did known tests pass? Spinal asks: is this suspected production risk real?
Known tests vs suspected risk
CI is your known-regression net. It runs the suite you maintain: unit tests, integration tests, type checks, linters, builds, migrations, smoke tests, and deployment checks.
That is essential. But a green pipeline mostly means the known checks still pass. It does not mean a new behavior risk was tested, especially when the code was generated by an agent and the risky scenario was not in the suite yet.
| Your CI | Spinal validation | |
|---|---|---|
| Question | Did the known suite pass? | Is the suspected production risk real? |
| Test selection | Predefined by the team | Generated for this PR and finding |
| Inputs | Repository and test environment | Intent, diff, repo rules, CI, logs, metrics, traces, SLOs |
| Runtime | Existing CI/CD pipeline | Customer CI/CD or dedicated sandbox |
| Output | Pass or fail for known checks | Risk rejected, reproduced, or inconclusive with evidence |
A concrete example
Suppose a coding agent updates SSO callback handling for a new identity-provider payload. The PR is clean. The existing auth tests pass. The known callback success path is covered.
The review finding is more specific: the change may have moved session expiry calculation from the refresh path into the callback path. If true, the system could accidentally change from a sliding idle timeout to a fixed session lifetime.
Existing CI may not catch that because the suite checks successful login, token parsing, and basic callback behavior. It may not simulate an already-active user returning at minute 20 and expecting the idle timeout to extend.
How Spinal chooses what to validate
Spinal does not generate a broad pile of tests. It starts with a review hypothesis and builds the smallest useful check around it.
| Input | Example for the SSO PR | Why it matters |
|---|---|---|
| Captured intent | Do not change session lifetime or refresh behavior | Defines what the PR must preserve |
| Diff | Expiry calculation moved in auth/session.ts | Identifies the suspected behavior change |
| Repo rules | Auth changes must preserve idle-timeout semantics | Turns team judgment into review criteria |
| Production signals | Login traces, refresh logs, session SLO boundary | Shapes a realistic scenario instead of a generic test |
| Validation profile | Node test command, auth fixtures, sandbox env vars | Determines where and how the check can run |
What the generated validation looks like
The output is a focused scenario, not a generic test suite rewrite:
Validation scenario
1. Create a session through the new IdP callback payload.
2. Simulate user activity at minute 20 using the observed refresh path.
3. Assert expires_at extends to preserve the 30-minute idle timeout.
4. Assert the user is not forced to re-authenticate before the expected idle window.The exact implementation depends on the repo profile. In one team, this runs as a Playwright or integration test in existing CI/CD. In another, it runs in a sandbox with production-shaped fixtures and scrubbed trace-derived inputs.
Where the check runs
Spinal validation should run where the result is meaningful and safe. That can be your existing CI/CD pipeline when the repo has the right test harness. It can also be a dedicated sandbox when the check needs services, realistic data shape, or isolation from normal pipeline constraints.
The important part is that the result returns to the review agent. The final PR review changes based on what happened.
| Result | Meaning | What the developer sees |
|---|---|---|
| Rejected | The suspected risk did not reproduce | Validation passed; evidence attached; merge confidence improves |
| Reproduced | The suspected risk is real | Needs changes; failing scenario and evidence are attached |
| Inconclusive | The check could not prove the risk either way | Residual risk remains; review explains what was missing |
How this works with coding agents
Claude Code, Codex, Cursor, and similar tools can help write tests. That is useful. But the validation target should not only come from the same coding session that produced the change.
Spinal starts from an independent review signal: captured intent, repo rules, production evidence, and the specific risk raised by the PR. The coding agent can still help fix the issue, but the quality gate is not just the author checking its own work.
The takeaway
CI guards the known. Spinal validates the newly suspected.
That distinction matters when coding agents create changes faster than teams can manually reason through every edge case. A green pipeline tells you the suite passed. Spinal tells you whether this PR's production-shaped risk was rejected, reproduced, or still unresolved.
Put a trust layer around your agents