Restore an instance when inbound activity races standby

[hiroTamada]

Summary

A standby guest is only restored when a new inbound connection arrives — wake-on-traffic does not exist. So a connection that is already open when the guest is suspended does not stall briefly, it hangs until the client gives up: no response, no close, no error, and every later request on that connection is lost too. Opening a second connection to the same guest works immediately, which makes it look like the first connection was individually poisoned.

executeStandby releases the controller lock for the pause and snapshot, which takes long enough for a request to land mid-transition. The failing path already hands that state back to the reconcile flow, but the successful path dropped it in clearStateLocked and left the guest suspended with a live connection pointed at it.

Change

After a successful standby, if inbound activity turned up while the snapshot was in flight, restore the instance instead of discarding that state. Reported as a raced_restored standby attempt, with a WARN log. A restore that fails with ErrInstanceNotFound drops controller state, matching how the standby path already handles a deleted instance.

RestoreInstance is new on the InstanceStore interface; the Linux provider maps instances.ErrNotFound onto autostandby.ErrInstanceNotFound the same way StandbyInstance does.

Scope

Deliberately narrow. Two adjacent hardening changes were cut from this PR and parked on hypeship/auto-standby-hardening-parked, to be reviewed separately if wanted:

• Confirming against conntrack before pausing. The tracked connection view is event-driven and the reconcile that re-reads the host table is cancelled once that view drains, so nothing compares it against reality between the countdown starting and the snapshot. Worth doing, but it adds a conntrack table read per standby attempt, and whether it helps at all depends on the open question below.
• Making TCPState.Active a denylist. Only NONE/TIME_WAIT/CLOSE would stop counting, so an unrecognised state keeps the VM awake rather than stranding a live flow. SYN_SENT2 (value 9, named TCPStateListen) is currently classified as finished, the same gap Count half-open inbound connections as auto-standby activity #312 closed for SYN_SENT. This is fleet-wide behaviour with cost implications, so it deserves its own review.

Neither is needed for the race this PR fixes.

Open question

There is a case where a live connection is not counted as active inbound at all, so the idle countdown starts and runs to completion while the client is mid-request. That is upstream of this fix and not addressed here — matchesInboundConnectionForEvent requires OriginalDestinationIP to equal the instance IP, which would not hold if a proxy hop to the guest is NAT'd. Worth resolving before adding more code in this area.

The structural fix remains wake-on-traffic: an inbound packet for a suspended guest triggering a restore would turn this entire family of bugs into a latency hiccup instead of a hang.

Testing

go test -race ./lib/autostandby/... passes, as do the auto-standby tests in cmd/api/api. Three new tests:

• inbound activity racing the snapshot restores the instance
• a raced restore for an instance deleted mid-transition drops controller state instead of erroring
• a quiet instance still reaches standby with no restore

The first two fail against the current controller. The remaining integration tests in cmd/api/api need a real hypervisor and registry and fail the same way on main in this environment — I did not run those.

🤖 Generated with Claude Code

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit cac391a. Configure here.}