bundle: Clear local state on destroy to prevent lineage mismatch

[janniklasrose]

Changes

Stacked on top of #6207 (which adds the reproducing acceptance test).

bundle destroy only deleted the remote state, leaving the local state file — and its lineage — behind. When the same bundle was later redeployed from a different machine that had no local state, the fresh deploy minted a new lineage that no longer matched the lingering local one, so every subsequent command failed with lineage mismatch in state files.

This removes the local direct-engine state file on a successful destroy, so no stale lineage is left behind. The next command then finds no local state and adopts the remote state cleanly.

Why delete the file rather than just blanking the lineage: blanking alone does not fix it. validateStates still string-compares the (now empty) local lineage against the remote one and mismatches, and even if empty lineages were skipped, the empty-but-higher-serial local state would out-rank the real remote state during winner selection and get deployed from. Deleting the file is a single, localized change that matches what a fresh machine looks like anyway — which is the exact situation being emulated.

Scoped to the direct engine: lineage (a per-deployment-history UUID) is a direct-engine concept, and terraform manages its own state lifecycle.

Reworked test

acceptance/bundle/state/state_present relied on the local direct state surviving a destroy to assert serial continuity. Its engine-selection intent (a higher-serial direct state keeps winning regardless of the DATABRICKS_BUNDLE_ENGINE env var) is unchanged; the mid-test destroy is dropped and the asserted final serial adjusts accordingly.

The reproducing test added in #6207 (acceptance/bundle/destroy/lineage-mismatch-after-redeploy) now shows machine A succeeding instead of erroring.

Tests

• Reworked state_present passes on both engine variants with identical output.
• 470 acceptance tests across state/destroy/invariant/wal/migrate pass; bundle/phases unit tests pass.

This pull request and its description were written by Isaac, an AI coding agent.

[eng-dev-ecosystem-bot]

Integration test report

Commit: f28302d

Run: 31376170828

	Env	❌​FAIL	🟨​KNOWN	💚​RECOVERED	🙈​SKIP	✅​pass	🙈​skip	Time
❌​	aws linux	1	1	3	4	292	1116	6:07
❌​	aws windows	1	1	3	4	294	1114	6:51
❌​	azure linux	1	1	3	4	291	1116	6:40
❌​	azure windows	1	1	3	4	293	1114	8:51
❌​	gcp linux	1	1		5	292	1116	7:35
❌​	gcp windows	1	1		5	294	1114	8:26

9 interesting tests: 4 SKIP, 3 RECOVERED, 1 KNOWN, 1 FAIL

	Test Name	aws linux	aws windows	azure linux	azure windows	gcp linux	gcp windows
🟨​	TestAccept	🟨​K	🟨​K	🟨​K	🟨​K	🟨​K	🟨​K
🙈​	TestAccept/bundle/invariant/no_drift	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S
❌​	TestAccept/bundle/resources/model_serving_endpoints/drift/recreated_same_name	❌​F	❌​F	❌​F	❌​F	❌​F	❌​F
🙈​	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S
🙈​	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S
🙈​	TestAccept/ssh/connection	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S
💚​	TestFetchRepositoryInfoAPI_FromRepo	💚​R	💚​R	💚​R	💚​R	🙈​S	🙈​S
💚​	TestFetchRepositoryInfoAPI_FromRepo/root	💚​R	💚​R	💚​R	💚​R
💚​	TestFetchRepositoryInfoAPI_FromRepo/subdir	💚​R	💚​R	💚​R	💚​R