[AWS] VPC endpoint refactor + pre-registered endpoints + naming overrides

[airizarryDB]

Summary

This branch refactors the AWS SRA's VPC endpoint handling and adds several quality-of-life flags. It also introduces backwards-compatible state migrations and updates the docs.

New AWS resources

• Service Direct endpoint (aws_vpc_endpoint.service_direct, databricks_mws_vpc_endpoint.service_direct): a front-end PrivateLink endpoint that customers' clients use to reach the workspace UI/API privately. Service Direct endpoints are commonly shared across workspaces in the same VPC, so SRA does not create one by default — set the new create_service_direct_vpce = true flag to opt in. Not available in GovCloud regions, where the resource is automatically skipped.

New optional tfvars

• create_service_direct_vpce (bool, default false): opt-in flag for SRA to create + register a Service Direct VPC endpoint for this workspace.
• custom_service_direct_vpce_id (string, default null): bring your own AWS Service Direct VPC endpoint ID (used in custom networking mode when you've already provisioned the endpoint).
• custom_general_access_mws_vpce_id, custom_scc_relay_mws_vpce_id, custom_service_direct_mws_vpce_id (string, default null): pre-registered Databricks-side MWS VPC endpoint IDs. When set, SRA skips the databricks_mws_vpc_endpoint registration step and wires the ID directly into the workspace network configuration. Use this when the AWS VPC endpoint has already been registered with the Databricks account (e.g., shared across workspaces in the same VPC).
• workspace_display_name (string, default null): human-readable workspace name shown in the Databricks UI. Defaults to resource_prefix when unset.
• custom_metastore_name (string, default null): name for the Unity Catalog metastore created by this deployment. Defaults to ${var.region}-unity-catalog when unset. Only used when metastore_exists = false.

Variable + resource renames (alignment with new naming)

VPC endpoint resources, module inputs, and tfvars variables were renamed to match the underlying Databricks PrivateLink endpoint nomenclature (general_access, scc_tunnel_dataplane_relay_access, service_direct) rather than the legacy backend_* names.

⚠️ Breaking change — tfvars rename

If you set either of these in your tfvars file, rename them before upgrading:

Old	New
custom_workspace_vpce_id	custom_general_access_vpce_id
custom_relay_vpce_id	custom_scc_relay_vpce_id

Wiring is identical — only the variable names changed.

Resource address renames (handled automatically)

The following Terraform resource addresses changed in this branch. moved blocks are included so existing deployments preserve their state on upgrade — no destroy/recreate plans, no out-of-band terraform state mv:

Old	New
aws_vpc_endpoint.backend_rest	aws_vpc_endpoint.general_access
aws_vpc_endpoint.backend_relay	aws_vpc_endpoint.scc_tunnel_dataplane_relay_access
databricks_mws_vpc_endpoint.backend_rest	databricks_mws_vpc_endpoint.general_access[0]
databricks_mws_vpc_endpoint.backend_relay	databricks_mws_vpc_endpoint.scc_tunnel_dataplane_relay_access[0]

The [0] index on the databricks_mws_vpc_endpoint resources reflects the addition of count (so the resources can be skipped when an MWS endpoint ID is provided).

Documentation

• aws/README.md: added pre-registered MWS endpoint guidance to the "Custom" networking mode bullet; added a "Service Direct" entry under Core AWS Components; added an "Optional Naming Overrides" subsection.
• docs/sra/docs/usage/AWS/components.mdx: mirrors the README updates for the Docusaurus site.

Tests

aws/tf/tests/mock_plan.tftest.hcl and aws/tf/tests/mock_plan_gov.tftest.hcl were updated to include the new variables (defaulted to null/false). Existing assertions are unchanged.

Related

• Feature request to the Databricks Terraform provider for a data.databricks_mws_vpc_endpoint source: databricks/terraform-provider-databricks#5715. When that lands, the "pre-registered MWS endpoint ID" path here can be simplified by looking endpoints up from state rather than having callers thread IDs in manually.

Test plan

• terraform fmt -recursive — clean
• terraform validate — passes against the existing mock test config
• terraform test -filter=tests/mock_plan.tftest.hcl — passes
• terraform test -filter=tests/mock_plan_gov.tftest.hcl — passes
• Fresh apply in a commercial region — workspace created with new resource names, no Service Direct endpoint created by default
• Fresh apply with create_service_direct_vpce = true in a supported region — Service Direct endpoint created and registered
• Upgrade apply from main on an existing deployment — moved blocks rewrite state, no destroy/recreate in the plan
• Apply with custom_general_access_mws_vpce_id set — databricks_mws_vpc_endpoint.general_access is skipped, mws_networks uses the provided ID directly
• Apply with workspace_display_name set — UI shows the override, not resource_prefix
• Apply with custom_metastore_name set (and metastore_exists = false) — metastore is created with the override name

[shreelshah12]

lgtm and tested on AWS Commercial

[airizarryDB]

Just completed testing, LGTM
AWS Commercial: ✅
AWS Govcloud Civilian: ✅
AWS Govcloud DoD: ✅