Skip to content

Conversation

@denik
Copy link
Contributor

@denik denik commented Nov 27, 2025

Changes

  • Extend structpath to support [key=‘value’] syntax.
  • Extend structdiff with extra parameter that maps path to keyFunc which is a function that returns key/value pair for a given slice element. They pair is used to identify the slice element.

  • Use this functionality on permissions resource in direct to solve drift due to backend reordering permissions.

Why

  • Prevent remote drift due to backend reordering slice elements.
  • Make diffs more readable / precise.

This is an alternative to always sorting the input, which destroys the references ($resources.jobs.foo.tasks[1] is not valid today because of sorting). In a follow up PR I’ll revert job tasks sorting #3212 and implement KeyedSlices instead.

@denik denik temporarily deployed to test-trigger-is November 27, 2025 09:51 — with GitHub Actions Inactive
@denik denik changed the title Treat some slices as maps; ignore permissions order change Treat some slices as maps; ignore permissions order Nov 27, 2025
@denik denik force-pushed the denik/path-key-syntax branch from 61a55e6 to 9ee67c5 Compare November 27, 2025 10:02
@denik denik temporarily deployed to test-trigger-is November 27, 2025 10:02 — with GitHub Actions Inactive
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Nov 27, 2025

Commit: 575f2ba

Run: 19741050772

Env 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 7 2 371 628 16:20
🟨​ aws windows 7 2 2 371 626 18:05
💚​ aws-ucws linux 7 2 514 513 21:39
💚​ aws-ucws windows 7 2 516 511 21:52
💚​ azure linux 1 4 371 627 14:22
💚​ azure windows 1 4 373 625 15:15
💚​ azure-ucws linux 1 4 510 512 23:14
💚​ azure-ucws windows 1 4 512 510 19:02
💚​ gcp linux 1 4 364 631 16:52
🔄​ gcp windows 3 4 364 629 15:12
13 failing tests:
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 🔄​f
🔄​ TestAccept/bundle/resources/experiments/basic ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/experiments/basic/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🙈​ TestAccept/bundle/run/app-with-job 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestAccept/bundle/templates/default-python/integration_classic ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.11 ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
Top 20 slowest tests (at least 2 minutes):
duration env testname
7:40 azure-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
7:34 azure-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
6:07 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
6:05 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:43 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:42 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:38 aws-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:32 azure windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:32 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:29 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:19 aws-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:04 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
4:51 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
4:38 azure-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
4:04 azure windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
4:03 azure-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
3:28 azure-ucws linux TestAccept/bundle/resources/synced_database_tables/basic
2:26 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
2:19 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
2:18 azure-ucws windows TestAccept/bundle/resources/synced_database_tables/basic

@denik denik changed the title Treat some slices as maps; ignore permissions order direct: Support configuration for unordered slices; use in permissions to ignor order Nov 27, 2025
@denik denik changed the title direct: Support configuration for unordered slices; use in permissions to ignor order direct: Support configuration for unordered slices; use in permissions to ignore order Nov 27, 2025
@denik denik temporarily deployed to test-trigger-is November 27, 2025 10:32 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is November 27, 2025 10:33 — with GitHub Actions Inactive
@denik denik marked this pull request as ready for review November 27, 2025 10:40
@denik denik temporarily deployed to test-trigger-is November 27, 2025 13:50 — with GitHub Actions Inactive

// [Optional] KeyedSlices returns a map from path patterns to KeyFunc for comparing slices by key instead of by index.
// Example: func (*ResourcePermissions) KeyedSlices(state *PermissionsState) map[string]any
KeyedSlices(state any) map[string]any
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like the implementation does not use the state parameter; shall it be removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, cleaned up 9cb6261

@denik denik temporarily deployed to test-trigger-is November 27, 2025 15:10 — with GitHub Actions Inactive
@denik denik requested a review from andrewnester November 27, 2025 15:11
@denik denik temporarily deployed to test-trigger-is November 27, 2025 15:23 — with GitHub Actions Inactive
@denik denik enabled auto-merge November 27, 2025 15:23
@denik denik added this pull request to the merge queue Nov 27, 2025
Merged via the queue into main with commit 0bc7fe3 Nov 27, 2025
13 checks passed
@denik denik deleted the denik/path-key-syntax branch November 27, 2025 16:05
@eng-dev-ecosystem-bot
Copy link
Collaborator

Commit: 0bc7fe3

Run: 19742129063

Env 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 7 3 1 404 615 47:51
🟨​ aws windows 7 2 3 1 404 613 47:07
💚​ aws-ucws linux 10 1 565 494 65:00
💚​ aws-ucws windows 10 1 567 492 65:19
🔄​ azure linux 3 3 3 402 614 49:12
🔄​ azure windows 3 3 3 404 612 50:12
💚​ azure-ucws linux 4 3 561 493 73:53
🔄​ azure-ucws windows 3 3 3 561 491 79:43
💚​ gcp linux 4 3 388 621 49:58
🔄​ gcp windows 3 3 3 388 619 52:04
16 failing tests:
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 💚​R 🔄​f 🔄​f 💚​R 🔄​f 💚​R 🔄​f
🔄​ TestAccept/bundle/resources/clusters/run/spark_python_task ✅​p ✅​p ✅​p ✅​p 🔄​f 🔄​f ✅​p 🔄​f ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/clusters/run/spark_python_task/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p 🔄​f 🔄​f ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/clusters/run/spark_python_task/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/dashboards/detect-change ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/dashboards/detect-change/DATABRICKS_BUNDLE_ENGINE=direct ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
💚​ TestAccept/bundle/run/app-with-job 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/run/app-with-job/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/run/app-with-job/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 50 slowest tests (at least 2 minutes):
duration env testname
15:11 azure-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
14:38 azure-ucws linux TestAccept/bundle/resources/permissions/factcheck/DATABRICKS_BUNDLE_ENGINE=terraform
13:50 azure-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
13:19 azure-ucws windows TestAccept/bundle/resources/clusters/run/spark_python_task/DATABRICKS_BUNDLE_ENGINE=terraform
13:13 gcp linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
13:13 gcp linux TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
12:04 aws windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
10:56 azure-ucws windows TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
10:41 azure linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
10:19 gcp windows TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
10:15 aws-ucws linux TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform
9:52 gcp windows TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
9:36 azure-ucws linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
9:31 azure-ucws windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
9:27 azure windows TestAccept/bundle/resources/permissions/factcheck/DATABRICKS_BUNDLE_ENGINE=terraform
8:45 azure-ucws windows TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=direct
8:40 aws windows TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=terraform
8:27 azure-ucws linux TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform
8:26 aws-ucws windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
8:24 aws windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
8:16 gcp windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
8:08 aws windows TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
8:08 gcp linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=direct/DATA_SECURITY_MODE=SINGLE_USER
8:05 azure-ucws linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
7:58 azure windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
7:57 gcp windows TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=terraform
7:54 gcp linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
7:48 azure windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
7:47 azure-ucws linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=direct/DATA_SECURITY_MODE=SINGLE_USER
7:45 gcp windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
7:44 gcp windows TestAccept/bundle/run/app-with-job/DATABRICKS_BUNDLE_ENGINE=terraform
7:40 aws-ucws linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
7:39 aws linux TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
7:36 azure linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
7:36 aws-ucws windows TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=direct
7:35 gcp linux TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=direct
7:34 aws-ucws linux TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=direct
7:32 gcp linux TestAccept/bundle/run/app-with-job/DATABRICKS_BUNDLE_ENGINE=direct
7:30 gcp windows TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=direct
7:28 gcp linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
7:28 azure-ucws windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
7:23 aws linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
7:15 gcp windows TestAccept/bundle/run/app-with-job/DATABRICKS_BUNDLE_ENGINE=direct
7:11 azure windows TestAccept/bundle/resources/clusters/run/spark_python_task/DATABRICKS_BUNDLE_ENGINE=terraform
7:10 azure-ucws windows TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform
7:07 aws-ucws windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
7:06 aws linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
7:05 azure linux TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
7:05 aws-ucws linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
7:05 aws-ucws linux TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants