Granular updates for model serving endpoints #3988
Open
+1,201
−19
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
shreyas-goenka
force-pushed
the
better-model-serving-update
branch
from
8f26a22 to
fd244e3
Compare
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Collaborator
|
Commit: dbdbdf5
9 failing tests:
Top 20 slowest tests (at least 2 minutes):
|
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
- Merged latest main branch which renamed DoUpdateWithChanges to DoUpdate - DoUpdate now has same signature with changes parameter - Updated all resource implementations to use DoUpdate - All model serving endpoint tests still passing
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
- Added 'type Changes = deployplan.Changes' alias in alert.go (same as main) - Updated all DoUpdate signatures to use *Changes instead of *deployplan.Changes - Removed unused deployplan imports where only the alias is used - Reduces diff with main branch
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
shreyas-goenka
requested review from
andrewnester,
anton-107,
denik and
pietern
as code owners
denik
reviewed
Nov 27, 2025
| trace update_file.py databricks.yml 'catalog_name: "first-inference-catalog"' 'catalog_name: "second-inference-catalog"' | ||
|
|
||
| trace $CLI bundle plan -o json > out.plan.$DATABRICKS_BUNDLE_ENGINE.json | ||
| trace $CLI bundle deploy |
Contributor
There was a problem hiding this comment.
Please add 'trace $CLI bundle plan' after deploys to check that there is no drift.
Contributor
Author
There was a problem hiding this comment.
These tests are local server only, because of the slow speed to spin up an endpoint. Would you prefer if we make one of these cloud or is there value in still having bundle plan anyways?
Contributor
There was a problem hiding this comment.
it's valuable locally as well but more so on cloud.
Perhaps we can enable one of them as CloudSlow?
Contributor
Author
There was a problem hiding this comment.
Yeah sounds good, can do for one.
| var err error | ||
|
|
||
| if r.hasFieldChange(changes, "tags") { | ||
| err = r.updateTags(ctx, id, config.Tags) |
Contributor
There was a problem hiding this comment.
It looks like this updates no longer run in parallel?
Contributor
Author
There was a problem hiding this comment.
Yes. It's not clear that these methods are safe to run in parallel so I'd rather be more conservative and do what terraform does.
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
This PR implements granular updates for model serving endpoints. Instead of always updating the entire endpoint configuration, the CLI now only sends updates for the specific fields that have changed. This are:
Why
Model serving endpoint updates are expensive operations. By sending only the changed fields, we reduce the scope of updates and improve deployment performance.
We also don't have guarentees that these API calls are safe to do in parallel. This matches the TF implementation: https://github.com/databricks/terraform-provider-databricks/blob/b0a2a1c6a1688498fd6a00c64003ef4948da21e8/serving/resource_model_serving.go#L366
Tests
Added comprehensive acceptance tests for various update scenarios:
Note: This PR depends on #3995 and should be merged after that PR is merged and this branch is rebased.
Note: These tests are local only because model serving endpoints take a long time (~30 minutes) to spin up and can be flaky. We can confirm though that the TF and DABs behavior matches.