Connections Pane: Databricks Support

[dfalbel]

This PR adds databricks support for the connections pane. Addresses #10430

Some Databricks interactions may take a while, like listing all catalogs because of cold starts. Eg if the warehouse is turned off, we may need to wait for it to turn on before the listing can be shown. This leads to RPC timeout errors.
To handle this, we've added a loading state for connections, that is shown while connections don't have any entries to display. We've also made list_objects never timeout, it's up for the implementation of the list_objects to timeout if needed and raise an error instead.

We've also added a similar loading state when clicking on the expand button. We now show a small spining icon that indicates that something is going on.

Databricks catalgos may have volumes which are leaf like nodes in the DB because they can't be expanded, but they are not table/view 'fields'. To handle this case, we've added a new optional field in the connections openRPC contract called 'has_children', if False it means that the obejct can no longer be expanded. By default this is assumed to be True unless the object 'kind' is 'field'.

To try it out, you can:

from databricks import sql
connection = sql.connect(
    server_hostname = "<host_name>",
    http_path              = "<http_path>",
    access_token       = "<PAT>" 
)
%connection_show connection

Feel free to ask questions for help with setting up.

Release Notes

New Features

• Added Connections Pane support for Databricks SQL Connector (Connections Pane: Databricks Support #10427).

Bug Fixes

• N/A

QA Notes

@:connections

[github-actions]

E2E Tests 🚀
This PR will run tests tagged with: @:critical @:connections

^{readme  valid tags}

[Copilot]

Pull Request Overview

This PR adds Databricks SQL Connector support to the Connections Pane, enabling users to browse catalogs, schemas, tables, views, and volumes from Databricks databases.

Key changes:

• Added has_children optional field to the OpenRPC contract to support leaf nodes like volumes that cannot be expanded
• Introduced loading and error states in the UI to handle slow operations and RPC timeouts
• Implemented DatabricksConnection class in Python backend with support for listing objects, fields, and previewing data

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
positron/comms/connections-backend-openrpc.json	Added optional has_children field to ObjectSchema
src/vs/workbench/services/languageRuntime/common/positronConnectionsComm.ts	Added TypeScript interface for has_children field
src/vs/workbench/services/positronConnections/common/interfaces/positronConnectionsInstance.ts	Updated interface to support undefined entries during loading
src/vs/workbench/services/positronConnections/browser/positronConnectionsInstance.ts	Updated to handle has_children field and undefined entries state
src/vs/workbench/contrib/positronConnections/browser/components/schemaNavigation.tsx	Added loading/error UI states and delayed spinner for expansion
src/vs/workbench/contrib/positronConnections/browser/components/schemaNavigation.css	Added styles for loading states and spinner animation
extensions/positron-python/python_files/posit/positron/connections_comm.py	Added has_children field to ObjectSchema model
extensions/positron-python/python_files/posit/positron/connections.py	Implemented DatabricksConnection class with full CRUD support
extensions/positron-python/python_files/posit/positron/tests/test_connections.py	Added comprehensive test suite for Databricks connections
extensions/positron-python/python_files/posit/test-requirements.txt	Added databricks-sql-connector dependency
extensions/positron-python/python_files/posit/pinned-test-requirements.txt	Pinned databricks-sql-connector version

[isabelizimm]

Tried it out! Things seem to work, but as you noted, it feels quite slow. To load the Connections pane took about a minute. Is this coming from the Databricks Python package? When I run queries in the Console, it doesn't feel as slow.

[dfalbel]

The reason why it's slow is that the first query sent to the databricks cluster is going to trigger the startup of the warehouse cluster. Our cluster is programmed to turn off the warehouse after 45 minutes of inactivity.

Subsequent queries are fast enought and that's probably why it didn't feel slow when you tried in the console. But it's also terribly slow if the warehouse is not started and you try to do any query using the python connector from the console.

Unfortunatelly, I don't think there's much we can do regarding cold starts. I don't think though this will be a common behavior for users that are connecting to a larger databricks cluster where the warehouse will probably be 'online' most of the time.

FWIW this also the behavior in the web app when running a query in stopped cluster:
(sorry the video is just 3 min in the waiting screen :P )

CleanShot.2025-11-13.at.12.36.10.mp4