Detect Databricks CLI version to gate --profile support

`--profile` on `databricks auth token` is a global Cobra flag, so old
CLIs (< v0.207.1) silently accept it and fail later with `cannot fetch
credentials` instead of `unknown flag: --profile`. The previous
error-based fallback never matched, leaving the `--host` fallback as
dead code.

This commit replaces the runtime fallback chain with version-based
capability detection:

* `CliVersion` carries a (major, minor, patch) triple plus an
  `UNKNOWN` sentinel and a default-dev-build (0,0,0) check.
* `DatabricksCliCredentialsProvider` runs `databricks version --output
  json` once per CLI path (cached on success only, with a 5s timeout)
  and gates `--profile` on >= v0.207.1; everything else falls back to
  `--host` with a precise warning.
* `CliTokenSource` is simplified to a single `cmd`; the
  `fallbackCmd` parameter and the runtime "unknown flag" retry loop are
  removed.

Mirrors the equivalent refactors in the Go and Python SDKs:
* databricks/databricks-sdk-go#1605
* databricks/databricks-sdk-py#1377

Co-authored-by: Isaac

Author: mihaimitrea-db

NEXT_CHANGELOG.md

@@ -7,11 +7,13 @@
 ### Breaking Changes
 
 ### Bug Fixes
+* Fixed Databricks CLI `--profile` fallback by detecting the CLI version at init time. The previous error-based detection was broken because `--profile` is a global Cobra flag silently accepted by old CLIs.
 
 ### Security Vulnerabilities
 
 ### Documentation
 
 ### Internal Changes
+* Detect Databricks CLI version at init time via `databricks version --output json`, enabling version-gated flag support. Successful detections are cached per CLI path; subprocess failures fall back to the most conservative command and are retried on the next call.
 
 ### API Changes

databricks-sdk-java/src/main/java/com/databricks/sdk/core/CliTokenSource.java

@@ -30,11 +30,6 @@ public class CliTokenSource implements TokenSource {
   private String accessTokenField;
   private String expiryField;
   private Environment env;
-  // fallbackCmd is tried when the primary command fails with "unknown flag: --profile",
-  // indicating the CLI is too old to support --profile. Can be removed once support
-  // for CLI versions predating --profile is dropped.
-  // See: https://github.com/databricks/databricks-sdk-go/pull/1497
-  private List<String> fallbackCmd;
 
   /**
    * Internal exception that carries the clean stderr message but exposes full output for checks.
@@ -58,24 +53,11 @@ public CliTokenSource(
       String accessTokenField,
       String expiryField,
       Environment env) {
-    this(cmd, tokenTypeField, accessTokenField, expiryField, env, null);
-  }
-
-  public CliTokenSource(
-      List<String> cmd,
-      String tokenTypeField,
-      String accessTokenField,
-      String expiryField,
-      Environment env,
-      List<String> fallbackCmd) {
-    super();
     this.cmd = OSUtils.get(env).getCliExecutableCommand(cmd);
     this.tokenTypeField = tokenTypeField;
     this.accessTokenField = accessTokenField;
     this.expiryField = expiryField;
     this.env = env;
-    this.fallbackCmd =
-        fallbackCmd != null ? OSUtils.get(env).getCliExecutableCommand(fallbackCmd) : null;
   }
 
   /**
@@ -158,22 +140,6 @@ public Token getToken() {
     try {
       return execCliCommand(this.cmd);
     } catch (IOException e) {
-      String textToCheck =
-          e instanceof CliCommandException
-              ? ((CliCommandException) e).getFullOutput()
-              : e.getMessage();
-      if (fallbackCmd != null
-          && textToCheck != null
-          && textToCheck.contains("unknown flag: --profile")) {
-        LOG.warn(
-            "Databricks CLI does not support --profile flag. Falling back to --host. "
-                + "Please upgrade your CLI to the latest version.");
-        try {
-          return execCliCommand(this.fallbackCmd);
-        } catch (IOException fallbackException) {
-          throw new DatabricksException(fallbackException.getMessage(), fallbackException);
-        }
-      }
       throw new DatabricksException(e.getMessage(), e);
     }
   }