Repository navigation
[improvement](fe) Secure root and admin bootstrap on new clusters - #70495
Open
CalvinKirs wants to merge 7 commits into
Open
CalvinKirs wants to merge 7 commits into
CalvinKirs wants to merge 7 commits into
Conversation
…ters ### Background Doris initializes root and admin with empty passwords. Client authentication currently accepts those credentials until an operator changes them; internal jobs use execution identities without a password handshake. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Enhancement Problem Summary: New clusters should restrict empty built-in credentials while preserving existing clusters' authentication behavior. Persist a secure-initialization marker with the first frontend journal record, restrict empty root to local MySQL password initialization, and refuse empty admin client logins. Deployment scripts receive explicit credentials, and CI configurations explicitly select legacy test behavior. ### Details - **Change map:** Production and infrastructure +1054/-165 lines; tests +703 lines, including 39 generated result-fixture lines; documentation +199 lines. Core changes cover authentication and password-reset sessions, cluster metadata, and password changes. The remaining changes connect container credentials, tools, and regression fixtures to those rules. - **Before and after:** New built-in accounts previously accepted empty credentials remotely. Securely initialized clusters now expose only a restricted local root bootstrap session; admin requires a password for client access. Internal JOB, MTMV, statistics, group commit, and token-authenticated audit loading retain their execution identities. - **Configs and defaults:** Add nonmutable allow_empty_password_for_root_admin=false for explicit test compatibility. Keep the existing skip_localhost_auth_check boolean and its false default as a local recovery setting. Validate initial_root_password before initial ROLE/VERSION creation; initialized clusters continue ignoring it. - **Impact on existing users:** Missing metadata markers retain the previous authentication policy. Startup warnings identify empty credentials and recovery/test settings, and root password changes report an empty admin account. New container deployments require initial credentials or explicit test configuration. ### Testing - Standard FE build with Checkstyle enabled completed successfully; untouched UI and BE Java extensions were excluded from packaging. - run-fe-ut.sh: BootstrapAuthenticationTest, 13 tests passed, including persistence/replay, authentication modes, SQL guards, password changes, and internal task contexts. - Regression framework build: 51 tests passed. Container authentication helper: 15 tests passed; modified shell/Python syntax checks passed. - Docker regression covers empty-credential denial, enabling admin, group commit, audit import, scheduled JOB, MTMV, statistics, FE restart, and explicit legacy test mode. Result fixtures were generated by run-regression-test.sh. - Local integration uses the cached amd64 BE 4.1.3 with be_exec_version=10 and experimental_enable_local_shuffle_planner=false to avoid newer master execution nodes. Neither setting changes authentication policy; both are confined to the local fixture. - Real MySQL probes cover local bootstrap restrictions, reset/prepare commands, immediate session unlock, and password changes forwarded through a follower. A real 4.1.3 journal-only metadata upgrade retains legacy logins. Invalid initial hashes fail before ROLE/VERSION and are absent from logs. ### Notes for Reviewers Start with BootstrapAuthPolicy, BootstrapPasswordGuard, and the first-frontend journal marker. Dedicated cloud and mTLS end-to-end deployment, and an actual supported-version downgrade round trip, were not exercised locally. Unknown optional Gson fields provide reader compatibility; a downgrade/checkpoint/re-upgrade policy-preservation guarantee is not claimed. ### Release note New clusters restrict empty root to local MySQL password initialization and reject empty admin client logins. Container initialization requires configured credentials or explicit test compatibility. Existing clusters preserve their authentication behavior and gain warnings for empty built-in credentials. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: Yes; repository design and container instructions updated; website/operator documentation must be coordinated for release. - Pre-submission status: Partially verified: local checks passed; dedicated cloud/mTLS deployment and downgrade round-trip validation remain for CI/release validation. ### AI Disclosure - Tool: Codex (GPT-6 family; exact model variant unavailable) - Generated: Code, tests, documentation, and this description - Human review: Pending
### Background Securely initialized clusters restrict empty built-in credentials and carry a password-reset requirement on local root sessions. Deployment adapters apply configured initial credentials before starting a new FE. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Bugfix Problem Summary: Failed password changes forwarded through a follower could clear its local session restriction, and certificate-forwarded loads and UDF-preferring function resolution missed the account or statement gates. The K8S main entrypoint skipped credential preparation, while Python password-file readers disagreed with shell readers on CRLF. Enforce the missing boundaries and test the actual affected routes. ### Details - **Frequency:** Failed forwarded password updates, verified certificate bypass, same-name UDF preference, K8S startup, and CRLF credential files trigger the respective gaps. - **Root cause:** Checks were attached to individual helpers or unresolved names without accounting for downstream status, parallel entrypoints, function binding, and file-decoding contracts. - **Why existing tests missed it:** Existing tests covered successful forwarding, ordinary function resolution, helper invocation, and LF-only files. - **Fix:** Require a successful master result before releasing a reset session; gate both forwarded certificate resolvers; reject unresolved connection functions under effective UDF preference; prepare credentials in the K8S main route; preserve password-file carriage returns consistently. - **Other paths and branches:** Cover both SET PASSWORD and ALTER USER, both forwarded certificate resolvers, existing K8S metadata, explicit test mode, and both Python readers. BE execution and wire definitions are unchanged. ### Testing - Standard FE-only build completed with Checkstyle enabled and zero violations. - run-fe-ut.sh: all 16 BootstrapAuthenticationTest cases passed, including failed forwarding, effective UDF preference, and actual forwarded certificate resolver branches. - Container helper tests: all 19 passed, including K8S main dispatch, missing credentials, configured secret, test mode, and existing metadata. - Both Docker regression suites passed without regenerating results. The secure suite now uses a CRLF password file and verifies account denial, admin enablement, group commit, audit import, JOB, MTMV, statistics, and FE restart. - Local Docker validation uses cached BE 4.1.3 with be_exec_version=10 and experimental_enable_local_shuffle_planner=false for execution compatibility. Authentication remains restrictive; these settings are local fixtures only. ### Notes for Reviewers Dedicated cloud and actual mTLS deployments and a supported downgrade round trip were not executed locally. Certificate resolver tests inject a verified decision through the existing service boundary. ### Release note Bootstrap restrictions now remain in force after rejected forwarded password updates and across certificate-forwarded loads and UDF preference. K8S initialization applies configured credentials, and deployment readers handle CRLF password files consistently. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: Yes; repository design updated; website/operator release coordination remains required. - Pre-submission status: Partially verified: local checks passed; dedicated cloud/mTLS deployment and downgrade round-trip validation remain for CI/release validation. ### AI Disclosure - Tool: Codex (GPT-6 family; exact model variant unavailable) - Generated: Code, tests, documentation, and this description - Human review: Pending
…iscovery ### Background A new container may create a Doris cluster or join one discovered through cloud metadata services. The FE determines which path applies after discovering its helper nodes. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Bugfix Problem Summary: Checking local VERSION files and CLI helper arguments in the container entrypoint can reject a valid cloud FE joining an existing cluster without an initial root secret. Defer the creation-only password requirement to the FE decision before its first ROLE/VERSION files, while retaining strict initialization defaults in official images. ### Details - **Frequency:** A metadata-less cloud follower or observer starts without a root secret and discovers its existing cluster through MS. - **Root cause:** The shell cannot identify a cluster creator before cloud membership discovery completes. - **Why existing tests missed it:** Local creator and helper tests did not model MS-discovered membership or the actual K8S dispatch with read-only configuration. - **Fix:** Add immutable require_initial_root_password, defaulted by the non-secret container startup marker. Validate missing credentials only on the authoritative creator path. Configuration can override the default; explicit empty-password test mode remains supported. - **Other paths and branches:** Prepare all FE entry routes once, preserve initialized metadata and read-only ConfigMaps, and leave joining FEs free to load existing authentication metadata. ### Testing - build.sh --fe: successful FE packaging and Checkstyle. - run-fe-ut.sh: BootstrapAuthenticationTest, 17 tests passed. - Python entrypoint/helper tests: 21 tests passed. - run-regression-test.sh: both bootstrap Docker suites passed with the new FE and cached BE 4.1.3. - Native FE probes: missing initial credentials rejected before ROLE/VERSION; corrected retry initialized successfully. Two-container observer probe: a fresh observer without initial credentials joined existing metadata with the container default enabled. - Native follower probes: rejected SET PASSWORD and ALTER USER retain bootstrap restrictions; successful retries release them. ### Release note Official container entrypoints default to requiring an initial root password only when creating a new cluster. Adding a cloud FE to an existing cluster does not require a root bootstrap secret. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: Yes; repository design and threat-model documentation updated. Public configuration and deployment documentation should accompany release. - Pre-submission status: Partially verified: local non-cloud runtime coverage completed; a full cloud MS deployment was not exercised. ### AI Disclosure - Tool: Codex (GPT-6) - Generated: Code, tests, documentation, and this commit description. - Human review: Pending maintainer review.
### Background New-cluster authentication hardening needs FE policy and session handling. Deployment adapters and local end-to-end fixtures can be maintained separately from the FE change. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Refactor Problem Summary: Reduce the patch to FE authentication, its core unit tests, documentation and the explicit test-mode settings needed by existing CI environments. Remove deployment/tool changes and newly added Docker cases, keeping their validation patches and records local. ### Details - **Change map:** The final aggregate contains 25 FE production files (+636/-63), one FE unit-test file (+373), nine CI compatibility settings (+25), and two documentation files (+178). No deployment/tool code or new Docker regression suite remains in the aggregate diff. - **Moves vs. logic changes:** Restore deployment, compose, framework credential support and tools to the base. Remove the container-only initial-password requirement and its test. Retained FE sources and unit tests exactly match the previously tested 7c4235c version. - **Behavior change:** FE can start without an initial root hash and enforces local password bootstrap on new marked clusters. Container entrypoints do not introduce a separate password requirement. Existing CI environments explicitly opt into empty-password test compatibility. - **Compatibility:** Legacy cluster policy, local recovery and internal JOB/MTMV identities retain the reviewed FE behavior. Deployment tools that depend on remote empty-password access require a separate adaptation. - **Performance impact:** None in the retained FE runtime paths. - **Rollback:** Restore the prior local commits if separately pursuing deployment integration; local validation patches preserve that work. ### Testing - Compared every retained FE file against 7c4235c: all 26 files are byte-for-byte identical. - Reused the recorded build.sh --fe packaging/Checkstyle success and run-fe-ut.sh BootstrapAuthenticationTest result: 16 tests, zero failures/errors. - Checked aggregate scope and whitespace. No new build or test run was needed for this scope-only reduction. - Earlier native/Docker validation records remain local; no new Docker cases are included in the patch. ### Release note New-cluster root/admin login restrictions are enforced by FE. Deployment and tool adaptations are outside this patch. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: Yes; repository design and threat model updated. Public deployment guidance should accompany the authentication change. - Pre-submission status: Partially verified: retained FE build/unit validation reused by exact source equality; full cloud MS/mTLS/downgrade scenarios were not exercised. ### AI Disclosure - Tool: Codex (GPT-6) - Generated: Scope reduction, documentation and this description. - Human review: Pending maintainer review.
### Background Native password authentication now carries an explicit client transport and returns whether the connection must reset its password. Some callers and test mocks still used the earlier void signatures. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Refactor Problem Summary: Remove four obsolete Auth/UserManager overloads and migrate the remaining callers to the explicit transport API. Correct existing DefaultAuthenticator tests that still mocked the old five-argument method, and remove an unused direct-password mock from MysqlProtoTest. ### Details - **Moves vs. logic changes:** Remove forwarding overloads, classify COM_CHANGE_USER as direct/proxied MySQL, and migrate existing test calls. The default-authenticator success test now checks both returned password-reset states. - **Behavior change:** Authentication policy is unchanged. COM_CHANGE_USER still rejects root/admin before password verification and clears reset state only after successful authentication as another user. Existing tests now exercise the method actually called by DefaultAuthenticator. - **Compatibility:** Internal Java callers are migrated together. SQL, wire protocols, metadata formats and the FE authentication SPI are unchanged. - **Performance impact:** Removes unused wrapper methods; no new query-path work. - **Rollback:** Reverting this commit restores the previous internal overloads and call sites. ### Testing - run-fe-ut.sh: 83 tests passed across DefaultAuthenticatorTest (5), MysqlProtoTest (14), UserAuthenticationBindingTest (21), BootstrapAuthenticationTest (16) and AuthenticatorManagerTest (27); FE main and test sources compiled successfully. - fe-code-style standalone Checkstyle for fe-core: zero violations. - Whole-tree caller/reference search and git diff --check passed. No deployment changes or BE build. ### Release note None. ### Check List (For Author) - Behavior changed: No authentication policy change (see Details). - Does this need documentation: No; internal API and test cleanup. - Pre-submission status: Ready: targeted FE compilation/tests and Checkstyle passed. ### AI Disclosure - Tool: Codex (GPT-6) - Generated: Code, tests and this description. - Human review: Pending maintainer review.
CalvinKirs
requested review from
924060929,
dataroaring,
englefly,
gavinchou,
liaoxin01,
morningman,
morrySnow and
starocean999
as code owners
November 12, 2026 05:57
### Background The bootstrap authentication design was included alongside the FE implementation. The implementation and user-visible behavior are described in the PR and threat model. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Build/CI/Chore Problem Summary: Remove docs/design/cluster-bootstrap-authentication.md from this PR as requested. Keep the existing-CI empty-password compatibility settings because current regression clients use root without a password. ### Details - **Affected:** Remove the standalone design document only. - **Developer and release impact:** No runtime, test, CI configuration or authentication policy change in this commit. - **License and third-party:** No dependency or license changes. - **Rollback:** Restore the document from the preceding commit if needed. ### Testing - Confirmed that the only source-tree change is removal of the document; no remaining tracked references to its path. - No tests rerun for a documentation-only deletion. Previous 83 FE tests and Checkstyle validation remain applicable to unchanged code. ### Release note None. ### Check List (For Author) - Behavior changed: No. - Does this need documentation: No; this commit removes the standalone design document. - Pre-submission status: Ready. ### AI Disclosure - Tool: Codex (GPT-6). - Generated: This commit description. - Human review: Pending maintainer review.
8 of 10 tasks
Member
Author
|
run buildall |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Doris creates root and admin with empty passwords. Operators can set a hashed
initial_root_passwordor set passwords after startup; without either, a newly initialized cluster accepts remote empty-password logins.What problem does this PR solve?
Issue Number: None
Related PR: None
PR Type: Enhancement
Problem Summary:
Make newly initialized clusters restrict empty-password built-in accounts. Empty root can use only direct loopback MySQL connections and must set a nonempty password before normal SQL; empty admin cannot authenticate as a client until a privileged user sets its password. Existing clusters retain their authentication behavior, and internal JOB, MTMV and statistics identities remain usable.
Details
allow_empty_password_for_root_admin=falsefor explicit test compatibility; it never bypasses a configured nonempty password. Keep the existing booleanskip_localhost_auth_check=falsefor local recovery. Marked clusters exclude PROXY protocol connections from bootstrap/recovery. Reject invalid nonemptyinitial_root_passwordvalues before writing ROLE/VERSION, without exposing the value.initial_root_password.Testing
run-fe-ut.sh: 83 passed, zero failures/errors/skips: DefaultAuthenticatorTest (5), MysqlProtoTest (14), UserAuthenticationBindingTest (21), BootstrapAuthenticationTest (16), AuthenticatorManagerTest (27).build.sh --feat18a0d71b219: BUILD SUCCESS, zero Checkstyle violations. The deployed package's 25 changed production classes match the compiled classes.MySQL: manual execution record
The following are excerpts from the author's actual manual session on MySQL port
32375. Passwords are redacted; welcome banners, repeated guidance and blank prompts are omitted.1. Empty root can connect locally, but must set a password before normal SQL. The same connection becomes usable immediately after setting it.
2. Empty admin is refused. Root's old empty credential is also refused after root sets its password.
3. Root enables admin by setting its password; admin then logs in and executes SQL.
ADBC / Arrow Flight SQL: executed inputs and saved results
The actual ADBC run used a separate fixture: FE MySQL
39945, FE Flight38985, BE Flight38965; Python 3.11, ADBC Flight SQL/manager 1.13.0, PyArrow 26.0.1, cached BE4.1.3-rc02-7126cf65d96. Each probe opened a fresh ADBC connection. Result tables below are rendered from values saved inprotocol-results.json.1. Before setting passwords, Flight rejects root and admin even over loopback.
127.0.0.1:38985UNAUTHENTICATEDUNAUTHENTICATED127.0.0.1:38985UNAUTHENTICATEDUNAUTHENTICATED2. After setting root's password through local MySQL, ADBC reads both an FE result and real BE data.
The harness executed
SET PASSWORD = PASSWORD(%s)through loopback MySQL port39945, binding the generated root password.SHOW DATABASESthen succeeded on that same connection.Executed over ADBC as root through both loopback and the FE node address:
Recorded FE result:
Recorded BE result:
Recorded Arrow schema:
id: int32,label: string. The FE result was checked withcursor.fetchone() == (42,); BE rows were fetched withcursor.fetch_arrow_table(). A deliberately wrong root password was rejected withUNAUTHENTICATED; admin still could not log in with an empty password.3. Admin activation, resetting passwords to empty, restart and test compatibility were then exercised.
ALTER USER 'admin' IDENTIFIED BY '[REDACTED]'42and the same three BE rows.SET PASSWORD FOR 'admin'@'%' = PASSWORD('')UNAUTHENTICATED.SET PASSWORD = PASSWORD('')UNAUTHENTICATED. Root's nonempty password was then restored through local MySQL.42and the BE rows; empty admin remains rejected.allow_empty_password_for_root_admin=true, and restart FE42and the BE rows;skip_localhost_auth_checkremains false.ADBC fixture setup, executed client code and complete PASS log
The fixture used new metadata and one healthy BE. A temporary hashed
initial_root_passwordwas used only to register the BE; root was reset to empty, the configuration entry removed, and FE restarted before the matrix. Admin remained empty and the secure-initialization marker persisted. Both bypass settings were false, and effective HTTP authentication was true. Local settings werebe_exec_version=10,enable_local_shuffle_planner=false, BEmem_limit=2G; client proxies were removed.Actual harness command (the local harness is not included in this PR):
Executed ADBC connection helper, copied from that harness:
Saved query-result fields:
{ "feConstant": 42, "beRows": [[1, "alpha"], [2, "beta"], [3, "gamma"]], "schema": "id: int32\nlabel: string" }The table was created and populated through MySQL before the positive ADBC probes:
Actual ADBC-related output from the final successful run:
Web and validation scope
Real Chromium exercised sign-in, the Playground's JDBC-backed SQL session, the same three-row BE query and sign-out. The run also covered Cookie/CSRF authentication, SQL-session creation/closure and the legacy Web SQL API, including empty-password rejection, admin activation, resetting passwords to empty, FE restart and explicit test compatibility.
The 35-check run used the isolated fixture. A later authenticated recheck of the manually modified demo returned
UNAUTHENTICATEDbefore querying and contributed zero passed checks. The isolated fixture was stopped afterwards, its compatibility setting restored to false and temporary credentials removed.Earlier local runtime checks covered direct/follower password-reset success and rejection, admin-empty JOB/MTMV execution across restart, statistics, audit import and group commit with cached BE 4.1.3. These were not rerun after the final internal API cleanup. Full cloud MS, a real mTLS provider matrix, downgrade round trips and a fresh COM_CHANGE_USER wire exchange were not exercised.
Notes for Reviewers
Start with
BootstrapAuthPolicy,BootstrapPasswordGuardand the first-FE marker journal/replay path inEnv. Optional fields reuse an existing operation code. Existing authenticated sessions are not revoked by password changes. Marker preservation through downgrade/checkpoint/re-upgrade is not promised.Release note
New clusters restrict empty-password root to local MySQL password initialization and reject empty-password admin client logins. Configure a hashed
initial_root_passwordbefore first startup for automation, or initialize root locally and set admin's password when needed. Existing clusters retain their behavior. Tests can explicitly enableallow_empty_password_for_root_admin.Check List (For Author)
AI Disclosure