Skip to content

[DBA-141] Add field validations on datasource adding#81

Merged
gasparian merged 11 commits intomainfrom
simon.karan/DBA-141-field-validations-on-datasource-adding
Mar 30, 2026
Merged

[DBA-141] Add field validations on datasource adding#81
gasparian merged 11 commits intomainfrom
simon.karan/DBA-141-field-validations-on-datasource-adding

Conversation

@SimonKaran13
Copy link
Copy Markdown
Member

@SimonKaran13 SimonKaran13 commented Mar 26, 2026

Summary

Datasource names with spaces and special characters were accepted in the form but caused errors after submission. This adds frontend validation so users get clear, immediate feedback.

validation-screenshot

Changes

Shared validation module

New validate_datasource_name() function enforcing name rules:

  • Folder segments (before the last /): alphanumeric, hyphens, underscores, dots
  • Source name (last segment): must start with a letter and contain only letters, digits, and underscores (matches the agent's SQL-safe constraint)

Also includes validate_port() (1–65535 range) and validate_hostname() (RFC 952/1123 + IPv4/IPv6 via stdlib ipaddress).

Files
  • src/databao_cli/features/datasource/validation.py

UI validation

Validate name before add/verify actions; check required config fields, port ranges, hostname format, and union property variants. The "Verify connection" button now also runs config field validation before attempting verification.

Files
  • src/databao_cli/features/ui/components/datasource_manager.py
  • src/databao_cli/features/ui/components/datasource_form.py

CLI validation

Validation loop in the interactive CLI workflow.

Files
  • src/databao_cli/workflows/datasource/add.py

Tests

Unit tests covering the validation module and validate_config_fields() (int vs port split, union properties, top-level key scoping, hostname-with-port rejection).

Files
  • tests/test_datasource_validation.py
  • tests/test_datasource_form_validation.py

Test Plan

  • make check passes
  • make test passes (140 tests)
  • UI tested with Playwright: spaces, special chars, empty name show correct errors

SimonKaran13 and others added 2 commits March 26, 2026 13:43
Add frontend validation for datasource names in both the Streamlit UI
and CLI workflow. Names with spaces, special characters, and other
invalid patterns are rejected with clear error messages before
submission instead of failing on the backend.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@SimonKaran13 SimonKaran13 self-assigned this Mar 26, 2026
SimonKaran13 and others added 2 commits March 26, 2026 14:03
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use the agent's source name pattern (^[A-Za-z][A-Za-z0-9_]*$) for the
last segment so names work unquoted in SQL. Add validate_port() and
validate_hostname() validators, applied automatically to config fields
based on their key name and type.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@gasparian gasparian self-requested a review March 26, 2026 17:10
@gasparian gasparian marked this pull request as ready for review March 27, 2026 10:16
@gasparian gasparian requested a review from a team March 27, 2026 10:16
@SimonKaran13 SimonKaran13 requested a review from Copilot March 27, 2026 10:17
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds shared datasource field validation to prevent invalid datasource names/config values from being accepted in the Streamlit UI and interactive CLI, improving feedback before submission and aligning test fixtures with the stricter naming rules.

Changes:

  • Introduces a shared validation module for datasource names plus host/port validation helpers.
  • Adds pre-submit validation in Streamlit datasource management and in the interactive CLI add workflow.
  • Adds unit tests for the shared validation module and updates e2e test fixtures to use underscore-based datasource names/ids.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/databao_cli/features/datasource/validation.py New shared validation helpers for datasource name, hostname, and port.
src/databao_cli/workflows/datasource/add.py Loops until a valid datasource name is entered in the CLI workflow.
src/databao_cli/features/ui/components/datasource_manager.py Validates datasource name/type before add/verify; validates required config fields before add.
src/databao_cli/features/ui/components/datasource_form.py Adds config-field validation (required fields + host/port checks).
tests/test_datasource_validation.py New unit tests for the shared validation module.
e2e-tests/tests/resources/sqlite_introspections.yaml Updates datasource id/comment to match underscore naming.
e2e-tests/tests/resources/duckdb_introspections.yaml Updates datasource id/comment to match underscore naming.
e2e-tests/src/databases/sqlite_utils.py Updates default sqlite datasource name to underscore form.
e2e-tests/src/databases/duckdb_utils.py Updates default duckdb datasource name to underscore form.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown
Contributor

@gasparian gasparian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "Save" path for existing datasources (datasource_manager.py:196-208) doesn't run any validation before persisting. A user could edit a datasource, clear a required field or type garbage into the port/host inputs, hit Save, and the invalid config gets written to disk without complaint. Seems like validate_config_fields() should be called here too, same as the "Add datasource" flow does.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

SimonKaran13 and others added 2 commits March 30, 2026 10:38
…fy path

- Use stdlib ipaddress for IP validation instead of colon heuristic
- Catch all whitespace (tabs, newlines, NBSP) not just spaces
- Split int parsing from port range validation (only port-named fields)
- Validate union property variants recursively
- Apply SKIP_TOP_LEVEL_KEYS only at top level, not nested
- Run config field validation on "Verify connection" path
- Update PR description to match actual name constraints
- Add tests for all changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings March 30, 2026 08:53
@SimonKaran13
Copy link
Copy Markdown
Member Author

SimonKaran13 commented Mar 30, 2026

@gasparian Addressed in 35eb85f — the "Save" path for existing datasources now runs validate_config_fields() before persisting, same as the "Add datasource" and "Verify connection" paths.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 30, 2026 09:39
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

SimonKaran13 and others added 2 commits March 30, 2026 14:11
…ant union fallback

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings March 30, 2026 12:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

@SimonKaran13 SimonKaran13 requested a review from gasparian March 30, 2026 13:13
@gasparian gasparian merged commit cecae36 into main Mar 30, 2026
10 checks passed
@gasparian gasparian deleted the simon.karan/DBA-141-field-validations-on-datasource-adding branch March 30, 2026 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants