fix: forward unknown settings for CHANGEABLE_IN_READONLY (#530)

[sankalpsthakur]

Summary

Fixes #530

Custom settings declared CHANGEABLE_IN_READONLY on a role (for example SQL_RO_my_rls_key used with row policies / getSetting) are often absent from system.settings for the restricted user. The client previously treated any missing or readonly setting as invalid and raised:

ProgrammingError: Setting SQL_RO_my_rls_key is unknown or readonly

when the same setting was passed via query(..., settings={...}). Embedding the setting in the SQL SETTINGS clause worked, because that path never goes through client validation.

This change forwards settings that are not present in system.settings for the current user to ClickHouse and lets the server accept or reject them. Known readonly settings still honor invalid_setting_action (error / drop / send).

This matches the maintainer suggestion on #530 that fine-grained discovery of CHANGEABLE_IN_READONLY is not practical for the client without extra privileges.

Test plan

• Unit tests in tests/unit_tests/test_driver/test_validate_settings.py covering unknown custom settings, known writable, known readonly (error/drop/send), matching readonly skip, optional transport drop, and transport pass-through. Confirmed the main unknown-setting case fails under the previous validation logic and passes with this change.
• Integration test test_changeable_in_readonly_custom_setting reproduces the issue setup (role with CHANGEABLE_IN_READONLY, readonly user, row policy) for both sync and async clients via param_client / call. Skips when the server lacks SQL_ custom settings or when running against cloud.
• ruff check / ruff format on touched files
• CI integration suite (requires Docker ClickHouse with custom_settings_prefixes=SQL_, as in this repo's docker-compose config)

Checklist

• Unit and integration tests covering the common scenarios were added
• A human-readable description of the changes was provided to include in CHANGELOG

[CLAassistant]

All committers have signed the CLA.

[joe-clickhouse]

Hey thanks @sankalpsthakur! This is good. I kept the core approach. It makes sense, as kinda discussed in #530 that the client can't discover CHANGEABLE_IN_READONLY settings and should just forward whatever it can't validate. I did push a few changes on top, though. Hope you don't mind!

1. I preserved invalid_setting_action='drop'. Your version forwarded unknown settings under every action. That removed a way for people to keep one settings dict portable across server versions, and our SQLAlchemy tests rely on it. Now, unknown settings forward under error and send but still drop under drop.

2. I added a guard for reserved request parameters. Once unknown settings forward by default, names like query, user, password, default_format, and the param_ bound-parameter namespace would be sent into the HTTP request where the server treats them as something other than settings. That can corrupt the query or a bound parameter. Those names now raise a client side ProgrammingError instead. This also restores the pre-change behavior of rejecting them under the default action.

3. I fixed the command request ordering. plan_command_request applied settings last. This would mean a forwarded setting could override a bound parameter or even external-data metadata. It now applies settings before the structural params which matches plan_query_request.

4. I made some test chnages. The integration tests failed as originally written. so I cleaned them up a bit and added some other safety guards, etc.