From c01f8b078c203b7a4a5325c9c310e5e8115aa03d Mon Sep 17 00:00:00 2001 From: James Date: Mon, 29 Jun 2026 18:21:23 +1000 Subject: [PATCH 1/3] docs: clarify column-level SELECT grants unsupported for MySQL/MariaDB CDC Column-scoped grants (GRANT SELECT (col1,col2)) break the capture: backfill and the per-table prerequisite check both run SELECT *, and CDC reads the binary log (global REPLICATION privilege) which carries every column regardless of grants. Point users to table-level SELECT + redaction to keep specific columns out of the pipeline. --- .../capture-connectors/MariaDB/MariaDB.md | 22 +++++++++++++++++ .../capture-connectors/MySQL/MySQL.md | 24 +++++++++++++++++++ 2 files changed, 46 insertions(+) diff --git a/site/docs/reference/Connectors/capture-connectors/MariaDB/MariaDB.md b/site/docs/reference/Connectors/capture-connectors/MariaDB/MariaDB.md index 1bd906d0617..30489a4b192 100644 --- a/site/docs/reference/Connectors/capture-connectors/MariaDB/MariaDB.md +++ b/site/docs/reference/Connectors/capture-connectors/MariaDB/MariaDB.md @@ -42,6 +42,8 @@ To meet these requirements, do the following: The `SELECT` permission can be restricted to just the tables that need to be captured, but automatic discovery requires `information_schema` access as well. +It cannot be restricted to a subset of columns within a table; see +[Column-level permissions](#column-level-permissions) for details. ```sql CREATE USER IF NOT EXISTS flow_capture IDENTIFIED BY 'secret'; @@ -105,6 +107,26 @@ However, you may find it appropriate to skip the backfill, especially for extrem In this case, you may turn off backfilling on a per-table basis. See [properties](#properties) for details. +## Column-level permissions + +The capture user's `SELECT` privilege can be limited to specific tables, but not to a +subset of columns within a table. Column-scoped grants do not restrict what the connector +captures, and they prevent it from running: + +- During backfill, the connector reads each table with `SELECT *`, which requires `SELECT` + on every column. A column-scoped grant such as `GRANT SELECT (col1, col2) ON db.tbl` + makes the backfill (and the connector's per-table validation check) fail with a + `cannot read from table` error. +- Ongoing changes are read from the binary log, not with `SELECT` queries. The binary log + is authorized by the global `REPLICATION CLIENT` and `REPLICATION SLAVE` privileges and + contains every column of a changed row, so column-level grants have no effect on what is + streamed during replication. + +Grant table-level `SELECT` (as shown in [Setup](#setup)) so the connector can read its +tables. To keep specific columns, such as sensitive fields, out of the pipeline, use +[redaction](/features/redaction.md) to drop or hash those fields at capture time so they +never land in the collection. + ## Configuration You configure connectors either in the Estuary web app, or by directly editing the catalog specification file. diff --git a/site/docs/reference/Connectors/capture-connectors/MySQL/MySQL.md b/site/docs/reference/Connectors/capture-connectors/MySQL/MySQL.md index c3fa9d0dc1f..0efafb159be 100644 --- a/site/docs/reference/Connectors/capture-connectors/MySQL/MySQL.md +++ b/site/docs/reference/Connectors/capture-connectors/MySQL/MySQL.md @@ -56,6 +56,8 @@ To meet these requirements, follow the steps for your hosting type. The `SELECT` permission can be restricted to just the tables that need to be captured, but automatic discovery requires `information_schema` access as well. +It cannot be restricted to a subset of columns within a table; see +[Column-level permissions](#column-level-permissions) for details. ```sql CREATE USER IF NOT EXISTS flow_capture @@ -151,6 +153,8 @@ CALL mysql.rds_set_configuration('binlog retention hours', 168); The `SELECT` permission can be restricted to just the tables that need to be captured, but automatic discovery requires `information_schema` access as well. +It cannot be restricted to a subset of columns within a table; see +[Column-level permissions](#column-level-permissions) for details. :::tip Your username must be specified in the format `username@servername`. @@ -213,6 +217,26 @@ However, you may find it appropriate to skip the backfill, especially for extrem In this case, you may turn off backfilling on a per-table basis. See [properties](#properties) for details. +## Column-level permissions + +The capture user's `SELECT` privilege can be limited to specific tables, but not to a +subset of columns within a table. Column-scoped grants do not restrict what the connector +captures, and they prevent it from running: + +- During backfill, the connector reads each table with `SELECT *`, which requires `SELECT` + on every column. A column-scoped grant such as `GRANT SELECT (col1, col2) ON db.tbl` + makes the backfill (and the connector's per-table validation check) fail with a + `cannot read from table` error. +- Ongoing changes are read from the binary log, not with `SELECT` queries. The binary log + is authorized by the global `REPLICATION CLIENT` and `REPLICATION SLAVE` privileges and + contains every column of a changed row, so column-level grants have no effect on what is + streamed during replication. + +Grant table-level `SELECT` (as shown in [Setup](#setup)) so the connector can read its +tables. To keep specific columns, such as sensitive fields, out of the pipeline, use +[redaction](/features/redaction.md) to drop or hash those fields at capture time so they +never land in the collection. + ## Configuration You configure connectors either in the Estuary web app, or by directly editing the catalog specification file. From 5e51a240527cc60d5b7304ee626d3cb8c1311ee6 Mon Sep 17 00:00:00 2001 From: James Date: Mon, 29 Jun 2026 21:33:24 +1000 Subject: [PATCH 2/3] docs: lead with binlog reason, rename section to 'Column-level permissions unsupported' --- .../capture-connectors/MariaDB/MariaDB.md | 19 ++++++++------- .../capture-connectors/MySQL/MySQL.md | 24 ++++++++++--------- 2 files changed, 23 insertions(+), 20 deletions(-) diff --git a/site/docs/reference/Connectors/capture-connectors/MariaDB/MariaDB.md b/site/docs/reference/Connectors/capture-connectors/MariaDB/MariaDB.md index 30489a4b192..85efd15dd78 100644 --- a/site/docs/reference/Connectors/capture-connectors/MariaDB/MariaDB.md +++ b/site/docs/reference/Connectors/capture-connectors/MariaDB/MariaDB.md @@ -42,8 +42,9 @@ To meet these requirements, do the following: The `SELECT` permission can be restricted to just the tables that need to be captured, but automatic discovery requires `information_schema` access as well. -It cannot be restricted to a subset of columns within a table; see -[Column-level permissions](#column-level-permissions) for details. +It cannot be restricted to a subset of columns within a table, because the connector +reads ongoing changes from the binary log, which always includes every column. See +[Column-level permissions unsupported](#column-level-permissions-unsupported) for details. ```sql CREATE USER IF NOT EXISTS flow_capture IDENTIFIED BY 'secret'; @@ -107,20 +108,20 @@ However, you may find it appropriate to skip the backfill, especially for extrem In this case, you may turn off backfilling on a per-table basis. See [properties](#properties) for details. -## Column-level permissions +## Column-level permissions unsupported The capture user's `SELECT` privilege can be limited to specific tables, but not to a -subset of columns within a table. Column-scoped grants do not restrict what the connector -captures, and they prevent it from running: +subset of columns within a table, because most of what the connector reads never goes +through `SELECT`: -- During backfill, the connector reads each table with `SELECT *`, which requires `SELECT` - on every column. A column-scoped grant such as `GRANT SELECT (col1, col2) ON db.tbl` - makes the backfill (and the connector's per-table validation check) fail with a - `cannot read from table` error. - Ongoing changes are read from the binary log, not with `SELECT` queries. The binary log is authorized by the global `REPLICATION CLIENT` and `REPLICATION SLAVE` privileges and contains every column of a changed row, so column-level grants have no effect on what is streamed during replication. +- The initial backfill does use `SELECT`, but reads each table with `SELECT *` (as does the + connector's per-table validation check), which requires `SELECT` on every column. A + column-scoped grant such as `GRANT SELECT (col1, col2) ON db.tbl` makes the backfill and + the validation check fail with a `cannot read from table` error. Grant table-level `SELECT` (as shown in [Setup](#setup)) so the connector can read its tables. To keep specific columns, such as sensitive fields, out of the pipeline, use diff --git a/site/docs/reference/Connectors/capture-connectors/MySQL/MySQL.md b/site/docs/reference/Connectors/capture-connectors/MySQL/MySQL.md index 0efafb159be..c9d7e1b74e3 100644 --- a/site/docs/reference/Connectors/capture-connectors/MySQL/MySQL.md +++ b/site/docs/reference/Connectors/capture-connectors/MySQL/MySQL.md @@ -56,8 +56,9 @@ To meet these requirements, follow the steps for your hosting type. The `SELECT` permission can be restricted to just the tables that need to be captured, but automatic discovery requires `information_schema` access as well. -It cannot be restricted to a subset of columns within a table; see -[Column-level permissions](#column-level-permissions) for details. +It cannot be restricted to a subset of columns within a table, because the connector +reads ongoing changes from the binary log, which always includes every column. See +[Column-level permissions unsupported](#column-level-permissions-unsupported) for details. ```sql CREATE USER IF NOT EXISTS flow_capture @@ -153,8 +154,9 @@ CALL mysql.rds_set_configuration('binlog retention hours', 168); The `SELECT` permission can be restricted to just the tables that need to be captured, but automatic discovery requires `information_schema` access as well. -It cannot be restricted to a subset of columns within a table; see -[Column-level permissions](#column-level-permissions) for details. +It cannot be restricted to a subset of columns within a table, because the connector +reads ongoing changes from the binary log, which always includes every column. See +[Column-level permissions unsupported](#column-level-permissions-unsupported) for details. :::tip Your username must be specified in the format `username@servername`. @@ -217,20 +219,20 @@ However, you may find it appropriate to skip the backfill, especially for extrem In this case, you may turn off backfilling on a per-table basis. See [properties](#properties) for details. -## Column-level permissions +## Column-level permissions unsupported The capture user's `SELECT` privilege can be limited to specific tables, but not to a -subset of columns within a table. Column-scoped grants do not restrict what the connector -captures, and they prevent it from running: +subset of columns within a table, because most of what the connector reads never goes +through `SELECT`: -- During backfill, the connector reads each table with `SELECT *`, which requires `SELECT` - on every column. A column-scoped grant such as `GRANT SELECT (col1, col2) ON db.tbl` - makes the backfill (and the connector's per-table validation check) fail with a - `cannot read from table` error. - Ongoing changes are read from the binary log, not with `SELECT` queries. The binary log is authorized by the global `REPLICATION CLIENT` and `REPLICATION SLAVE` privileges and contains every column of a changed row, so column-level grants have no effect on what is streamed during replication. +- The initial backfill does use `SELECT`, but reads each table with `SELECT *` (as does the + connector's per-table validation check), which requires `SELECT` on every column. A + column-scoped grant such as `GRANT SELECT (col1, col2) ON db.tbl` makes the backfill and + the validation check fail with a `cannot read from table` error. Grant table-level `SELECT` (as shown in [Setup](#setup)) so the connector can read its tables. To keep specific columns, such as sensitive fields, out of the pipeline, use From c673586fd53da8d3b6f739cba79393af62d89722 Mon Sep 17 00:00:00 2001 From: James Date: Fri, 3 Jul 2026 09:10:25 +1000 Subject: [PATCH 3/3] docs: fold column-level SELECT note into setup per review (aeluce) Replace the standalone 'Column-level permissions unsupported' section with a reworded permissions sentence in the setup steps: SELECT must be table-level not column-level, plus a one-line pointer to redaction for keeping columns out. --- .../capture-connectors/MariaDB/MariaDB.md | 29 ++------------ .../capture-connectors/MySQL/MySQL.md | 38 ++++--------------- 2 files changed, 12 insertions(+), 55 deletions(-) diff --git a/site/docs/reference/Connectors/capture-connectors/MariaDB/MariaDB.md b/site/docs/reference/Connectors/capture-connectors/MariaDB/MariaDB.md index 85efd15dd78..2ebe6a74cae 100644 --- a/site/docs/reference/Connectors/capture-connectors/MariaDB/MariaDB.md +++ b/site/docs/reference/Connectors/capture-connectors/MariaDB/MariaDB.md @@ -40,11 +40,10 @@ To meet these requirements, do the following: 1. Create the `flow_capture` user with replication permission, and the ability to read all tables. -The `SELECT` permission can be restricted to just the tables that need to be -captured, but automatic discovery requires `information_schema` access as well. -It cannot be restricted to a subset of columns within a table, because the connector -reads ongoing changes from the binary log, which always includes every column. See -[Column-level permissions unsupported](#column-level-permissions-unsupported) for details. +Grant `SELECT` on all tables or restrict it to the tables to be captured. `SELECT` +permissions must be at the table level, not the column level. Automatic discovery also +requires `information_schema` access. To keep specific columns, such as sensitive fields, +out of the capture, use [redaction](/features/redaction.md) rather than column-level grants. ```sql CREATE USER IF NOT EXISTS flow_capture IDENTIFIED BY 'secret'; @@ -108,26 +107,6 @@ However, you may find it appropriate to skip the backfill, especially for extrem In this case, you may turn off backfilling on a per-table basis. See [properties](#properties) for details. -## Column-level permissions unsupported - -The capture user's `SELECT` privilege can be limited to specific tables, but not to a -subset of columns within a table, because most of what the connector reads never goes -through `SELECT`: - -- Ongoing changes are read from the binary log, not with `SELECT` queries. The binary log - is authorized by the global `REPLICATION CLIENT` and `REPLICATION SLAVE` privileges and - contains every column of a changed row, so column-level grants have no effect on what is - streamed during replication. -- The initial backfill does use `SELECT`, but reads each table with `SELECT *` (as does the - connector's per-table validation check), which requires `SELECT` on every column. A - column-scoped grant such as `GRANT SELECT (col1, col2) ON db.tbl` makes the backfill and - the validation check fail with a `cannot read from table` error. - -Grant table-level `SELECT` (as shown in [Setup](#setup)) so the connector can read its -tables. To keep specific columns, such as sensitive fields, out of the pipeline, use -[redaction](/features/redaction.md) to drop or hash those fields at capture time so they -never land in the collection. - ## Configuration You configure connectors either in the Estuary web app, or by directly editing the catalog specification file. diff --git a/site/docs/reference/Connectors/capture-connectors/MySQL/MySQL.md b/site/docs/reference/Connectors/capture-connectors/MySQL/MySQL.md index c9d7e1b74e3..93eb1dae403 100644 --- a/site/docs/reference/Connectors/capture-connectors/MySQL/MySQL.md +++ b/site/docs/reference/Connectors/capture-connectors/MySQL/MySQL.md @@ -54,11 +54,10 @@ To meet these requirements, follow the steps for your hosting type. 1. Create the `flow_capture` user with replication permission, and the ability to read all tables. -The `SELECT` permission can be restricted to just the tables that need to be -captured, but automatic discovery requires `information_schema` access as well. -It cannot be restricted to a subset of columns within a table, because the connector -reads ongoing changes from the binary log, which always includes every column. See -[Column-level permissions unsupported](#column-level-permissions-unsupported) for details. +Grant `SELECT` on all tables or restrict it to the tables to be captured. `SELECT` +permissions must be at the table level, not the column level. Automatic discovery also +requires `information_schema` access. To keep specific columns, such as sensitive fields, +out of the capture, use [redaction](/features/redaction.md) rather than column-level grants. ```sql CREATE USER IF NOT EXISTS flow_capture @@ -152,11 +151,10 @@ CALL mysql.rds_set_configuration('binlog retention hours', 168); 3. Using [MySQL workbench](https://docs.microsoft.com/en-us/azure/mysql/single-server/connect-workbench) or your preferred client, create the `flow_capture` user with replication permission, and the ability to read all tables. -The `SELECT` permission can be restricted to just the tables that need to be -captured, but automatic discovery requires `information_schema` access as well. -It cannot be restricted to a subset of columns within a table, because the connector -reads ongoing changes from the binary log, which always includes every column. See -[Column-level permissions unsupported](#column-level-permissions-unsupported) for details. +Grant `SELECT` on all tables or restrict it to the tables to be captured. `SELECT` +permissions must be at the table level, not the column level. Automatic discovery also +requires `information_schema` access. To keep specific columns, such as sensitive fields, +out of the capture, use [redaction](/features/redaction.md) rather than column-level grants. :::tip Your username must be specified in the format `username@servername`. @@ -219,26 +217,6 @@ However, you may find it appropriate to skip the backfill, especially for extrem In this case, you may turn off backfilling on a per-table basis. See [properties](#properties) for details. -## Column-level permissions unsupported - -The capture user's `SELECT` privilege can be limited to specific tables, but not to a -subset of columns within a table, because most of what the connector reads never goes -through `SELECT`: - -- Ongoing changes are read from the binary log, not with `SELECT` queries. The binary log - is authorized by the global `REPLICATION CLIENT` and `REPLICATION SLAVE` privileges and - contains every column of a changed row, so column-level grants have no effect on what is - streamed during replication. -- The initial backfill does use `SELECT`, but reads each table with `SELECT *` (as does the - connector's per-table validation check), which requires `SELECT` on every column. A - column-scoped grant such as `GRANT SELECT (col1, col2) ON db.tbl` makes the backfill and - the validation check fail with a `cannot read from table` error. - -Grant table-level `SELECT` (as shown in [Setup](#setup)) so the connector can read its -tables. To keep specific columns, such as sensitive fields, out of the pipeline, use -[redaction](/features/redaction.md) to drop or hash those fields at capture time so they -never land in the collection. - ## Configuration You configure connectors either in the Estuary web app, or by directly editing the catalog specification file.