Repository navigation
Key schema definition checks for CreateTable and UpdateTable - #595
Open
LeeroyHannigan wants to merge 4 commits into
Open
LeeroyHannigan wants to merge 4 commits into
LeeroyHannigan wants to merge 4 commits into
Conversation
LeeroyHannigan
requested review from
amrith,
c33howard,
jcshepherd,
pdf-amzn,
robinnsc and
yesyayen
as code owners
October 26, 2026 03:34
LeeroyHannigan
force-pushed
the
fix/517-519-520-stream-update-table-sentences
branch
from
October 30, 2026 09:07
8e3d8f0 to
6eff502
Compare
CreateTable and an UpdateTable GSI Create refused an undefined key attribute with one sentence that named only the first missing key. Amazon DynamoDB (measured 2026-10-25 and 2026-10-26 at 01:30 and 01:36 UTC, each request twice) applies these rules: - A name defined twice in AttributeDefinitions answers `Attribute Name is duplicated: <name>`, naming the first name whose second definition comes first, before any key check. ExtendDB accepted duplicates. - The table's key schema is checked first, then each LSI's, then each GSI's. A schema with more keys than there are definitions answers `Invalid KeySchema: Some index key attribute have no definition`; otherwise a key with no definition answers `Some index key attributes are not defined in AttributeDefinitions`, listing every key of that schema and every definition. The two checks live in the new validation/key_definitions.rs and both operations call them. The service orders the two lists in a way that follows neither the request nor the alphabet; ExtendDB lists them in request order. The UpdateTable unit test row that expected the listing sentence for one definition and two keys now expects the key schema sentence, as measured.
Dual-target wire tests from the 2026-10-25 and 2026-10-26 measurements: UpdateTable GSI Create with a missing hash or range key, both keys, fewer definitions than keys and names defined twice; CreateTable with missing keys on the table, an LSI and GSIs in each order, fewer definitions than keys and names defined twice; and an UpdateTable that redefines the table's own key with another type, which keeps the stored type. The lists in the sentence are compared as sets, since the service's order is not the request's.
…order documented Amazon DynamoDB reports a CreateTable billing fault (throughput with PAY_PER_REQUEST, or none with PROVISIONED) before a duplicated or missing definition (measured 2026-10-26 at 02:27 and 02:34 UTC, each case twice). The billing check now runs before the definition checks. The duplicate check compared every definition with every earlier one, and UpdateTable runs it once per GSI Create: 8,000 definitions with 200 Creates took 22.9 s. It now makes one pass with a set of seen names and still names the first name whose second definition comes first. The module comment states where the service checks a duplicate (after the billing checks, the one-name-twice rule and the length rule) and where ExtendDB does (after the key schema shape checks). docs/differences-from-dynamodb.md records the order of the two lists in the listing sentence.
CreateTable with PAY_PER_REQUEST and throughput beside a duplicated name, a missing key, or fewer definitions than keys answers the billing sentence (measured 2026-10-26 at 02:27 and 02:34 UTC). The redefinition test waits for its new index to be active before the table is deleted, since the service refuses DeleteTable while an index is being created, and writes an item with the stored key type. The measurement times in the docstring are corrected to 01:27 and 01:29 UTC, plus the two later runs.
LeeroyHannigan
force-pushed
the
fix/index-key-definition-sentences
branch
from
October 31, 2026 22:28
ea5045e to
2bb47d6
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #554. Fixes #555. Stacked on #548, which owns the UpdateTable sentences this changes.
CreateTable and an UpdateTable GSI
Createnow checkAttributeDefinitionsagainst each key schema with DynamoDB's rules and sentences, and refuse a name defined twice.What the service answers
Measured on 2026-10-25 and on 2026-10-26 at 01:27, 01:29, 02:27 and 02:34 UTC in us-east-1, each request twice with the same answer. No request created or changed a table: every CreateTable named an existing table, and every UpdateTable carried a key with no definition or a duplicate the service refuses.
Attribute Name is duplicated: <name>, naming the first name whose second definition comes first ([b, a, b, a]namesb). It comes before a missing key and applies to two definitions of one name with different types. The service checks it after the rule that one attribute cannot be both HASH and RANGE and after the length rule below, and before the other key schema checks.Invalid KeySchema: Some index key attribute have no definition. This is the rule behind fix(streams, update-table): CreationRequestDateTime; measured UpdateTable sentences and validation order #548's empty-member row and the "two or more missing keys" row of The Keys and AttributeDefinitions lists in the undefined-index-key sentence differ from the service #554: definitions[pk]for keysg,ranswer it, while[pk, z]for the same keys answer the listing sentence below.One or more parameter values were invalid: Some index key attributes are not defined in AttributeDefinitions. Keys: [...], AttributeDefinitions: [...], with every key of that schema and every definition.Createis accepted, and the table keeps its type (UpdateTable accepts a redefined table key type and a duplicated AttributeDefinitions entry #555's first row).The service orders the two lists in a way that is neither the request's nor alphabetical and does not depend on the attribute types:
[pk, g]renders[pk, g],[g, pk]renders[g, pk],[pk, r]renders[r, pk], and keyshk,rkrender[rk, hk]. ExtendDB lists keys in key schema order and definitions in request order.What changed
crates/core/src/validation/key_definitions.rs(new):validate_unique_attribute_definitionsandvalidate_key_schema_defined, with unit tests.crates/core/src/validation/mod.rs: CreateTable calls them for the table, each LSI and each GSI in that order, in place of the loop that named the first missing key of all schemas together. The unusedformat_attr_defsgoes. The table billing check now runs before the definition checks, so a billing fault is reported first as on the service; before this change a billing fault was reported after a missing key.crates/engine/src/update_table.rs: the GSICreatebranch keeps fix(streams, update-table): CreationRequestDateTime; measured UpdateTable sentences and validation order #548's sentence for an absent member and calls the two checks for the rest.crates/engine/src/update_table_tests.rs: fix(streams, update-table): CreationRequestDateTime; measured UpdateTable sentences and validation order #548's row for one definition and two keys expected the listing sentence; it now expects the key schema sentence, as measured. fix(streams, update-table): CreationRequestDateTime; measured UpdateTable sentences and validation order #548's row for one undefined name and two keys is removed, since it is the same case, and two rows are added (a listing with every key, a duplicate).docs/differences-from-dynamodb.md: a row for the order of the two lists.The redefinition row needed no code change: the backends already keep the stored definition of a name the table has.
Related behaviour
Measured differences that exist on #548's head and on
mainas well:Tests
tests/test_key_schema_definitions.py: 31 dual-target wire tests from the measurements. The lists in the listing sentence are compared as sets. The redefinition test waits for its index to be active before the table is deleted.Run locally at the head
2bb47d61b, on a SQLite server and a PostgreSQL server built from it:6eff5029f: 27 failed, 4 passed. The 4 that pass there are the one-key UpdateTable listing ([g]against[pk]), the one-key CreateTable listing ([pk]against[x]), the billing fault beside a duplicate (fix(streams, update-table): CreationRequestDateTime; measured UpdateTable sentences and validation order #548 has no duplicate check) and the redefinition row.tests/matchingupdate_table,create_table,table_operations,gsi,lsi,throughputorbilling, with the new file, and the sixtests/pythonfiles matchingtable,gsi,lsi,indexorstream: 163 passed on each backend. The 5 tests oftests/test_gsi_async_queue.pyneed a PostgreSQL connection string and stop at setup on SQLite; the 5 oftests/test_gsi_async_queue_sqlite.pyskip on PostgreSQL.tests/test_gsi_async.pycalls the settings CLI at./target/release/extenddb, which these runs provided.cargo +1.97.0 fmt --all --check,cargo +1.97.0 clippy --locked --workspace --all-targets -- -D warnings,cargo +1.97.0 check --locked --workspace --all-targets,cargo +1.88.0 check --locked --workspace, and theextenddb-coreandextenddb-engineunit tests (698 and 185 passed).CI runs the new file in
run-integration,run-integration-sqlite,mongodb-pytestandcassandra-pytest.Files
crates/core/src/validation/mod.rsgoes from 3,803 to 3,802 lines andcrates/engine/src/update_table.rsfrom 384 to 355. The new checks are in the 169-linekey_definitions.rs.Rebase on #548's head
Rebased onto #548's head
6eff5029f, which now sits onmain817b2405d. The two copies of #548's commits this branch carried are dropped; they differ from #548's own commits only by the author address and by context linesmainchanged. The four commits of this branch apply without conflicts and their patches are unchanged.Between the two bases,
mainchanged one code file this branch touches. #401 replaced the online index limit sentence invalidate_vector_index_updateswith theONLINE_INDEX_LIMITconstant, incrates/core/src/validation/mod.rs, about 360 lines above the hunk this branch changes there. That function runs on UpdateTable's vector index path, before the GSI checks, as "Related behaviour" says, and it adds no check to CreateTable. #522 added two rows to the tag operations table ofdocs/differences-from-dynamodb.md; this branch's row is in the Control Plane table above it. The order of the checks described above holds on CreateTable and on the UpdateTable GSICreatebranch after the rebase.CI passed all 29 checks on
2bb47d61b.Merging
This is based on #548's branch. After #548 merges, retarget it to
main.git merge-tree --write-treeof the head against the 33 open PRs that changevalidation/mod.rs,update_table.rs,update_table_tests.rsordocs/differences-from-dynamodb.md, compared with the same merge of #548's head, adds conflicts with eight. #481, #485 and #700 each add apub useon the line this adds at the top ofvalidation/mod.rs; keep both lines. #399, #507, #515, #531 and #550 each add a row to the Control Plane table ofdocs/differences-from-dynamodb.mdafter the same line as this change's row; keep both rows. The other conflicts are the same with or without this change; #474 movesvalidate_gsi_updatesout ofupdate_table.rs, so it carries the new GSICreatebranch to its new place.