Fine tune read/write locks when locking tables#5404
Fine tune read/write locks when locking tables#5404melton-jason wants to merge 5 commits intomainfrom
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR refines the table‐locking mechanism to use separate read and write locks during the autonumbering process, reducing unnecessary full‐table write locks on shared tables.
- lock_tables now accepts distinct read/write sets and constructs the LOCK TABLES SQL accordingly.
- do_autonumbering is updated to only write‐lock the target table and read‐lock all others.
- tree_extras helper functions have been cleaned up and replaced with metadata‐based checks.
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| specifyweb/specify/tree_extras.py | Removed unused imports, replaced regex checks with db_table suffix logic, added is_treetable |
| specifyweb/specify/lock_tables.py | Updated lock_tables signature to (read_tables, write_tables), built separate read/write clauses |
| specifyweb/specify/autonumbering.py | Switched to new lock_tables API, renamed helpers and added get_tables_to_read_lock and get_tree_tables_to_lock |
Comments suppressed due to low confidence (2)
specifyweb/specify/autonumbering.py:47
- [nitpick] The variable name
writing_tois inconsistent with the function parameterwrite_tables. Consider renaming it towrite_tablesfor clarity and consistency.
writing_to = set([obj._meta.db_table])
specifyweb/specify/lock_tables.py:12
- The new read/write locking API has several edge cases (no write tables, no read tables, overlapping sets). Please add unit or integration tests to verify correct SQL generation and behavior across all combinations.
def lock_tables(read_tables: Set[str], write_tables: Set[str]):
| write_statement = ','.join( | ||
| [table + ' write' for table in write_tables]) + ',' if len(write_tables) > 0 else '' | ||
| read_statement = ','.join( | ||
| [table + ' read' for table in final_read_tables]) |
There was a problem hiding this comment.
When write_tables is non-empty and read_tables is empty, write_statement ends with a trailing comma and read_statement is empty, resulting in invalid SQL (e.g., LOCK TABLES t1 write,;). Handle the empty-read case explicitly or strip off the extra comma before executing.
| [table + ' read' for table in final_read_tables]) | |
| [table + ' read' for table in final_read_tables]) | |
| # Remove trailing comma from write_statement if read_statement is empty | |
| if not read_statement: | |
| write_statement = write_statement.rstrip(',') |
| @@ -86,3 +94,6 @@ def get_tables_from_field_path(model: str, field_path: str) -> list[str]: | |||
| table = datamodel.get_table_strict(other_model) | |||
|
|
|||
| return tables | |||
|
|
|||
| def get_tree_tables_to_lock(tree_table: str) -> set[str]: | |||
There was a problem hiding this comment.
[nitpick] Hardcoding related table names (f'{tree_table}def', f'{tree_table}defitem') may break if conventions evolve. Consider deriving these names from Django model metadata or a centralized utility.
|
Superseded by #7671 |
Addresses part of #5337
Fixes #4148
From #5337 (comment)
Now when autonumbering, Specify will only issue a write lock on the table which will be updated, read-locking all other tables. This will allow other users of Specify to read from common tables (collection, discipline, etc.) while some other user is in the process of autonumbering one or more records.
Previously, these tables would be inaccessible to other users until the autonumbering process completed. This created a massive problem when using the WorkBench and relying on autonumbering: a WorkBench validation or upload could take a significant amount of time to complete, meaning Specify would not be usable for other users.
In the future, we can extend the capability further and leverage Row Locks for even more fine-tuned control for some tables.
Checklist
and self-explanatory (or properly documented)
Testing instructions
Coming Soon