Bring changes from `v7.11.4` to `main` by melton-jason · Pull Request #7683 · specify/specify7

melton-jason · 2026-02-04T19:34:31Z

Brings the changes from #7671 / v7.11.4 to main.

Checklist

Self-review the PR after opening it to make sure the changes look good and
self-explanatory (or properly documented)
Add relevant issue to release milestone
Add pr to documentation list
Add automated tests

Testing instructions

Follow the testing instructions from Prevent broken transactions when autonumbering (#7671) (provided below) and ensure the functionality is correct with the updated branch:

Sample Database

To speed up setup for testing this Pull Request, i've set up a testing database that contains all resources required for testing:
Feel free to use the database!

User - spadmin
Password - testuser

Collections:

Fish

In the SCC Division

In the Ichthyology Discipline

Catalog Number of the format FSH-#########

One CollectionObject WorkBench Data Set

One Accession WorkBench Data Set

Plants

In the SCC Division

In the Botany Discipline

Catalog Number of the format #########

CO -> text2 of the format AAA-NNN-#########

Where A can be any letter and N can be any number

Locality -> text1 of the format #########

Two Collection Object WorkBench Data Sets

One Locality Data Set

Vascular Plants

In the SCC Division

In the Botany Discipline

CO -> text2 of the format AAA-NNN-#########

Where A can be any letter and N can be any number

Locality -> text1 of the format #########

One Collection Object WorkBench Data Set

Mammals

In the Test Division

In the Mammology Discipline

One Accession WorkBench Data Set

Where Accessions are scoped to Division:

issue-6490_scoped_accession_testing.sql.zip

Where Accessions are globally scoped:

issue-6490_global_accession_testing.sql.zip

General Concurrency with WorkBench

Start a WorkBench Upload or Validation operation on a sufficiently large Data Set (the operation needs to be in process while the below steps of General Concurrency are completed)
Open Specify in a new tab, window, or browser
Open a DataEntry form of the same table as the base table as the WorkBench Data Set, where one or fields have an autonumbering field format
Save the Data Entry record and ensure the record saves successfully
Reload the page and ensure Specify loads and still remains accessible

Concurrent Autonumbering with the WorkBench

Start a WorkBench Upload operation on a sufficiently large Data Set (the operation needs to be in process while some of the below steps of Concurrent Autonumbering with the WorkBench are completed) that contains two or more fields that will be autonumbered
Open Specify in a new tab, window, or browser
Open a DataEntry form of the same table as the base table as the WorkBench Data Set, where one or fields have an autonumbering field format
Save the Data Entry record and ensure the record saves successfully
Wait for the WorkBench Upload operation to complete
Ensure that the value for the autonumbered field from Data Entry is skipped in the WorkBench upload results.

Concurrent Autonumbering with Non Collection Scoping
Review Table Scoping Hiearchy for an overview on table scopes

With the Sample Database, this would be using the Locality Data Set in the Plants Collection with creating a new Locality in the Vascular Plants Collection

Start a WorkBench Upload operation on a sufficiently large Data Set (the operation needs to be in process while some of the below steps of Concurrent Autonumbering with Non Collection Scoping are completed) where the base table is not scoped to Collection that contains one or more fields that will be autonumbered
Open Specify in a new tab, window, or browser
Switch to a different Collection which is in the same scope as the records being uploaded
Open a DataEntry form of the same table as the base table as the WorkBench Data Set where one or more fields have an autonumbering field format
Save the Data Entry record and ensure the record saves successfully
Wait for the WorkBench Upload operation to complete
Ensure that the value for the autonumbered field from Data Entry is skipped in the WorkBench upload results.

Autonumbering with Variable Table Scoping

In a database with more than one Division where Accessions are scoped to Division

Start a WorkBench Upload operation on a sufficiently large Accession Data Set (the operation needs to be in process while some of the below steps of Concurrent Autonumbering with Non Collection Scoping are completed) where a field on Accession is being autonumbered
Open Specify in a new tab, window, or browser
Switch to a different Collection which is in a different Division as the records being uploaded
Open an Accession DataEntry form
Save the Accession and ensure the record saves successfully
Wait for the WorkBench Upload operation to complete
Ensure that the Accessions in different Divisions are independently auto-numbered (e.g., Accessions in different Divisions should not share the same autonumbering)

In a database with more than one Division where Accessions are globally scoped

Start a WorkBench Upload operation on a sufficiently large Accession Data Set (the operation needs to be in process while some of the below steps of Concurrent Autonumbering with Non Collection Scoping are completed) where a field on Accession is being autonumbered
Open Specify in a new tab, window, or browser
Switch to a different Collection which is in a different Division as the records being uploaded
Open an Accession DataEntry form
Save the Accession and ensure the record saves successfully
Wait for the WorkBench Upload operation to complete
Ensure that the value for the autonumbered field from Data Entry is skipped in the WorkBench upload results in the other Division
Generally ensure that autonumbering respects the scoping of Accessions (e.g, using DataEntry, create Accessions in different scopes and ensure they're autonumbered as expected)

General

Please also do general testing focused around autonumbering, especially with concurrent WorkBench uploads!

… when building a case statement for the QB.

…lumn

See #6490, #5337 Replaces #5404 Fixes #4148, #7560 This branch is largely the application of #7455 on `v7.11.3`

Initially from de6d9ec in #7399

Based largely off the initial implementation at 5ed701a from #7399

Addresses #5337 Fixes #4148, #7560

…scope model

emenslin

Looks good! I did run into some migration problems, but since that could be a test panel issue I will approve this, but it is something that should probably be looked into.

bhumikaguptaa

Everything worked as expected! I also came across a few migration errors while testing for the accession auto-numbering.

Iwantexpresso

I have tested the indecent division and found out that the independent autonomy based on divisions is not consistent

Screen.Recording.2026-02-10.at.4.46.47.PM.mov

this is a fresh version of the db and also there has been a weird crash in which it considered an already existing accession number as already existing even though they were from different divisions

trimmed.error.mov

here is the crash report if it give any insigne to the room of this problem
Edit: I have found out that this is happening after you attempt to autonumber into the test div twice in a row

Specify 7 Crash Report - 2026-02-10T22_58_30.487Z.txt

See related (and first) #7683

melton-jason · 2026-02-12T18:27:07Z

Thank you all for the reviews!

There some very subtle behavior going on behind the scenes to cause that behavior in your review @Iwantexpresso!

Ultimately, what's happening is tied to the fact that you can really only be logged into one collection as one user at a time for Specify across all tabs. That is, the Collection and User you are logged in as is shared across all tabs of Specify for that instance.
You can observe this behavior by opening two tabs of Specify, switching Collections in one, and refreshing the other. The Collection will appear to be "automatically" be switched to the other Collection, but the actual switching of Collections happened as soon as any tab of the Specify instance switches collection.

v7.11.4-prerelease_switch_collection.mov

So when you switch collections in one tab and navigate back to an older tab that is in some other collection, Specify will treat requests as having come from the collection of the prior (newer) tab.

Behind the scenes, Specify "globally" (with the perspective of the browser) manages which collection the user is signed in to via a Cookie, which is tab independent.

specify7/specifyweb/backend/context/middleware.py

Line 32 in 0fed5bb

collection_id = int(request.COOKIES.get('collection', ''))

(This Cookie is set whenever the user explicitly chooses a Collection to login to. There's a backend helper function called set_collection_cookie you can look for usages for if you're curious about how this is implemented in the code!)

You can actually see which Collection you are "logged in to" by using your browser's developer tools. In the below demonstration of the bug, I inspect the Cookies present on the request made to the backend. Even though I appear to be in two separate Collections, because I last logged in to the Fish Collection (CollectionID = 32769) all request made from that point will also be made from the Fish Collection:

v7.11.4-prerelease_bts.mov

For more information, you can view the browsers docs on how to use their Developer Tools to inspect Cookies:
Chrome Dev Tools
Firefox Dev Tools

The general "expected" behavior is to explicitly switch Collections whenever you are dealing with more than one Collection:

v7.11.4-prerelease_expected.mov

There are probably a lot of nasty bugs that could arise if you expect to be logged into more than one Collection at a time for a Specify instance...

I will note, however, that this specific bug does not happen on main:

v7.11.4-prerelease_main.mov

The main difference between this branch and main is that we are storing the highest autonumbering value for the Collection that performed the autonumbering for 5 seconds. Because you are technically making the request within the same Collection and saving the other Accession within 5 seconds, autonumbering sees the first Accession Number and thinks it is in the same scope.

@specify/ux-testing
To summarize, the bug described in @Iwantexpresso review #7683 (review) occurs when:

You have a tab of Specify that is logged into some Collection
In another tab of the same instance you explicitly switch Collections to some other Collection in a different scope
In one of the two tabs you create a record that is autonumbered
Within 5 seconds, without changing Collections you create a record of the same table in the other tab that is also autonumbered

I can try and modify the Autonumbering code to first prioritize the scoping of the record being saved (i.e., use the field values of the record being saved) rather than the Collection the user is logged into (and fallback to the latter if the former can not be easily determined), though this may result in other subtle bugs because "easy to determine" is itself pretty difficult to determine 😅.

How often do you think this will happen in practice (a production instance)?

emenslin · 2026-02-12T19:17:32Z

You have a tab of Specify that is logged into some Collection

In another tab of the same instance you explicitly switch Collections to some other Collection in a different scope

In one of the two tabs you create a record that is autonumbered

Within 5 seconds, without changing Collections you create a record of the same table in the other tab that is also autonumbered

I can try and modify the Autonumbering code to first prioritize the scoping of the record being saved (i.e., use the field values of the record being saved) rather than the Collection the user is logged into (and fallback to the latter if the former can not be easily determined), though this may result in other subtle bugs because "easy to determine" is itself pretty difficult to determine 😅.

How often do you think this will happen in practice (a production instance)?

With all those points in mind I would imagine it is very unlikely for a user to run into this, and if fixing this one issue would likely cause other bugs I think it is probably not worth it to fix it, at least right now. I'd like to hear other opinions, especially @grantfitzsimmons, but personally I would think it's unlikely enough that it's not worth fixing.

grantfitzsimmons · 2026-02-12T19:23:36Z

I agree with @emenslin, this is an edge case that is likely not to be encountered in production.

melton-jason · 2026-02-12T19:49:21Z

I would generally agree it is probably not likely to happen, but maybe that is just a result of being set in the Specify Way of Things.
Ideally and at a higher level, this edge case would be resolved if we ever do add support for being logged into more than one Collection at a given time, but that is probably not on the table any time soon.

We might be introducing much more prevalent bugs if this edge case is explicitly fixed.

I could decrease the threshold in which Specify will forget the highest autonumbering value in the same Scope relative to the Collection they're logged into (e.g., less than 5 seconds) to help mitigate this Issue, but it still has to be sufficiently long enough for instances (generally regardless of performance) to handle the genuine case of concurrent autonumbering with the WorkBench and Data Entry.

At the moment it seems like general sentiment is that the edge case is not going to be addressed, at least in the v7.12.0 release.

grantfitzsimmons · 2026-02-12T20:25:08Z

@melton-jason @Iwantexpresso #7695

Iwantexpresso · 2026-02-16T15:40:19Z

Thank you all for the reviews!

There some very subtle behavior going on behind the scenes to cause that behavior in your review @Iwantexpresso!

Ultimately, what's happening is tied to the fact that you can really only be logged into one collection as one user at a time for Specify across all tabs. That is, the Collection and User you are logged in as is shared across all tabs of Specify for that instance. You can observe this behavior by opening two tabs of Specify, switching Collections in one, and refreshing the other. The Collection will appear to be "automatically" be switched to the other Collection, but the actual switching of Collections happened as soon as any tab of the Specify instance switches collection.

v7.11.4-prerelease_switch_collection.mov

So when you switch collections in one tab and navigate back to an older tab that is in some other collection, Specify will treat requests as having come from the collection of the prior (newer) tab.

Behind the scenes, Specify "globally" (with the perspective of the browser) manages which collection the user is signed in to via a Cookie, which is tab independent.

specify7/specifyweb/backend/context/middleware.py

Line 32 in 0fed5bb

collection_id = int(request.COOKIES.get('collection', ''))

(This Cookie is set whenever the user explicitly chooses a Collection to login to. There's a backend helper function called set_collection_cookie you can look for usages for if you're curious about how this is implemented in the code!)

You can actually see which Collection you are "logged in to" by using your browser's developer tools. In the below demonstration of the bug, I inspect the Cookies present on the request made to the backend. Even though I appear to be in two separate Collections, because I last logged in to the Fish Collection (CollectionID = 32769) all request made from that point will also be made from the Fish Collection:

v7.11.4-prerelease_bts.mov

For more information, you can view the browsers docs on how to use their Developer Tools to inspect Cookies: Chrome Dev Tools Firefox Dev Tools

The general "expected" behavior is to explicitly switch Collections whenever you are dealing with more than one Collection:

v7.11.4-prerelease_expected.mov

There are probably a lot of nasty bugs that could arise if you expect to be logged into more than one Collection at a time for a Specify instance...

I will note, however, that this specific bug does not happen on main:

v7.11.4-prerelease_main.mov

The main difference between this branch and main is that we are storing the highest autonumbering value for the Collection that performed the autonumbering for 5 seconds. Because you are technically making the request within the same Collection and saving the other Accession within 5 seconds, autonumbering sees the first Accession Number and thinks it is in the same scope.

@specify/ux-testing To summarize, the bug described in @Iwantexpresso review #7683 (review) occurs when:

You have a tab of Specify that is logged into some Collection

In another tab of the same instance you explicitly switch Collections to some other Collection in a different scope

In one of the two tabs you create a record that is autonumbered

Within 5 seconds, without changing Collections you create a record of the same table in the other tab that is also autonumbered

I can try and modify the Autonumbering code to first prioritize the scoping of the record being saved (i.e., use the field values of the record being saved) rather than the Collection the user is logged into (and fallback to the latter if the former can not be easily determined), though this may result in other subtle bugs because "easy to determine" is itself pretty difficult to determine 😅.

How often do you think this will happen in practice (a production instance)?

yeah this totally makes sense. at first I did not fully process how uncommon it might actually be, but realistically speaking, The Pr itself still addresses its main purpose. I probably should've made it a comment than requesting changes, my apologies! still thank you for taking your time Explaining it in such detail it helps a lot!

acwhite211 and others added 30 commits September 30, 2025 12:33

Refactor picklist handling to improve null handling and type coercion…

90b3b66

… when building a case statement for the QB.

Apply format.py changes from issue-7456

ee9d106

Apply changes from adding stats_counts context api

29e01a2

Apply group_concat and blank_nulls changes

2d9dd2c

apply changes from issue-7510

cb1b9dd

Apply new changes for new renumber_tree function

fb5d5e9

Update docker-compose.yml for mariadb 11.8

9151d0a

Fix blank_nulls wrapping numeric CatalogNumber cast instead of raw co…

b77fb62

…lumn

fix customized formatters

dcecec6

fix: Use named locks when autonumbering

4b9f0c3

See #6490, #5337 Replaces #5404 Fixes #4148, #7560 This branch is largely the application of #7455 on `v7.11.3`

chore: simplify autonumber interface

12eb463

feat: backport Redis to v7.11.3 base

dfa7d6a

Initially from de6d9ec in #7399

feat: add utilities for interacting with Redis

f294d5b

Based largely off the initial implementation at 5ed701a from #7399

feat: allow using a Manager to handle named locking

4303545

fix: better handle subsequent requests to get same lock

94fd657

fix: resolve erronous fillin_year in apply_autonumbering

0dd769a

refactor: specify type in apply_autonumbering function

6fad8f9

feat: integrate autonumbering locks with Workbench

4de5c06

Addresses #5337 Fixes #4148, #7560

feat: allow getting scope from model classes

8fbec8b

feat: add more scoping tables

3870a0f

fix: properly respect scope when setting autonumbering keys

e2edb11

fix: resolve typo in static method decorator

d48a7a0

fix: implement comparison operators for scope type

957ba26

fix: internally always use strings for scope types when autonumbering

0039829

feat: spin up a redis instance during backend tests

75964c1

fix: resolve more failing tests

f6e5550

fix: make scopetype helper method static

43fc35d

fix: use ScopeType ints when creating attachments

654c0cd

fix: handle edge case where inferred scope is not related object

04a6922

fix: handle case when lock dispatcher is not present when autonumbering

7942817

github-project-automation bot added this to General Tester Board Feb 4, 2026

github-project-automation bot moved this to 📋Back Log in General Tester Board Feb 4, 2026

melton-jason added this to the 7.12.0 milestone Feb 4, 2026

melton-jason added 3 commits February 5, 2026 18:59

Merge remote-tracking branch 'origin/main' into v7.11.4-prerelease

ba2c8cc

fix: use app and label name instead of object reference to determine …

cdf0bd1

…scope model

chore: remove unused imports

997724e

melton-jason requested review from a team February 6, 2026 01:59

melton-jason marked this pull request as ready for review February 6, 2026 01:59

emenslin approved these changes Feb 9, 2026

View reviewed changes

emenslin requested a review from a team February 9, 2026 17:07

bhumikaguptaa self-requested a review February 9, 2026 21:50

bhumikaguptaa approved these changes Feb 9, 2026

View reviewed changes

Iwantexpresso self-requested a review February 10, 2026 22:43

Iwantexpresso requested changes Feb 10, 2026

View reviewed changes

github-project-automation bot moved this from 📋Back Log to Dev Attention Needed in General Tester Board Feb 10, 2026

melton-jason added a commit that referenced this pull request Feb 11, 2026

fix: resolve redis connection in test

355dda1

See related (and first) #7683

melton-jason requested a review from Iwantexpresso February 12, 2026 19:54

Merge branch 'main' into v7.11.4-prerelease

ceeed19

grantfitzsimmons mentioned this pull request Feb 12, 2026

Auto-numbering collisions across divisions when working in multiple tabs #7695

Open

acwhite211 approved these changes Feb 12, 2026

View reviewed changes

grantfitzsimmons merged commit 81966d3 into main Feb 13, 2026
14 checks passed

grantfitzsimmons deleted the v7.11.4-prerelease branch February 13, 2026 20:52

github-project-automation bot moved this from Dev Attention Needed to ✅Done in General Tester Board Feb 13, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bring changes from `v7.11.4` to `main`#7683

Bring changes from `v7.11.4` to `main`#7683
grantfitzsimmons merged 41 commits intomainfrom
v7.11.4-prerelease

melton-jason commented Feb 4, 2026 •

edited

Loading

Uh oh!

emenslin left a comment

Uh oh!

bhumikaguptaa left a comment

Uh oh!

Iwantexpresso left a comment •

edited

Loading

Uh oh!

melton-jason commented Feb 12, 2026 •

edited

Loading

Uh oh!

emenslin commented Feb 12, 2026

Uh oh!

grantfitzsimmons commented Feb 12, 2026

Uh oh!

melton-jason commented Feb 12, 2026 •

edited

Loading

Uh oh!

grantfitzsimmons commented Feb 12, 2026

Uh oh!

Uh oh!

Iwantexpresso commented Feb 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Comments

Conversation

melton-jason commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Testing instructions

Sample Database

Uh oh!

emenslin left a comment

Choose a reason for hiding this comment

Uh oh!

bhumikaguptaa left a comment

Choose a reason for hiding this comment

Uh oh!

Iwantexpresso left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

melton-jason commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

emenslin commented Feb 12, 2026

Uh oh!

grantfitzsimmons commented Feb 12, 2026

Uh oh!

melton-jason commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

grantfitzsimmons commented Feb 12, 2026

Uh oh!

Uh oh!

Iwantexpresso commented Feb 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Comments

melton-jason commented Feb 4, 2026 •

edited

Loading

Iwantexpresso left a comment •

edited

Loading

melton-jason commented Feb 12, 2026 •

edited

Loading

melton-jason commented Feb 12, 2026 •

edited

Loading