Skip to content

Request first_non_empty fields when fetching Esri layers - #117

Merged
iandees merged 2 commits into
openaddresses:masterfrom
jeffdefacto:esri-first-non-empty-fields
Oct 6, 2026
Merged

iandees merged 2 commits into
openaddresses:masterfrom
jeffdefacto:esri-first-non-empty-fields

Conversation

@jeffdefacto

Copy link
Copy Markdown
Contributor

For ESRI sources, EsriRestDownloadTask.fields_from_conform_function decides which fields go into outFields. Only join and format are treated as functions that take a list of fields. Every other function falls through to v.get('field'). first_non_empty also uses a fields list, so it adds None, and none of its fields are downloaded.

Conform then fails in row_fxn_first_non_empty:

File "openaddr/conform.py", line 1201, in row_fxn_first_non_empty
    if row[field] and row[field].strip():
KeyError: 'TOWN_VIL_1'

This turned up in openaddresses/openaddresses#9973 (run https://batch.openaddresses.io/run/16806). Two ESRI sources using first_non_empty for city failed. Two sibling sources with a plain city mapping passed. That PR now uses a plain mapping as a workaround.

Change

  • Add first_non_empty to the multi-field branch in fields_from_conform_function.
  • Add a first_non_empty case to test_field_names_to_request.

Testing

  • python -m unittest openaddr.tests.cache: all pass.
  • The new assertion fails without the fix.

🤖 Generated with Claude Code

jeffdefacto and others added 2 commits October 2, 2026 15:31
fields_from_conform_function only treated join and format as
multi-field functions, so first_non_empty fell through to
v.get('field') and contributed None. Esri downloads then omitted the
listed fields and conform failed with a KeyError.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@iandees
iandees merged commit e27fd5d into openaddresses:master Oct 6, 2026
1 check passed
@iandees

iandees commented Oct 6, 2026

Copy link
Copy Markdown
Member

This is deployed.

jeffdefacto added a commit to openaddresses/openaddresses that referenced this pull request Oct 13, 2026
About 800 rows in each layer have a blank TOWN_VIL_1. Use
first_non_empty to fall back to PO_NAME. This was left out of #9973
because batch-machine didn't request first_non_empty fields for ESRI
sources; fixed in openaddresses/batch-machine#117 (10.5.0).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants