Small fixes: 1-sec FCAS columns, Excel close, Pandas deprecations#81
Merged
Conversation
Cherry-picked from #67 (commit 1b57257). The companion test in tests/test_data_fetch_methods.py is dropped because PR #72 deleted that file when porting the suite to the offline fixture architecture — a fixture-based regression test is added in a follow-up commit on this branch. AEMO publishes 1-second contingency FCAS RRP and LOCALDISPATCH columns in DISPATCHPRICE, DISPATCHREGIONSUM, TRADINGPRICE, TRADINGREGIONSUM and TRADINGLOAD since the October 2023 introduction of the very-fast service. Master already lists the bid-side LOWER1SEC/RAISE1SEC entries in DISPATCHLOAD and NEXT_DAY_DISPATCHLOAD; this fills in the matching price-side and region-side columns so they're included in the default select_columns for those tables. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… test Two follow-ups to the cherry-picked commits on this branch: 1. The Pandas freq="5T" → "5min" rename in the previous commit only touched tests/test_performance_stats.py. The same deprecated alias was still in src/nemosis/custom_tables.py in two places (plant_stats and trading_and_dispatch_cost). Apply the same rename there so the warning is silenced from the library side, not just the tests. 2. The 1-second FCAS commit dropped its companion test (target file was deleted by PR #72). Replace it with a fixture-based regression test in tests/end_to_end_table_tests/test_dispatch_price.py that uses the 2024-12 fixture (post-October-2023 cutover) and confirms RAISE1SECRRP and LOWER1SECRRP arrive in the output when no explicit select_columns is passed — locking in the contract that the default column list for DISPATCHPRICE includes them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bundles three independent small fixes cherry-picked from #67, plus follow-ups.
RAISE1SECRRP/LOWER1SECRRPand*1SECLOCALDISPATCHto the default column list ofTRADINGLOAD,TRADINGPRICE,TRADINGREGIONSUM,DISPATCHPRICE,DISPATCHREGIONSUM. Auto-merged cleanly with master's existing 1-sec entries inDISPATCHLOAD/NEXT_DAY_DISPATCHLOAD(different tables)._pd.ExcelFile(...)inwith— resource-leak fix. Straight pick.custom_tables.py: groupby key access via.nameinstead oflist(group[col])[0], andapply(...)now passesinclude_groups=False. Alsofreq="5T"→"5min"intest_performance_stats.py.Follow-ups in commit 4
freq="5T"rename to the two remaining call-sites insrc/nemosis/custom_tables.py(plant_statsandtrading_and_dispatch_cost). Commit 3 only fixed the test file; the library itself was still emitting the deprecation warning.tests/end_to_end_table_tests/test_dispatch_price.py. The commit-1 source change dropped its companion test (target file was deleted by Port test suite to offline fixture-based architecture #72). Replacement uses the 2024-12 fixture (post-October-2023 1-sec cutover, confirmed to containRAISE1SECRRP/LOWER1SECRRPin the source CSV) and asserts both columns arrive in the output when no explicitselect_columnsis passed.Test plan
uv run pytest tests/— 368 pass, 1 skipped, 1 unrelated warning (existingpd.concatdeprecation intest_processing_info_maps.py, untouched by this PR)test_one_second_fcas_columns_included_by_defaultpassesDropped from #67's commit set
tests/test_data_fetch_methods.pymodifications from1b57257(file deleted by Port test suite to offline fixture-based architecture #72; replaced by fixture-based test above)