Skip to content

Unable to obtain DUDETAILSUMMARY #62

@mdavis-xyz

Description

@mdavis-xyz

Summary

I'm trying to get DUDETAILSUMMARY. When setting and end date past the available data, no data is returned. I expect at least the most recent available file to be returned.

As of today (7/1/2026) the most recent file is PUBLIC_ARCHIVE#DUDETAILSUMMARY#FILE01#202511010000.zip

import datetime as dt
from nemosis import cache_compiler, dynamic_data_compiler
import os


start_time = '2025/01/01 00:00:00'
end_time = '2026/01/01 00:00:00'
table = 'DUDETAILSUMMARY'

dynamic_dir = "data/test/dynamic"
os.makedirs(dynamic_dir, exist_ok=True)
df = dynamic_data_compiler(start_time, end_time, table, dynamic_dir)

Even though the timespan includes multiple eligible files, I get error:

INFO: Compiling data for table DUDETAILSUMMARY
INFO: Downloading data for table DUDETAILSUMMARY, year 2025, month 12
WARNING: PUBLIC_ARCHIVE#DUDETAILSUMMARY#FILE01#202512010000 not downloaded
WARNING: Loading data from data/test/dynamic\PUBLIC_ARCHIVE#DUDETAILSUMMARY#FILE01#202512010000.feather failed.
INFO: Downloading data for table DUDETAILSUMMARY, year 2026, month 01
WARNING: PUBLIC_ARCHIVE#DUDETAILSUMMARY#FILE01#202601010000 not downloaded
WARNING: Loading data from data/test/dynamic\PUBLIC_ARCHIVE#DUDETAILSUMMARY#FILE01#202601010000.feather failed.

---------------------------------------------------------------------------
NoDataToReturn                            Traceback (most recent call last)
Cell In[21], line 3
      1 dynamic_dir = "data/test/dynamic"
      2 os.makedirs(dynamic_dir, exist_ok=True)
----> 3 df = dynamic_data_compiler(start_time, end_time, table, dynamic_dir)
      4 df

File [~\venv\Lib\site-packages\nemosis\data_fetch_methods.py:135](http://localhost:8889/lab/tree/~/venv/Lib/site-packages/nemosis/data_fetch_methods.py#line=134), in dynamic_data_compiler(start_time, end_time, table_name, raw_data_location, select_columns, filter_cols, filter_values, fformat, keep_csv, parse_data_types, rebuild, **kwargs)
    133     return all_data
    134 else:
--> 135     raise NoDataToReturn(
    136         (
    137             f"Compiling data for table {table_name} failed. "
    138             + "This probably because none of the requested data "
    139             + "could be download from AEMO. Check your internet "
    140             + "connection and that the requested data is archived on: "
    141             + "https://nemweb.com.au/ see nemosis.defaults for table specific urls."
    142         )
    143     )

NoDataToReturn: Compiling data for table DUDETAILSUMMARY failed. This probably because none of the requested data could be download from AEMO. Check your internet connection and that the requested data is archived on: https://nemweb.com.au/ see nemosis.defaults for table specific urls.

I suspect nemosis is trying to deduplicate this standing data table for me, but there's a bug in the date calculations to figure out which file to use.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions