LLC4320 Data Anomaly (Theta variable contains extreme invalid values)

[chenqiulin20011015]

While using xmitgcm.llcreader.ECCOPortalLLC4320Model() to load LLC4320 data, I encountered significant data anomalies in the Theta variable (sea surface temperature).`import xmitgcm.llcreader as llcreader
import matplotlib.pyplot as plt
import dask
import numpy as np

model = llcreader.ECCOPortalLLC4320Model()
dask.config.set(**{'array.slicing.split_large_chunks': False})

ds = model.get_dataset(
varnames=['Theta'],
k_levels=[0],
type='latlon',
iter_start=model.iter_start,
iter_stop=model.iter_start + model.iter_step,
read_grid=True
)

sst = ds.Theta.isel(time=0)

i_start, i_end = 1824, 4703
j_start, j_end = 4040, 6857
sst_cropped = sst.isel(i=slice(i_start, i_end), j=slice(j_start, j_end))

i_vals = np.arange(i_start, i_end)
j_vals = np.arange(j_start, j_end)
I, J = np.meshgrid(i_vals, j_vals)

plt.figure(figsize=(10, 6))
plt.pcolormesh(I, J, sst_cropped[0, :, :], cmap='coolwarm', shading='auto')
plt.colorbar(label='SST (°C)')
plt.title('SST (Theta k=0)')
plt.xlabel('i index')
plt.ylabel('j index')
plt.tight_layout()
plt.show()
`

I'm using:

• Python version: 3.11.5
• xmitgcm version: [0.5.2]
  How can I solve this problem

[jywang31]

I have also encountered this issue — not only with Theta, but with several other variables as well. The data retrieved using xmitgcm.llcreader contains abnormally large and seemingly invalid values. I would greatly appreciate any efforts to resolve this issue at your earliest convenience. @rabernat @raphaeldussin @cspencerjones

[jywang31]

This issue might be caused by corrupted original data from the ECCO Data Portal. I tried downloading some 2D variables such as Eta from https://data.nas.nasa.gov/ecco/eccodata/llc_4320/compressed/, and the values I read were garbled, like 4e-30.

[Yanyu-Fly]

Hello, I have the same problem as you. When I use model = llcreader.ECCOPortalLLC4320Model() to read the Eta variable, the downloaded data will have maximum and minimum values. This situation has lasted for two weeks since I found it. This did not happen last month. The data I read last month was normal.
Now, not only the variable data read has problems, but also the coordinate grid data read has problems. Here are my codes and running results:

from xmitgcm import llcreader
model = llcreader.ECCOPortalLLC4320Model()
print(f": {model.varnames}")
for face in range(13):
xc = ds_coords.XC.sel(face=face).values
yc = ds_coords.YC.sel(face=face).values
print(f"Face {face}:")
print(f" Longitude range: {xc.min()} ~ {xc.max()}")
print(f" Latitude range: {yc.min()} ~ {yc.max()}")

['Eta', 'KPPhbl', 'oceFWflx', 'oceQnet', 'oceQsw', 'oceSflux', 'oceTAUX', 'oceTAUY', 'PhiBot', 'Salt', 'SIarea', 'SIheff', 'SIhsalt', 'SIhsnow', 'SIuice', 'SIvice', 'Theta', 'U', 'V', 'W']
Face 0:
Longitude range: nan ~ nan
Latitude range: nan ~ nan
Face 1:
Longitude range: -1.37911665594459e-12 ~ 26675727302656.0
Latitude range: nan ~ nan
Face 2:
Longitude range: nan ~ nan
Latitude range: -2.5861761769375864e+38 ~ 2.5861824644845635e+38
Face 3:
Longitude range: -2.5927202963962046e+38 ~ 2.592716037090188e+38
Latitude range: nan ~ nan
Face 4:
Longitude range: -6.929719927224742e-13 ~ 13404039282688.0
Latitude range: nan ~ nan
Face 5:
Longitude range: -2.5927207020443967e+38 ~ 2.5927207020443967e+38
Latitude range: -2.5861761769375864e+38 ~ 2.5861824644845635e+38
Face 6:
Longitude range: nan ~ nan
Latitude range: -2.588007678524796e+38 ~ 2.5880316117681285e+38
Face 7:
Longitude range: nan ~ nan
Latitude range: -2.586174351520722e+38 ~ 2.5861806390676992e+38
Face 8:
Longitude range: -1.3870009741429046e-12 ~ 26828456591360.0
Latitude range: nan ~ nan
Face 9:
Longitude range: nan ~ nan
Latitude range: nan ~ nan
Face 10:
Longitude range: nan ~ nan
Latitude range: -2.5861820588363715e+38 ~ 2.5861826673086596e+38
Face 11:
Longitude range: -1.385552480040464e-12 ~ 26800572858368.0
Latitude range: nan ~ nan
Face 12:
Longitude range: nan ~ nan
Latitude range: nan ~ nan

[jywang31]

Hi, I contacted NASA's data portal team by email — they’re aware of the issue. There’s a bug in their web application that's affecting file downloads. They’re testing a fix now and should roll it out soon.
Hopefully it won’t take too long!

[chenqiulin20011015]

Yes, I received the same response as well. Hopefully they’ll be able to fix it soon.

[anthony-meza]

Has there been an update on this?

I'm getting "FileNotFound" errors when trying to access 2160 data through llcreader

[chenqiulin20011015]

According to NASA's response, their data portal is currently experiencing a very high volume of access requests, and the backend system is struggling to keep up. They mentioned that they are migrating to a new storage solution, which will significantly improve the performance and reliability of the data portal in the near future.

If you try several times, the download should eventually work. Although the transfer speed may be slow, it is generally still possible to complete the download.