update all WCS resources and documentation

[dylanbeaudette]

Many of the WCS resources are out of date and the new grid systems have shifted slightly. I'm working on PR to address the many changes that need to be made in soilDB. The new data are nearly ready on SoilWeb.

• soil color, including new OCONUS grids
• ISSR800
• SSURGO-derivative mukey grids
• STATSGO2, grid resolution bump
• SSURGO/STATSGO2 hybrid grids
• RSS (Update RSS mosaic for WCS #463)
• Just before the next CRAN release, ensure WCS URLs point to load-balancer.
• After next CRAN release, delete old grids and mapfile used by WCS.

For the near-future, investigate the use of STAC vs. ad-hoc documentation of these data.

[brownag]

Please also address this issue: #347

[dylanbeaudette]

#458 will only address soil color WCS, tackling WCS updates for mukey grids and ISSR800 in the next PR. After that I'll test shifting to SoilWeb-hosted WCS specifications.

[brownag]

Sounds good, you know I am excited for that last bit. Have you considered using STAC ? I recall reading one of the TODO comments alluding to that. That seems like it would make these data a lot more discoverable outside of the R package and the downloads page on SoilWeb.

[dylanbeaudette]

Thanks. I would like to investigate STAC in the near future. For now, I'll focus on making sure that all of the WCS resources are up to date, and that changes are made to make the update process as automated as possible. You are correct, these data should be more "discoverable".

[brownag]

Regarding the series of recent changes, I think at a minimum we should be using DescribeCoverage WCS endpoint to obtain CRS, extent, and resolution info, rather than including that in hard-coded metadata as we currently do. Related to this, I think a few more parts of our ad hoc metadata could be stored in metadata blocks of the WCS itself.

Notably, if you run DescribeCoverage it looks like the units of measure are not correct for any of the current WCS items.

Also, it looks like the GetCapabilities endpoint fails for ISSR800 WCS. Is this new behavior/related to in-progress updates? I get an exception:

msWCSGetCoverageMetadata20(): Unable to access file.

[dylanbeaudette]

Good ideas. After I get the mukey grid changes finished, I'll update all of the WCS metadata. I need to track down what is causing differences in the GetCapabilities requests:

• https://soilmap4-1.lawr.ucdavis.edu/cgi-bin/mapserv?map=/data1/website/wcs/soilcolor.map&SERVICE=WCS&VERSION=2.0.1&REQUEST=GetCapabilities
• https://soilmap4-1.lawr.ucdavis.edu/cgi-bin/mapserv?map=/data1/website/wcs/mukey-grids.map&SERVICE=WCS&VERSION=2.0.1&REQUEST=GetCapabilities
• https://soilmap4-1.lawr.ucdavis.edu/cgi-bin/mapserv?map=/data1/website/wcs/issr800.map&SERVICE=WCS&VERSION=2.0.1&REQUEST=GetCapabilities

The funky units of measure are the Mapserver/WCS 2.0 defaults. Still investigating that.

The requests are VERY slow, not sure if this is a Mapserver issue or related to the large size of the files in question. While it would be nice for these to work as expected (I'll make sure they do), I'd rather rely on a server-side e.g. JSON specification that can be loaded instantly. I'm still weighing options for how a single specification is translated into mapfile directives and WCS output.

[brownag]

The requests are VERY slow, not sure if this is a Mapserver issue or related to the large size of the files in question.

Appears to be much faster via load balancer URL

[dylanbeaudette]

Agreed. Also a little faster now after some server-side changes. Still, GetCapabilities and DescribeCoverage aren't all that useful at this stage. Despite plenty of tinkering and additions to the mapfile.

I'll continue to pursue the value of WCS built-in metadata, after I upgrade Mapserver to the current release. A custom JSON file, or fully-implemented STAC is more flexible and likely the best route forward.

[dylanbeaudette]

Upgraded SoilWeb Mapserver from 7.6.x → 8.6.x. Will continue testing WCS metadata reporting, but so far it seems too inflexible for what I have in mind.