Configurable WFS version and CITE compliant WFS requests

[offtherailz]

Description

MapStore sends XML that does not validate against the schema of the version it claims to speak. On a GeoServer with CITE strict compliance enabled, which is needed to pass the INSPIRE validation tests, every WFS request is rejected and the attribute table, the widgets and the rest show nothing.
Some of that is fixable on 1.1.0 and some is not. Paginated GetFeature is standard only from 2.0.0: startIndex is not in the 1.1.0 schema, so on a strictly validating server there is no way to paginate while staying on 1.1.0. GeoServer accepts it today as a vendor extension, and MapStore depends on that for the attribute table (epics/featuregrid.js), the filter autocomplete (observables/autocomplete.js) and epics/geoProcessing.js.

So the goal is to make WFS 2.0.0 actually usable, per layer and per service, and to fix the XML we generate for both versions. There is partial 2.0 code in the tree that was never activated and cannot be switched on as it stands.

An investigation branch where to take inspiration is done here.
Here a mock of config and catalog that need to be update contextially with the new UI. Need to send a updated mockup before starting. Notice that after changes, the position of the tools may vary a lot. Please refer to the latest UI for position of the section indicated. In fact the mockup uses only a selector for WFS type.

WFS Layer settings

WMS Layer Settings

What has to be done

1. Fix the nine violations listed below.
2. Introduce search.wfsVersion (with layer.wfsVersion as fallback), detected from GetCapabilities when a layer is added, overridable per layer and per catalog service.
3. Thread that version into the requests. Today observables/wfs.js builds the request builder once at module level with wfsVersion: "1.1.0" hardcoded, so no layer value can reach a query.
4. Keep 1.1.0 behaviour intact, startIndex included. Standard pagination is what a layer on 2.0.0 gets.
5. Add the version selector to the layer settings and to the WFS catalog advanced settings and WMS catalog too.

Known Violations to fix

here the violations we need to correct in order to make the filters ane queries strictly compliant.

	Violation	Reference
1	version="2.0" sent in GetFeature instead of "2.0.0"	WFS 2.0.0 §7.2
2	Filters use the ogc: prefix instead of fes: on 2.0	FES 2.0.0 §7.1
3	SortBy uses the wfs: prefix, where 1.1.0 wants ogc: and 2.0 wants fes:	FES 2.0.0 §6.3
4	SortOrder is "A" instead of "ASC" or "DESC"	FES 2.0.0 Table 3
5	The namespace of a workspace prefixed type name is never declared	XML Namespaces §3
6	And wrapper around a single child predicate	FES 2.0.0 §7.9.3
7	GetCapabilities asks for version="1.1.1" (observables/wfs.js:52)	WFS §8.3
8	Widget and chart filters use ogc: inside a WFS 2.0 wrapper	FES 2.0.0 §7.1
9	GML 3.2 geometries carry no gml:id	GML 3.2.1 §9.2.1

Numbers 3, 5 and 6 are invalid on 1.1.0 too and are semantically neutral to fix, so they land on both versions. A tenth complaint from the same server, workspace and layer names containing underscores, is a server side naming problem.

Two candidates are not violations and need no work: ogc:FeatureId is valid in FES 1.1.0, and serverType: "no-vendor" is unrelated, since maxFeatures and count are standard parameters.

Native WFS layers

A WFS layer from the catalog has search: {url, type: "wfs"}, so the field fits. One from a saved map, from WMC or from a hand written config may have no search, which is why layer.wfsVersion has to work as a fallback and why the lookup belongs in a helper next to getSearchUrl and getWFSLayerName.

The UI has a gap: in #12802 the "WFS linked service" panel exists only for type === 'wms' (settings/General.jsx:268), while the editable URL field covers wms and wfs. A native WFS layer needs a container of its own for the selector, next to its URL field.

Acceptance criteria

• A FULL TEST have to be executed before the delivery
  • With CITE flag
    • must fail with proper error on pagination (e.g. table)
      • WMS legacy layers
      • WFS legacy layers
      • WMS layers set with 1.1.0
      • WFS layers set with 1.1.0
    • must work (also pagination)
      • WMS layers set with 2.0
      • WFS layers set with 2.0
  • Without CITE flag, all the layer above should work.
  • WFS-T need to be tested too (feature grid editing). In this case requests need to fallback to 1.1.0, with a proper error handling in case of CITE compliancy error, that have to be reported and analyized to check if a solution is possible.
• A layer with no search.wfsVersion behaves exactly as today, (startIndex included), and layer.wfsVersion works as a fallback
• search.wfsVersion is settable per layer and per catalog service, survives in the map configuration, and a service default merges into search instead of replacing it
• A layer added from the catalog gets the highest version the service advertises, resolved once at add time rather than per request
• The version reaches the request for both WMS layers with a linked service and native WFS layers, through a helper that also works when search is absent
• WFS/RequestBuilder uses one version check accepting 2.0 and 2.0.0, and emits count, typeNames and fes: on 2.0.x against maxFeatures, typeName and ogc: on 1.x
• On 2.0.x: version="2.0.0", fes: in layer and widget filters, gml:id on GML 3.2 geometries, and pagination through the standard startIndex and count
• On both versions: no single child And, SortBy with the prefix its version requires, after Filter, with ASC or DESC, and the type name namespace declared
• Generated payloads validate against the XSD of their version, covered by unit tests
• The layer settings expose the version with a re-detect action, in the "WFS linked service" panel for WMS layers and in a new section for native WFS layers, reusing the components from #12571 , #12573 Support of distinct search-typename in WMS layers and way to edit urls of layers  #12802
• The WFS catalog advanced settings expose the service default, including an auto option
• Attribute table, widgets, cross layer filter, dependency and extent resolution, snapping and layer download all work against a GeoServer with CITE strict compliance enabled
• Docs updated: docs/user-guide for the selector, docs/developer-guide/maps-configuration.md for the new property

Things to pay attention:

• observables/wfs.js:52 toLayerCapabilitiesURL uses 1.1.1 that do not exist. Use the layer's one (used only by cross layer filter). Need to check that also cross layer filters use the right approach, this path needs to be version aware, with a proper unit test.
• Check that whenever the version is 2.0 or 2.0.0 the generated request is identical (somewhere checks are 2. sometimes are asking to explicit "2.0". Uniform these 2 behaviours.
• Rendering a WFS layer on the map is verified against a strictly CITE validating service, in both the default and the no-vendor configuration (2nd should not use cql filter), with a layer filter applied.
• A filter that cannot be honoured by the service produces a visible failure rather than a map or a table showing unfiltered data.

Other useful information

Most of the work exists on an investigation branch, touching utils/ogc/Filter/filter.js, utils/ogc/WFS/RequestBuilder.js, epics/wfsquery.js, epics/widgetsbuilder.js, api/catalog/WFS.js, the catalog advanced settings and the English translations. What is left is tests and a run against a real server.

Related: #12571, #12573, #12774, #12787, #12802.

Reference: internal support request about CITE and INSPIRE compliance on a customer GeoServer.