Let clients retrieve the shape of served resources #6

challenge / use case

Clients have no way to retrieve the @metreeca/blue shape behind an endpoint. Without it, they can't find out the
structure, constraints and links of the resources they consume unless they keep a copy of the shape definitions that
the server already holds.

The target consumer is a generic graph BI interface able to connect to any Gate server: it is compiled against no
model, so it discovers the structure of the data it explores, and the links it may follow, from the server alone.
UIs sharing the model package with the server import the shapes directly and need none of this.

proposal

Publish the whole server model at a well-known endpoint, provisionally /.well-known/gate, listing the shapes the
server serves and the endpoints serving them:

{
  "endpoints": { "/offers/": 0, "/vendors/": 2 },
  "shapes": [
    { "kind": "resource", "members": { "items": { "kind": "property", "range": { "shape": { "kind": "reference", "target": 1 } } } } },
    { "kind": "resource", "members": { "vendor": { "kind": "property", "range": { "shape": { "kind": "reference", "target": 3 } } } } },
    …
  ]
}

• Shapes are collected with blue model() from the entry shapes of the endpoints, each listed once: cyclic models
  (SKOSConcept.broader references SKOSConcept itself) are listed in full, and shapes refined on the spot for one
  endpoint, as in reference(() => resource(Resource, { … })), are listed apart from the shapes they extend
• Function-valued shapes are resolved; value shapes are inlined, while resource shapes, whether referenced or embedded,
  are encoded by identifier, so references point to an entry of the listing rather than expanding it
• The client fetches the model once and follows references locally: no request per referenced shape and no drill-down
  syntax; shapes change only with deployments, so the response is cached with an ETag
• The endpoint map is declared on the server, for instance server({ model: { "/offers/": Offers, … } }): drivers
  are anonymous handlers mounted by user routers, and Express doesn't expose nested mount paths ahead of time, so
  server() has no other way to know which shape is served where

Open points:

• Shape identifiers: positions in model() order, stable within a deployment and covered by the ETag, or names, which
  would need support in blue, as shapes carry no name of their own
• Endpoint path: /.well-known/ suffixes are meant to be registered with IANA (RFC 8615), though unregistered use is
  common; a Gate-specific suffix keeps clear of other uses
• Per-endpoint shapes: whether GET /resource? is kept, returning the identifier of the endpoint shape within the
  model, or dropped
• The declared endpoint map restates the routing: whether drivers could take part in declaring it without leaving it
  incomplete

alternatives

• Per-endpoint shapes with client-driven drill-down: each endpoint serves its own shape with references closed,
  and clients expand referenced shapes with a template marked by a query prefix, as in
  GET /offers/?!{"vendor":{},"items":{"product":{}}}; set aside as a generic BI client exploring freely would need a
  request per step to discover the model it could fetch at once. Within this option, an Accept media type was set
  aside as it shares the URL with data and needs Vary: Accept, poorly honoured by CDNs, and a reserved template key
  as it conflates shape and data templates
• Self-registering drivers: each driver adds its shape and mount path to the endpoint map the first time it serves
  a request, keeping routing as the only source of truth; set aside as the map stays incomplete until every endpoint
  has been hit
• IRI patterns in the serialised shape, letting clients dereference referenced shapes themselves: references carry
  only target, and the optional pattern lives on the target resource shape; patterns are rare in real-world
  models, constrain identifiers rather than naming endpoints, can't be dereferenced without an actual identifier, and
  are unavailable for shapes refined on the spot for one endpoint

notes

• Related to Automatically publish OpenAPI specs derived from shape-based models #2 (OpenAPI specs derived from shapes) and Include JSON-LD @context in REST responses derived from shape-based models #3 (JSON-LD @context derived from shapes), which derive
  other descriptions from the same shapes

• Requires a JSON serialisation of shapes from @metreeca/blue

• Related to Support introspection queries for property paths qest#7 (introspection queries for property paths), which addresses model discovery from within
  the query language rather than by serving the shape

findings

Shapes are plain objects and serialise to JSON as they are, except where a member holds a shape as a function rather
than as an object: JSON.stringify drops functions at any depth, so the shape they stand for is lost.

The uncalled form is the usual one in real-world models: in a sample production model, shapes are declared as factory functions (export function SKOSConcept() { return resource(…) }) and members reference them
without calling them, roughly 38 uncalled value shapes against 2 called ones, 11 uncalled embedded shapes, and every
reference() target.

Consequences for the implementation:

• Serving the shape requires an encoder that resolves every function-valued shape, not only reference targets
• Reference targets must not be expanded recursively, as models are cyclic: the model lists each shape once and
  references point to it by identifier
• Should per-endpoint shapes be kept: GET /resource and GET /resource? can't be told apart from URL.search,
  which is empty in both cases, so the handler has to inspect the raw request URL, and the driver and query() docs,
  which state that an empty query retrieves the default view, need revising to match; for GET /collection/? the
  driver serves the container shape, not the item shape