Enhancement CAT-FR-SF-04 - disabling SHACL validation and refactoring

README.md

@@ -1,3 +1,18 @@
 # XFSC Documentation
 
-This repository contains the XFSC documentation about the components, architecture and more. 
+This repository contains the XFSC documentation about the components, architecture and more.
+
+## Rendering docs locally
+
+A Dockerized AsciiDoc toolchain with Mermaid diagram support is available under [`tools/asciidoctor/`](tools/asciidoctor/). Build the image once, then use it to render any document:
+
+```bash
+# Build
+docker build -t xfsc-asciidoctor tools/asciidoctor/
+
+# Render to HTML
+docker run --rm -v "$(pwd)":/documents xfsc-asciidoctor \
+  asciidoctor -r asciidoctor-diagram path/to/document.adoc
+```
+
+See the [full README](tools/asciidoctor/README.md) for PDF output and more examples.

federated-catalogue/src/docs/architecture/chapters/01_introduction_and_goals.adoc

@@ -20,6 +20,12 @@ The requirements for implementing the Gaia-X Federated Catalogue are given in it
 * The specification of a requirement turned out to be unsatisfiable for conceptual or practical reasons.
 * The understanding of an underlying concept had evolved during the further work of the Gaia-X Technical Committee and its working groups after the original publication of the requirements specification.
 
+The catalogue specification has been updated with new requirements, identified by IDs starting with CAT-FR-*.
+While these requirements are not fully implemented yet, they are planned to be completed by 2026.
+The catalogue's architecture is designed to support the implementation of these new requirements.
+In cases of conflict, the newer requirements take precedence over the older ones.
+
+For more details, refer to the document "Enhancement of the XFSC Federated Catalogue" footnote:[Enhancement of the XFSC Federated Catalogue. 2025. https://github.com/eclipse-xfsc/docs/blob/9a178c30f76610e5924fd72c39d259d6a8ae0461/federated-catalogue/src/docs/CAT%20Enhancement/CAT_Enhancement_Specifications%20v1.0.pdf].
 
 === Quality Goals
 

federated-catalogue/src/docs/architecture/chapters/05_building_block_view.adoc

@@ -74,9 +74,21 @@ The Graph Database can be considered as a kind of search index. The single sourc
 
 Generically returning the Self-Description files containing claims that influence query response is not possible. To get the relevant Self-Description files, the query to the Graph Database can be formulated to return the Gaia-X entity that is the __credentialSubject__ of a Verifiable Credential. Then this can be used as a filter for the Self-Description endpoint, to download the Self-Description file.
 
-Neo4j is used as implementation of the Graph database.
+The graph database backend is **pluggable**. The implementation is selected via the `graphstore.impl` configuration property:
 
-[IMPORTANT] 
+[options="header",cols="1,2,2"]
+|===
+| Value | Backend | Query Language
+| `neo4j` | Neo4j (default) | openCypher
+| `fuseki` | Apache Jena Fuseki | SPARQL-star
+| `dummy` | No-op (for testing) | —
+|===
+
+Both backends implement the same `GraphStore` interface. Core services contain no backend-specific logic.
+
+===== Neo4j backend
+
+[IMPORTANT]
 .Limitation: queries to non-Enterprise Neo4j Graph database return empty record when no results found, rather than empty list.
 ====
 When there is no data in the Graph database, i.e., no claims extracted from Self-Description, there is still a configuration node for the neosemantics modulefootnote:[https://neo4j.com/labs/neosemantics/4.0/config/[Configuring Neo4j to use RDF data]], which enables Neo4j to support the RDF data model, which is required here.
@@ -89,6 +101,10 @@ However this revoke operation is only supported in Neo4j enterprise.footnote:[ht
 We chose not to implement a workaround that involves query rewriting, as this may have harmful side effects.
 ====
 
+===== Fuseki backend
+
+The Fuseki backend uses Apache Jena with RDF-star. Claims are stored as RDF-star triple nodes: `<< subject predicate object >> cred:credentialSubject credentialSubjectUri`. Queries must use SPARQL-star syntax. Results are shuffled unless the query contains an `ORDER BY` clause.
+
 ==== File Store
 The File Store is responsible to persist all file based content submitted to the catalogue. These are Self-Descriptions and Schemas.
 
@@ -112,7 +128,14 @@ The catalogue implementation is structured into two main components _fc-service-
 | Service | Component, controlling the workflows and interactions between the subcomponents.
 | Authorization | Handling the authorization, and user management in combination with the _Service_ and _Authentication_ component.
 | Authentication | This component stores the users and participant metadata.
-| Verification | Responsible for applied the defined verification rules on the Self-Descriptions. Also extracts the claims from the Verifiable Presentation (_Claim Extractor_). Also contains a component to periodically revalidate submitted Self-Descriptions (_Revalidation Service_).
+| Verification
+a| Responsible for applying the defined verification rules on the Self-Descriptions and extracting claims from Verifiable Presentations.
+
+* **Verification pipeline:** Syntactic, semantic, and cryptographic verification of Self-Descriptions. The SD is checked as a whole — if VC verification fails, the entire SD verification fails.
+* **Strategy pattern:** `VerificationServiceImpl` acts as a thin Context that delegates to a `VerificationStrategy` implementation (currently `CredentialVerificationStrategy`).
+* **Schema Validation Service:** Automatic SHACL schema validation has been disabled in the upload flow and extracted to a dedicated service.
+* **Revalidation Service:** Periodically revalidates submitted Self-Descriptions.
+* **Claim Extractor:** Extracts claims from Verifiable Presentations.
 | Graph Store | Interface and logic for communicating with the Graph DB.
 
 | Schema Store
@@ -173,83 +196,108 @@ Notes:
 
 * The SD is checked as a whole. If the verification of the VCs fails, the verification of the whole SD fails.
 
-[source,java]
-----
-/**
- * Validation Self-Description interface.
- */
-@Service
-public interface VerificationService {
+[mermaid, width=2000]
+....
+classDiagram
+    class VerificationService {
+        <<interface>>
+        +verifyParticipantSelfDescription(payload) VerificationResultParticipant
+        +verifyOfferingSelfDescription(payload) VerificationResultOffering
+        +verifyResourceSelfDescription(payload) VerificationResultResource
+        +verifySelfDescription(payload) VerificationResult
+        +verifySelfDescription(payload, verifySemantics, verifySchema, verifyVPSig, verifyVCSig) VerificationResult
+        +extractClaims(payload) List~SdClaim~
+        +verifySelfDescriptionAgainstSchema(payload, schema) SchemaValidationResult
+        +verifySelfDescriptionAgainstCompositeSchema(payload) SchemaValidationResult
+    }
+    note for VerificationService "verifySelfDescriptionAgainstSchema and\nverifySelfDescriptionAgainstCompositeSchema\nare deprecated — use SchemaValidationService directly"
+....
 
-  /**
-   * The function validates the Self-Description as JSON and tries to parse the json handed over.
-   *
-   * @param payload ContentAccessor to SD which should be syntactically validated.
-   * @return a Participant metadata validation result. If the validation fails, the reason explains the issue.
-   */
-  VerificationResultParticipant verifyParticipantSelfDescription(ContentAccessor payload) throws VerificationException;
+[#_verification_strategy_pattern]
+===== Verification Strategy Pattern
 
-  /**
-   * The function validates the Self-Description as JSON and tries to parse the json handed over.
-   *
-   * @param payload ContentAccessor to SD which should be syntactically validated.
-   * @return a Verification result. If the verification fails, the reason explains the issue.
-   */
-  VerificationResultOffering verifyOfferingSelfDescription(ContentAccessor payload) throws VerificationException;
+The `VerificationServiceImpl` follows the Strategy pattern. It acts as a thin Context that maintains per-request verification toggles (`verifySemantics`, `verifySchema`, `verifyVPSignature`, `verifyVCSignature`) and delegates all verification and claim extraction logic to a `VerificationStrategy` injected via Spring's `@Qualifier`.
 
-  /**
-   * The function validates the Self-Description as JSON and tries to parse the json handed over.
-   *
-   * @param payload ContentAccessor to SD which should be syntactically validated.
-   * @return a Self-Description metadata validation result. If the validation fails, the reason explains the issue.
-   */
-  VerificationResult verifySelfDescription(ContentAccessor payload) throws VerificationException;
+The `resolveStrategy(ContentAccessor)` method selects the appropriate strategy for a given payload. Currently it always resolves to `CredentialVerificationStrategy`.
 
-  /**
-   * The function validates the Self-Description against the composite schema.
-   *
-   * @param payload ContentAccessor to SD which should be validated.
-   * @return the result of the semantic validation.
-   */
-  SemanticValidationResult verifySelfDescriptionAgainstCompositeSchema(ContentAccessor payload);
+[mermaid, width=2000]
+....
+classDiagram
+    class VerificationServiceImpl {
+        -verifySemantics : boolean
+        -verifySchema : boolean
+        -verifyVPSignature : boolean
+        -verifyVCSignature : boolean
+        +resolveStrategy(payload) VerificationStrategy
+    }
+    class VerificationStrategy {
+        <<interface>>
+        +verifySelfDescription(payload, strict, expectedClass, flags) VerificationResult
+        +extractClaims(payload) List~SdClaim~
+        +setBaseClassUri(baseClass, uri) void
+    }
+    class CredentialVerificationStrategy {
+        +verifySelfDescription(payload, strict, expectedClass, flags) VerificationResult
+        +extractClaims(payload) List~SdClaim~
+    }
+    VerificationServiceImpl --> VerificationStrategy : delegates to
+    VerificationStrategy <|.. CredentialVerificationStrategy
+....
 
-  /**
-   * The function validates the Self-Description as JSON and tries to parse the json handed over.
-   *
-   * @param payload
-   * @param verifySemantics
-   * @param verifySchema
-   * @param verifySignatures
-   * @return
-   * @throws VerificationException
-   */
-  VerificationResult verifySelfDescription(ContentAccessor payload,
-      boolean verifySemantics, boolean verifySchema, boolean verifySignatures) throws VerificationException;
+`VerificationServiceImpl` injects the strategy via `@Qualifier("credentialVerificationStrategy")`, providing compile-time safety. The `resolveStrategy(ContentAccessor)` method currently always resolves to `CredentialVerificationStrategy`.
 
-  /**
-   * Extract claims from the given payload. This does not do any validation of the payload.
-   *
-   * @param payload The payload to extract claims from.
-   * @return The list of extracted claims.
-   */
-  List<SdClaim> extractClaims(ContentAccessor payload);
-}
-----
+[#_upload_schema_validation_configuration]
+===== Schema Validation Configuration
+
+Automatic SHACL schema validation during asset upload is disabled by default. This behavior is controlled by the `schema` verification property.
 
+[options="header",cols="2,1,3"]
+|===
+| Property | Default | Description
+| `federated-catalogue.verification.schema` | `false` | Enable/disable automatic SHACL schema validation during asset upload.
+|===
+
+**When disabled (default):**
+
+* Assets are stored regardless of SHACL Shape conformity
+* No validation errors on upload for non-conforming assets
+* SHACL Shapes remain uploadable and available for on-demand validation.
+
+**When enabled:**
+
+* Assets are validated against all stored SHACL Shapes during upload
+* Upload is rejected if the asset is invalid against any stored shape
+
+[#_schema_validation_service]
+===== Schema Validation Service
+
+Automatic schema validation has been **disabled in the upload verification flow**. The Schema Validation Service was extracted from the `VerificationService` to provide on-demand SHACL schema validation of stored assets.
+
+[mermaid, width=2000]
+....
+classDiagram
+    class SchemaValidationService {
+        <<interface>>
+        +validateSelfDescriptionAgainstSchema(payload, schema) SchemaValidationResult
+        +validateSelfDescriptionAgainstCompositeSchema(payload) SchemaValidationResult
+        +validateClaimsAgainstSchema(claims, schema) SchemaValidationResult
+        +validateClaimsAgainstCompositeSchema(claims) SchemaValidationResult
+    }
+....
 
 ===== Revalidation Service
 
 This component periodically revalidates existing Self-Descriptions. It checks if the signatures are still valid (e.g. not expired or referencing DID is still available). If the validation of a Self-Description fails, it's either set to state _deprecated_ or _eol_. This will remove the Self-Description from the Graph Database.
 
-Semantic revalidation is not needed here. It is done every time a new Schema is added.
+Semantic revalidation is not needed here. Schema revalidation against SHACL shapes is performed by the `SchemaValidationService` every time a new Schema is added or updated.
 
 ===== Claim Extractor
 
 This component is used by the _Verification Service_ to extract Claims from Self-Descriptions. Those claims are returned by the _Verification Service_ in the _Verification Result_.
 
 ==== Graph Store
 
-The Graph Store is the interface for interacting with the Graph Database. It receives claims (extracted from Self-Descriptions) and adds them to the graph database. The Graph Database only contains claims from active Self-Descriptions and offers an openCypher query interface. External ontologies are not queried when processing requests.
+The Graph Store is the interface for interacting with the Graph Database. It receives claims (extracted from Self-Descriptions) and adds them to the graph database. The Graph Database only contains claims from active Self-Descriptions and offers a query interface (openCypher for the Neo4j backend, SPARQL-star for the Fuseki backend). External ontologies are not queried when processing requests.
 
 [source,java]
 ----