Problems uncovered by the did:webvh test suite

[swcurran]

Please investigate and address the issues raised in the did:webvh test suite -- my fork. Here are a list of the resolution issues for the DIF Java implementation:

Test	Log Producer	Status	Notes
basic-create	python	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
basic-create	ts	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
basic-update	java-eecc	❌ FAIL	resolve error: Invalid DID log: versionTime must be after previous entry at entry 2
basic-update	python	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
basic-update	ts	⚠️ XFAIL	TS COMPAT: nextKeyHashes:[] — TS serialises empty list; Java library may reject on update validation
deactivate	java-eecc	❌ FAIL	resolve error: Invalid DID log: versionTime must be after previous entry at entry 2
deactivate	python	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
deactivate	ts	⚠️ XFAIL	TS COMPAT: nextKeyHashes:[] — TS serialises empty list; Java library may reject on update validation
key-rotation	java-eecc	❌ FAIL	resolve error: Invalid DID log: versionTime must be after previous entry at entry 2
key-rotation	python	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
key-rotation	ts	⚠️ XFAIL	TS COMPAT: nextKeyHashes:[] — TS serialises empty list; Java library may reject on update validation
multi-update	java-eecc	❌ FAIL	resolve error: Invalid DID log: versionTime must be after previous entry at entry 3
multi-update	python	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
multi-update	ts	⚠️ XFAIL	TS COMPAT: nextKeyHashes:[] — TS serialises empty list; Java library may reject on update validation
multiple-update-keys	java-eecc	❌ FAIL	resolve error: Invalid DID log: versionTime must be after previous entry at entry 2
multiple-update-keys	python	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
multiple-update-keys	ts	⚠️ XFAIL	TS COMPAT: nextKeyHashes:[] — TS serialises empty list; Java library may reject on update validation
portable	python	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
portable	ts	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
portable-move	python	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
portable-move	ts	⚠️ XFAIL	TS COMPAT: nextKeyHashes:[] — TS serialises empty list; Java library may reject on update validation
pre-rotation	python	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
pre-rotation	ts	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
pre-rotation-consume	java-eecc	❌ FAIL	resolve error: Invalid DID log: signing key not in active updateKeys at entry 2
pre-rotation-consume	python	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
pre-rotation-consume	rust	❌ FAIL	resolve error: Invalid DID log: signing key not in active updateKeys at entry 2
pre-rotation-consume	ts	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
services	java-eecc	❌ FAIL	resolve error: Invalid DID log: versionTime must be after previous entry at entry 2
services	python	❌ FAIL	LIB BUG: 0.2.0 NPEs on witness:{} in parameters (Python/TS write empty witness object; library expects absent field)
services	ts	⚠️ XFAIL	TS COMPAT: nextKeyHashes:[] — TS serialises empty list; Java library may reject on update validation
witness-threshold	rust	❌ FAIL	resolve error: Invalid witness proofs: insufficient witness proofs for entry 1-QmdDTDXMshdRwAbnN7b22BKAUy1CvQsq6y6zQqSP7reDJ7: need 1, got 0
witness-update	java-eecc	❌ FAIL	resolve error: Invalid DID log: versionTime must be after previous entry at entry 2
witness-update	rust	❌ FAIL	resolve error: Invalid witness proofs: missing witness proof for entry 1-QmayLd7TkamnZhA8EbQCP6aUvU7YvGW1xW2iauRFC47ziP
witness-update	ts	⚠️ XFAIL	TS COMPAT: nextKeyHashes:[] — TS serialises empty list; Java library may reject on update validation

There are a series of issues:

1. The Null Pointer Exception (NPE) in the library. The spec allows for an empty list of objects or array, but does not permit null. Looks like in places that is not being correctly handled in the Java implementaiton.
2. A versionTime difference with the Java-EECC implementation. Might be with that implementation, but other implementations have successfully processed all/most of the EECC-produced logs.
3. The Java resolver is failing to accept that having "nextKeyHashes" : [] is valid in any entry. Not having it is fine per the spec, but having it with "[]" is also fine. Note that "nextKeyHashes" : null is not valid per the spec.
4. Pre-rotation handling vs. what is done by the Rust and Java-EECC implementation.
5. Handling the Rust witnesses. Not exactly sure what is wrong, but I think it could be due to "witness pruning". In the did:webvh spec, the DID Controller may "prune" the witness file such that only the most recent one (or -- special case, two) proofs are in the file. Any proof for an entry is implies that the witness "approves" all prior entries and so there is no need to retain those proofs.

Background on the test suite:

• The "vectors" folder has a list of tests (first column above), that includes a YAML set of instructions for building the DID
  • Within each vector folder is a folder for each implementation that contains the DID Log (and witnesses, as appropriate) and resolution DIDDocs from the DID Log, as produced by the implementation. So the "ts" folder has the Log and resolution results from running the TS implementation.
• The "implementations" folder has driver source code for each implementation. Each driver runs the following:
  • Generate all of the test DIDs and then resolve the DIDs -- updating the files stored in the "vectors/" folder.
  • For each of the test and each other implementation, resolve the DIDs using the Logs generated by each. So for "ts", it processes the Logs of the "ts", "python", "rust", "java" and "java-eecc" implementations -- a total of 13 * 5 tests
• The "implementations" folder for each implementation has a "status.md" that shows the status of every generation test by the library, and every resolution tests.
  • Status values are PASS, DIFF (for resolution), SKIP, XFAIL and FAIL
  • DIFF is for resolution, when the resolution result(s) (DIDDocs) from one library doesn't match that of the source library. These are often acceptable errors and I hope to add a way to suppress those after verification that the DIFFs are not material.
  • XFAIL means that a known issue is causing the problem that needs to be fixed.
  • FAIL is when either the library can't generate a DID, or can't process the DID Log from another implementation. The latter is particularly bad.

It would also be a good idea to look at the implementation source code to make sure it is valid. It is all AI generated.

Please update and I'll update the test suite repo.

[swcurran]

There is also a couple of generation errors. Not sure if the notes are accurate (AI generated, as noted above), but the other implementations succeeded in creating these DIDs.

Test Case	Status	Notes
multiple-update-keys	⚠️ SKIP	multiple-update-keys at create time not supported by ivir3zam API
pre-rotation-consume	⚠️ SKIP	resolver rejected generated log: Invalid DID log: signing key not in active updateKeys at entry 2

[IVIR3zaM]

Hey @swcurran,

I fixed the issues on this PR, please have a look