Documentation Overhaul

[APickledWalrus]

Problem

This pull request looks at three problems:

1. The documentation features available across Skript are highly variable. For example, Event SyntaxInfos have direct documentation methods, while all other syntaxes must use annotations. Others only support certain properties. While the Documentable interface was introduced as an attempt to resolve these issues, it is not consistently used, and it is not fully featured (not equivalent with syntax).
2. Documentation output is inconsistent and missing a lot of information. JSONGenerator was an improvement in terms of output format, but it is unnecessarily difficult to add additional data. There is also no customization offered for documentation developers, leading to the usage of completely proprietary tools (though the creation of these tools is not an issue).
3. Our existing site is quite outdated and cannot address current documentation needs. This PR provides the necessary internal work for https://beta-docs.skriptlang.org/syntaxes/ to be "completed".

Solution

This pull request completely reworks how Documentation is attached throughout the codebase. It focuses on a unified implementation that separates documentation and generation/output. Below, I'll walk through the actual API classes, and then I'll cover the outputs for the implementing classes.

There is a lot to cover, so I may miss some things. Please feel free to ask a lot of questions, and I'll update this description as necessary.

This pull request is also available on our repository for testing under version 2.16.0-feature-docs-overhaul.

Documentation API

DocumentationAdapter

A DocumentationAdapter is used for extracting documentation information out of Skript's classes. Documentation data is written to an adapter in a key-value format. Keys are strings, while values can be anything. The adapter is also scoped, which is used to define sections of documentation. For example, when all expressions are written to an adapter, they are written within an "expressions" scope. The same applies for other types of syntax.

Skript provides a default implementation which can be created using the static of methods available on the interface.

There are two relevant write methods:

1. write(String, Object) - Writes a single key-value pair to the adapter. One important thing to note is that the default implementation has special handling.
2. write(Documentable) - Writes a Documentable (described further below) to this adapter. A Documentable represents something that provides (outputs) a chunk of documentation (many key-value pairs). One thing to note is that, for the default implementation, a Documentable written using the write(String, Object) method will cause automatic scoping. That is, the method will instead enter a new scope (using the provided key as the name), write the documentable, and then exit that scope.

There are two relevant scoping methods:

1. enterScope(String) - Enters a new scope (section) for documentation to be written in.
2. exitScope() - Exits the last entered scope. This should typically be paired with an enterScope call. For example: enter a new scope, write some data, exit that scope.

There is also a reference system for elements to reference each other. For example, when an Expression outputs its return type, it is actually a reference to the ClassInfo. References are not resolved until all documentation has been written, allowing it to correctly handle ID collisions. Current ID collision is rather simple (the adapter simply appends a -N prefix). A reference can be obtained by calling adapter.reference(<thing>) where <thing> is the object being referenced.

Of course, the other key part is actually obtaining all of the data written to an adapter. Data is stored on an adapter as a String-Object map encompassing the existing system. This map is accessed by calling the dataMap() method. Going back to the Expression example with scoping I mentioned above, expression-related data would be available on this level and obtained using dataMap.get("expressions"). Expressions are stored as another Map, where the keys represent the IDs of the expressions written. The data map can be used by a DocumentationGenerator (described below) to be output in some format.

Currently, the adapter dumps documentation on a per-addon basis (one adapter per addon). I may consider changing this for allowing bulk generation though.

Documentable

This is a new interface that replaces the existing Documentable interface, which has been deprecated and reworked to provide compatibility. Something that is Documentable can output documentation data. That is, it can write information to a DocumentationAdapter. It defines 4 methods:

1. write(DocumentationAdapter) - This method is to be implemented by classes for them to write documentation data, such as name, examples, etc...
2. preWrite(DocumentationAdapter) - This method is not required. It is called before write. Some Documentables, when written, may perform actions such as entering new scopes. It is useful to separate this behavior from the primary write method, as it allows the overriding of the write method to provide additional data without breaking the scope management. For example, the Expression SyntaxInfo overrides the write implementation of SyntaxInfo because it also provides return type information.
3. postWrite(DocumentationAdapter - This method is not required. It is called after write. As mentioned with preWrite, this can be used for things such as scope management.
4. canWrite(DocumentationAdapter) - This method is not required. It allows controlling whether any of the other write methods are called. For example, if a class will only write its documentation under certain conditions, it can control it here rather than needing to create separate implementations.

Consider this example:

public class MyThing implements Documentable {

	public String id() {
		/* implementation goes here */
	}

	@Override
	public void preWrite(DocumentationAdapter adapter) {
		adapter.enterScope(id());
	}

	@Override
	public void write(DocumentationAdapter adapter) {
		adapter.write("foo", "bar");
		adapter.write("numbers", List.of(1, 2, 3));
	}

	@Override
	public void postWrite(DocumentationAdapter adapter) {
		adapter.exitScope();
	}

}

Documentation

On the data side of things, this interface exists to define a standard container of Documentation information (and what that information is). It is a greatly extended version of the existing (not the new one) Documentable interface. It describes all of the standard information one expects such as name, description, and examples. It also supports storing additional data in the form of Documentable objects. As the name implies, this enables storing additional data without having to reimplement the class. There are a few uses of this in the codebase that I cover below.

We also define a builder interface, and provide default implementations for both. Thus, elements will store Documentation objects directly. This ensure consistencies in the available methods and prevents significant implementation duplication across classes. Documentation supports being converted back into builder form (Documentation#toBuilder).

On the interface is a NONE Documentation object that can be used for intentionally marking an element as having no documentation. It is permitted to convert this object back into a builder to add additional information. This will NOT prevent. Thus, the Documentation#isNoDocs(Documentation) method should be used for checking whether a Documentation should be written.

It is also important to note that it is NOT a goal of this pull request to deprecate or remove the existing documentation annotations. Those are fully preserved, and a conversion method is provided: Documentation#of(Class). This will read the annotations and apply the relevant data. This means that annotations can now be used with non-syntax classes (e.g. ClassInfo) if desired.

Below is an example of both approaches to creating a Documentation object:

// There is a direct method for creating a simple Documentation with the typical information of
// name, description, since, examples
Documentation docs = Documentation.of(
	"My Thing",
	"""
	Line 1 of description
	Line 2 of description
        """
	"INSERT VERSION",
	"""
	Example 1
	""",
	"""
	Example 2
	"""
);

// There is the full-featured builder approach as well
Documentation docs = Documentation.builder()
	.name("My Thing")
	.description("""
		Line 1 of description
		Line 2 of description
		""")
	.addSince("INSERT VERSION")
	.addExample("""
		Example 1
		""")
	.addExample("""
		Example 2
		""")
	.build();

DocumentationDocumentable

This is a Documentable that has a Documentation object. It defines an additional documentation() method for obtaining the object's Documentation. It overrides the default write methods to handle scoping and output. That is, it writes its values under a scope determined from documentation().id() (or autoID() is no ID is explicitly set).

DocumentationGenerator

This is a small interface to describe methods related to generating an output from a DocumentationAdapter. It defines a single method:

• generate(Path) - Generates a documentation file (in some format, TBD by implementation) at the provided path.

It has a static method for obtaining our default implementation, the JSONGenerator (described below).

JSONGenerator

Currently, this implementation is internal-only and has no additional methods. It simply dumps the adapter's data map as JsonElements. It provides two additional fields as well:

1. version - A JsonObject with major and minor fields. This describes the version of the JSONGenerator (currently 3.0, incrementing from the old JSONGenerator class's 2.0 version).
2. source - A JsonObject with name and version fields. This describes the addon the documentation was generated for.

As expected then, the old JSONGenerator class has been deprecated, though it retains full functionality. The schema is the same as how the data map is described below for our implementations). You can also review our new documentation site's description of the output at https://github.com/SkriptLang/docs/blob/master/src/components/syntaxes/documentation.ts. This does not cover the high-level, but is is essentially:

version: {
  major: number,
  minor: number,
},
source: {
  name: string,
  version: string,
},
expressions: {
  [key: string]: Documentation, // key is same as Documentation.id
},
// same for other types of syntaxes, experiments, properties, etc.

Implementations

This API has been consistently applied throughout Skript's codebase. Most relevant classes make use of Documentable or DocumentationDocumentable. Methods inconsistent with the new structure have been deprecated, though full compatibility is retained. Below, I cover each class.

The diagram below (work in progress) provides an overview of the default outputs of Skript's classes:

SyntaxRegistry

SyntaxRegistry implements Documentable. It writes keys, scoping for each. For example, it will enter a scope for the Expression key, write all SyntaxInfos under that key, and then exit the scope. By default (the interface), it writes the STRUCTURE, SECTION, EFFECT, CONDITION, and EXPRESSION keys. Our default implementation writes all keys stored on it but STATEMENT. There is an extended write method for writing specific keys.

SyntaxInfo

SyntaxInfo implements DocumentationDocumentable. SyntaxInfo's builder now has a method for setting the Documentation. Direct origin related methods have been deprecated, since it is now part of Documentation.

SyntaxInfo overrides the write method to also write the following:

• patterns: List of strings from patterns. Patterns are compiled and stringified using the SkriptPattern stringification API (excludes parse tags, type flags).

There is also a Bukkit-specific "additional data" available for syntaxes: Events. It is a replacement for the @Events annotation. It implements Documentable and writes the following:

• events: List of references to Event SyntaxInfos

Skript's default gen-docs command will automatically resolve this data for syntaxes that implement EventRestrictedSyntax.

Expression

Expression overrides the write method to also write the following:

• returnType: returnType()

Structure

Structure overrides the write method to also write the following:

• entryValidator() (scoping is left to EntryValidator)
• nodeType: nodeType.name()

Event

Event overrides the write method to also write the following:

• cancellable: boolean (whether the event can be cancelled using EffCancelEvent)
• eventValues: list of EventValue (see below for output)

Classes

Since there is no registry for ClassInfos, there is a static write method available on Classes.
It writes the following under the scope types:

• All class infos

ClassInfo

ClassInfo implements DocumentationDocumentable. It overrides write to also write the following:

• properties: List of String-Object maps each containing:
  • property: Reference to a Property
  • origin: Origin for the thing that provided the Property implementation for this ClassInfo
  • description: Description for this Property on this ClassInfo (it's specific functionality).

The other unique feature of ClassInfo is that it has usage which is similar to pattern, but not part of Documentation. Thus, it is available to add as an additional data with the Usage subclass. Example:

docsBuilder.addData(Usage.of("usage info here"));

Usage is a Documentable that writes the following:

• usage: usage()

ExperimentRegistry

ExperimentRegistry implements Documentable. It writes the following under the experiments scope:

• All experiments

Experiment

Experiment implements DocumentationDocumentable. It overrides the write method to also write the following:

• phase: phase().name()
• pattern: string (same approach as SyntaxInfo)

FunctionRegistry

FunctionRegistry implements Documentable. It writes the following under the functions scope:

• All API-registered functions (i.e. not script functions)

Function (org.skriptlang version)

Function implements Documentable. It writes the following:

• returnType: signature().returnType() (only if not null)

It writes the following under the parameters scope:

• All parameters

DefaultFunction

DefaultFunction implements DocumentationDocumentable.
It writes the defaults of DocumentableDocumentable and Function.

JavaFunction

JavaFunction implements DocumentationDocumentable.
It writes the defaults of DocumentableDocumentable and Function.

Parameter

Parameter implements Documentable. It writes the following under the name() (the result of it) scope:

• name: name()
• type: type()
• plural: type().isArray()

It writes the following under the modifiers scope:

• All documentable modifiers

Modifier

Some modifiers implement Documentable.

OPTIONAL is Documentable. It writes the following:

• optional: true

RANGED is documentable. It writes the following under the ranged scope:

• min: Friendly string representation of min()
• max: Friendly string representation of max()

PropertyRegistry

PropertyRegistry implements Documentable. It writes the following under the properties scope:

• All properties

Property

Property implements DocumentationDocumentable. It overrides the write method to also write the following:

• types: List of references to ClassInfos implementing the property
• syntaxes: List of references to SyntaxInfos implementing the property

It also defines an "additional data" for Documentation to be used by syntaxes: RelatedProperty.
This is an equivalent for the existing annotation, @RelatedProperty. It implements Documentable and writes the following:

• relatedProperty: Reference to property()

EntryValidator

EntryValidator implements Documentable. It writes the following under the entries scope:

• All entry data

EntryData

EntryData implements Documentable. It writes the following under the getKey() (the result of it) scope:

• key: getKey()
• optional: isOptional()
• multiple: supportsMultiple

EventValue

EventValue implements Documentable. An event value writes:

• type: valueClass()
• plural: valueClass().isArray()
• time: time().name() (Enum name)
• patterns: List of strings (same approach as SyntaxInfo)
• supportedChangeModes: List of enum names (compiled via hasChanger)

EntityData

Since there is no registry for EntityData, there is a static write method available on EntityData.
It writes the following under the scope entitydatas:

• All non-simple EntityDatas

EntityDataInfo

This is the internal data structure representing an EntityData registration.
It dumps the same information as SyntaxInfo.
Currently however, it will only ever have 4 complete entries:

• Skript-based Origin
• Name
• ID
• Patterns

Site Switch

This pull request updates the documentation GitHub actions to start targeting the new docs repository (beta-docs). Once this pull request is merged, we can finalize the transition to the new site. It also makes the switch to the javadocs repository (javadocs.skriptlang.org/skript). Javadocs generation is now a separate action. It currently supports nightly javadocs, though we can debate this.

Final Notes

This pull request also includes a cleanup of Documentation in multiple places, including Experiments and Properties.

testDifferentOriginUnregistration was removed in SyntaxRegistryTest because Origin is relevant for how SyntaxRegister orders syntaxes (Skript vs not). We may want to reconsider the equality here.

Testing Completed

The output of the new JSON generator is used for https://beta-docs.skriptlang.org/syntaxes/ (both Skript and skript-worldguard outputs). Compatibility measures have been verified through existing tests.

---

Completes:

• When using multiline examples the docs generator breaks #8383 (in that HTMLGenerator is removed)
  Related: none
  AI assistance: none

[CrebsTheCoder]

Good job pickle

[AnOwlBe]

pickle is finally almost free from the docs..