Parse F# 6 index syntax xs[i] as an index_expression

[pcshrosbree]

Fixes #242.

Since F# 6, xs[i] with the [ touching the expression is an index. FSC's LexFilter turns that [ into HIGH_PRECEDENCE_BRACK_APP, and its checker treats an adjacent list as an index. f [i] with a space applies f to a list. The grammar gave both the same application_expression with a list_expression argument, while the older xs.[i] got an index_expression.

This is a syntactic reading. FSC resolves an adjacent [ by type, which a parser cannot see. I ran these through dotnet fsi:

• (xs)[1], f(10)[1] and xss[1][0] index.
• fs[0][7] with a function in fs[0], and (f)[5], apply, with no diagnostic.
• f[1; 2] with a function f applies, with the informational FS3365 asking for a space.

index_expression is right for the common case and is what the issue asks for first. The issue's alternative, an application marked as adjacent, would avoid asserting an index where the receiver is a function. I can switch to it if you prefer.

Change

• index_expression gains a second form: an expression followed by a touching [ (token.immediate), then the index or slice, then ]. Its body is scoped like a list's (_paren_indent … _dedent), so it may span lines, as ys[ + an indented let + ] does in an existing test.
• The touching [ has no lexical precedence. Tree-sitter ranks lexical precedence above match length, so any precedence would let it beat the longer [| and [] tokens: f[|1|] (an application, as in FSC) would lex as f[ + |1|], and null : string[] would lose its array type. Against a list's [ of the same length, the earlier-defined rule wins.
• f[] stays an application to the empty list. FCS parses it so, and there is nothing to index with. A touching [ immediately followed by ] is an empty list_expression. That form lives only in _low_prec_app's argument slot: in list_expression itself it would also claim every ([ and [[ at an expression start, and it did when I tried.

Tests

New tests in test/corpus/expr.txt:

• index versus application: xs[1], g [1; 2], g xs[0] (the index binds tighter than the application), xs[0][1], m["k"], (xs)[0];
• a touching array literal and an empty list stay application arguments: g[|1|], TreeNode[], [Seq[]; x]. This test passes on main too.

A third new test covers an under-indented index body (xs[ / 1 + / 2 / ] followed by a declaration). That pins the list-style scope: with a plain indent scope, the body breaks.

Every input parses without diagnostics in FCS (Fantomas.FCS 7.0.5). FCS marks the first bracket in each case as an atomic application, but it leaves the outer one non-atomic in (xs)[0], f(x)[0] and xs[0][1]. The compiler still indexes there, as the fsi runs above show.

Known limits

• A comment before a touching [. f (* c *)[0] becomes an index. The immediate token sees no whitespace once the comment extra is consumed.
• f[1; 2]. It becomes an index whose body is a sequential_expression, where FSC (with FS3365) applies f to a two-element list.
• From-end indexing. xs[^1] remains an ERROR, as on main.

12 existing tests change their expected trees. Each recorded xs[i] as an application of xs to a list:

• dot expression ((A[1]).B)
• index expression (test[test])
• index list expressions and index single list expressions ([2; 3; 4][1])
• array list expressions and array single list expressions ([|2; 3; 4|][1])
• call function from list of functions (fs[0] 0)
• index list with value declaration (a multi-line index body)
• match expression 3 rules (basePath[1])
• list index with slice ranges (xs[..1] and others)
• short for expression in computation expression and its let then variant (vars[key])

In each, the application of a list becomes an index_expression, and every new tree is error-free.

Corpus

• check:baseline: nothing parses worse. ProjectGeneration.fs goes from 1 to 0 error nodes and E_productioncoverage03.fs from 10 to 6.
• check:invalid: all recorded errors are still found.
• parser.c grows 4.0%.

[pcshrosbree]

This PR is one of a set; #255 maps them all, with their dependencies and merge order.