SILE picks the "wrong" Libertinus Math if several are available

[KMickle]

I hit a problem with the current SILE version: it was not able to render any math. The error message clearly (as clearly as stack traces go) indicated the font metrics table for the math font was nil.
I read that "Libertinus Math" is considered the default font for SILE math. I installed that to no avail. I then downloaded the example math code and it did not render either.
Later I discovered that there are actually two versions of "Libertinus Math" in the wild: the ttf version and the otf version. The ttf version won't do. The otf version works fine.
Now the most confusing part: the library responsible for the font matching prefers ttf Libertinus over otf, so math rendering fails if both ttf and otf are available.
It would be good if the manual stated which fonts are "good" for math. Also, it is probably wise to scan all the font matches and pick the "best" match, not the "first".
I suspect this problem extends to non-math fonts, too, but not so noticeable there.

[Omikhleia]

Greetings.
Try running SILE with -d fonts (maybe other debug flags would be interesting too, but that's a start) and see what it reports. How SILE resolves fonts is system-dependent (e.g. using fontconfig for font matching on Linux) and might not have to do with SILE if the "wrong" font is picked at "system"-level...
Another options, of course, is to set font folders properly (e.g., I use SILE with a curated list of fonts in Docker images, just to be sure I have "somewhat" reproducible builds on my dev. platforms).

[Omikhleia]

if both ttf and otf are available.

N.B. Also try to check (with otfinfo or similar tools) which versions of the fonts are used. Unless there's a huge version discrepancy (e.g. some fairly old version), it might be a question to ask to the Libertinus folks, since even the latest version includes both TTF and OTF builds (what's the subtle difference for end-users?)

[KMickle]

It hasn't been two weeks since I started setting SILE up, so all the named fonts should be fresh.

Running with -d fonts is actually how I discovered the problem. It showed SILE has found the ttf Libertinus Math and stopped there.

I'll try to obtain an MWE when I'm doing the next setup.

Thank you for the answer! I need a while to think and experiment on that.

[KMickle]

Okay, I have some new input on the problem.

I have set SILE up on one more machine. Was quite a quest, as it refused to build claiming 'Gentium Plus' was not found. The symptoms were as follows:

1. Gentium Plus font files were in /usr/share/fonts;
2. Gentium Plus font files were readable for all users;
3. fc-match did not find Gentium Plus;
4. fc-match DID FIND Gentium Plus as root user.

The problem turned out to be a per-user config file in ~/.config/fontconfig that prohibited the use of Gentium Plus explicitly. Whether I did that a while back or not -- I cannot recall.

This is not directly related to the problem; I just don't know where else to put it.

Here is what happened next. I have three instances of the "Libertinus Math" font in my fonts folder:

It seems, fontconfig thinks they're all good (fc-match "Gentium Plus") and happily returns the first font in the list. Here, otf font is the first one, so SILE works.

I then deleted the first font. The second one is a ttf. SILE failed with Error: runtime error: /usr/local/share/sile/packages/math/base-elements.lua:61: attempt to index local 'mathTable' (a nil value)

I then deleted the second font. The third one is an otf, so SILE started typesetting math successfully once again.

It would seem, fontconfig is not sophisticated enough for tools like SILE. fontconfig just picks "any Leela", but it obviously won't do here. Honestly, I was close to quitting at the point when it didn't work. I am afraid, many people would.

[Omikhleia]

fontconfig is not sophisticated enough for tools like SILE.

It would probably lead to similar problems for any other software using fonts on a given system. (I am fairly sure I once hit a similar issue with LibreOffice and/or Gimp, however these find the fonts...). (And beyond fonts, there may be issues witth Lua versions, other tools such as Graphviz, &c.).

I am not sure what is "actionable" for SILE in this issue. (?)

[KMickle]

Similar effect -- yes. Similar problems -- no.

For other software it may impact how stuff looks. Here it causes the software (SILE!) to fail with an unprocessed exception (not even a sane error message!), with all the official requirements met. Which is a bad thing.

As I see it, it boils down to these two points:

1. fontconfig is obviously inadequate for SILE's needs: it just tries to pick whatever looks like what the user wants, but SILE requires more than that. As I see it, fontconfig is designed this way on purpose, to avoid a failure at any cost. Ironically, this type of behavior causes SILE to -- exactly -- fail. So it would seem to me, SILE is either misusing this tool or underusing its features. The latter means fontconfig might be able to sort through font types and math measurements table support, as long as SILE asks for exactly what it means. The word is "might", because I did not check and have no idea.
2. Crashing with an unhandled exception is a no-go whenever this happens in production.
3. Since SILE can only use special kind of fonts for math typesetting, that should at least be noted in the User Manual.

As to how I would progress:

1. Handle the exception at base-elements.lua:61 and drop a reasonable message aboud what happened and why;
2. Put an emphasis in the User Manual (math package) that the font being an otf and having [whatever properties it must have] is a strict requirement, and the ttf counterparts of these fonts should be uninstalled or disabled;
3. Check if fontconfig suite can look for the needed font not only by name, but also by type and by required technical data. If it can, SILE should make use of it.

My Lua skills are currently at "let's look up how we use a Lua loop statement" at the moment, so I dare not try to do it myself at the moment, nor I'm in position to make suggestions, being totally new to the community. If I may be of any help with some specific simple tasks, please let me know.

[Omikhleia]

not even a sane error message!
(...) Handle the exception at base-elements.lua:61 and drop a reasonable message about what happened and why;

I wholly agree, the error should be better caught, and much earlier even, and with a better message. I also had it once (but on a branch toying with experiments and unusual fonts, yet I had a hard time understanding the problem wasn't my modified code, but just one of the fonts...)
That's a good first issue actually. It's free open source software, contributions should be welcome.

(I won't comment now on the other points -- They go beyond my own skills and/or opinions)

[KMickle]

I only hope it shall help in development and eventually in forging a faithful userbase. The tool is the only faithful descendant of TeX, it totally deserves a chance to thrive

[Omikhleia]

Just for extra reference, this is actually a know issue on Libertinus Math, which caused headaches in other typesetting engines too: alerque/libertinus#571