Intercal is probably the oldest esolang and it's a really interesting test case. Its whole schtick is that it's supposed to be nothing like any other programming language. So it has interesting semantics like flow control being done through "abstain" and "reinstate" which blocks certain instructions from the interpreter. The operators are select, mingle and "unary binary" operations which do not exist in any other programming language except select is pext in x86 assembly. The output format is obscure and is built around a tape head going around the wrong side of a circular tape.
Famously this is the language where you have to be the right level of polite for the compiler to accept the program.
It's a language where just writing basic things requires a lot of out of the box thinking but it's definitely possible for humans to wrap their head around it. So that's probably the perfect test for if LLMs can reason in a strange programming language with minimal training data.
Intercal is probably the oldest esolang and it's a really interesting test case. Its whole schtick is that it's supposed to be nothing like any other programming language. So it has interesting semantics like flow control being done through "abstain" and "reinstate" which blocks certain instructions from the interpreter. The operators are select, mingle and "unary binary" operations which do not exist in any other programming language except select is pext in x86 assembly. The output format is obscure and is built around a tape head going around the wrong side of a circular tape.
Famously this is the language where you have to be the right level of polite for the compiler to accept the program.
It's a language where just writing basic things requires a lot of out of the box thinking but it's definitely possible for humans to wrap their head around it. So that's probably the perfect test for if LLMs can reason in a strange programming language with minimal training data.