Embed notebooks into Scala CLI?

[susliko]

Hi 🖖🏻
Scala CLI is a great tool for prototyping, implementing PoCs and other sorts of scripting tasks.
However, it doesn't cover a major set of use cases related to analytics and ML.

These are usually covered by notebook environments (like Jupyter, Zeppelin or Polynote).
Typical workflow for them is as follows:

• Open an interactive session in a browser tab.
• Write code in cells with some code completion support.
• Execute cells in any order. Cells share common state.

So, can't we implement some scala-cli notebook command, which would open a browser tab and offer a similar experience?
With code completion via Metals (e.g. like in Scastie), dependency management via using directives and some toolkit with a set of solid libraries for plotting, operating csv files, etc.

[SethTisue]

I would assume this would be out of scope for the scala-cli project itself unless it were already available as a separate project.

Like once it exists, there could be a scala-cli subcommand that opens it. But it would have to exist first. (And yes, it would be fantastic if it did!)

This would be comparable to how scala-cli serves as a launcher for a whole slew of separately built projects: the Scala 2 and 3 compilers and REPLs and Scaladoc tools, scalafmt, scalafix (soon: #2968), Coursier... all of which existed before scala-cli did.

One thing that does exist already in the notebook space is Almond: https://almond.sh , which runs inside Jupyter. Do you think it would be useful for scala-cli to be able to launch Almond as it presently exists?

[susliko]

Yeah, implementing notebooks from scratch would require an enormous amount of resources and it's totally reasonable to build on an existing foundation like Polynote and Almond.

Polynote:

• Depends on: jvm, python3, pip, and a bunch of Python libraries.
• Has code completion.
• Has some support for dependency management (but I've found no documentation on this).
• Is close to being an abandoned project judging by code activity and activity in gitter channel.

Almond Jupyter kernel:

• Depends on: Jupyter (and hence on python3 and pip).
• Does not have a code completion. But looks like there was a successful attempt to make Metals work via jupyterlab-lsp
• Traditionally used ivy notation to manage dependencies, but some support for using dependencies have been added.

So, Almond indeed seems to be the only viable option here.

I'm a bit concerned about Python dependencies, as Scala CLI is able to download and set up everything automatically for other commands.