Switch DB from better-sqlite3 to libsql

[Lokowitz]

Community Contribution License Agreement

By creating this pull request, I grant the project maintainers an unlimited,
perpetual license to use, modify, and redistribute these contributions under any terms they
choose, including both the AGPLv3 and the Fossorial Commercial license terms. I
represent that I have the right to grant this license for all contributed content.

Description

Hey guys,
this is related to the issue that better-sqlite3 is not supporting async anymore.
This PR is an example how we could do the step to libsql with a minimum of effort.
This will require all users to migrate first to the latest available version (current 1.12.2) because I deleted all old migration files for sql because otherwise they need to be completely rewritten.
I added a check in server/setup/migrationsSqlite.ts to don't skip this task.

If you don't want to go this way and want to rewrite all migration files, than feel free to take over because this is out of my skills :-D

Cheers Marvin

[Lokowitz]

@oschwartz10612 fixed the merge conflicts. So feel free to review :)

[oschwartz10612]

Thanks @Lokowitz!

Not opposed to switching to libsql necessarily. Forgetting if we discussed this earlier... is there a way we can keep better-sqlite3 and adjust the code to not complain about the async functions or that migration would be too difficult?

[marcschaeferger]

@oschwartz10612 Yes, there are ways to keep better-sqlite3 for example by refactoring everything to fully synchronous transactions or by pinning to 11.9.1 (which effectively means no proper Node 24 support). We could even introduce our own transaction wrapper for the async/sync differences. All of those options would technically work, but they either lock us into an older better-sqlite3 version or require a non-trivial refactor and avoiding re-introducing async transaction callbacks. From a maintenance and future-proofing perspective, I still think adopting libSQL is the cleaner and less risky path.

What I would not do, though, is delete the old migration files. In theory we can keep all versions compatible and allow a direct upgrade from e.g. 1.10 → 1.13 without forcing users through 1.12.2 first. Drizzle will happily run all missing migrations in order as long as all the scriptsSqlite/*.ts files are still present and work with the new libSQL driver.

That does mean a bit more work (adapting and testing the legacy migrations), but in return we don’t artificially constrain users to a specific intermediate version just because we dropped older migrations from the codebase.

You can see some discussion in the following Issue #1492