fix(index): compile in-process when the serving image has no guest rootfs - #12
Closed
philocalyst wants to merge 21 commits into
Closed
philocalyst wants to merge 21 commits into
philocalyst wants to merge 21 commits into
Conversation
buildRustPackage refuses to vendor turso_versioning-0.8.0-pre.7 without an outputHashes entry, so the nix2container backend image cannot build. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
smolvm's submodules are declared as git@github.com URLs. The sandboxed fetch has no ssh client, so cargo vendoring of the backend image failed. A git insteadOf rewrite sends those clones to https://github.com/. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
The HTTPS submodule checkout changes the smolfile/smolvm git tree. Pin the fixed-output hash Nix computed for that tree. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
The lockfile has a crates.io trustfall and a git fork at the same version. importCargoLock linked both to one directory name, and the second link failed inside the read-only first crate. A name collision keeps the later tree under its store path. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
importCargoLock cannot name both trustfall 0.8.1 crates, and cargo's git fetch missed the HTTPS rewrite that fetchgit uses. Vendor with cargoHash and point GIT_CONFIG_GLOBAL at the GitHub insteadOf config so smolvm submodules clone over HTTPS. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
The serving binary is feature-gated. The image package now passes buildFeatures = [ "server" ] through mkRustService. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
The serving crate graph uses #![feature] on the fenix stable toolchain. RUSTC_BOOTSTRAP=1 is the same switch the workspace tests already use. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
The host LLVM cannot select avx512.vpdpbusd when crates target the native CPU. Pin RUSTFLAGS to x86-64-v3 so the serving binary links. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
Thin LTO re-lowers avx512.vpdpbusd after the target feature is dropped, and this LLVM cannot select that intrinsic. Release LTO stays off for the nix2container serving image. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
qdrant-edge compiles its vnni kernels on their own. Thin LTO and a forced x86-64-v3 target both ask LLVM to select vpdpbusd without that feature. Drop the CPU pin and leave release LTO off. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
This LLVM cannot lower vpdpbusd even in per-function target_feature objects. Set qdrant_edge_no_avx512_vnni for mkRustService and skip the VNNI kernels at compile time; runtime dispatch keeps the AVX2 path. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
…hits A lib.rs-style card needs the signals that ordered the hit. Package search already ranks with dependents, and the wire hit now carries that count plus keywords and license from the same facets. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
The script checks health, an empty query, a serde search page, and that dependents do not rise as scores fall. It exits 2 when nothing is listening. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
Three indexed crates with a shared keyword sort by dependents. A dependents mass that stays under the popularity clamp beats a download count just below that mass. An exact name beats a keyword when popularity ties. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
The gitignored doltlite.c amalgamation is pinned in manifest.toml but absent from the flake source snapshot. Add a fixed-output fetch and copy the three engine files into workspace/vendor/doltlite during mkRustService postPatch so nudox-serve links the real DoltLite catalog engine. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
The server requires Qdrant at assembly time. Pin the upstream musl release binary (nixpkgs qdrant fails AVX-512 codegen here) and add nudox-backend-run to start Qdrant then nudox-serve with durable paths under /var/lib/nudox. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
Linux serving images have no Smolvm golden. Choosing the cage only because the host is Linux left every package Unindexed. Use the in-process producer unless NUDOX_GUEST_ROOTFS is set, and create /tmp before the first tempfile. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
The live probe now ingests serde, polls until the package is Stored, and requires a package card with description, keywords, or license plus a Serialize symbol hit. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
In-process Rust indexing runs cargo metadata and build-script checks. The live serde job reached Stored only after those tools and a linker were on PATH. The image now carries them, and the container probe reads the HTTP status from the response instead of a lost subshell variable. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
Indexing spawns tantivy merge threads. With the SIGPROF sampler armed, nudox-serve aborts during serde compile. OTEL_SDK_DISABLED leaves logs and /metrics up. Co-authored-by: Miles Wirht <philocalyst@users.noreply.github.com>
This was referenced Sep 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Linux serving image has no Smolvm guest rootfs, so every package failed in the cage and stayed unindexed. Compile strategy now follows whether
NUDOX_GUEST_ROOTFSis provisioned:The image entrypoint creates
/tmpbefore the first tempfile, and now also putscargo,rustc, and the gccccwrapper onPATHwith a writableCARGO_HOME. In-process Rust indexing runscargo metadataand build-scriptcargo check. Without the linker, serde failed inCompilingwithlinker cc not found.tests/docker/lib_rs_indexed.shingests serde, waits until it is Stored, and requires a package card plus aSerializesymbol hit.Tests
RUSTC_BOOTSTRAP=1 cargo test -p index --features server --lib select_compile_strategy— 3 passed.Against a container with cargo, rustc, and cc on PATH,
tests/docker/lib_rs_indexed.shpassed: serde Stored, description "A generic serialization/deserialization framework", keywords, licensemit or apache-2.0, downloads present, and a Serialize symbol hit. The image rebuild bakes those tools in so the container no longer depends on a host mount.Need help on this PR? Tag
@codesmithwith what you need. Autofix is disabled.