vscode & Roo Code support -- sessionId= is required #8

elasticdotventures · 2025-05-30T07:47:16Z

No description provided.

…ired sessionId= should not be required

d6e

lgtm

I realized I overlooked several issues

elasticdotventures · 2025-05-31T04:12:09Z

No worries. Are you going to merge it? (i'll switch back to your version rather than running on my fork)

d6e

Okay yeah, actually I think it's fine

Output is truncated to fit the specified token limit using Hugging Face's pretrained bert-base-cased tokenizer. 🦨 Skunky: Truncation is by character, not true token boundary; see code comment for future improvement. All changes build cleanly and are isolated to the explicit requirements.

version 0.2.0; --tldr --max_tokens & new list_crate_items (uses rust analyzer)

elasticdotventures · 2025-07-05T07:59:05Z

FYI - not sure why this wasn't merged, but I've continued to add features to my fork.

feature/list crate items

Co-authored-by: elasticdotventures <[email protected]>

- Update Dockerfile to use Debian slim instead of Alpine - Improve entrypoint.sh with better error handling - Add ci.yml workflow for builds/tests on PRs and main - Add release-and-publish.yml for multi-arch Docker + cross-compiled binaries - Update set-version.sh to use shell instead of Python for better portability - Update README.md with comprehensive Docker and release documentation Co-authored-by: elasticdotventures <[email protected]>

Co-authored-by: elasticdotventures <[email protected]>

…-compilation (#7) * Initial plan * Update Docker to Debian and add comprehensive CI/CD workflows Co-authored-by: elasticdotventures <[email protected]> * Fix GitHub Actions output redirection syntax Co-authored-by: elasticdotventures <[email protected]> * Update GitHub Actions to use newer versions and improve cross-compilation setup Co-authored-by: elasticdotventures <[email protected]> * Add Docker and pkgx MCP configuration examples to README Co-authored-by: elasticdotventures <[email protected]> --------- Signed-off-by: Brian Horakh <[email protected]> Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: elasticdotventures <[email protected]>

* Initial plan * Fix YAML syntax errors in rust.yml workflow Co-authored-by: elasticdotventures <[email protected]> * Fix apply_tldr logic to properly handle multiple LICENSE/VERSION sections Co-authored-by: elasticdotventures <[email protected]> --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: elasticdotventures <[email protected]>

* wip: testing pkgx & docker builds * feat: pkgx mcp registry * feat: pkgx --------- Signed-off-by: Brian Horakh <[email protected]>

Copilot

Pull Request Overview

This PR adds comprehensive support for VSCode and Roo Code MCP integration by making the sessionId query parameter optional in the HTTP SSE server, along with several major feature additions including TLDR mode, token counting, item enumeration, and complete Docker/CI/CD infrastructure.

Key Changes:

Made sessionId optional in HTTP SSE endpoint with automatic session creation
Added TLDR mode to filter out LICENSE/VERSION sections from documentation
Implemented token-aware truncation with configurable max_tokens limit
Added list_crate_items tool for enumerating crate contents
Introduced comprehensive Docker support with multi-arch builds and GHCR publishing
Added release automation with cross-compiled binaries

Reviewed Changes

Copilot reviewed 25 out of 28 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
src/transport/http_sse_server/http_sse_server.rs	Made sessionId optional, added automatic session creation for POST requests without sessionId
src/tools/tldr.rs	New module to filter LICENSE and VERSION sections from documentation output
src/tools/item_list.rs	New tool to enumerate items (structs, enums, traits, functions) in Rust crates
src/tools/mod.rs	Added count_tokens function using Hugging Face tokenizer
src/tools/docs/docs.rs	Added tldr and max_tokens support to DocRouter with post-processing logic
src/bin/cratedocs.rs	Added version command, tldr/max_tokens flags, list_crate_items tool support, and comprehensive tests
tests/integration_tests.rs	Updated tool count from 3 to 4, changed HTML assertions to markdown
Dockerfile	New multi-stage Debian-based build with entrypoint script
docker/entrypoint.sh	Shell script supporting http/stdio modes with environment configuration
.github/workflows/release-and-publish.yml	Automated release workflow with multi-arch Docker builds and cross-compiled binaries
.github/workflows/docker.yml	Docker build workflow with MCP registry publishing
.github/workflows/ci.yml	CI workflow for build and test automation
README.md	Comprehensive rewrite with Docker, pkgx, versioning, and MCP integration examples
server.json	MCP server manifest for registry publishing
Cargo.toml	Updated package name, version, repository, and dependencies
scripts/set-version.sh	Python-based version bumping script for Cargo.toml and Cargo.lock

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-15T23:14:27Z

.github/workflows/docker.yml

+        with:
+          context: .
+          push: true
+          platforms: linux/amd64


The Docker workflow builds only for linux/amd64 (line 54), while the release workflow builds for both linux/amd64 and linux/arm64. This inconsistency means that pushes to main will only have amd64 images, but tagged releases will have both architectures. Consider aligning the platforms across both workflows for consistency.

Suggested change

platforms: linux/amd64

platforms: linux/amd64,linux/arm64

Copilot · 2025-11-15T23:14:28Z

src/tools/tldr.rs

+    // Match any heading (for ending the skip)
+    let heading_re = Regex::new(r"^\s*#+").unwrap();
+    // Match <detail> tags including start, end, and inline attributes
+    let detail_tag_re = Regex::new(r"<[/]?detail.*?>").unwrap();


The regex pattern r"<[/]?detail.*?>" will only match lowercase "detail" tags. HTML tags are case-insensitive, so this should use the case-insensitive flag: r"(?i)<[/]?detail.*?>" to properly match <Detail>, <DETAIL>, etc.

Suggested change

let detail_tag_re = Regex::new(r"<[/]?detail.*?>").unwrap();

let detail_tag_re = Regex::new(r"(?i)<[/]?detail.*?>").unwrap();

Copilot · 2025-11-15T23:14:28Z

src/tools/docs/docs.rs

+                                if let Some(last_space) = truncated.rfind(' ') {
+                                    truncated.truncate(last_space);
+                                }
+                                truncated.push_str(" 内容被截断");


The Chinese text "内容被截断" (meaning "content truncated") should be internationalized or use English for consistency with the rest of the codebase. Consider using "... [truncated]" or a similar English message.

Suggested change

truncated.push_str(" 内容被截断");

truncated.push_str(" [content truncated]");

Copilot · 2025-11-15T23:14:28Z

docker/entrypoint.sh

+# If explicit args provided, run with those
+if [ "$#" -gt 0 ]; then
+  exec /usr/local/bin/cratedocs "$@"
+fi
+
+# default behavior: start in selected mode
+case "$MODE" in
+  http)
+    if [ "${DEBUG}" = "true" ]; then
+      exec /usr/local/bin/cratedocs http --address "$ADDRESS" --debug
+    else
+      exec /usr/local/bin/cratedocs http --address "$ADDRESS"
+    fi
+    ;;
+  stdio)
+    exec /usr/local/bin/cratedocs stdio
+    ;;
+  *)
+    echo "Unknown CRATEDOCS_MODE: $MODE" >&2
+    exit 2
+    ;;
+esac


The entrypoint script has duplicate logic for handling the MODE variable. Lines 8-16 execute the command, but then lines 18-38 have another case statement that would never be reached due to the exec call. This dead code should be removed.

Suggested change

# If explicit args provided, run with those

if [ "$#" -gt 0 ]; then

exec /usr/local/bin/cratedocs "$@"

fi

# default behavior: start in selected mode

case "$MODE" in

http)

if [ "${DEBUG}" = "true" ]; then

exec /usr/local/bin/cratedocs http --address "$ADDRESS" --debug

else

exec /usr/local/bin/cratedocs http --address "$ADDRESS"

fi

;;

stdio)

exec /usr/local/bin/cratedocs stdio

;;

*)

echo "Unknown CRATEDOCS_MODE: $MODE" >&2

exit 2

;;

esac

# (lines 17-39 removed; no replacement needed)

Copilot · 2025-11-15T23:14:28Z

README.md

+cargo run --bin cratedocs test --tool search_crates --query logger --format json
+
+# Summarize output (strip LICENSE and VERSION sections, limit tokens)
+cargo run --bin cratedocs test --tool lookup_crate --crate-name tokio --tldr --max_tokens 48000


In the "tldr" mode documentation, the --max_tokens flag should use underscores consistently with the actual parameter name. However, CLI flags typically use hyphens (kebab-case), so this should be --max-tokens to match standard CLI conventions.

Suggested change

cargo run --bin cratedocs test --tool lookup_crate --crate-name tokio --tldr --max_tokens 48000

cargo run --bin cratedocs test --tool lookup_crate --crate-name tokio --tldr --max-tokens 48000

Copilot · 2025-11-15T23:14:29Z

src/transport/http_sse_server/http_sse_server.rs

+                let app_clone = app.clone();
+                let task_session_id = new_session_id_arc.clone();
+                tokio::spawn(async move {
+                    let router = RouterService(DocRouter::new());


The DocRouter::new() is used here, but the new session creation logic doesn't respect the tldr and max_tokens settings that might be configured elsewhere in the application. Consider passing these settings or using a consistent configuration mechanism across the application.

Suggested change

let router = RouterService(DocRouter::new());

// Retrieve tldr and max_tokens from app_clone or config

let tldr = app_clone.tldr;

let max_tokens = app_clone.max_tokens;

let router = RouterService(DocRouter::with_config(tldr, max_tokens));

Copilot · 2025-11-15T23:14:30Z

src/tools/mod.rs

+
+pub fn count_tokens(text: &str) -> Result<usize, tokenizers::Error> {
+    // 🦨 skunky: This loads the tokenizer from Hugging Face Hub every call; cache for production.
+    let tokenizer = Tokenizer::from_pretrained("bert-base-cased", None)?;


The tokenizer is loaded from Hugging Face on every call. As the comment notes, this should be cached for production use. Consider implementing a lazy_static or once_cell pattern to cache the tokenizer instance across calls to avoid repeated downloads and initialization overhead.

Suggested change

pub fn count_tokens(text: &str) -> Result<usize, tokenizers::Error> {

// 🦨 skunky: This loads the tokenizer from Hugging Face Hub every call; cache for production.

let tokenizer = Tokenizer::from_pretrained("bert-base-cased", None)?;

use once_cell::sync::Lazy;

static TOKENIZER: Lazy<Result<Tokenizer, tokenizers::Error>> = Lazy::new(|| {

Tokenizer::from_pretrained("bert-base-cased", None)

});

pub fn count_tokens(text: &str) -> Result<usize, tokenizers::Error> {

// 🦨 skunky: Tokenizer is now cached using once_cell::sync::Lazy for production use.

let tokenizer = TOKENIZER.as_ref()?;

Copilot · 2025-11-15T23:14:30Z

src/tools/item_list.rs

+
+/// Utility function to download and cache crate source.
+async fn download_and_cache_crate(crate_name: &str, version: &str) -> Result<String> {
+    let cache_dir = Path::new("./cache");


The cache directory ./cache is relative to the current working directory, which can cause issues when the binary is run from different locations. Consider using a system-appropriate cache directory (e.g., user's cache directory via dirs crate) or making this configurable.

Copilot · 2025-11-15T23:14:30Z

README.md

+      "args": [
+        "-y",
+        "mcp-remote@latest",
+        "http://127.0.0.1:3000/sse?sessionId=",


The URL parameter in the example shows sessionId= with no value. According to the code changes, sessionId is now optional and should either be omitted entirely or have a value. The empty value sessionId= may cause parsing issues. Update the example to either remove sessionId= or provide a placeholder value like sessionId=abc123.

Suggested change

"http://127.0.0.1:3000/sse?sessionId=",

"http://127.0.0.1:3000/sse?sessionId=abc123",

Copilot · 2025-11-15T23:14:30Z

.github/workflows/release-and-publish.yml

+      - name: Upload x86_64 binary to release
+        uses: softprops/action-gh-release@v1
+        with:
+          files: target/x86_64-unknown-linux-gnu/release/cratedocs
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Upload aarch64 binary to release
+        uses: softprops/action-gh-release@v1
+        with:
+          files: target/aarch64-unknown-linux-gnu/release/cratedocs


The uploaded binaries have identical names (cratedocs) for both architectures, which means the second upload will overwrite the first. The filenames should include the architecture (e.g., cratedocs-x86_64 and cratedocs-aarch64) or be placed in separate directories to prevent conflicts.

elasticdotventures and others added 3 commits May 30, 2025 07:43

sessionId= should not be required

05c1878

Merge pull request #1 from PromptExecution/feature/sessionId-not-requ…

1e287a7

…ired sessionId= should not be required

example with stdio

30545c1

d6e previously approved these changes May 30, 2025

View reviewed changes

added install

d92a961

d6e approved these changes May 31, 2025

View reviewed changes

elasticdotventures and others added 9 commits July 5, 2025 05:45

--tldr checkpoint 1

faa44e7

--tldr checkpoint 2

304ff1d

added count_tokens

4afe1e7

added 内容被截断

f891b41

improved readme

04aeafd

list_crate_items

e98ed75

list_crate_items

96442c1

Merge pull request #2 from PromptExecution/feature/tldr

34f5e7a

version 0.2.0; --tldr --max_tokens & new list_crate_items (uses rust analyzer)

elasticdotventures and others added 12 commits July 5, 2025 08:15

checkpoint 1, broken

f37981e

checkpoint 2, working

ae1fc42

checkpoint, moved list_crate_items moved to tools

fd18fe7

checkpoint, 1 test fails

fc3f2f9

added version tool

b2df171

--tldr added <detail> tag stripping

579f0c1

checkpoint, syn in - but missing

58b7680

list_crate_items appears to work!

a13d2be

Merge pull request #3 from PromptExecution/feature/list-crate-items

948a515

feature/list crate items

tdlr didn't work with stdio

ac46dde

added tldr.rs

5e5ba5a

added --max_tokens ### to stdio mode

c48014e

elasticdotventures and others added 10 commits November 13, 2025 08:25

wip: testing pkgx & docker builds

a9b7fb2

Initial plan for pkgx build support

78da690

Co-authored-by: elasticdotventures <[email protected]>

Fix crate name references in binary and tests

df250b4

Co-authored-by: elasticdotventures <[email protected]>

Add pkgx support with package.yml configuration

c2ce396

Co-authored-by: elasticdotventures <[email protected]>

Add PR summary documentation

9e780c6

Co-authored-by: elasticdotventures <[email protected]>

Update PR summary with branch instructions

9eb42c1

Co-authored-by: elasticdotventures <[email protected]>

feat/build pkgx install (#9)

9584b65

* wip: testing pkgx & docker builds * feat: pkgx mcp registry * feat: pkgx --------- Signed-off-by: Brian Horakh <[email protected]>

Copilot AI review requested due to automatic review settings November 15, 2025 23:09

Copilot started reviewing on behalf of elasticdotventures November 15, 2025 23:10 View session

Copilot finished reviewing on behalf of elasticdotventures November 15, 2025 23:12

Copilot AI reviewed Nov 15, 2025

View reviewed changes

	let detail_tag_re = Regex::new(r"<[/]?detail.*?>").unwrap();
	let detail_tag_re = Regex::new(r"(?i)<[/]?detail.*?>").unwrap();

	truncated.push_str(" 内容被截断");
	truncated.push_str(" [content truncated]");

	cargo run --bin cratedocs test --tool lookup_crate --crate-name tokio --tldr --max_tokens 48000
	cargo run --bin cratedocs test --tool lookup_crate --crate-name tokio --tldr --max-tokens 48000

-                    let router = RouterService(DocRouter::new());
+                    // Retrieve tldr and max_tokens from app_clone or config
+                    let tldr = app_clone.tldr;
+                    let max_tokens = app_clone.max_tokens;
+                    let router = RouterService(DocRouter::with_config(tldr, max_tokens));

	"http://127.0.0.1:3000/sse?sessionId=",
	"http://127.0.0.1:3000/sse?sessionId=abc123",

vscode & Roo Code support -- sessionId= is required #8

Are you sure you want to change the base?

vscode & Roo Code support -- sessionId= is required #8

Uh oh!

Conversation

elasticdotventures commented May 30, 2025

Uh oh!

d6e left a comment

Choose a reason for hiding this comment

Uh oh!

elasticdotventures commented May 31, 2025

Uh oh!

d6e left a comment

Choose a reason for hiding this comment

Uh oh!

elasticdotventures commented Jul 5, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants