@nmnmcc/toolbox

Composable middleware for structured LLM calls with TypeScript

Introduction

@nmnmcc/toolbox is a TypeScript library for building structured, type-safe LLM applications with composable middleware. It provides a functional approach to defining language model interactions with strong type inference, schema validation via Zod, and a middleware pattern inspired by web frameworks.

Key Features:

Type-safe - Full TypeScript support with automatic type inference
Composable - Middleware-based architecture for building complex behaviors
Schema-driven - Uses Zod for runtime validation and structured outputs
Extensible - Easy to create custom middlewares and initializers

Installation

npm install @nmnmcc/toolbox zod openai

The library has peer dependencies on zod (^4) and openai (^6).

TypeScript Compatibility

This library requires TypeScript 5.0+ with strict mode enabled.

Package version	TypeScript	Node.js
0.4.x+	5.0+	18+

The `describe` Function

Migration Notes

This release aligns the documentation with the current runtime and type definitions in the codebase. If you're upgrading from earlier versions, note the following changes:

context.messages -> context.history: The middleware context now exposes a history property which is an array of tuples [message, completions], where completions is an array of OpenAI chat completions associated with the message.
Finalizers now receive the full LanguageModelMiddlewareContext and return the parsed output value (not the context object).
memory middleware options: max_messages was renamed to max_history.
For logging or middleware examples that need access to tokens/usage, use result.history to extract the latest completion usage:

const usage = result.history.at(-1)?.[1].at(-1)?.usage;

The describe function is the core of the library. It has two forms:

Describing Regular Functions

Wrap any function with metadata for use as tools in ReAct agents:

import { z } from "zod";
import { describe } from "@nmnmcc/toolbox";

const add_numbers = describe(
	{
		name: "add_numbers",
		description: "Add two numbers together",
		input: z.object({
			a: z.number().describe("First addend"),
			b: z.number().describe("Second addend"),
		}),
		output: z.object({ sum: z.number().describe("Sum of both numbers") }),
	},
	async ({ a, b }) => {
		return { sum: a + b };
	},
);

const result = await add_numbers({ a: 1, b: 2 });
console.log(result.sum); // 3

Describing LLM-Powered Functions

Create functions backed by language models with a middleware chain:

import { z } from "zod";
import { describe } from "@nmnmcc/toolbox";
import { initializer } from "@nmnmcc/toolbox/initializers/initializer";
import { finalizer } from "@nmnmcc/toolbox/finalizers/finalizer";
import { retry } from "@nmnmcc/toolbox/middlewares/retry";
import { logging } from "@nmnmcc/toolbox/middlewares/logging";

const summarize = describe(
	{
		name: "summarize",
		description: "Summarize the provided text",
		input: z.object({ text: z.string().describe("Text to summarize") }),
		output: z.object({ summary: z.string().describe("A concise summary") }),
		model: "gpt-4o",
		temperature: 0.7,
	},
	[
		initializer(
			"You are a helpful assistant that summarizes text concisely.",
		),
		logging(),
		retry(2),
		finalizer(),
	],
);

const result = await summarize({ text: "Long article text goes here..." });
console.log(result.summary);

Type Signature:

type Description<Input, Output> = {
	name: string;
	description: string;
	input: Input; // Zod schema
	output: Output; // Zod schema
};

type LanguageModelDescription<Input, Output> = Description<Input, Output> & {
	model: string; // OpenAI model name
	temperature?: number;
	max_tokens?: number;
	// ... any OpenAI ChatCompletionCreateParams
	client?: OpenAI; // Optional custom OpenAI client
};

// For regular functions
function describe<Input, Output>(
	description: Description<Input, Output>,
	implementation: (input: z.input<Input>) => Promise<z.output<Output>>,
): Described<Input, Output>;

// For LLM-powered functions
function describe<Input, Output>(
	description: LanguageModelDescription<Input, Output>,
	imports: [initializer, ...middlewares, finalizer],
): Described<Input, Output>;

Extending the Library

The library is designed to be extended with custom middleware. This section explains how to create your own middleware to add custom behavior to your LLM calls.

Creating Custom Middleware

Middleware in this library follows a pattern similar to Express.js or Koa. Each middleware is a function that receives a context and a next function, allowing you to:

Inspect or modify the request context before calling the LLM
Call next(context) to continue the chain
Inspect or modify the completion result after the LLM call
Short-circuit the chain by returning early (e.g., for caching)

Middleware Interface

import type {
	LanguageModelMiddleware,
	LanguageModelMiddlewareContext,
	LanguageModelMiddlewareNext,
	LanguageModelOutputContext,
} from "@nmnmcc/toolbox";

type LanguageModelMiddleware<Input, Output> = (
	context: LanguageModelMiddlewareContext<Input, Output>,
	next: LanguageModelMiddlewareNext<Input, Output>,
) => Promise<LanguageModelMiddlewareContext<Input, Output>>;

Context Structure:

type LanguageModelMiddlewareContext<Input, Output> = {
	description: LanguageModelDescription<Input, Output>;
	initializer: LanguageModelInitializer<Input, Output>;
	middlewares: LanguageModelMiddleware<Input, Output>[];
	finalizer: LanguageModelFinalizer<Input, Output>;
	usage: OpenAI.CompletionUsage;
	input: z.output<Input>;
	tools?: OpenAI.Chat.Completions.ChatCompletionFunctionTool[];
	history: [
		OpenAI.Chat.ChatCompletionMessageParam,
		OpenAI.Chat.Completions.ChatCompletion[],
	][];
};
type LanguageModelOutputContext<Input, Output> =
	& LanguageModelMiddlewareContext<Input, Output>
	& { output: z.output<Output> };

Custom Middleware Examples

Example 1: Simple Logging Middleware

import type { LanguageModelMiddleware } from "@nmnmcc/toolbox";

const simple_logger = <Input, Output>(): LanguageModelMiddleware<
	Input,
	Output
> => {
	return async (context, next) => {
		console.log(`[${context.description.name}] Starting call`);
		const start = Date.now();

		const result = await next(context);

		const elapsed = Date.now() - start;
		console.log(`[${context.description.name}] Completed in ${elapsed}ms`);

		return result;
	};
};

Example 2: Response Transformation Middleware

import type { LanguageModelMiddleware } from "@nmnmcc/toolbox";

const add_prefix = <Input, Output>(
	prefix: string,
): LanguageModelMiddleware<Input, Output> => {
	return async (context, next) => {
		const result = await next(context);

		// Modify the response content
		const message = result.history.at(-1)?.[1].at(-1)?.choices[0]?.message;
		if (message?.content) {
			message.content = prefix + message.content;
		}

		return result;
	};
};

Example 3: Caching Middleware

import type { LanguageModelMiddleware } from "@nmnmcc/toolbox";

const simple_cache = <Input, Output>(
	store: Map<string, any>,
): LanguageModelMiddleware<Input, Output> => {
	return async (context, next) => {
		const cache_key = `${context.description.name}:${
			JSON.stringify(context.input)
		}`;

		// Check cache
		const cached = store.get(cache_key);
		if (cached) {
			console.log("Cache hit!");
			return cached;
		}

		// Call LLM and cache result
		const result = await next(context);
		store.set(cache_key, result);

		return result;
	};
};

Example 4: Error Handling Middleware

import type { LanguageModelMiddleware } from "@nmnmcc/toolbox";

const error_handler = <Input, Output>(
	on_error: (error: Error) => void,
): LanguageModelMiddleware<Input, Output> => {
	return async (context, next) => {
		try {
			return await next(context);
		} catch (error) {
			on_error(error as Error);
			throw error;
		}
	};
};

Example 5: Request Modification Middleware

import type { LanguageModelMiddleware } from "@nmnmcc/toolbox";

const add_context = <Input, Output>(
	additional_context: string,
): LanguageModelMiddleware<Input, Output> => {
	return async (context, next) => {
		// Add additional context to history
		const modified_context = {
			...context,
			history: [
				...context.history,
				[{ role: "system" as const, content: additional_context }, []],
			],
		};

		return await next(modified_context);
	};
};

Example 6: Composing Multiple Middlewares

import type { LanguageModelMiddleware } from "@nmnmcc/toolbox";

const aggregator = <Input, Output>(
	...middlewares: LanguageModelMiddleware<Input, Output>[]
): LanguageModelMiddleware<Input, Output> => {
	return async (context, next) => {
		const chain = middlewares.reduceRight(
			(prev, curr) => (ctx) => curr(ctx, prev),
			next,
		);
		return chain(context);
	};
};

// Usage
const standard_middlewares = aggregator(logging(), retry(3), timeout(30000));

Built-in Components

This library comes with several built-in components to help you get started quickly. For detailed documentation, see:

Initializers - Convert input into initial message arrays
- initializer - Standard initializer with system prompt
Middlewares - Composable middleware for common patterns
- cache - Cache LLM responses
- memory - Maintain conversation history
- retry - Retry failed calls
- logging - Log execution metrics
- timeout - Add timeouts
- react - ReAct pattern for tool calling
- otel - OpenTelemetry tracing
- aggregator - Compose multiple middlewares
Finalizers - Extract output from completions
- finalizer - Standard JSON parser

Build & release

This repo is wired up as a turborepo that runs tsdown builds and tsc checks per package.

pnpm run build runs turbo run tsdown, which sequentially executes each package's tsdown --filter packages/<name> script. The shared tsdown.config.ts ensures a consistent Node 18 target, declaration generation, and dist output for every package.
pnpm run typecheck runs turbo run check, which invokes tsc --noEmit inside every package.
Each package publishes the contents of dist. After building, you can publish individual packages with standard pnpm --filter <name> publish commands (or whatever release flow you prefer).

License

LGPL-2.1-or-later

Name		Name	Last commit message	Last commit date
Latest commit History 25 Commits
.vscode		.vscode
packages		packages
.gitignore		.gitignore
.prettierrc.json		.prettierrc.json
LICENSE		LICENSE
README.md		README.md
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
tsconfig.json		tsconfig.json
tsconfig.tsbuildinfo		tsconfig.tsbuildinfo
tsdown.config.ts		tsdown.config.ts
turbo.json		turbo.json

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

@nmnmcc/toolbox

Introduction

Table of Contents

Installation

TypeScript Compatibility

The `describe` Function

Migration Notes

Describing Regular Functions

Describing LLM-Powered Functions

Extending the Library

Creating Custom Middleware

Middleware Interface

Custom Middleware Examples

Example 1: Simple Logging Middleware

Example 2: Response Transformation Middleware

Example 3: Caching Middleware

Example 4: Error Handling Middleware

Example 5: Request Modification Middleware

Example 6: Composing Multiple Middlewares

Built-in Components

Build & release

License

About

Uh oh!

Releases

Contributors 2

Uh oh!

Languages

License

NMNMCC/toolbox

Folders and files

Latest commit

History

Repository files navigation

@nmnmcc/toolbox

Introduction

Table of Contents

Installation

TypeScript Compatibility

The describe Function

Migration Notes

Describing Regular Functions

Describing LLM-Powered Functions

Extending the Library

Creating Custom Middleware

Middleware Interface

Custom Middleware Examples

Example 1: Simple Logging Middleware

Example 2: Response Transformation Middleware

Example 3: Caching Middleware

Example 4: Error Handling Middleware

Example 5: Request Modification Middleware

Example 6: Composing Multiple Middlewares

Built-in Components

Build & release

License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Contributors 2

Uh oh!

Languages

The `describe` Function