Extract structured data and insights from documents
using LLMs and decision models

Describe, don't prompt.

Quickstart · How it works · Agentic workflows · Changelog · Documentation

^{An illustrative contract. Every extracted fact is traced back to its source.}

Note

New in 1.0: decision models. Next to LLMs, ContextGem now works with decision models, which answer closed questions (yes/no, one of your labels, a level of a scale) with the probability of each possible answer, fast and at a low cost. A model group sends each aspect and concept to the model that suits it, and any model can fall back on others: see LLMs and decision models.

Upgrading from 0.x? Some classes and parameters are renamed or removed, some defaults have changed, and Python 3.10 is no longer supported. Follow the migration guide, or point a coding agent at llms.txt, the index of the documentation.

ContextGem is a free, open-source framework that does the work around the models for you: the prompts, the data models, the source references and the justifications, with minimal code.

Why ContextGem

Reliable extraction from documents usually means writing prompts, designing validation models, mapping outputs back to the source, orchestrating multi-step pipelines and tracking usage across models. ContextGem has all of this built in, in one declarative pipeline that you can reuse and serialize.

No prompts to write Prompts are generated from your descriptions of what to extract.	Typed, validated output Text, numbers, dates, booleans, labels, ratings and JSON objects.	Every fact traced to its source Items map back to their paragraphs or sentences, with the reasoning on request.
Nested context Aspects, sub-aspects and concepts focus the model on the text that matters.	The right model for each question An LLM where it takes reasoning, a decision model where a fast, low-cost answer is enough.	Answers with probabilities A decision model gives the probability of each possible answer, and you set the threshold.

Read more on the motivation behind ContextGem.

LLMs and decision models

ContextGem works with two kinds of models. You describe what to extract in the same way for both, and each aspect and concept goes to the model that suits it.

^{An illustrative service agreement: the open question goes to the LLM, the closed ones to the decision model.}

	LLM	Decision model
How it answers	Writes the answer, which ContextGem validates into typed items	Gives the probability of each possible answer, and writes no text
What it extracts	Aspects and every type of concept, from text and images	Aspects, and Boolean, label and rating concepts, from text
With each item	References to paragraphs or sentences, and a justification on request	The probabilities of all the possible answers, and a decision threshold that you set for the model, or for each aspect and concept
Use it for	Open values, and answers that take reasoning	Closed questions in volume, where a fast, low-cost answer is enough
Models	Any cloud or local LLM through LiteLLM: OpenAI, Anthropic, Google, Azure OpenAI, xAI, Ollama, LM Studio and more	TypeSafe Jev, OpenAI Decisions API, Microsoft Foundry and more, cloud or local
Read more	Supported LLMs, configuration, extraction methods	Decision models

Choosing a model. ContextGem is tested with hosted LLMs such as gpt-6-luna (OpenAI) and claude-haiku-5-5 (Anthropic), and with decision models such as Jev (TypeSafe). A smaller LLM, such as many that run locally, may not follow the detailed extraction instructions: check it on a sample of your documents, and see the troubleshooting guide if it falls short. Before you give questions to a decision model, compare it with an LLM on a sample of your documents: see when to use a decision model.

Quick start

uv add contextgem  # or: pip install -U contextgem

Extract with an LLM

An LLM extracts aspects and values of any type. Below, it extracts two things from a contract: an aspect, the payment terms, which comes back as the paragraphs on payments, and a concept, the anomalies, which span the whole document and which retrieval-based approaches can miss. Each anomaly comes with a justification and the sentence it was found in.

import os

from contextgem import Aspect, Document, DocumentLLM, StringConcept


doc = Document(
    raw_text=(
        "Consultancy Agreement\n"
        "This agreement between Company A (Supplier) and Company B (Customer)...\n"
        "The term of the agreement is 1 year from the Effective Date...\n"
        "The Supplier shall provide consultancy services as described in Annex 2...\n"
        "The Customer shall pay the Supplier within 30 calendar days of receiving an invoice...\n"
        "The purple elephant danced gracefully on the moon while eating ice cream.\n"  # 💎 anomaly
        "Time-traveling dinosaurs will review all deliverables before acceptance.\n"  # 💎 another anomaly
        "This agreement is governed by the laws of Norway...\n"
    ),
)

doc.add_aspects(
    [
        Aspect(
            name="Payment terms",
            description="Clauses on payments, fees and invoices",
        )
    ]
)
doc.add_concepts(
    [
        StringConcept(
            name="Anomalies",
            description="Anomalies in the document",
            add_references=True,
            reference_depth="sentences",
            add_justifications=True,
            justification_depth="brief",
        )
    ]
)

llm = DocumentLLM(
    model="openai/gpt-6-luna",  # or another LLM, cloud or local
    api_key=os.environ.get("CONTEXTGEM_OPENAI_API_KEY"),
)
doc = llm.extract_all(doc)

payment_terms = doc.get_aspect_by_name("Payment terms").extracted_items
# 1 item, the paragraph: "The Customer shall pay the Supplier within 30 calendar
#     days of receiving an invoice..."

anomalies = doc.get_concept_by_name("Anomalies").extracted_items
# 2 items. The first one:
# .value: "The statement that a purple elephant danced on the moon while eating
#     ice cream is anomalous and unrelated to the consultancy agreement."
# .justification: "It is an implausible, fantastical statement in an otherwise
#     contractual document."
# .reference_sentences: the sentence of the purple elephant

^{What the example above extracts: an aspect gives back the document's own text, a concept gives new values with their sources.}

Decide with a decision model

A decision model answers closed questions (yes/no, one of your labels, a level of a scale) with the probability of each possible answer. Below, it extracts an aspect from a contract, the termination clauses, by deciding for each paragraph whether it is one, and answers two questions about the contract, concepts, in one request.

import os

from contextgem import (
    Aspect,
    BooleanConcept,
    Document,
    DocumentDecisionModel,
    LabelConcept,
)


doc = Document(
    raw_text=(
        "Service Agreement\n"
        "Either party may terminate this Agreement for any reason by giving 30 days' "
        "written notice to the other party.\n"
        "This Agreement is governed by the laws of the State of New York.\n"
    ),
)

doc.add_aspects(
    [
        Aspect(
            name="Termination",
            description="Clauses on termination of the agreement",
        )
    ]
)
doc.add_concepts(
    [
        BooleanConcept(
            name="Termination for convenience",
            description="Either party may terminate the agreement without cause",
        ),
        LabelConcept(
            name="Notice form",
            description="Required form of the termination notice",
            labels=["Written", "Email", "Any form", "Not specified"],
            classification_type="multi_class",
        ),
    ]
)

decision_model = DocumentDecisionModel(
    model="typesafe/jev-latest",  # or another decision model, cloud or local
    api_key=os.environ.get("CONTEXTGEM_TYPESAFE_API_KEY"),
)
# The aspect: a yes/no question for each paragraph. The concepts: both in one request.
doc = decision_model.extract_all(doc)

termination_clauses = doc.get_aspect_by_name("Termination").extracted_items
# 1 item, the paragraph: "Either party may terminate this Agreement for any reason..."
# .probabilities: {True: 0.99, False: 0.01} (rounded); the other two paragraphs
#     had 0.26 and 0.03, below the decision threshold of 0.5

convenience = doc.get_concept_by_name("Termination for convenience").extracted_items[0]
# .value: True
# .probabilities: {True: 0.98, False: 0.02} (rounded)

notice_form = doc.get_concept_by_name("Notice form").extracted_items[0]
# .value: ["Written"]
# .probabilities: {"Written": 1.0, "Email": 0.0, "Any form": 0.0, "Not specified": 0.0}

More examples: aspects, sub-aspects, concepts of an aspect, concepts from images, aspects containing concepts, aspects with a decision model, a decision model and an LLM in one model group, multi-model pipelines and a chat interface with tool calling.

How it works

^{An illustrative contract: what you define, and what is extracted.}

1. Wrap your content in a Document: text, images, or both.
2. Say what to extract. Aspects pull out the parts of the document on a topic, as paragraphs or sentences. Concepts extract typed values from the whole document or from an aspect, with optional references and justifications. Keep a set of both as a reusable extraction pipeline.
3. Run a model. One call to an LLM, a decision model or a model group of both extracts everything. A group sends each aspect and concept to the model of its role, e.g. the closed questions to a decision model and the rest to an LLM, and each model can have fallbacks.

Read more on how ContextGem works in the documentation.

In agentic workflows

An agent plans and works across documents and steps. ContextGem reads one document in depth and returns typed values with their sources, or the probabilities of the answers to closed questions. The two meet at three points:

^{Where ContextGem runs in an agentic workflow.}

• At ingestion: extract the metadata of each new document once, so that agents find documents by their type, parties or dates.
• In a tool: an agent calls an extraction and gets the values with the sentences they came from, without the document in its context. The tool is a Python function, for ContextGem's own chat, an agent framework or an MCP server.
• At a decision point: a decision model answers a closed question with the probability of each answer, so that the workflow branches on a value that you defined and knows how sure the model is of it. Before the turn of an agent with many tools, it can also pick the groups of tools that the request needs.

See Agentic workflows in the documentation for an example of each.

Every paragraph is read

ContextGem is built for in-depth analysis of individual documents. There is no search step that picks the chunks that look relevant, as in RAG: the model reads every paragraph of the text it extracts from, all at once or in batches of paragraphs of a size you set, so no fact is missed because a search ranked its passage low. This avoids the retrieval inconsistencies that make RAG struggle with complex concepts and nuanced insights.

The trade-off: ContextGem does not query across documents or search a corpus. For these, RAG frameworks (e.g. LlamaIndex, Haystack) remain more appropriate.

How it is tested

Besides tests that replay recorded responses of real models, Hypothesis property tests run whole extractions on generated model outputs, including what a model should never return: IDs that are not in the prompt, values of the wrong type or not valid JSON, and responses cut off at the max output tokens. They check that such an output is rejected and retried, or reported with an error or a warning, and that it never yields extracted items. A change that can affect the quality of the extraction is measured on evaluation cases, i.e. complex documents with reference answers, against a recorded baseline of quality and cost. What exact matching cannot compare (narrative values, justifications and text fields) is judged by a decision model, Jev: it answers one yes/no question at a time with a probability, and is checked on answers whose verdict is known before it is used. See Testing.

Documentation

Full documentation: contextgem.shcherbak.ai

Get started	Define what to extract	Run models	Optimize
Installation Quickstart Advanced usage How it works Migrating to v1.0	Documents Aspects Concepts Extraction pipelines Serialization	Supported LLMs LLM configuration Extraction methods Decision models Chat with tools Agentic workflows	Accuracy Speed Cost Long documents Choosing the right LLM(s)

For LLMs and coding agents, llms.txt indexes the documentation, with a Markdown copy of each page.

For visual architecture maps and natural language Q&A on the codebase, see DeepWiki.

Community

Need help or want to discuss? Start a thread in GitHub Discussions.	Found a bug or have a feature request? Open an issue.
Want to contribute? Contributions are welcome, from a fixed typo to a new feature: see the Contributor Guidelines.	Security The project is scanned with CodeQL and Bandit, and with Snyk as needed: see the security policy.

Acknowledgements

ContextGem relies on these excellent open-source packages: aiolimiter, colorlog, docstring-parser, fastjsonschema, genai-prices, Jinja2, litellm, pillow, pydantic, python-ulid, tenacity and wtpsplit-lite.

License

Apache 2.0 License - see the LICENSE and NOTICE files for details.

© 2025 Shcherbak AI AS — Enterprise AI Engineering. We build AI agents to fit your enterprise — each verified, constrained, and governed.

More from shcherbak_ai

License compatibility checks for dependency trees, fast enough for CI.

Network egress control for CLI tools and agent workflows.

Questions or collaboration ideas? Reach out on LinkedIn or X. Built with ❤️ in Oslo, Norway.