diff --git a/donna/artifacts/intro.md b/.agents/donna/intro.md similarity index 100% rename from donna/artifacts/intro.md rename to .agents/donna/intro.md diff --git a/donna/artifacts/research/specs/report.md b/.agents/donna/research/specs/report.md similarity index 100% rename from donna/artifacts/research/specs/report.md rename to .agents/donna/research/specs/report.md diff --git a/donna/artifacts/research/work/research.md b/.agents/donna/research/work/research.md similarity index 100% rename from donna/artifacts/research/work/research.md rename to .agents/donna/research/work/research.md diff --git a/donna/artifacts/rfc/specs/design.md b/.agents/donna/rfc/specs/design.md similarity index 100% rename from donna/artifacts/rfc/specs/design.md rename to .agents/donna/rfc/specs/design.md diff --git a/donna/artifacts/rfc/specs/request_for_change.md b/.agents/donna/rfc/specs/request_for_change.md similarity index 100% rename from donna/artifacts/rfc/specs/request_for_change.md rename to .agents/donna/rfc/specs/request_for_change.md diff --git a/donna/artifacts/rfc/work/design.md b/.agents/donna/rfc/work/design.md similarity index 100% rename from donna/artifacts/rfc/work/design.md rename to .agents/donna/rfc/work/design.md diff --git a/donna/artifacts/rfc/work/do.md b/.agents/donna/rfc/work/do.md similarity index 100% rename from donna/artifacts/rfc/work/do.md rename to .agents/donna/rfc/work/do.md diff --git a/donna/artifacts/rfc/work/plan.md b/.agents/donna/rfc/work/plan.md similarity index 100% rename from donna/artifacts/rfc/work/plan.md rename to .agents/donna/rfc/work/plan.md diff --git a/donna/artifacts/rfc/work/request.md b/.agents/donna/rfc/work/request.md similarity index 100% rename from donna/artifacts/rfc/work/request.md rename to .agents/donna/rfc/work/request.md diff --git a/donna/artifacts/usage/artifacts.md b/.agents/donna/usage/artifacts.md similarity index 100% rename from donna/artifacts/usage/artifacts.md rename to .agents/donna/usage/artifacts.md diff --git a/donna/artifacts/usage/cli.md b/.agents/donna/usage/cli.md similarity index 100% rename from donna/artifacts/usage/cli.md rename to .agents/donna/usage/cli.md diff --git a/donna/artifacts/usage/worlds.md b/.agents/donna/usage/worlds.md similarity index 93% rename from donna/artifacts/usage/worlds.md rename to .agents/donna/usage/worlds.md index f0278040..e4a94a27 100644 --- a/donna/artifacts/usage/worlds.md +++ b/.agents/donna/usage/worlds.md @@ -21,7 +21,7 @@ s3 buckets, git repositories, databases, etc. Default worlds and there locations are: -- `donna` — `donna.artifacts` — the subpackage with artifacts provided by Donna itself. +- `donna` — `/.agents/donna` — the project-local bundled Donna specs installed from `donna/fixtures/specs` by workspace init/update. - `home` — `~/.donna/home` — the user-level donna artifacts, i.e. those that should be visible for all workspaces on this machine. - `project` — `/.donna/project` — the project-level donna artifacts, i.e. those that are specific to this project. - `session` — `/.donna/session` — the session world that contains the current state of work performed by Donna. diff --git a/.donna/project/core/top_level_architecture.md b/.donna/project/core/top_level_architecture.md index 0f493828..d649e863 100644 --- a/.donna/project/core/top_level_architecture.md +++ b/.donna/project/core/top_level_architecture.md @@ -26,7 +26,8 @@ The code is separated by layers/subsystems into subpackages: - `donna.cli` — code that implements the `donna` CLI tool, its commands, arguments parsing, etc. - `donna.primitives` — code that implements basic building blocks for Donna's behavior: concrete implementations of various classes from the `donna.machine`. - `donna.lib` — module that contains constructed primitives to be used in donna artifacts by referencing them by python import path. Like `donna.lib.workflow`, `donna.lib.goto`, etc. -- `donna.artifacts` — artifacts that are distributed with Donna itself: specifications of how it works, predefined workflows, etc. +- `donna.fixtures.skills` — bundled skills that are distributed with Donna and synced into project workspaces under `.agents/skills`. +- `donna.fixtures.specs` — bundled Donna specifications and workflows that are distributed with Donna and synced into project workspaces under `.agents/donna`. ## Data structures diff --git a/changes/unreleased.md b/changes/unreleased.md index 211c857f..f25175de 100644 --- a/changes/unreleased.md +++ b/changes/unreleased.md @@ -1,10 +1,14 @@ ### Migration +- Run `donna workspaces update` in existing projects so bundled Donna specs are installed into `.agents/donna` for the new filesystem-backed `donna` world. - Update your scripts and specs to use external tools or direct file edits to create, update, move, copy, or delete world artifacts instead using removed Donna commands. ### Changes - `--tag` option is replaced with `--predicate` in all CLI commands that accept artifact patterns. +- Removed the Python donna world. + - Added workspace spec dumping into `.agents/donna` on `donna workspaces init` and `donna workspaces update`. + - Reconfigured the default `donna` world to load bundled specs from `.agents/donna` through the filesystem world and removed the Python world implementation. - Removed world artifact mutation support. - Removed `donna artifacts` mutation commands and the supporting artifact-mutation code paths. - Removed `readonly` world-artifact mutability modeling from workspace config and world abstractions. @@ -13,5 +17,10 @@ ### Breaking Changes +- Donna no longer exposes bundled specs through the Python-backed `donna` world; `donna workspaces init|update` now sync them into `.agents/donna`. - `donna artifacts` no longer supports `update`, `copy`, `move`, or `remove`. - Donna no longer mutates world artifacts through workspace APIs or world configuration. + +### Removals + +- Removed the Python world implementation and the `donna.artifacts` package-backed source of bundled Donna specs. diff --git a/donna/__init__.py b/donna/__init__.py index 0cc59761..e69de29b 100644 --- a/donna/__init__.py +++ b/donna/__init__.py @@ -1 +0,0 @@ -donna_artifacts_root = "donna.artifacts" diff --git a/donna/artifacts/__init__.py b/donna/artifacts/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/donna/artifacts/usage/__init__.py b/donna/artifacts/usage/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/donna/cli/commands/workspaces.py b/donna/cli/commands/workspaces.py index 53367e85..6f15d4e4 100644 --- a/donna/cli/commands/workspaces.py +++ b/donna/cli/commands/workspaces.py @@ -4,7 +4,7 @@ import typer from donna.cli.application import app -from donna.cli.types import SkillsOption +from donna.cli.types import SkillsOption, SpecsOption from donna.cli.utils import cells_cli from donna.protocol.cell_shortcuts import operation_succeeded from donna.protocol.cells import Cell @@ -23,20 +23,20 @@ def _resolve_target_dir() -> pathlib.Path: @workspaces_cli.command(help="Initialize Donna workspace.") @cells_cli -def init(skills: SkillsOption = True) -> Iterable[Cell]: +def init(skills: SkillsOption = True, specs: SpecsOption = True) -> Iterable[Cell]: target_dir = _resolve_target_dir() - initialize_workspace(target_dir, install_skills=skills).unwrap() + initialize_workspace(target_dir, install_skills=skills, install_specs=specs).unwrap() return [operation_succeeded("Workspace initialized successfully")] @workspaces_cli.command(help="Update Donna workspace files.") @cells_cli -def update(skills: SkillsOption = True) -> Iterable[Cell]: +def update(skills: SkillsOption = True, specs: SpecsOption = True) -> Iterable[Cell]: target_dir = _resolve_target_dir() - update_workspace(target_dir, install_skills=skills).unwrap() + update_workspace(target_dir, install_skills=skills, install_specs=specs).unwrap() return [operation_succeeded("Workspace updated successfully")] diff --git a/donna/cli/types.py b/donna/cli/types.py index a330a23f..4986b18e 100644 --- a/donna/cli/types.py +++ b/donna/cli/types.py @@ -168,6 +168,14 @@ def _parse_input_path(value: str) -> pathlib.Path: ), ] +SpecsOption = Annotated[ + bool, + typer.Option( + "--specs/--no-specs", + help="Enable or disable Donna specs updates in `.agents/donna`.", + ), +] + InputPathArgument = Annotated[ pathlib.Path, diff --git a/donna/fixtures/specs/intro.md b/donna/fixtures/specs/intro.md new file mode 100644 index 00000000..2371158c --- /dev/null +++ b/donna/fixtures/specs/intro.md @@ -0,0 +1,59 @@ +# Introduction to the Donna tool + +```toml donna +kind = "donna.lib.specification" +``` +This document provides an introduction to the Donna — a CLI tool that helps manage the work of AI agents like Codex. + +## Overview + +Donna is designed to to invert control flow: instead of agents deciding what to do next, the `donna` tells agents what to do next by following predefined workflows. + +The core idea is that most of high-level workflows are more algorithmic than it may seem at first glance. For example, it may be difficult to fix a particular type issue in the codebase, but the overall process of polishing the codebase is quite linear: + +1. Ensure all tests pass. +2. Ensure the code is formatted correctly. +3. Ensure there are no linting errors. +4. Go to the step 1 if you changed something in the process. +5. Finish. + +We may need coding agents on the each step of the process, but there no reason for agents to manage the whole grooming loop by themselves — it take longer time, spends tokens and may lead to confusion of agents. + +## Artifact Tags + +To simplify searching for artifacts by their semantics, Donna allows tagging artifacts with semantically valuable keywords. Artifacts in `donna:*` world use the next set of tags. + +Artifact type tags: + +- `workflow` — workflow artifact — is set automatically by Donna. +- `specification` — specification artifact — is set automatically by Donna. + +## Instructions + +1. On start of the YOUR session you **MUST** read and understand instruction on using the Donna tool `{{ donna.lib.view("donna:usage:cli") }}`. It **MUST** be a one time operation. Do not repeat it unless you forget how to use the tool. +2. If you need to perform a work with Donna, you **MUST** select an appropriate Donna workflow to perform the work and run it. +3. If there is no appropriate workflow, ask the developer for a precise instructions on what to do. +4. If you are executing a workflow operation and need to perform a complex action or changes, you SHOULD search for an appropriate workflow and run it as a child workflow — it is the intended way to use Donna. +5. Run to list all workflows: `{{ donna.lib.list("**", predicate='"workflow" in section.tags') }}` +6. Run to list all specifications: `{{ donna.lib.list("**", predicate='"specification" in section.tags') }}` + +## Journaling + +You MUST use `donna journal write` to track your actions and thoughts, according the description in `{{ donna.lib.view("donna:usage:cli") }}`. + +Journaling is a required part of workflow execution. An action request MUST be considered incomplete until required journal records are written. + +Journaling lifecycle for each non-trivial action request: + +1. Start intent (`Goal:`) before substantial work begins. +2. Progress updates (`Step:`) at significant phase boundaries. +3. Concrete edits (`Change:`) after meaningful source/artifact update batches. +4. Completion handoff (`Step:`) before calling `sessions action-request-completed`. + +Journal records MUST be change/decision-oriented and SHOULD be sufficient for another agent to continue work without re-discovery. + +If you perform a long operation (e.g., exploring the codebase, designing a solution) that takes more than 10 seconds, you MUST journal your progress. + +You MUST use `donna journal view --lines 100` to read the last records after you compress your context. + +If your work is interrupted and you resume later, you MUST first journal `Resume context and next action`. diff --git a/donna/fixtures/specs/research/specs/report.md b/donna/fixtures/specs/research/specs/report.md new file mode 100644 index 00000000..2fdcc17b --- /dev/null +++ b/donna/fixtures/specs/research/specs/report.md @@ -0,0 +1,163 @@ +# Format of the Research Report document + +```toml donna +kind = "donna.lib.specification" +``` + +This document describes the format and structure of a Research Report document used by Donna workflows from `donna:research:*` namespace. + +## Overview + +Donna introduces a group of workflows located in `donna:research:*` namespace that organize the process of researching a problem, collecting information, analyzing it, synthesizing options, and producing a final solution. + +Session-related research artifacts MUST be stored as `session:research:`, unless the developer or parent workflow specifies a different location. The `` MUST be unique within the session. + +The agent (via workflows) creates the artifact and updates it iteratively as the research process progresses. + +## Research report structure + +The research report is a Donna artifact (check `{{ donna.lib.view("donna:usage:artifacts") }}`) with the next structure: + +- **Primary section** -- title and short description of the research problem. +- **Original problem description** -- original problem statement from the developer or parent workflow. +- **Formalized problem description** -- formalized version of the problem statement. +- **Goals** -- list of goals the research should achieve. +- **Desired form of final solution** -- description of the expected form and constraints for the final solution. +- **Solution space** -- description of analysis axes and synthesis dimensions. +- **Information to collect** -- list of information required to research the problem. +- **Information sources** -- list of sources that can provide the required information. +- **Collected information** -- gathered information with source references. +- **Analysis** -- analysis of collected information along the specified axes. +- **Synthesized solutions** -- synthesized solution options along the specified dimensions. +- **Evaluation** -- evaluation of synthesized solutions against the goals. +- **Final solution** -- final solution in the desired form. + +## General language and format + +- You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. +- You MUST follow `{{ donna.lib.view("donna:usage:artifacts") }}`. +- You MUST follow the structure specified in this document. + +### List format + +- If a section is described as a list, it MUST contain only a single markdown list. +- Each list item MUST be concise and clear. +- Each list item SHOULD be atomic and focused on a single aspect. +- Reviewer MUST be able to tell if the list item statement is true or false by inspecting the resulting artifacts and behavior. + +Common approaches to improve list items: + +- Split a single item with an enumeration into multiple items with a single focus. +- Transform an abstract item into a concrete one by referencing specific artifacts, measurable criteria, verifiable conditions, etc. + +### Trusted inputs + +Some sections of the research report MUST be based on trusted inputs. Trusted inputs are: + +- Original problem description from the developer or parent workflow. +- Statements from the research report itself. +- Existing project documentation, code, and artifacts. +- External standards, when they define constraints or best practices for the project domain. +- Documentation of third-party libraries, frameworks, or tools when they describe constraints or best practices. +- Primary research sources (datasets, reports, official publications) were used to collect the required information. + +## `Primary` section + +- Title MUST be concise and reflect the essence of the research problem. +- Description MUST provide a brief overview of the problem, its purpose, and why research is needed. + +## `Original problem description` section + +- This section MUST contain the original problem description from the developer or from the parent workflow. +- The problem description MUST NOT be modified by agents. + +## `Formalized problem description` section + +- The section MUST contain a clear professional high-level description of the problem based on the original description. +- The section MUST be limited to a single paragraph with a few sentences. +- The section MUST explain what someone gains after the problem is solved and how they can see it working. + +## `Goals` section + +- This section MUST contain a list of goals that the research should achieve. +- Each goal MUST be grounded in the formalized problem description. + +Goal quality criteria: + +- A goal MUST be a desired end state, outcome, or result. +- A goal MUST define what ultimately should be true, not how to achieve it. + +Examples: + +- Bad: `- Investigate database options.` +- Good: `- Identify a database option that meets the project's scalability requirements.` + +## `Desired form of final solution` section + +- The section MUST be grounded in the formalized problem description and the goals. +- The section MUST describe the expected form of the final solution (for example: recommendation, decision matrix, implementation plan, or specification). +- The section MUST specify any required structure, formatting, or constraints on the final solution. +- The section SHOULD be a short list or a short paragraph, whichever is clearer for the problem. + +## `Solution space` section + +- The section MUST describe the axes along which collected information will be analyzed and the dimensions along which solutions will be synthesized. +- The section MUST contain two subsections: **Analysis axes** and **Synthesis dimensions**. +- Each axis or dimension MUST be grounded in the goals or the formalized problem description. + +### `Analysis axes` subsection + +- The subsection MUST contain a list of analysis axes. +- Each axis MUST describe a single perspective or criterion used to analyze information. +- Each axis SHOULD be phrased so it is clear how to apply it to collected information. + +### `Synthesis dimensions` subsection + +- The subsection MUST contain a list of synthesis dimensions. +- Each dimension MUST describe a single perspective or criterion used to synthesize solution options. +- Each dimension SHOULD make the comparison between options easier. + +## `Information to collect` section + +- The section MUST contain a list of information items required to research the problem. +- Each item MUST be specific enough to be collected or verified from sources. +- Each item MUST be grounded in the formalized problem description or the goals. + +## `Information sources` section + +- The section MUST contain a list of sources that can provide the required information. +- Each source entry MUST include a short identifier and a brief description of the source. +- Each source entry SHOULD include access method, scope, and reliability notes if relevant. +- Each source MUST be relevant to at least one item from **Information to collect**. + +## `Collected information` section + +- The section MUST contain the collected information mapped to the items from **Information to collect**. +- Each collected information item MUST reference one or more source identifiers from **Information sources**. +- The section MUST make it clear which information items are satisfied and which are missing. +- If a required information item cannot be collected, the section MUST state that explicitly and explain why. + +## `Analysis` section + +- The section MUST analyze the collected information along the **Analysis axes**. +- The analysis MUST be organized so each axis can be reviewed independently. +- The analysis MUST clearly separate observed facts from inferences or assumptions. +- The analysis format MUST fit the **Desired form of final solution** and should make downstream synthesis straightforward. + +## `Synthesized solutions` section + +- The section MUST present synthesized solutions or options in a format consistent with the **Synthesis dimensions**. +- Each solution SHOULD reference the analysis items that justify it. +- The synthesis format MUST fit the **Desired form of final solution**. + +## `Evaluation` section + +- The section MUST evaluate each synthesized solution against the **Goals**. +- The evaluation MUST make trade-offs explicit and identify risks or uncertainties. +- The evaluation MUST result in a clear comparison between solutions. + +## `Final solution` section + +- The section MUST present the final solution in the form specified in **Desired form of final solution**. +- The final solution MUST be justified by the evaluation results. +- If the evaluation does not allow a confident final solution, the section MUST state the remaining uncertainties and what additional information would resolve them. diff --git a/donna/fixtures/specs/research/work/research.md b/donna/fixtures/specs/research/work/research.md new file mode 100644 index 00000000..12a46588 --- /dev/null +++ b/donna/fixtures/specs/research/work/research.md @@ -0,0 +1,198 @@ +# Research workflow + +```toml donna +kind = "donna.lib.workflow" +start_operation_id = "start" +``` + +Workflow for performing research on a given question/problem/situation. Collects relevant information, analyzes it, synthesizes possible options, and produces an answer/solution/deliverables. + +The purpose of this workflow is to provide an information for making decisions or producing solutions based on researched data. It can be used: + +- When the developer explicitly asks to conduct research. +- When other workflows need to perform research as a subtask. + +## Start + +```toml donna +id = "start" +kind = "donna.lib.request_action" +fsm_mode = "start" +``` + +1. Read the specification `{{ donna.lib.view("donna:usage:artifacts") }}` if you haven't done it yet. +2. Read the specification `{{ donna.lib.view("donna:research:specs:report") }}` if you haven't done it yet. +3. `{{ donna.lib.goto("ensure_problem_description_exists") }}` + +## Ensure problem description exists + +```toml donna +id = "ensure_problem_description_exists" +kind = "donna.lib.request_action" +``` + +At this point, you SHOULD have a clear description of the problem in your context. I.e., you know what you need to do in this workflow. + +1. If you have a problem description in your context, `{{ donna.lib.goto("prepare_artifact") }}`. +2. If you have no problem description in your context, but you know it is in one of `session:*` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_artifact") }}`. +3. If you have no problem description in your context, and you don't know where it is, ask the developer to provide it. After you get the problem description, `{{ donna.lib.goto("prepare_artifact") }}`. + +## Prepare research artifact + +```toml donna +id = "prepare_artifact" +kind = "donna.lib.request_action" +``` + +1. Based on the problem description you have, suggest an artifact name in the format `session:research:`. `` MUST be unique within the session. +{# TODO: we can add donna.lib.list('session:*') here as the command to list all artifacts in session #} +2. Create the artifact and specify an original problem description in it. +3. `{{ donna.lib.goto("formalize_research") }}` + +## Formalize Research + +```toml donna +id = "formalize_research" +kind = "donna.lib.request_action" +``` + +1. Using your knowledge about the project and the original problem description, formulate a format description of the problem +2. Update the artifact with the formalized problem description. +3. `{{ donna.lib.goto("set_primary_section_of_the_research_artifact") }}` + +## Set primary section of the research artifact + +```toml donna +id = "set_primary_section_of_the_research_artifact" +kind = "donna.lib.request_action" +``` + +1. Based on the formalized problem description, update the primary (h1) section of the research artifact to reflect the problem being researched. So the agents can effectively find and understand the research artifact using Donna. +2. `{{ donna.lib.goto("formulate_goals") }}` + +## Formulate Goals + +```toml donna +id = "formulate_goals" +kind = "donna.lib.request_action" +``` + +1. Based on the formalized problem description, formulate a list of goals that the problem solution should achieve. +2. Update the artifact with the list of goals. +3. `{{ donna.lib.goto("fomalize_form_of_final_solution") }}` + +## Formalize form of final solution + +```toml donna +id = "fomalize_form_of_final_solution" +kind = "donna.lib.request_action" +``` + +1. Based on the formalized problem description and the list of goals, formulate the desired form of the final solution. +2. Update the artifact with the desired form of the final solution. +3. `{{ donna.lib.goto("describe_solution_space") }}` + +## Describe solution space + +```toml donna +id = "describe_solution_space" +kind = "donna.lib.request_action" +``` + +To produce a solution, we should understand how to analyze the data and how to synthesize results into the final solution. + +For this, we need to list the axes along which we will analyze the data and the dimensions along which we will synthesize the results. + +1. List the analysis axes. +2. List the synthesis dimensions. +3. Update the artifact with the description of the solution space. +4. `{{ donna.lib.goto("describe_information_to_collect") }}` + +## Describe information to collect + +```toml donna +id = "describe_information_to_collect" +kind = "donna.lib.request_action" +``` + +1. Based on the formalized problem description and the list of goals, list the information to be collected to research the problem. +2. Update the artifact with the description of the information to collect. +3. `{{ donna.lib.goto("list_information_sources") }}` + +## List information sources + +```toml donna +id = "list_information_sources" +kind = "donna.lib.request_action" +``` + +1. Based on the formalized problem description and the list of goals, formulate a list of information sources that can help to research the problem. +2. Update the artifact with the list of information sources. +3. `{{ donna.lib.goto("collect_information") }}` + +## Collect information + +```toml donna +id = "collect_information" +kind = "donna.lib.request_action" +``` + +1. For each information source from the list: + 1. For each piece of required information from the description: + 1. If the source can not provide this piece of information, skip it. + 2. Access the source and collect the required information. + 3. Update the artifact with the collected information. +2. `{{ donna.lib.goto("analyze_information") }}` + +## Analyze information + +```toml donna +id = "analyze_information" +kind = "donna.lib.request_action" +``` + +1. Using the analysis axes from the solution space description, analyze the collected information. Choose the format that best represents the analysis results and suits the required solution form. +2. Update the artifact with the analysis results. +3. `{{ donna.lib.goto("synthesize_solutions") }}` + +## Synthesize solutions + +```toml donna +id = "synthesize_solutions" +kind = "donna.lib.request_action" +``` + +1. Using the synthesis dimensions from the solution space description, synthesize possible solutions based on the analysis results. Choose the format that best represents the synthesized solutions and suits the required solution form. +2. Update the artifact with the synthesized solutions. +3. `{{ donna.lib.goto("evaluate_solutions") }}` + +## Evaluate solutions + +```toml donna +id = "evaluate_solutions" +kind = "donna.lib.request_action" +``` + +1. Evaluate the synthesized solutions against the goals from the formalized problem description. +2. Update the artifact with the evaluation results. +3. `{{ donna.lib.goto("produce_final_solution") }}` + +## Produce final solution + +```toml donna +id = "produce_final_solution" +kind = "donna.lib.request_action" +``` + +1. Based on the evaluation results, produce the final solution in the desired form. +2. Update the artifact with the final solution. +3. `{{ donna.lib.goto("finish") }}` + +## Finish + +```toml donna +id = "finish" +kind = "donna.lib.finish" +``` + +Research workflow completed. Provide the final research output to the developer. diff --git a/donna/fixtures/specs/rfc/specs/design.md b/donna/fixtures/specs/rfc/specs/design.md new file mode 100644 index 00000000..020d0422 --- /dev/null +++ b/donna/fixtures/specs/rfc/specs/design.md @@ -0,0 +1,141 @@ +# Format of the Design document + +```toml donna +kind = "donna.lib.specification" +``` + +This document describes the format and structure of a Design document used to design changes to a project proposed in a Request for Change (RFC). This document is an input for planning and will be treated as strong recommendations for the implementation of the proposed change. + +## Overview + +Donna introduces a group of workflows located in `donna:rfc:*` namespace that organize the process of proposing, reviewing, and implementing changes to a project via RFC and Design documents. + +You create a Design document to explicitly describe the exact changes you want to make to the project in order to implement the RFC. + +If not otherwise specified, Design documents for the session MUST be stored as `session:design:` artifacts in the session world. + +**The Design document MUST list exact changes to the project that will be implemented.** E.g. concrete function names and signatures, file paths, data structures, etc. + +**The Design document MAY skip implementation details.** E.g. it may skip the exact implementation of a function body, but it should specify the function signature and its purpose; It may use pseudocode to describe the logic of a function, but it should not skip describing the logic at all. + +The Design document MUST NOT be a high-level description of the problem and solution. A high-level description of the problem and solution should be provided in the RFC document. + +**Identifiers in this document MUST follow project-specific naming conventions.** Before working on a Design document, you should read the artifacts with project guidelines for naming and code style, if any exist. + +## Design document structure + +The RFC document is Donna artifact (check `{{ donna.lib.view("donna:usage:artifacts") }}`) with the next structure: + +- **Primary section** — title and short description of the proposed change. +- **Inputs** — list of input documents that are relevant for the proposed change, starting from the RFC document. +- **Architecture changes** — list of high-level architectural changes that MUST be implemented. +- **Highlevel code changes** — list of high-level code changes that MUST be implemented: modules to add/modify, functions to add/modify, classes to add/modify, etc. This section should not include low-level implementation details, but it should include enough details to understand the scope of the change and its impact on the project. +- **Data structure changes** — list of changes to data structures that MUST be implemented. +- **Interface changes** — list of changes to interfaces (e.g. function signatures, API endpoints, etc.) that MUST be implemented. +- **Tests changes** — list of autotests that MUST be implemented/updated/removed. Only if the project already uses autotests. +- **Documentation changes** — list of changes in the documentation. Only if the project already has documentation. +- **Other changes** — list of other changes that do not fit into the previous sections, but are still relevant for the proposed change. +- **Order of implementation** — a proposed order of implementation of the changes listed in the previous sections. This section should be treated as a recommendation (from the author of the Design document), not a strict requirement. + +## General language and format + +- You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. +- You MUST follow `{{ donna.lib.view("donna:usage:artifacts") }}`. +- You MUST follow the structure specified in this document. + +### List format + +- If a section is described as a list, it MUST contain only a single markdown list. +- Each list item MUST be concise and clear. +- Each list item SHOULD be atomic and focused on a single aspect. +- Reviewer MUST be able to tell whether the list item statement is true or false by inspecting the resulting artifacts and behavior. + +Examples: + +- Bad: `- Implement a new module.` +- Bad: `- The interface MUST include button A, button B, and dropdown C.` +- Good: `- Implement a module "a.b.c" that is responsible for X, Y, and Z.` +- Good: `- Add a button class "ButtonA" to the module "a.b.c"` +- Good: `- "ButtonA" class MUST have a method "click()" that performs action X when called.` + +Common approaches to improve list items: + +- Split a single item with an enumeration into multiple items with a single focus. +- Transform an abstract item into a concrete one by specifying measurable criteria or user-visible behavior. + +## `Primary` section + +- Title MUST be concise and reflect the essence of the proposed change. Derive it from the RFC. +- Description MUST provide an essence of the proposed change. + +## `Inputs` section + +- The section MUST contain a list of documents that are relevant to the proposed change. +- The first item in the list MUST be the original RFC document that describes the problem and the proposed solution. +- Other items in the list MAY include other documentation, code files, external links, etc., that are relevant for the proposed change. + +## `Architecture changes` section + +- The section MUST contain a free-form but precise and grounded description of the high-level architectural changes that MUST be implemented. +- Along the text, you may add code snippets, diagrams, and other visual aids to make the description clearer and more precise. + +## `High-level code changes` section + +- The section MUST contain a list of high-level changes in the code. +- The level of abstraction: `add a module A`, `remove the class B`, `change the implementation of a function C`. +- The section MUST list only the most important changes that are significant cornerstones of the proposed change. +- The section MAY omit low-level details, such as the small or utilitarian functions, minor refactorings, etc. + +## `Data structure changes` section + +- The section MUST list exact changes to data structures that MUST be implemented. +- Each change MUST be accompanied by a description of the purpose of the change and its impact on the project. + +Examples of statements about structure changes: + +- Bad: `- Change the data structure of the project.` +- Bad: `- Update the class A.` +- Good: `- Add a field "x" to the class "A".` +- Good: `- Change the type of the field "y" in the class "B" from "int" to "str".` +- Good: `- Add structure "C" with fields "a", "b", and "c".` + +## `Interface changes` section + +- The section MUST list exact changes to interfaces that MUST be implemented. +- Each change MUST be accompanied by a description of the purpose of the change and its impact on the project. + +Examples of statements about interface changes: + +- Bad: `- Change the interface of functions in the project.` +- Bad: `- Update the API endpoints.` +- Good: `- Add a new API endpoint "/api/v1/resource" that accepts POST requests with JSON body containing fields "a", "b", and "c".` +- Good: `- Change the signature of the function "foo" from "foo(x: int) -> str" to "foo(x: int, y: str) -> str".` + +## `Tests changes` section + +- If the project does not use autotests, this section MUST contain a statement `No changes in tests are required, since the project does not use autotests.` +- If the project uses autotests, this section MUST contain a list of autotests that MUST be implemented/updated/removed. +- Each changes piece of logic MUST have at least one corresponding autotest that verifies its correctness and prevents regressions in the future. +- Each added/updated branch of logic MUST have at least one corresponding autotest that verifies its correctness and prevents regressions in the future. + +Examples of statements about test changes: + +- Bad: `- Add tests for the new module.` +- Bad: `- Update tests for the changed function.` +- Good: `- Add a test "test_foo_with_valid_input" that verifies that the function "foo" returns the expected result when called with valid input.` +- Good: `- Add a test "test_foo_success_path" that verifies that the function "foo" returns the expected result when called with input that follows the success path.` + +## `Documentation changes` section + +- If the project does not have documentation, this section MUST contain a statement `No changes in documentation are required, since the project does not have documentation.` +- If the project has documentation, this section MUST contain a list of changes in the documentation that MUST be implemented. + +## `Other changes` section + +- The section MAY contain a list of other changes that do not fit into the previous sections, but are still relevant for the proposed change. +- The section MUST be a single statement: `No other changes are required, since all relevant changes are covered in the previous sections.` if there are no other changes to mention. + +## `Order of implementation` section + +- The section MUST contain a proposed order of implementation of the changes listed in the previous sections. +- The section MUST refer only to identifiers mentioned in the previous sections, and it MUST NOT introduce new identifiers or entities that are not mentioned in the previous sections. diff --git a/donna/fixtures/specs/rfc/specs/request_for_change.md b/donna/fixtures/specs/rfc/specs/request_for_change.md new file mode 100644 index 00000000..9c56893a --- /dev/null +++ b/donna/fixtures/specs/rfc/specs/request_for_change.md @@ -0,0 +1,266 @@ +# Format of the Request for Change document + +```toml donna +kind = "donna.lib.specification" +``` + +This document describes the format and structure of a Request for Change (RFC) document used to propose changes to a project by Donna workflows from `donna:rfc:*` namespace. This document is an input for a Design document creation. + +## Overview + +Donna introduces a group of workflows located in `donna:rfc:*` namespace that organize the process of proposing, reviewing, and implementing changes to a project via RFC and Design documents. + +You create RFC documents to propose changes to the project. + +If not otherwise specified, RFC documents for the session MUST be stored as `session:rfc:` artifacts in the session world. + +## RFC structure + +The RFC document is Donna artifact (check `{{ donna.lib.view("donna:usage:artifacts") }}`) with the next structure: + +- **Primary section** — title and short description of the proposed change. +- **Original description** — original description of the requested changes from the developer or parent workflow. +- **Formal description** — formal description of the requested changes. +- **Goals** — list of goals that the proposed change aims to achieve. +- **Objectives** — list of objectives that need to be accomplished to achieve the goals. +- **Constraints** — list of constraints that must be considered while implementing the proposed change. +- **Requirements** — list of requirements that the proposed change must fulfill. +- **Acceptance criteria** — list of criteria that define when the proposed change is considered complete and successful. +- **Solution** — list of statements about how the final result should look like. +- **Verification** — list of statements about how to verify each objective, constraint, requirement, and acceptance criterion. +- **Deliverables** — list of deliverables that should be produced as part of the proposed change. +- **Action items** — unordered list of atomic actions/changes that must be performed to implement the proposed change. + +## General language and format + +- You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. +- You MUST follow `{{ donna.lib.view("donna:usage:artifacts") }}`. +- You MUST follow the structure specified in this document. + +### List format + +- If a section is described as a list, it MUST contain only a single markdown list. +- Each list item MUST be concise and clear. +- Each list item SHOULD be atomic and focused on a single aspect. +- Reviewer MUST be able to tell whether the list item statement is true or false by inspecting the resulting artifacts and behavior. + +Examples: + +- Bad: `- Improve performance and fix bugs.` +- Bad: `- The interface MUST include button A, button B, and dropdown C.` +- Good: `- Performance test MUST show at least 20% improvement in response time under load.` +- Good: `- Fix bug A causing a crash when input is empty.` +- Good: `- The interface MUST include button A that triggers action X.` +- Good: `- The interface MUST include button B that triggers action Y.` + +Common approaches to improve list items: + +- Split a single item with an enumeration into multiple items with a single focus. +- Transform an abstract item into a concrete one by specifying measurable criteria or user-visible behavior. + +### Trusted inputs + +Some sections of the RFC document MUST be based on trusted inputs. Trusted inputs are: + +- Original description from the developer or parent workflow. +- Statements from the RFC document itself. +- Existing project documentation, code, and artifacts. +- External standards, when they define constraints or best practices for the project domain. +- Documentation of third-party libraries, frameworks, or tools when they describe constraints or best practices. + +## `Primary` section + +- Title MUST be concise and reflect the essence of the proposed change. +- Description MUST provide a brief overview of the proposed change, its purpose, and its expected impact. + +## `Original description` section + +- This section MUST contain the original request from the developer or from the parent workflow. +- The request MUST NOT be modified by agents. + +## `Formal description` section + +- The section MUST contain a clear professional high-level description of the work to be done based on the developer's request. +- The section MUST be limited to a single paragraph with a few sentences. +- The section MUST explain what someone gains after these changes and how they can see it working. + State the user-visible behavior the change will enable. + +## `Goals` section + +- This section MUST contain a list of goals that the proposed change aims to achieve. + +The goal quality criteria: + +- A goal MUST be a desired end state, outcome, or result. +- A goal MUST define what ultimately should be true, not how to achieve it. + +Examples: + +- Bad: `- Implement authentication system.` +- Good: `- Protect user data from unauthorized access.` + +## `Objectives` section + +- The section MUST contain a list of objectives that need to be completed to achieve the goals. +- Each goal MUST have a set of objectives that, when all achieved, ensure the goal is met. +- Each goal MUST have 2–6 objectives, unless the goal is demonstrably trivial (≤1 artifact, no dependencies). + +Objective quality criteria: + +- An objective MUST be a specific, measurable condition that must be true for a goal to be satisfied. +- An objective MUST describe an achieved state, capability, artifact, or behavior. + +Examples: + +- Bad: `- Create user authentication and authorization system.` +- Bad: `- Design user authentication flow.` +- Good: `- Specification for user authentication flow exists.` +- Good: `- User is able to log in using email and password.` + +## `Constraints` section + +- The section MUST contain a list of constraints that the changes MUST respect. +- The section MAY be empty only if no constraints are explicitly known. +- A constraint MUST be derived from trusted inputs. Agents MUST NOT invent constraints. + +Constraint quality criteria: + +- A constraint MUST be a hard limit on the solution space. +- A constraint MUST be externally imposed by technology, policy, compatibility, time, budget, etc. +- A constraint MUST NOT describe desired behavior or outcomes. +- A constraint MUST be non-negotiable within the scope of the RFC. + +Examples: + +- Bad: `- The system should be easy to maintain.` +- Bad: `- Use clean architecture.` +- Good: `- The solution MUST be compatible with Python 3.12.` +- Good: `- The solution MUST NOT introduce new runtime dependencies.` +- Good: `- The solution MUST follow the specification project:specs:abc` +- Good: `MUST not change public CLI flags` + +## `Requirements` section + +- The section MUST contain a list of requirements that the proposed change MUST fulfill. + +Requirement quality criteria: + +- A requirement MUST be a single atomic condition, capability, or feature that the system MUST meet, provide, or exhibit after the change is implemented. +- A requirement MUST be directly testable. +- A requirement MUST be stated independently of implementation details. +- A requirement MUST NOT restate goals or objectives verbatim. + +Examples: + +- Bad: `- Improve security.` +- Bad: `- Implement OAuth.` +- Good: `- The system MUST reject authentication attempts with invalid credentials.` +- Good: `- The system MUST log all failed authentication attempts with timestamp and user identifier.` + +## `Acceptance criteria` section + +- The section MUST contain a list of acceptance criteria used to determine whether the proposed change is complete and successful. +- An acceptance criterion MUST be derived explicitly from statements in the RFC document. Agents MUST NOT invent acceptance criteria. + +Acceptance criteria quality criteria: + +- An acceptance criterion MUST be a single atomic check that results in a pass/fail outcome. +- An acceptance criterion check MUST be about a single artifact: a single file exists, a single test passes, a single behavior observed, etc. +- An acceptance criterion MUST NOT describe implementation steps, internal design decisions, or "how" to achieve the result. + +Examples: + +- Bad: `- All requirements are implemented.` +- Bad: `- The feature works as expected.` +- Good: `- The artifact with documentation for X exists.` +- Good: `- The autotest for requirement Y exists.` +- Good: `- All autotests pass without errors.` +- Good: `- The tool can run on Python 3.12 without errors.` + +## `Solution` section + +- The section MUST contain a list of statements describing how the system should look like/behave after the proposed changes are implemented. +- The section MUST NOT establish an order of implementation steps. +- The full solution MUST ensure the truth/validity of all statements in the RFC. + +Solution statement quality criteria: + +- A solution statement MUST be a specific change in a specific artifact, behavior, or capability. +- A solution statement MUST NOT prescribe implementation steps. +- A solution statement SHOULD be phrased in the present tense, describing an achieved state. + +Examples: + +- Bad: `- First implement the database schema, then add API endpoints.` +- Bad: `- Use framework X to handle authentication.` +- Good: `- The system exposes an authentication API that accepts credentials and returns an access token on success.` +- Good: `- User-facing documentation describes how to configure and use the authentication feature.` + +## `Verification` section + +- The section MUST contain a list of checks that MUST be passed to prove that the work is complete and correct. +- Each verification statement MUST map to a single item from **Objectives**, **Constraints**, **Requirements**, or **Acceptance criteria**. +- Each **Objective**, **Constraint**, **Requirement**, or **Acceptance criterion** MUST map to a single verification statement. +- Each verification statement MUST describe *how* the corresponding item can be verified. + +Verification quality criteria: + +- A verification statement MUST be a concrete verification step or check. +- A verification statement MUST be automatable if it is possible. +- A verification statement SHOULD be verifiable by agents without human intervention. +- A verification statement MUST result in a boolean outcome (verified / not verified). +- A verification statement MUST reference specific artifacts, commands, tests, or observable behavior. + +Examples: + +- Bad: `- Verify that authentication works correctly.` +- Bad: `- Review the implementation manually.` +- Good: `- Run test suite `tests/auth/test_login.py`; all tests MUST pass.` +- Good: `- Inspect artifact `project:specs:authenticationd`; it MUST exist and contain section "Login flow".` +- Good: `- Execute CLI command `tool login` with invalid credentials; command MUST exit with non-zero code.` + +## `Deliverables` section + +- The section MUST contain a list of concrete artifacts that MUST exist after the changes are implemented. +- A deliverable MUST be derived from trusted inputs. Agents MUST NOT invent deliverables. + +Deliverable quality criteria: + +- A deliverable MUST be a tangible, single artifact or a single part of an artifact, not an activity or process. +- A deliverable MUST be independently identifiable and verifiable. +- Each deliverable MUST be phrased as an existence statement using normative + language: "MUST include …", "MUST exist …", etc. +- Explicitly add source files as deliverables only when the task is specifically + about creating or modifying those files (e.g., "MUST add docs/cli.md …"). + +Examples: + +- Bad: `- Implement authentication code` +- Bad: `- Refactor auth module.` +- Good: `- Module app/auth/authentication.py exists.` +- Good: `- Donna artifact project:specs:authentication exists.` +- Good: `- Test suite tests/auth/ exists.` + +## `Action items` section + +- The section MUST contain an unordered list of atomic actions. + +Action item quality criteria: + +- An action item MUST be a single change that can be applied or executed. +- An action item MUST be small enough to be completed without further decomposition. +- An action item MUST be phrased as an imperative sentence. +- An action item MUST affect a specific artifact, file, config, etc. +- An action item MUST reference concrete paths, identifiers, or commands. +- An action item MUST be actionable by an agent. + +Examples: + +- Bad: `- Work on authentication.` +- Bad: `- Improve security everywhere.` +- Bad: `- Fix the bugs A` +- Good: `- Create an artifact project:specs:authentication with sections "Login flow" and "Token lifecycle".` +- Good: `- Add test file tests/auth/test_login.py covering invalid credential cases.` +- Good: `- Implement test tests/auth/test_login.py:TestLogin:test_invalid_credentials.` +- Good: `- Update CLI help text to include login command description.` +- Good: `- Implement function app/auth/authentication.py:authenticate_user().` diff --git a/donna/fixtures/specs/rfc/work/design.md b/donna/fixtures/specs/rfc/work/design.md new file mode 100644 index 00000000..54ec9bb5 --- /dev/null +++ b/donna/fixtures/specs/rfc/work/design.md @@ -0,0 +1,121 @@ +# Create a Design document + +```toml donna +kind = "donna.lib.workflow" +start_operation_id = "start" +``` + +This workflow creates a Design document artifact based on an RFC and aligned with `donna:rfc:specs:design`. + +## Start Work + +```toml donna +id = "start" +kind = "donna.lib.request_action" +fsm_mode = "start" +``` + +1. Read the specification `{{ donna.lib.view("donna:rfc:specs:design") }}` if you haven't done it yet. +2. Read the specification `{{ donna.lib.view("donna:usage:artifacts") }}` if you haven't done it yet. +3. `{{ donna.lib.goto("ensure_rfc_artifact_exists") }}` + +## Ensure RFC artifact exists + +```toml donna +id = "ensure_rfc_artifact_exists" +kind = "donna.lib.request_action" +``` + +At this point, you SHOULD have a clear RFC to design. + +1. If you have an RFC artifact id in your context, view it and `{{ donna.lib.goto("prepare_design_artifact") }}`. +2. If you have no RFC artifact id in your context, but you know it is in one of `{{ donna.lib.list("session:**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_design_artifact") }}`. +3. If you have no RFC artifact id in your context, and you don't know where it is, ask the developer to provide the RFC artifact id or to create a new RFC. After you get it and view the artifact, `{{ donna.lib.goto("prepare_design_artifact") }}`. + +## Prepare Design artifact + +```toml donna +id = "prepare_design_artifact" +kind = "donna.lib.request_action" +``` + +1. If the name of the artifact is not specified explicitly, assume it to be `session:design:`, where `` SHOULD correspond to the RFC slug. +2. Save the next template into the artifact, replace `` with appropriate values. + +~~~ +# + +```toml donna +kind = "donna.lib.specification" +``` + +<short description of the proposed design> + +## Inputs + +- <RFC artifact id> + +## Architecture changes + +## High-level code changes + +## Data structure changes + +## Interface changes + +## Tests changes + +## Documentation changes + +## Other changes + +## Order of implementation +~~~ + +3. `{{ donna.lib.goto("initial_fill") }}` + +## Initial Fill + +```toml donna +id = "initial_fill" +kind = "donna.lib.request_action" +``` + +1. Read the specification `{{ donna.lib.view("donna:rfc:specs:design") }}` if you haven't done it yet. +2. Read the RFC artifact selected in the previous step if you haven't done it yet. +3. Analyze the project if needed to understand the requested change context. +4. Fill in all sections of the Design draft artifact. +5. Ensure the first item in `Inputs` section is the RFC artifact id. +6. `{{ donna.lib.goto("review_design_format") }}` + +## Review Design Format + +```toml donna +id = "review_design_format" +kind = "donna.lib.request_action" +``` + +1. List mismatches between the Design artifact and the Design specification `{{ donna.lib.view("donna:rfc:specs:design") }}`. +2. For each mismatch, make necessary edits to the Design draft artifact to ensure compliance. +3. `{{ donna.lib.goto("review_design_content") }}` + +## Review Design Content + +```toml donna +id = "review_design_content" +kind = "donna.lib.request_action" +``` + +1. Read the Design document and identify gaps, inconsistencies, or areas for improvement in accordance with the RFC and current project context. Use `{{ donna.lib.view("donna:research:work:research") }}` workflow if you need to make a complex decision. +2. Make necessary edits to the Design draft artifact to address identified issues. +3. If there were changes made on this step or the previous `review_design_format` step `{{ donna.lib.goto("review_design_format") }}`. +4. If no changes were made, `{{ donna.lib.goto("finish") }}`. + +## Complete Draft + +```toml donna +id = "finish" +kind = "donna.lib.finish" +``` + +Design draft is complete and ready for implementation planning. diff --git a/donna/fixtures/specs/rfc/work/do.md b/donna/fixtures/specs/rfc/work/do.md new file mode 100644 index 00000000..c39d9294 --- /dev/null +++ b/donna/fixtures/specs/rfc/work/do.md @@ -0,0 +1,121 @@ + +# Do Request For Change work + +```toml donna +kind = "donna.lib.workflow" +start_operation_id = "estimate_complexity" +``` + +This workflow uses a description of a problem or changes required from the developer or parent workflow to: + +1. Create a Request for Change (RFC) artifact. +2. Create a Design document artifact based on the RFC. +3. Plan the work required to implement the designed changes. +4. Execute the planned work. + +## Estimate complexity + +```toml donna +id = "estimate_complexity" +kind = "donna.lib.request_action" +``` + +1. If the developer or parent workflow explicitly specified that the work is complex, `{{ donna.lib.goto("create_rfc") }}`. +2. If the developer or parent workflow explicitly specified that the work is simple, `{{ donna.lib.goto("fast_planning") }}`. +3. Make a rough estimate of the complexity of the requested changes. +4. If the complexity is small, `{{ donna.lib.goto("fast_planning") }}`. +5. Otherwise, `{{ donna.lib.goto("create_rfc") }}`. + +Examples of changes with small complexity: + +- A single file edit with clear instructions. +- A small addition to an existing function or class. +- A minor configuration change. +- A small documentation update. +- A simple bug fix with a known solution. + +## Fast planning + +```toml donna +id = "fast_planning" +kind = "donna.lib.request_action" +``` + +1. Draft a simple plan to implement the requested changes directly, without creating a formal RFC artifact. +2. `{{ donna.lib.goto("plan_rfc_work") }}`. + +## Create RFC + +```toml donna +id = "create_rfc" +kind = "donna.lib.request_action" +fsm_mode = "start" +``` + +1. Choose a workflow to create an RFC artifact. +2. Run the chosen workflow. +3. After completing the workflow `{{ donna.lib.goto("design") }}`. + +## Design + +```toml donna +id = "design" +kind = "donna.lib.request_action" +``` + +1. Choose a workflow to create a Design document artifact based on the RFC created in the previous step. +2. Run the chosen workflow. +3. After completing the workflow `{{ donna.lib.goto("plan_rfc_work") }}`. + +## Plan RFC work + +```toml donna +id = "plan_rfc_work" +kind = "donna.lib.request_action" +``` + +1. Choose the workflow to plan the work. If you created a Design document in the previous step, use it as a basis. +2. Run the chosen workflow. +3. Ensure you know the workflow id created in the previous step (default is `session:execute_rfc` if not specified). +4. After completing the workflow `{{ donna.lib.goto("execute_rfc_work") }}`. + +## Execute RFC work + +```toml donna +id = "execute_rfc_work" +kind = "donna.lib.request_action" +``` + +1. Run the workflow created by the plan step (default: `session:execute_rfc`) and complete it. +2. After completing the workflow `{{ donna.lib.goto("polish_changes") }}`. + +## Polish changes + +```toml donna +id = "polish_changes" +kind = "donna.lib.request_action" +``` + +1. Find the workflow to polish the changes made. +2. Run the chosen workflow if you found one. +3. `{{ donna.lib.goto("log_changes") }}`. + +## Log changes + +```toml donna +id = "log_changes" +kind = "donna.lib.request_action" +``` + +1. Find the workflow to log the changes made in the scope of this RFC. +2. Run the chosen workflow if you found one. +3. `{{ donna.lib.goto("finish") }}`. + +## Finish + +```toml donna +id = "finish" +kind = "donna.lib.finish" +``` + +RFC workflow execution completed. Report the outcome to the developer. diff --git a/donna/fixtures/specs/rfc/work/plan.md b/donna/fixtures/specs/rfc/work/plan.md new file mode 100644 index 00000000..180ab8dd --- /dev/null +++ b/donna/fixtures/specs/rfc/work/plan.md @@ -0,0 +1,76 @@ + +# Plan work on base of Design & Request for Change documents + +```toml donna +kind = "donna.lib.workflow" +start_operation_id = "start" +``` + +This workflow plans the work required to implement a specified Design document. The RFC document SHOULD be used as a helper context. The result of this workflow is a new workflow in the `session:*` world with detailed steps to implement the designed changes. + +## Start Work + +```toml donna +id = "start" +kind = "donna.lib.request_action" +fsm_mode = "start" +``` + +1. Read the Design document that the developer or parent workflow wants you to implement. +2. Read the RFC document that the developer or parent workflow wants you to implement, if it exists. +3. Read the specification `{{ donna.lib.view("donna:usage:artifacts") }}` if you haven't done it yet. +4. `{{ donna.lib.goto("prepare_workflow_artifact") }}` + +## Prepare workflow artifact + +```toml donna +id = "prepare_workflow_artifact" +kind = "donna.lib.request_action" +``` + +1. If the name of the artifact is not specified explicitly, assume it to `session:plans:<short-id-equal-to-design-slug>`. +2. Create a workflow with the next operations: + - Start + - A step for each action point in the RFC document and each item in the `Order of implementation` in Design document with the goal to minimize dependencies between steps and introduce changes incrementally. + - Add additional steps if the design requires it. + - (`id="review_changes"`) A gate step to start reviewing the changes. + - A gate step to start reviewing the deliverables. + - A step per deliverable to check if it exists and is correct. If the deliverable is missing or incorrect, the step MUST specify how to fix it and `goto` to the `review_changes` step. + - A gate step to start verification. + - A step for each verification statement to check it. If the statement is not satisfied, the step MUST specify how to fix it and `goto` to the `review_changes` step. + - A finish step. +3. `{{ donna.lib.goto("review_workflow") }}` + +## Review workflow + +```toml donna +id = "review_workflow" +kind = "donna.lib.request_action" +``` + +For each of the steps in the workflow you created: + +1. If the step can fail, add instructions on how to fix the issue and `goto` to the appropriate step. +2. If the step requires both research and implementation, split it into two steps: research step and implementation step. +3. If the step required editing multiple artifacts (multiple files, multiple functions, etc.), split it into multiple steps, one per change required. +4. If the step is too big or complex, split it into multiple smaller steps. +5. Fux & improve naming and IDs in the step's title and body. + +Naming rules: + +- A title of operation MUST always have meaning without knowing context: no sequential numbers, no generic titles like `Plan part 1`. +- If you mention an ID from the RFC or the Design document, you MUST include a human-readable name of the entity you refer. E.g. not `O1` but `O1 (Implement a new module "a.b.c")` + +After all steps are reviewed: + +1. If the changes were introduced, `{{ donna.lib.goto("review_workflow") }}`. +2. If no changes were introduced, `{{ donna.lib.goto("finish") }}`. + +## Finish + +```toml donna +id = "finish" +kind = "donna.lib.finish" +``` + +Work plan finalized. Ready to execute the planned workflow. diff --git a/donna/fixtures/specs/rfc/work/request.md b/donna/fixtures/specs/rfc/work/request.md new file mode 100644 index 00000000..4a612ba2 --- /dev/null +++ b/donna/fixtures/specs/rfc/work/request.md @@ -0,0 +1,124 @@ + +# Create a Request for Change + +```toml donna +kind = "donna.lib.workflow" +start_operation_id = "start" +``` + +This workflow creates a Request for Change (RFC) document based on a description of the problem or the required changes. + +## Start Work + +```toml donna +id = "start" +kind = "donna.lib.request_action" +fsm_mode = "start" +``` + +1. Read the specification `{{ donna.lib.view("donna:rfc:specs:request_for_change") }}` if you haven't done it yet. +2. Read the specification `{{ donna.lib.view("donna:usage:artifacts") }}` if you haven't done it yet. +3. `{{ donna.lib.goto("ensure_work_description_exists") }}` + +## Ensure work description exists + +```toml donna +id = "ensure_work_description_exists" +kind = "donna.lib.request_action" +``` + +At this point, you SHOULD have a clear description of the problem in your context. I.e. you know what you need to do in this workflow. + +1. If you have a problem description in your context, `{{ donna.lib.goto("prepare_rfc_artifact") }}`. +2. If you have no problem description in your context, but you know it is in one of `{{ donna.lib.list("session:**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_rfc_artifact") }}`. +3. If you have no problem description in your context, and you don't know where it is, ask the developer to provide it. After you get the problem description, `{{ donna.lib.goto("prepare_rfc_artifact") }}`. + +## Prepare RFC artifact + +```toml donna +id = "prepare_rfc_artifact" +kind = "donna.lib.request_action" +``` + +1. If the name of the artifact is not specified explicitly, assume it to be `session:rfc:<short-problem-related-identifier>`, where `<short-problem-related-identifier>` MUST be unique within the session. +2. Save the next template into the artifact, replace `<variables>` with appropriate values. + +~~~ +# <Title> + +```toml donna +kind = "donna.lib.specification" +``` + +<short description of the proposed change> + +## Original description + +<original description of the requested changes from the developer or parent workflow> + +## Formal description + +## Goals + +## Objectives + +## Constraints + +## Requirements + +## Acceptance criteria + +## Solution + +## Verification + +## Deliverables + +## Action items +~~~ + +3. `{{ donna.lib.goto("initial_fill") }}` + +## Initial Fill + +```toml donna +id = "initial_fill" +kind = "donna.lib.request_action" +``` + +1. Read the specification `{{ donna.lib.view("donna:rfc:specs:request_for_change") }}` if you haven't done it yet. +2. Analyze the project if needed to understand the context of the requested change. +3. Based on the problem description you have, fill in all sections of the RFC draft artifact. +4. `{{ donna.lib.goto("review_rfc_format") }}` + +## Review RFC Format + +```toml donna +id = "review_rfc_format" +kind = "donna.lib.request_action" +``` + +1. List mismatches between the RFC artifact and the RFC specification `{{ donna.lib.view("donna:rfc:specs:request_for_change") }}`. +2. For each mismatch, make necessary edits to the RFC draft artifact to ensure compliance with the RFC specification. +3. `{{ donna.lib.goto("review_rfc_content") }}` + +## Review RFC Content + +```toml donna +id = "review_rfc_content" +kind = "donna.lib.request_action" +``` + +1. Read the RFC document and identify any gaps, inconsistencies, or areas for improvement in the content in accordance with the current project context. Use `{{ donna.lib.view("donna:research:work:research") }}` workflow if you need to make a complex decision. +2. Make necessary edits to the RFC draft artifact to address identified issues. +3. If there were changes made on this step or the previous `review_rfc_format` step `{{ donna.lib.goto("review_rfc_format") }}`. +4. If no changes were made, `{{ donna.lib.goto("finish") }}`. + +## Complete Draft + +```toml donna +id = "finish" +kind = "donna.lib.finish" +``` + +RFC draft is complete and ready for planning or review. diff --git a/donna/fixtures/specs/usage/artifacts.md b/donna/fixtures/specs/usage/artifacts.md new file mode 100644 index 00000000..b87059b2 --- /dev/null +++ b/donna/fixtures/specs/usage/artifacts.md @@ -0,0 +1,271 @@ +# Default Text Artifacts Behavior + +```toml donna +kind = "donna.lib.specification" +``` + +This document describes the default format and behavior of Donna's text artifacts. +This format and behavior is what should be expected by default from an artifact if not specified otherwise. + +## Overview + +An artifact is any text or binary document that Donna manages in its worlds. For example, via CLI commands `donna -p <protocol> artifacts …`. + +The text artifact has a source and one or more rendered representations, produced in specific rendering modes. + +— The source is the raw text content of the artifact as it is stored on disk or in remote storage. +- The representation is the rendered version of the artifact for a specific rendering mode. In practice, the same source is rendered in `view` mode for CLI display, `execute` mode for workflow execution, and `analysis` mode for internal parsing and validation (see "Rendering artifacts"). + +To change the artifact, developers and agents edit its source. + +To get information from the artifact, developers, agents and Donna view one of its representations (typically via the view rendering mode). + +**If you need an information from the artifact, you MUST view its representation**. Artifact sources are only for editing. + +Read the specification `{{ donna.lib.view("donna:usage:cli") }}` to learn how to work with artifacts via Donna CLI. + +## Source Format and Rendering + +The source of the text artifact is a Jinja2 template of a Markdown document. + +When rendering the artifact, Donna processes the Jinja2 template with a predefined context (at minimum `render_mode` and `artifact_id`, and optionally `current_task`/`current_work_unit` during workflow execution), then renders the resulting Markdown content into the desired representation based on the selected rendering mode. + +**Artifact source should not use Jinja2 inheritance features** like `{{ "{% extends %}" }}` and `{{ "{% block %}" }}`. + +Donna provides a set of special directives that can and MUST be used in the artifact source to enhance its behavior. Some of these directives are valid for all artifacts, some are valid only for specific section kinds. + +Here are some examples: + +- `{{ "{{ donna.lib.view(<artifact-id-pattern>) }}" }}` — references another artifact (supports `*` and `**` wildcard patterns). In `view`/`execute` modes it renders an exact CLI command to view the artifact; in `analysis` mode it renders a `$$donna ... $$` marker used for internal parsing. +- `{{ "{{ donna.lib.list(<artifact-id-pattern>) }}" }}` — references artifact listing by pattern (supports `*` and `**` wildcard patterns). In `view`/`execute` modes it renders an exact CLI command to list artifacts; in `analysis` mode it renders a `$$donna ... $$` marker used for internal parsing. +- `{{ "{{ donna.lib.goto(<workflow-operation-id>) }}" }}` — references the next workflow operation to execute. In `view`/`execute` modes it renders an exact CLI command to advance the workflow; in `analysis` mode it renders a `$$donna goto ... $$` marker used to extract workflow transitions. + +## Jinja2 rendering + +Donna allows all of Jinja2 expressions in artifact sources, except inheritance-related once: `{{ "{% extends %}" }}` , `{{ "{% block %}" }}`, etc. + +Donna intentionally hides some parts of the source in the rendered output, but they remain visible in the source files themselves (on filesystem): + +- fenced code blocks with the `donna` marker (they contain technical information for the Donna, not information for the agent). +- Jinja2 comments like `{{ "{# ... #}" }}`. + +## Rendering artifacts + +Donna renders the same artifact source into different representations depending on the rendering mode. The mode is internal to Donna (users do not select it directly) and controls how directives are expanded and which metadata is included. + +- `view` — default representation used when the CLI loads artifacts for display (`artifacts view`, `artifacts list`, `artifacts validate`). This is the human/agent-facing output. +- `execute` — representation used when Donna executes workflow operations (`sessions run`). It renders directives with task/work-unit context so the resulting text is actionable for the agent. +- `analysis` — internal representation used during parsing and validation. It emits `$$donna ... $$` markers so Donna can extract workflow transitions and other structured signals. + +## Structure of a Text Artifact + +Technically, any valid Markdown document is a valid text artifact. + +However, Donna assigns special meaning to some elements of the Markdown document to provide enhanced behavior and capabilities. + +### Sections + +Artifact is divided into multiple sections: + +- H1 header and all text till the first H2 header is considered the `head section` of the artifact. +- Each H2 header and all text till the next H2 header (or end of document) is considered a `tail section` of the artifact. + +Head section provides a description of the artifact and its purpose and MUST contain a configuration block of the artifact. The head section is also the artifact's `primary section` and is used when Donna needs to show a brief summary of the artifact, for example, when listing artifacts or when an operation targets the artifact without specifying a section. + +Tail sections describes one of the components of the artifact and CAN contain configuration blocks as well. Configuration blocks placed in subsections (h3 and below) count as part of the parent tail section. + +The content of the header (text after `#` or `##`) is considered the section title. + +Donna always interprets the head section as a general description of the artifact and treats it as the primary section. + +Donna interprets a tail section according to the primary section kind and configuration blocks in that section. + +### Configuration Blocks + +Configuration blocks are fenced code blocks with specified primary format, followed by the `donna` keyword and, optionally, list of properties. + +The supported primary formats are: TOML, JSON, YAML. **You MUST prefer TOML for configuration blocks**. + +The configuration block properties format is `property1 property2=value2 property3=value3"`, which will be parsed into a dictionary like: + +```python +{ + "property1": True, + "property2": "value2", + "property3": "value3", +} +``` + +The content of the block is parsed according to the primary format and interpreted according its properties. + +Configuration blocks are parsed by Donna and removed from rendered Markdown representations (see "Jinja2 rendering"); they remain in the source for editing and inspection on the file system. + +Fences without `donna` keyword are considered regular code blocks and have no special meaning for Donna. + +### Configuration Merging + +When a section contains multiple configuration blocks, Donna merges them in document order. + +- The merge is applied per section: the head section is merged independently, and each tail section has its own merged configuration. +- Config blocks are merged in the order they appear; later blocks override earlier keys. +- The merge is shallow: if a key maps to a nested object, a later block replaces the whole value (there is no deep merge). +- Config blocks in subsections (H3 and below) belong to their parent H2 tail section and are merged into that section's configuration. + +### Artifact Tags + +Artifacts can include semantic tags via a `tags` field in the section configuration. Tags are a list of strings and default to an empty list `[]` when omitted. + +Tags are used for deterministic artifact filtering and discovery (for example, via `donna -p <protocol> artifacts list ... --predicate '"workflow" in section.tags'`). Tags are typically attached to the primary section and describe the artifact as a whole. + +The canonical list of standard tags is documented in `donna:intro`. + +## Section Kinds, Their Formats and Behaviors + +### Header section + +Header section MUST contain a config block with a `kind` property. The `kind` MUST be a full Python import path pointing to the primary section kind instance. + +Example (`donna` keyword skipped for examples): + +```toml +kind = "donna.lib.specification" +``` + +Header section MUST also contain short human-readable description of the artifact outside of the config block. + +### Kind: Specification + +Specification artifacts describe various aspects of the project in a structured way. + +Currently there is no additional structure or semantics for this kind of artifact. + +### Kind: Workflow + +Workflow artifacts describe a sequence of operations that Donna and agents can perform to achieve a specific goal. + +Workflow is a Finite State Machine (FSM) where each tail section describes one operation in the workflow. + +Donna validates workflows by ensuring the start operation exists, reachable sections are valid operations, final operations have no outgoing transitions, and non-final operations have at least one outgoing transition. It does not currently report unreachable sections. + +Workflow start operation MUST be declared in the workflow head-section config via `start_operation_id` +and MUST reference an existing operation section. + +Example (`donna` keyword skipped for examples): + +```toml +kind = "donna.lib.workflow" +start_operation_id = "start_operation" +``` + +Each tail section MUST contain config block with `id` and `kind` properties that specifies the identifier and kind of the operation. + +Example (`donna` keyword skipped for examples): + +```toml +id = "operation_id" +kind = "donna.lib.request_action" +``` + +The title of the workflow section MUST be a short human-readable description of the operation in the form of an imperative verb phrase, for example, `Implement the feature X`, `Create a document Y`. + +#### Kind: Operation + +The title of the operation section MUST be a short human-readable description of the operation in the form of an imperative verb phrase, for example, `Run tests`, `Format the codebase`, `Implement function X in the module Y`.x + +##### `donna.lib.request_action` + +`donna.lib.request_action` operation indicates that Donna will request the agent to perform some action. + +The content of the tail section is the text instructions for the agent on what to do. + +Example of the instructions: + +``` +1. Run `some cli command` to do something. +2. If no errors encountered `{{ '{{ donna.lib.goto("next_operation") }}' }}` +3. If errors encountered `{{ '{{ donna.lib.goto("error_handling_operation") }}' }}` + +Here may be any additional instructions, requirements, notes, references, etc. +``` + +`donna.lib.goto` directive will be rendered in the direct instruction for agent of what to call after it completed the action. + +**The body of the operation MUST contain a neat strictly defined algorithm for the agent to follow.** + +##### `donna.lib.run_script` + +`donna.lib.run_script` operation executes a script from the operation body without agent/user interaction. + +The body of the operation MUST include exactly one fenced code block whose info string includes `<language> donna script`. +Any other text in the operation body is ignored. + +Script example: + +```bash donna script +#!/usr/bin/bash + +echo "Hello, World!" +``` + +Configuration options: + +```toml +id = "<operation_id>" +kind = "donna.lib.run_script" + +save_stdout_to = "<variable_name>" # optional +save_stderr_to = "<variable_name>" # optional + +goto_on_success = "<next_operation_id>" # required +goto_on_failure = "<next_operation_id>" # required +goto_on_code = { # optional + "1" = "<next_operation_id_for_code_1>" + "2" = "<next_operation_id_for_code_2>" +} + +timeout = 60 # optional, in seconds +``` + +Routing rules: + +- Exit code `0` routes to `goto_on_success`. +- Non-zero exit codes first check `goto_on_code`, then fall back to `goto_on_failure`. +- Timeouts are treated as exit code `124`. + +Scripts are executed with the current project root as working directory. + +When `save_stdout_to` and/or `save_stderr_to` are set, the operation stores captured output in the task context +under the specified variable names. + +##### `donna.lib.output` + +`donna.lib.output` operation emits its body as an output cell and then continues to the configured next step. + +The body of the operation is rendered as an output cell during execution. + +Configuration options: + +```toml +id = "<operation_id>" +kind = "donna.lib.output" +next_operation_id = "<next_operation_id>" # required +``` + +##### `donna.lib.finish` + +`donna.lib.finish` operation indicates that the workflow is finished. + +The body of the operation is rendered as an output cell before the workflow completes. + +Each possible path through the workflow MUST end with this operation. + +## Directives + +Donna provides multiple directives that MUST be used in the artifact source to enhance its behavior. + +Here they are: + +1. `{{ "{{ donna.lib.view(<artifact-id-pattern>) }}" }}` — references another artifact (supports `*` and `**` wildcard patterns). In `view`/`execute` modes it renders an exact CLI command to view the artifact; in `analysis` mode it renders a `$$donna ... $$` marker. +2. `{{ "{{ donna.lib.list(<artifact-id-pattern>) }}" }}` — references artifact listing by pattern (supports `*` and `**` wildcard patterns). In `view`/`execute` modes it renders an exact CLI command to list artifacts; in `analysis` mode, it renders a `$$donna ... $$` marker. +3. `{{ "{{ donna.lib.goto(<workflow-operation-id>) }}" }}` — references the next workflow operation to execute. In `view`/`execute` modes it renders an exact CLI command to advance the workflow; in `analysis` mode, it renders a `$$donna goto ... $$` marker used for transition extraction. +4. `{{ "{{ donna.lib.task_variable(<variable_name>) }}" }}` — in `view` mode renders a placeholder note about task-variable substitution, in `execute` mode renders the actual task-context value (or an explicit error marker if missing), and in `analysis` mode renders a `$$donna task_variable ... $$` marker. diff --git a/donna/fixtures/specs/usage/cli.md b/donna/fixtures/specs/usage/cli.md new file mode 100644 index 00000000..bf21ad4b --- /dev/null +++ b/donna/fixtures/specs/usage/cli.md @@ -0,0 +1,235 @@ +# Donna Usage Instructions + +```toml donna +kind = "donna.lib.specification" +``` + +This document describes how agents MUST use Donna CLI to manage and perform their workflows. + +**Agents MUST follow the instructions and guidelines outlined in this document precisely.** + +## Overview + +`donna` is a CLI tool that helps manage the work of AI agents like OpenAI Codex. + +It is designed to invert control flow: instead of agents deciding what to do next, the Donna tells agents what to do. The tool achieves this by following predefined workflows that describe how to perform various tasks. One may look at workflows as hierarchical state machines (HSM) that guide agents through complex processes step by step. + +The core idea is that most high-level workflows are more algorithmic than it may seem at first glance. For example, it may be difficult to fix a particular problem in the codebase, but the overall process of polishing it is quite linear: + +1. Run tests, if they fail, fix the problems. +2. Format the code. +3. Run linters, if there are issues, fix them. +4. Go to the step 1 if you changed something in the process. +5. Finish. + +We may need coding agents on the each step of the process, but there no reason for agents to manage the whole loop by themselves — it takes longer time, spends tokens and confuses agents because they need to reason over long contexts. + +## Primary rules for agents + +- Donna stores all project-related data in `.donna` directory in the project root. +- All work is always done in the context of a session. There is only one active session at a time. +- You MUST always work on one task assigned to you. +- You MUST keep all the information about the session in your memory. +- You always can ask the `donna` tool for the session details if you forget something. + +## CLI + +### Protocol + +Protocol selects the output formatting and behavior of Donna's CLI for different consumers (humans, LLMs, automation). +When an agent invokes Donna, it SHOULD use the `llm` protocol (pass an `-p llm` argument) unless the developer explicitly instructs otherwise. + +### Project root + +`-r <project-root>` sets the project root explicitly for any command (long form: `--root`). +If it is omitted, Donna discovers the project root by searching from the current working directory upwards for the `.donna` workspace directory. +Use this option when you run Donna from outside the project tree or when you want to target a specific project. + +### Protocol cells + +Donna communicates its progress and requests by outputting inrofmation organized in "cells". There are two kinds of cells output: + +- Log cells — `DONNA LOG: <log-message>` — one line messages describing what Donna is doing. Mostly it is an information about the next operation being executed. +- Info cells — multiline cells with structured header and freeform body. + +An example of an info cell: + +``` +--DONNA-CELL eZVkOwNPTHmadXpaHDUBNA BEGIN-- +kind=action_request +media_type=text/markdown +action_request_id=AR-65-bd + +<here goes the multiline markdown content of the action request> + +--DONNA-CELL eZVkOwNPTHmadXpaHDUBNA END-- +``` + +Donna can omit log cell start and end markers if a command produces only a single cell. + +Donna renders cells differently, depending on the protocol used. + +### Commands + +There are four sets of commands: + +- `donna -p <protocol> workspaces …` — manages workspaces. Most-likely it will be used once per your project to initialize it. +- `donna -p <protocol> sessions …` — manages sessions. You will use these commands to start, push forward, and manage your work. +- `donna -p <protocol> artifacts …` — manages artifact discovery, reading, and validation. +- `donna -p <protocol> journal …` — manages session actions journal. You will use these commands to log and inspect the history of actions performed during the session. + +Use: + +- `donna -p <protocol> <command> --help` to get the list of available subcommands. +- `donna -p <protocol> <command> <subcommand> --help` to get the help on specific subcommand. + +### Workspaces + +Run `donna -p <protocol> workspaces init [<directory-path>]` to initialize Donna workspace in the given directory. If `<directory-path>` is omitted, Donna will initialize workspace in the current working directory. + +It is a one time operation you need to perform once per project to create a place where Donna will store all its data. + +### Starting sessions + +The developer is responsible for starting a new session. + +You are allowed to start a new session in the next cases: + +1. There is no active session. +2. The developer explicitly instructed you to start a new session. + +You start session by calling `donna -p <protocol> sessions start`. + +### Session flow + +After the session starts you MUST follow the next workflow to perform your work: + +1. List all possible workflows with command `donna -p <protocol> artifacts list`. +2. Choose the most appropriate workflow for the task you are going to work on or ask the developer if you are not sure which workflow to choose. +3. Start chosen workflow by calling `donna -p <protocol> sessions run <workflow-id>`. +4. Donna will output descriptions of all operations it performs to complete the work. +5. Donna will output **action requests** that you MUST perform. You MUST follow these instructions precisely. +6. When you done processing an action request, call `donna -p <protocol> sessions action-request-completed <action-request-id> <next-full-operation-id>` to report request completion. `<next-full-operation-id>` MUST contain full identifier of the next operation, like `<world>:<artifact>:<operation-id>`. +7. After you complete an action request, Donna will continue workflow execution and output what you need to do next. + +You MUST continue following Donna's instructions until the workflow is completed. + +### Session state + +- `donna -p <protocol> sessions status` — get the status of the current session. +- `donna -p <protocol> sessions details` — get detailed information about the current session, including list of active action requests. +- `donna -p <protocol> sessions start` — start a new session. This command resets session state AND removes all session-level artifacts. +- Run `donna -p <protocol> sessions reset` to reset the current session. This command resets session state BUT keeps all session-level artifacts. Use this command when you need to restart the worklow but keep all the artifacts you created during the session. + +### Starting work + +If the developer asked you to do something new: + +- Run `donna -p <protocol> sessions status` to get the status of the current session. +- If there is no active session, start a new session by calling `donna -p <protocol> sessions start`. +- If the session is active and there are unfinished work in it, you MUST ask the developer whether to continue the work in the current session or start a new one. +- If the session is active and there are no unfinished work in it, follow the instructions in the `Session flow` section to choose and start a new workflow. + +### Continuing work + +If the developer asked you to continue your work, you MUST call `donna -p <protocol> sessions continue` to get your instructions on what to do next. + +If Donna tells you there is no work left, you MUST inform the developer that there is no work left in the current session. + +### Working with artifacts + +An artifact is a markdown document with extra metadata stored in one of the Donna's worlds. + +Use the next commands to work with artifacts: + +- `donna -p <protocol> artifacts list [<artifact-pattern>]` — list all artifacts corresponding to the given pattern. If `<artifact-pattern>` is omitted, list all artifacts in all worlds. Use this command when you need to find an artifact or see what artifacts are available. +- `donna -p <protocol> artifacts view <artifact-pattern>` — get the meaningful (rendered) content of all matching artifacts. This command shows the rendered information about each artifact. Use this command when you need to read artifact content. +- `donna -p <protocol> artifacts validate [<artifact-pattern>]` — validate all artifacts corresponding to the given pattern. If `<artifact-pattern>` is omitted, validate all artifacts in all worlds. + +Donna does not mutate artifacts stored in worlds. Developers and external tools are responsible for creating, updating, moving, copying, or deleting world artifacts before Donna reads or validates them. + +Commands that accept an artifact pattern (`artifacts list`, `artifacts view`, `artifacts validate`) also accept `--predicate/-p <python-expression>` to filter by artifact primary section. The expression is evaluated as `bool` with `section` global available (for example: `--predicate '"workflow" in section.tags'`). + +The format of `<artifact-pattern>` is as follows: + +- full artifact identifier: `<world>:<artifact>` +- `*` — single wildcard matches a single level in the artifact path. Examples: + - `*:artifact:name` — matches all artifacts named `artifact:name` in all worlds. + - `world:*:name` — matches all artifacts with id `something:name` in the `world` world. +- `**` — double wildcard matches multiple levels in the artifact path. Examples: + - `**:name` — matches all artifacts with id ending with `:name` in all worlds. + - `world:**` — matches all artifacts in the `world` world. + - `world:**:name` — matches all artifacts with id ending with `:name` in the `world` world. + +### Working with journal + +Use the next commands to work with session journal: + +- `donna -p <protocol> journal write <message>` — record a single new entry to the journal with the given **single-line** `message` (newlines are not allowed). Donna automatically adds a timestamp and other relevant information to the journal entry. +- `donna -p <protocol> journal view [--lines N] [--follow]` — display journal records. + +Agents MUST use `donna -p <protocol> journal write <message>` to log: + +- Goals of the long-running agent-side operations: `Goal: <goal description>`. +- Significant steps of the long-running agent-side operations: `Step: <phase progress or completion handoff>`. +- Significant thoughts during the long-running operations: `Thought: <important thought>`. +- Significant assumptions during the long-running operations: `Assumption: <important assumption>`. +- Changes in the project source code or in the project structure: `Change: <what changed and where>`. + +For each non-trivial action request, agents MUST follow this journaling contract: + +1. Write exactly one `Goal:` record at action-request start. +2. Write `Step:` records at significant phase boundaries. If an action request describes a multi-step process, there MUST be at least one `Step:` record per specified step and one `Step:` record for the completion handoff. +3. Write `Change:` records after each meaningful source update batch. +4. Write one final `Step:` record immediately before `sessions action-request-completed`. + +Agents MUST consider these cases as significant phase boundaries: + +- A work phase expected to take more than 10 seconds. +- Transition from analysis/research to implementation/editing. +- Transition to a new step in a multi-step process described in the action request. +- Start or completion of a multi-file or multi-artifact change batch. +- A decision that changes implementation direction. + +Before `sessions action-request-completed`, agents MUST check journal completeness for the current action request. + +Agents MUST NOT log: + +- CLI commands they execute. +- Elementary/trivial steps. + +## IMPORTANT ON DONNA TOOL USAGE + +**Strictly follow described command syntax** + +**You MUST follow `donna` call conventions specified in**, by priority: + + 1. Direct instructions from the developer. + 2. `AGENTS.md` document. + 3. Specifications in `project:` world. + 4. This document. + +**All Donna CLI commands MUST include an explicit protocol selection using `-p <protocol>`.** Like `donna -p llm <command>`. + +**All Donna CLI commands MUST be run from the project root or its subdirectories unless you pass `-r <project-root>`.** + +If you are not running from the project root or its subdirectories, add `-r <project-root>` to point Donna to the correct project. + +**Pass text arguments to the tool in quotes with respect to escaping.** The tool MUST receive the exact text you want to pass as an argument. + +Use one of the next approaches to correctly escape text arguments: + +``` +# option 1 +donna -p <protocol> <command> <...> $'# Long text\n\nwith escape sequences...' + +# option 2 +donna -p <protocol> <command> <...> \ + "$(cat <<'EOF' +# Long text + +with escape sequences... +EOF +)" + +``` diff --git a/donna/fixtures/specs/usage/worlds.md b/donna/fixtures/specs/usage/worlds.md new file mode 100644 index 00000000..e4a94a27 --- /dev/null +++ b/donna/fixtures/specs/usage/worlds.md @@ -0,0 +1,43 @@ +# Donna World Layout + +```toml donna +kind = "donna.lib.specification" +``` + +This document describes how Donna discovers and manages its dynamic and/or external artifacts. +Including usage docs, work workflows, operations, current work state and additional code. + +## Overview + +In order to function properly and to perform in a full potential, Donna relies on a set of artifacts +that guide its behavior and provide necessary capabilities. + +These artifacts are represented as text files, primary in Markdown format, however other text-based +formats can be used as well, if explicitly requested by the developer or by the workflows. + +Donna discovers these artifacts by scanning the "worlds" specified in `<project-root>/.donna/config.toml` +as `worlds` list. Most of worlds are filesystem folders, however other world types can be implemented such as: +s3 buckets, git repositories, databases, etc. + +Default worlds and there locations are: + +- `donna` — `<project-root>/.agents/donna` — the project-local bundled Donna specs installed from `donna/fixtures/specs` by workspace init/update. +- `home` — `~/.donna/home` — the user-level donna artifacts, i.e. those that should be visible for all workspaces on this machine. +- `project` — `<project-root>/.donna/project` — the project-level donna artifacts, i.e. those that are specific to this project. +- `session` — `<project-root>/.donna/session` — the session world that contains the current state of work performed by Donna. + +All worlds have a free layout, defined by developers who own the particular world. + +## Artifact Access + +Donna has read access to artifacts stored in worlds. It discovers, fetches, renders, and validates world artifacts, but it does not create, update, move, copy, or delete them. + +Developers and external tools are responsible for mutating world artifacts before Donna reads or validates them. + +Donna still writes its own session state and journal data in the `session` world, but that internal state storage is separate from world-artifact mutation. + +## `<world>:intro` artifact + +It is a recommended practice to provide a short introductory artifact `intro.md` at the root of each world. + +So, the agent can load descriptions of all worlds in a single command like `donna -p llm artifacts view "*:intro"`. diff --git a/donna/lib/worlds.py b/donna/lib/worlds.py index e1988cea..42e85bc8 100644 --- a/donna/lib/worlds.py +++ b/donna/lib/worlds.py @@ -1,7 +1,5 @@ """Shared world constructor instances for default configuration.""" from donna.workspaces.worlds.filesystem import FilesystemWorldConstructor -from donna.workspaces.worlds.python import PythonWorldConstructor filesystem = FilesystemWorldConstructor() -python = PythonWorldConstructor() diff --git a/donna/workspaces/config.py b/donna/workspaces/config.py index ee1364b7..482ce0d7 100644 --- a/donna/workspaces/config.py +++ b/donna/workspaces/config.py @@ -55,9 +55,9 @@ def _create_default_worlds() -> list[WorldConfig]: WorldConfig.model_validate( { "id": WorldId("donna"), - "kind": "donna.lib.worlds.python", + "kind": "donna.lib.worlds.filesystem", "session": False, - "package": "donna", + "path": pathlib.Path(".agents") / "donna", } ), WorldConfig.model_validate( diff --git a/donna/workspaces/initialization.py b/donna/workspaces/initialization.py index 9ba6f0d6..24605ad9 100644 --- a/donna/workspaces/initialization.py +++ b/donna/workspaces/initialization.py @@ -15,6 +15,8 @@ SKILLS_ROOT_DIR = pathlib.Path(".agents") / "skills" DONNA_SKILL_FIXTURE_DIR = pathlib.Path("fixtures") / "skills" +DONNA_SPECS_ROOT_DIR = pathlib.Path(".agents") / "donna" +DONNA_SPECS_FIXTURE_DIR = pathlib.Path("fixtures") / "specs" # this list must only increase in size, # do not remove old items from it, since users may upgrade from older versions of Donna @@ -36,6 +38,17 @@ def _sync_donna_skill(project_dir: pathlib.Path) -> None: shutil.copytree(source_dir, project_dir / SKILLS_ROOT_DIR, dirs_exist_ok=True) +def _sync_donna_specs(project_dir: pathlib.Path) -> None: + source = importlib.resources.files("donna").joinpath(*DONNA_SPECS_FIXTURE_DIR.parts) + target_dir = project_dir / DONNA_SPECS_ROOT_DIR + + if target_dir.exists(): + shutil.rmtree(target_dir) + + with importlib.resources.as_file(source) as source_dir: + shutil.copytree(source_dir, target_dir) + + @unwrap_to_error def initialize_runtime( # noqa: CCR001 root_dir: pathlib.Path | None = None, @@ -84,7 +97,9 @@ def initialize_runtime( # noqa: CCR001 @unwrap_to_error def initialize_workspace( - project_dir: pathlib.Path, install_skills: bool = True + project_dir: pathlib.Path, + install_skills: bool = True, + install_specs: bool = True, ) -> Result[None, core_errors.ErrorsList]: """Initialize the physical workspace for the project (`.donna` directory).""" project_dir = project_dir.resolve() @@ -118,11 +133,18 @@ def initialize_workspace( if install_skills: _sync_donna_skill(project_dir) + if install_specs: + _sync_donna_specs(project_dir) + return Ok(None) @unwrap_to_error -def update_workspace(project_dir: pathlib.Path, install_skills: bool = True) -> Result[None, core_errors.ErrorsList]: +def update_workspace( + project_dir: pathlib.Path, + install_skills: bool = True, + install_specs: bool = True, +) -> Result[None, core_errors.ErrorsList]: project_dir = project_dir.resolve() workspace_dir = project_dir / config.DONNA_DIR_NAME @@ -132,4 +154,7 @@ def update_workspace(project_dir: pathlib.Path, install_skills: bool = True) -> if install_skills: _sync_donna_skill(project_dir) + if install_specs: + _sync_donna_specs(project_dir) + return Ok(None) diff --git a/donna/workspaces/worlds/python.py b/donna/workspaces/worlds/python.py deleted file mode 100644 index f0569f7c..00000000 --- a/donna/workspaces/worlds/python.py +++ /dev/null @@ -1,191 +0,0 @@ -import importlib -import importlib.resources -import pathlib -from collections.abc import Iterable -from typing import TYPE_CHECKING, cast - -from donna.core.errors import ErrorsList -from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import ArtifactId, FullArtifactId, FullArtifactIdPattern, WorldId -from donna.domain.types import Milliseconds -from donna.workspaces import errors as world_errors -from donna.workspaces.artifacts_discovery import ArtifactListingNode, list_artifacts_by_pattern -from donna.workspaces.worlds.base import RawArtifact -from donna.workspaces.worlds.base import World as BaseWorld -from donna.workspaces.worlds.base import WorldConstructor - -if TYPE_CHECKING: - from donna.machine.artifacts import Artifact - from donna.workspaces.artifacts import ArtifactRenderContext - from donna.workspaces.config import WorldConfig - - -class PythonRawArtifact(RawArtifact): - content: bytes - - def get_bytes(self) -> bytes: - return self.content - - @unwrap_to_error - def render( - self, full_id: FullArtifactId, render_context: "ArtifactRenderContext" - ) -> Result["Artifact", ErrorsList]: - from donna.workspaces.config import config - - source_config = config().get_source_config(self.source_id).unwrap() - return Ok(source_config.construct_artifact_from_bytes(full_id, self.get_bytes(), render_context).unwrap()) - - -class Python(BaseWorld): - id: WorldId - session: bool = False - package: str - artifacts_root: str - - def _resource_root(self) -> importlib.resources.abc.Traversable | None: - package = self.artifacts_root - - try: - return importlib.resources.files(package) - except ModuleNotFoundError: - return None - - def _artifact_listing_root(self) -> ArtifactListingNode | None: - root = self._resource_root() - if root is None: - return None - - return cast(ArtifactListingNode, root) - - def _resolve_artifact_file( - self, artifact_id: ArtifactId - ) -> Result[importlib.resources.abc.Traversable | None, ErrorsList]: - parts = str(artifact_id).split(":") - if not parts: - return Ok(None) - - resource_root = self._resource_root() - if resource_root is None: - return Ok(None) - - *dirs, file_name = parts - resource_dir = resource_root - - for part in dirs: - resource_dir = resource_dir.joinpath(part) - - if not resource_dir.is_dir(): - return Ok(None) - - from donna.workspaces.config import config - - supported_extensions = config().supported_extensions() - matches = [ - entry - for entry in resource_dir.iterdir() - if entry.is_file() - and entry.name.startswith(f"{file_name}.") - and pathlib.Path(entry.name).suffix.lower() in supported_extensions - ] - - if not matches: - return Ok(None) - - if len(matches) > 1: - return Err([world_errors.ArtifactMultipleFiles(artifact_id=artifact_id, world_id=self.id)]) - - return Ok(matches[0]) - - def has(self, artifact_id: ArtifactId) -> bool: - resolve_result = self._resolve_artifact_file(artifact_id) - if resolve_result.is_err(): - return True - - return resolve_result.unwrap() is not None - - @unwrap_to_error - def fetch(self, artifact_id: ArtifactId) -> Result[RawArtifact, ErrorsList]: # noqa: CCR001 - resource_path = self._resolve_artifact_file(artifact_id).unwrap() - if resource_path is None: - return Err([world_errors.ArtifactNotFound(artifact_id=artifact_id, world_id=self.id)]) - - extension = pathlib.Path(resource_path.name).suffix - from donna.workspaces.config import config - - source_config = config().find_source_for_extension(extension) - if source_config is None: - return Err( - [ - world_errors.UnsupportedArtifactSourceExtension( - artifact_id=artifact_id, - world_id=self.id, - extension=extension, - ) - ] - ) - - return Ok( - PythonRawArtifact( - source_id=source_config.kind, - content=resource_path.read_bytes(), - ) - ) - - @unwrap_to_error - def has_artifact_changed(self, artifact_id: ArtifactId, since: Milliseconds) -> Result[bool, ErrorsList]: - return Ok(False) - - def list_artifacts(self, pattern: FullArtifactIdPattern) -> list[ArtifactId]: # noqa: CCR001 - return list_artifacts_by_pattern( - world_id=self.id, - root=self._artifact_listing_root(), - pattern=pattern, - ) - - # TODO: How can the state be represented in the Python world? - def read_state(self, name: str) -> Result[bytes | None, ErrorsList]: - return Err([world_errors.WorldStateStorageUnsupported(world_id=self.id)]) - - def write_state(self, name: str, content: bytes) -> Result[None, ErrorsList]: - return Err([world_errors.WorldStateStorageUnsupported(world_id=self.id)]) - - def journal_reset(self) -> Result[None, ErrorsList]: - return Err([world_errors.WorldStateStorageUnsupported(world_id=self.id)]) - - def journal_add(self, content: bytes) -> Result[None, ErrorsList]: - return Err([world_errors.WorldStateStorageUnsupported(world_id=self.id)]) - - def journal_read(self, lines: int | None = None, follow: bool = False) -> Iterable[Result[bytes, ErrorsList]]: - yield Err([world_errors.WorldStateStorageUnsupported(world_id=self.id)]) - - @unwrap_to_error - def initialize(self, reset: bool = False) -> Result[None, ErrorsList]: - super().initialize(reset=reset).unwrap() - return Ok(None) - - def is_initialized(self) -> bool: - return True - - -class PythonWorldConstructor(WorldConstructor): - def construct_world(self, config: "WorldConfig") -> Python: - package = getattr(config, "package", None) - - if package is None: - raise ValueError(f"World config '{config.id}' does not define a python package") - - module = importlib.import_module(str(package)) - artifacts_root = getattr(module, "donna_artifacts_root", None) - - if artifacts_root is None: - raise ValueError(f"Package '{package}' does not define donna_artifacts_root") - - if not isinstance(artifacts_root, str): - raise ValueError(f"Package '{package}' defines invalid donna_artifacts_root") - - return Python( - id=config.id, - package=str(package), - artifacts_root=artifacts_root, - session=config.session, - )