Skip to content

FEAT: Add remove_seeds_from_memory_async to memory - #1

Open
blahdeblahde wants to merge 624 commits into
mainfrom
blahdeblahde-remove-seeds-from-memory-api
Open

FEAT: Add remove_seeds_from_memory_async to memory#1
blahdeblahde wants to merge 624 commits into
mainfrom
blahdeblahde-remove-seeds-from-memory-api

Conversation

@blahdeblahde

@blahdeblahde blahdeblahde commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Description

PyRIT memory can add seed prompt datasets (via add_seed_datasets_to_memory_async / add_seeds_to_memory_async) and query them with get_seeds, but there is no counterpart for removing seeds. Today users have to drop into raw SQL, which is error-prone and differs across the supported backends (DuckDB, SQLite, Azure SQL).

This PR adds remove_seeds_from_memory_async, which accepts the same filter parameters as get_seeds (dataset_name, dataset_name_pattern, added_by, harm_categories, authors, groups, source, value_sha256, data_types, seed_type, parameters, metadata, prompt_group_ids) and returns the number of seeds removed.

Recommended workflow — preview, then delete with the same filters:

# See what matches before deleting
to_delete = memory.get_seeds(dataset_name="illegal")

# Remove them; returns the count deleted
removed = await memory.remove_seeds_from_memory_async(dataset_name="illegal")

Implementation

  • Extracted the filter-building logic out of get_seeds into a shared private helper (_build_seed_filter_conditions) so both query and removal build conditions from a single source and cannot drift.
  • remove_seeds_from_memory_async deletes matching rows in a single transaction using the SQLAlchemy ORM (rollback on error), so it behaves consistently across DuckDB, SQLite, and Azure SQL.

Safety

  • At least one filter must be provided; a no-filter call raises ValueError to prevent accidentally wiping the entire seed database.

Tests and Documentation

Teststests/unit/memory/memory_interface/test_interface_remove_seeds.py covers:

  • exact dataset_name match
  • dataset_name_pattern (SQL LIKE)
  • multi-filter narrowing (dataset_name + added_by)
  • no-filter call raises ValueError
  • zero-match returns 0 and deletes nothing
  • harm_categories (list field) and source filters

Documentation — added a "Removing Seeds from the Database" section to doc/code/memory/8_seed_database.py (and the synced .ipynb) demonstrating the preview-then-remove workflow.

rlundeen2 and others added 30 commits July 3, 2026 05:55
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ng/TAP (microsoft#1377)

Co-authored-by: Robert Fitzpatrick <fitzpatrickr@microsoft.com>
Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
Co-authored-by: Robert Fitzpatrick <robertfitzpatrick@Roberts-MacBook-Pro.local>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ft#2130)

Co-authored-by: Behnam Ousat <behnamousat@microsoft.com>
Co-authored-by: Behnam Ousat <behnamousat@microsoft.com>
…rom memory (microsoft#2120)

Co-authored-by: Behnam Ousat <behnamousat@microsoft.com>
…rosoft#2137)

Signed-off-by: WatchTree-19 <119982314+WatchTree-19@users.noreply.github.com>
Co-authored-by: Richard Lundeen <rlundeen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…icrosoft#2134)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: WatchTree-19 <119982314+WatchTree-19@users.noreply.github.com>
Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rosoft#2111)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…icrosoft#2132)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…lper (1/3) (microsoft#2125)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tch group (microsoft#2149)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… and attacks (microsoft#2140)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…atch group across 1 directory (microsoft#2148)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…icrosoft#2151)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…esponseHandler (microsoft#2146)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: jsong468 <songjustin@microsoft.com>
romanlutz and others added 30 commits August 14, 2026 22:46
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a8b8c380-b64e-4ea7-a731-fb8c3b1778ab
…#2399)

Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
Copilot-Session: c9450170-29e5-4918-b1bc-71f6d9f46402
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7b3a8d42-bfe2-4b5e-90ae-fc211028c2ad
Co-authored-by: Eve Kazarian <evekazarian@microsoft.com>
Co-authored-by: Richard Lundeen <rlundeen@microsoft.com>
Co-authored-by: Richard Lundeen <137218279+rlundeen2@users.noreply.github.com>
Copilot-Session: 63d0ddad-791c-4301-8502-8f6b9dc8a88a
Copilot-Session: 9846b003-bdd8-48bb-a501-bf4f3a58b171
)

Co-authored-by: Bradley Menz <menz0069@flinders.edu.au>
Co-authored-by: Richard Lundeen <137218279+rlundeen2@users.noreply.github.com>
Co-authored-by: Richard Lundeen <rlundeen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Adam Lin <adam@agentthreatrule.org>
Co-authored-by: Adrian Gavrila <adrian-gavrila@outlook.com>
Co-authored-by: hannahwestra25 <hannahwestra@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ts (microsoft#1351)

Co-authored-by: Richard Lundeen <rlundeen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Richard Lundeen <137218279+rlundeen2@users.noreply.github.com>
Copilot-Session: 2164ffd5-f965-47b1-93f9-29c058dfbb01
Copilot-Session: 669aaa53-c978-4479-aea3-b2215bb42041
…de (microsoft#2407)

Co-authored-by: VishnuR23 <19866703+VishnuR23@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…#2403)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…icrosoft#2423)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… to 0.16.3 (microsoft#2421)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…rosoft#2425)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 73cdd687-fe68-4ab2-8e0c-5ae03c6aeb3b
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Richard Lundeen <rlundeen@microsoft.com>
Co-authored-by: Richard Lundeen <137218279+rlundeen2@users.noreply.github.com>
Copilot-Session: 00d2ef6d-0445-4a59-af21-ca032a7eb915
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d6f9fd6d-38b7-4841-bb94-c608033c1c04
Copilot-Session: 1e5bf78a-8e94-4839-b6bf-7f29a6a19a38
Copilot-Session: 7affb573-e048-445a-9152-0b46e997dd89
Copilot-Session: 9b95a2c8-cdc2-4e94-9776-cbea2a2b41ff
Add executed outputs to the remove_seeds example cell in 8_seed_database
and remove the 'deleting one modality' bullet that describes a still-valid
group rather than an error, per review feedback.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f9e3cc10-5c77-4384-92d9-d27dad9baea2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.