mtg-mechanics-parser

mtg-mechanics-parser is a rule-based parsing and scoring system for creature cards in Magic: The Gathering.

The project transforms raw card data from Scryfall into structured gameplay features by parsing a card's Oracle text, identifying game mechanics, extracting quantitative features, and assigning a weighted power score.

This project is a continuation on our previous work on scoring creature cards in Magic: The Gathering.

Unlike traditional tabular datasets, Magic card text encodes structured game rules in natural-language form, including actions, triggered events, costs, restrictions, and continuous effects. As a result, evaluating card strength is a non-trivial task even for experienced Magic: The Gathering players, as it requires interpreting mechanics embedded in text rather than explicit structured fields.

Additionally, similar gameplay mechanics can be expressed through many distinct textual patterns. This makes rule-based parsing challenging, requiring a large set of pattern-matching rules and feature-specific extraction logic.

To address this challenge, this project implements a complete processing pipeline from scratch:

• Dataset cleaning and building
• Oracle text parsing
• Ability segmentation and classification
• Mechanic-specific feature extraction
• Feature vector generation
• Weighted card scoring

No external NLP or machine learning libraries are used for mechanic detection; all parsing, classification, and feature extraction logic is implemented using domain-specific systems designed for Magic: The Gathering card text.

The parser currently analyzes creature cards and detects a wide range of gameplay mechanics including removal, card advantage, token generation, mana production, reanimation, counters, triggered abilities, activated abilities, and global effects.

The resulting feature representations can be used for card evaluation, dataset analysis, machine learning experiments, and gameplay complexity research.

You can read our paper on this project here.

Why This Project Exists

Magic: The Gathering contains tens of thousands of unique cards whose functionality is primarily expressed through natural-language Oracle text.

While datasets such as Scryfall provide extensive card metadata, many important gameplay characteristics are not available as structured features. For example, determining whether a card generates card advantage, creates tokens, removes opposing permanents, or reanimates creatures often requires interpreting the card's rules text.

By converting card text into structured mechanic-level features, those nuanced gameplay characteristics can be analyzed computationally.

The long-term goal is to provide a foundation for card evaluation, power-level estimation, and machine learning models operating on gameplay-relevant card characteristics.

Setup

Install Python 3.10 or newer and Git. For a new checkout, choose the block for your shell and run it in the directory where you want the repository created. If you already have the repo open, skip the clone and cd lines.

Windows PowerShell

If PowerShell blocks activation scripts, use the no-activation block below instead.

git clone https://github.com/AgentSamSA/mtg-mechanics-parser.git
cd mtg-mechanics-parser
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -e .
mtg-parser --help

Windows setup without activation (works when scripts are disabled):

git clone https://github.com/AgentSamSA/mtg-mechanics-parser.git
cd mtg-mechanics-parser
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -e .
Set-Alias mtg-parser "$PWD\.venv\Scripts\mtg-parser.exe"
mtg-parser score --card "Birds of Paradise"

The alias lasts for the current PowerShell session. In a new terminal, run it again from the repository root:

Set-Alias mtg-parser "$PWD\.venv\Scripts\mtg-parser.exe"

macOS / Linux (Bash or Zsh)

git clone https://github.com/AgentSamSA/mtg-mechanics-parser.git
cd mtg-mechanics-parser
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .
mtg-parser --help

For the examples below, keep this environment active and run commands from the repository root (the directory containing pyproject.toml). In a new terminal, activate the environment again. If PowerShell blocks activation, use the commands that run without activation.

Documentation

See the documentation index for detailed guides:

• Commands and presentation demos: all commands, options, JSON output, and troubleshooting.
• Research notebook setup: full dependencies, kernel selection, and TensorFlow/Keras troubleshooting.
• Architecture: pipeline stages, data objects, and Python usage.
• Data preparation: downloads, cleaned schema, and dataset scope.
• Ability parsing: normalization, keyword routing, and ability classification.
• Feature extraction: extractor reference and adding mechanics.
• Extractor audit: correctness fixes, known issues, and improvement priorities.
• Scoring: exact weights, calculations, and interpretation.

Command-Line Walkthrough

The CLI uses the same parsing, extraction, and scoring code as the notebook. Examples use the installed mtg-parser command, with a hyphen. mtg_parser with an underscore is the Python package name, not the terminal command.

Show general help or the score command's options:

mtg-parser --help
mtg-parser score --help

Offline presentation demo

The checked-in cleaned dataset supports these examples without downloading data or launching Jupyter. Run each block separately to inspect the stages.

Parse ability types, conditions, costs, and effects:

mtg-parser parse --card "Birds of Paradise"

Extract mechanic features:

mtg-parser features --card "Birds of Paradise"

Explain the total score and each ability's weighted contributions:

mtg-parser score --card "Birds of Paradise"

The last command reports a total of 4: power/toughness 1, keywords 1, and abilities 2 (mana production). Scores approximate the project's manual rubric rather than measuring competitive win rate.

Explore cards and sets

Score all Bloomburrow cards in the input dataset:

mtg-parser score --set blb

Pick a reproducible random card from Bloomburrow:

mtg-parser score --set blb --random --seed 7

Pick a random set and show its first five cards in dataset order:

mtg-parser score --random-set --seed 7 --limit 5

Export scores or features. These commands create results/ if needed:

mtg-parser score --format json --output "results/scores.json"
mtg-parser features --set blb --format json --output "results/blb-features.json"

--card matches exact names, ignoring case. With no selection flags, commands process the entire input. Random choices default to seed 42. JSON output is always an array, including for a single card.

Selection searches the local input dataset. The default contains 857 rows across lea, ons, lrw, ktk, iko, and blb. See the command reference for custom input files and all options.

Download and rebuild the dataset

This optional workflow downloads missing bulk data, extracts creatures, and rebuilds the cleaned CSV. It can take longer and requires network access when raw data is missing. The offline demos above do not require this step.

mtg-parser prepare

For individual stages, custom paths, or a refresh of existing raw files, follow the data workflow.

Cleaning joins creature metadata to the manual-points CSV by card name and set. It rebuilds the labeled research dataset; it does not include every creature from the bulk download. Existing raw intermediates are reused.

Run the tests

python -m unittest discover -s tests -v

System Pipeline

mtg-mechanics-parser processes Magic: The Gathering card data from Scryfall and converts a card's Oracle text into structured gameplay features that can be used to estimate card power.

The main challenge is that identical gameplay mechanics can appear in many syntactically different forms, requiring rule-based pattern matching rather than simple keyword detection.

Raw Scryfall Data
        ↓
Creature Card Filtering
        ↓
Dataset Cleaning
        ↓
Ability Parsing
        ↓
Card Context Construction
        ↓
Feature Extraction
        ↓
Feature Scoring
        ↓
Final Card Power Score

Stage 1: Data Preparation

Raw card data is downloaded from Scryfall and is filtered to include only creature cards. The dataset is then cleaned to remove unnecessary fields, invalid records, and filter down to only the columns we are interested in.

Stage 2: Ability Parsing

Oracle text is segmented into individual ability blocks and converted into structured Ability objects. This separates activated abilities, triggered abilities, and static abilities for downstream processing.

Stage 3: Card Context Construction

Relevant card information such as mana cost, power/toughness, keywords, and parsed abilities is assembled into a unified Card object that is passed through the extraction pipeline.

Stage 4: Feature Extraction

Feature extractors analyze card text and identify gameplay mechanics such as:

• Card draw
• Token generation
• Mana production
• Removal effects
• Reanimation
• Damage effects
• Global effects
• Counter placement
• Activated abilities
• Triggered abilities

Each extractor produces structured feature values that quantify the complexity and gameplay impact of the card.

Stage 5: Scoring

Extracted features are converted into weighted scores using predefined scoring rules. Individual component scores are combined into a final card power score.

The resulting score can be used for downstream analysis, comparison, and evaluation of creature card power level.

Project Structure

docs/                            # Parser guides and command reference

data/
  processed/                     # Processed dataset
  raw/                           # Raw dataset sources from Scryfall

notebooks/
  IMT575_Final_Project_v2.ipynb  # Final project walkthrough and analysis
  assets/
    card_images/                 # Example Magic card images
    diagrams/                    # Card anatomy and pipeline diagrams

tests/                           # CLI integration tests

src/
  mtg_parser/
    __main__.py                  # Python module entry point
    cli.py                       # Standalone pipeline and demo commands
    constants/
      feature_weights.py         # Weights for scoring ability features
      keyword_weights.py         # Weights for scoring keyword features
      mechanics.py               # Gameplay mechanic related constants
      searches.py                # Regex searches to identify ability features
    data/
      __init__.py                # Package initialization
      build_dataset.py           # Builds and saves cleaned dataset
      cleaning.py                # Cleans filtered dataset
      download.py                # Fetches raw dataset
      extract_creatures.py       # Extracts only creature cards from raw dataset
    features/
      extractors/
        __init__.py              # Package initialization
        activated.py             # Activated ability features
        bounce.py                # Permanent card bounce features
        card_advantage.py        # Card advantage features
        damage.py                # Direct damage features
        destroy.py               # Hard removal features
        global_effects.py        # Global effect features
        impulse_draw.py          # Impulsive draw features
        keyword_counters.py      # Keyword counter features
        mana_production.py       # Mana production/reduction features
        minus_xx.py              # -X/-X features
        reanimate.py             # Graveyard reanimation features
        special.py               # Special features
        static.py                # Static ability features
        tokens.py                # Creature token features
        triggered.py             # Triggered ability features
      utils/
        counters_utils.py        # Keyword counter helpers
        mana_utils.py            # Mana ability helpers
        parsing.py               # General feature text parsing helpers
        token_utils.py           # Creature token helpers
      ability_features.py        # AbilityFeatures class definition
      card_features.py           # CardFeatures class definition
      detect_zone.py             # Identify gameplay zone in text
      encode_keywords.py         # One-hot encoding keyword columns
      feature_extractor.py       # Feature extractor pipeline
      keyword_features.py        # KeywordFeatures class definition
      pattern_builder.py         # Identify global ability text patterns
      pipeline.py                # Holds feature extractor pipeline initialization
      registry.py                # Holds our feature list
    parsing/
      __init__.py                # Package initialization
      ability.py                 # Ability class definition
      keyword_router.py          # Keyword line identifier
    pipeline/
      __init__.py                # Package initialization
      build_context.py           # Build card context from row
      card_ability_bundle.py     # CardAbilityBundle class definition
      extract_ability_blocks.py  # Get ability blocks from text
      orchestrator.py            # Pipeline methods to process each row
    scoring/
      models/
        ability_score.py         # AbilityScore class definition
        card_score.py            # CardScore class definition
      __init__.py                # Package initialization
      ability_scorer.py          # Score ability features for each card
      card_scorer.py             # Score combined features for each card
      keyword_scorer.py          # Score keyword features for each card
      pt_scorer.py               # Score power/toughness features for each card
    utils/
      __init__.py                # Package initialization
      normalizer.py              # Normalize keywords
      paths.py                   # Set project paths
      text_preprocessing.py      # Preprocess Oracle text

Package Overview

data/

Responsible for dataset acquisition, cleaning, and preprocessing.

---

parsing/

Responsible for converting Oracle text into structured ability representations.

• Ability types are identified via an Ability object which parses the ability from the Oracle text

---

pipeline/

Coordinates end-to-end processing of individual cards.

• build_context identifies which lines of text are keyword abilities to avoid double scoring them. While Scryfall preprocesses keywords into a separate column, they retain the keywords inside Oracle text
• card_ability_bundle defines a CardAbilityBundle object which holds certain card-level metadata and a list of Ability objects for the parsed cards
• extract_ability_blocks builds out the list of ability blocks to be parsed downstream

---

features/

Responsible for extracting gameplay-relevant mechanics from card text. Outputs structured feature vectors describing card functionality.

• Each extractor inside features/extractors is responsible for extracting their respective feature from the ability text. For instance, destroy.py is responsible for extracting any removal effects (destroy/exile)
• AbilityFeature, CardFeature, and KeywordFeature objects are defined in order to score the cards
• Regex strings were utilized in order to match ability text to features. These strings are stored under constants/searches.py

---

scoring/

Converts extracted features into numeric power scores. Utilizes that card's Feature objects and weights to determine final score.

---

constants/

Stores configuration values used throughout the project, including feature weights and feature extractor pipeline initialization.

---

utils/

Shared utility functions used across multiple modules.

Known Limitations

As Magic text is incredibly complex, there are certain cases that are currently not handled by our parser. Our goal is to eventually be able to handle each of these edge cases, but our current list includes:

• Transforming/double face cards
• Certain ability downsides
• Cases of nonscoring text, such as text that exists purely as part of card instantiation (e.g. "This creature enters with an oil counter on it."). See cards like Evolved Spinoderm
• Many joke cards with mechanics not in the formally defined rules

In general, the parser tends to overscore cards relative to their actual power level, due to the current limitations on handling downsides and nonscoring text.
This parser also currently only handles creature cards, as they are relatively simple to work with, and most importantly, include a quantifiable set of values (power/toughness) that allows us to establish a baseline.

Other card types, such as instants and sorceries, can be incredibly complex and can include very unique text that is hard to score, such as is the case with cards like Indomitable Creativity.

Acknowledgements

Special thanks to Gavin Gray Valentine, whose Youtube video is what sparked interest in this project. We can recommend his channel, Distraction Makers, as a source of valuable game design discussion and insight.