[WIP] Prototype for new technology data

.codespell.ignore

@@ -0,0 +1,4 @@
+# SPDX-FileCopyrightText: The technology-data authors
+#
+# SPDX-License-Identifier: MIT
+CAF

.github/instructions/copilot-instructions.md

@@ -0,0 +1,44 @@
+# Copilot Coding Agent Instructions for `technologydata`
+
+<!--
+SPDX-FileCopyrightText: The technology-data authors
+
+SPDX-License-Identifier: MIT
+
+-->
+
+## Project Overview
+- `technologydata` is a Python package for energy system modellers to screen, harmonize, and transform techno-economic input data for energy system models.
+- Major use cases: data validation, harmonization, transformation to model-ready formats, and provenance/audit tracing.
+- Core classes: `DataPackage`, `Technology`, `Parameter`, `Source`, and `SourceCollection` (see `technologydata/`).
+- Data flows: Input data (JSON or DataFrame) → `DataPackage`/`Technology` objects → validation/transformation → output for modeling/analysis.
+
+## Key Workflows
+- **Install dependencies:** Use `uv` (see `pyproject.toml`, `uv.lock`). Example: `uv sync`.
+- **Activate virtual environment:** Use `source .venv/bin/activate` before running commands.
+- **Linting/formatting:** Use `ruff` (`ruff check .` and `ruff format .`).
+- **Type checking:** Use `mypy`.
+- **Testing:** Use `pytest` (tests in `test/`). Fixtures in `test/conftest.py`. Run: `pytest`.
+- **Pre-commit hooks:** Set up with `pre-commit` using `.pre-commit-config.yaml`, use this preferably over using `mypy` and `ruff` manually.
+- **Docs:** Built with `mkdocs` (see `docs/`).
+
+## Project-Specific Patterns
+- **Validation:** Instantiating `Technology` or `DataPackage` triggers schema validation. Use `.check_consistency()` and `.calculate_parameters()` for further checks and derivations.
+- **Parameter Derivation:** Use `.calculate_parameters()` to fill in missing values based on rules.
+- **Unit Handling:** Uses `pint` for units and `pydeflate` for currency/inflation adjustments. Custom `UnitRegistry` in `technologydata/utils/units.py`.
+- **Data Provenance:** Each `Parameter` and `Technology` tracks its source and transformation history.
+- **Data Input:** Prefer JSON conforming to the internal schema. See `test/test_data/` for examples.
+
+## External Integrations
+- `pydeflate` for currency/inflation
+- `pint` for units
+- `hdx.location.country` for country/currency codes
+- `savepagenow` for web archiving
+
+## References
+- Design: `docs/design.md`
+- Class diagram: `docs/class-diagram.puml`
+- Example data: `test/test_data/`
+
+---
+If any conventions or workflows are unclear, please ask for clarification or check `docs/design.md` for rationale and details.

.github/workflows/ci.yaml

@@ -1,64 +1,41 @@
-name: CI
+# SPDX-FileCopyrightText: The technology-data authors
+#
+# SPDX-License-Identifier: MIT
+name: Technologydata prototype CI
 
 on:
   push:
-    branches: [master]
+    branches: [prototype-2]
   pull_request:
-    branches: [master]
-  schedule:
-  - cron: "0 5 * * 0"
-
-
-# Cancel any in-progress runs when a new run is triggered
-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}
-  cancel-in-progress: true
+    branches: [prototype-2]
 
 jobs:
-  build:
-    name: CI
+  test:
     runs-on: ubuntu-latest
 
-    defaults:
-      run:
-        shell: bash -l {0}
-
     steps:
-    - name: Checkout code
+    - name: Checkout repository
       uses: actions/checkout@v4
 
-    - name: Setup micromamba
-      uses: mamba-org/setup-micromamba@v2
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
       with:
-        micromamba-version: latest
-        environment-file: environment.yaml
-        log-level: debug
-        init-shell: bash
-        cache-environment: true
-        cache-downloads: true
+        python-version: "3.13"
 
-    - name: Run unit tests
+    - name: Cache Python dependencies
+      uses: actions/cache@v3
+      with:
+        path: ~/.cache/pip
+        key: ${{ runner.os }}-pip-${{ hashFiles('**/uv.lock') }}
+        restore-keys: |
+          ${{ runner.os }}-pip-
+    - name: Install package and dependencies
       run: |
-        make unit-test
+        python -m pip install uv
+        uv lock
+        uv sync
 
-    - name: Test snakemake workflows
+    - name: Run unit tests
       run: |
-        make test
-
-    - name: Upload artifacts
-      uses: actions/upload-artifact@v4
-      with:
-        name: technology-data
-        path: outputs/
-
-    - name: Check for differences in generated outputs
-      run: |-
-        git config --global --add safe.directory $GITHUB_WORKSPACE
-        git add outputs/
-        if ! git diff --cached --quiet; then
-          echo "Generated outputs differ from the branch. Please commit rerun `snakemake` and commit the changes to the `outputs/*.csv` files."
-          git diff --cached
-          exit 1
-        else
-          echo "No differences found in generated outputs."
-        fi
+        uv run pytest

.gitignore

@@ -1,26 +1,77 @@
-# SPDX-FileCopyrightText: Contributors to technology-data <https://github.com/pypsa/technology-data>
+# SPDX-FileCopyrightText: The technology-data authors
 #
-# SPDX-License-Identifier: GPL-3.0-only
+# SPDX-License-Identifier: MIT
 
+# Byte-compiled / optimized / DLL files
+*__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual environments
+.env/
+.venv/
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# PyInstaller
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+.pytest_cache/
+
+# PyCharm
+.idea/
+
+# VS Code
+.vscode/
+
+# macOS
 .DS_Store
-.snakemake
-_build
-.idea
-.vscode
-gurobi.log
-*.log
-*/__pycache__
-
-## Core latex/pdflatex auxiliary files:
-*.aux
-*.lof
-*.log
-*.lot
-*.fls
-*.out
-*.toc
-*.fmt
-*.fot
-*.cb
-*.cb2
-.*.lb
+
+# Jupyter Notebook
+.ipynb_checkpoints/
+
+# mypy
+.mypy_cache/
+.dmypy.json
+
+# folder for misc developments
+dev/