Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 51 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,42 +302,22 @@ serve = "uvicorn api.main:app --host 0.0.0.0"
migrate = "alembic upgrade head"
```

running `pdm dockerize` from the workspace root installs every workspace member,
### Building a single image for the whole workspace

Running `pdm dockerize` from the workspace root installs every workspace member,
along with their transitive dependencies, into the output directory,
and exposes their console scripts (subject to `include_bins`/`exclude_bins`).

Members are installed as regular (**non-editable**) packages,
so the resulting image is self-contained and never references build-time host paths.

### Running from the workspace root

Like `pdm install`, `pdm lock` and `pdm sync`, the `dockerize` command must be run from the workspace root,
since members share a single lockfile and a single environment:

```console
$ cd packages/api && pdm dockerize
[PdmUsageError]: `pdm dockerize` can only be run from the workspace root: /path/to/my-workspace
```

Building one image per member is not supported: a workspace produces a single image.
Use `include`/`exclude` to select which scripts that image exposes.

### Configuration is read from the workspace root only

**The `tool.pdm.dockerize` and `tool.pdm.scripts` sections are only read from the workspace root `pyproject.toml`.**
Those sections are ignored in members.

### Which members are included

A member is installed when it is matched by `[tool.pdm.workspace] members`,
has a `project.name` and is a distribution
(ie. neither `distribution = false` nor `package-type = "application"`).

Members are treated as implicit `default` group dependencies,
so they are excluded by selections dropping that group, such as `--no-default`.

### Dockerfile

Members must be present in the build stage so their wheels can be built:
make sure your `.dockerignore` does not exclude the members directory.

Expand All @@ -347,6 +327,54 @@ COPY packages/ ./packages/
RUN pdm dockerize
```

### Building one image per member

Running `pdm dockerize` from a member, or targeting it with `--project`,
builds an image for **that member only**:

```console
$ cd packages/api && pdm dockerize
$ pdm --project packages/api dockerize # equivalent
```

- the member dependencies are installed into `packages/api/dist/docker/lib`,
including the **sibling members it depends on**, as non-editable packages;
- the members it does not depend on are **not** installed;
- like any dockerized project, the member itself is **not** installed:
its sources are exposed through `PYTHONPATH`, so they have to be in the image.

```dockerfile
COPY packages/api /app
COPY packages/api/dist/docker /app
WORKDIR /app
ENTRYPOINT ["/app/entrypoint"]
```

Declare a dependency on a sibling member by name:

```toml
[project]
name = "api"
dependencies = ["shared"]
```

### Where the configuration comes from

| From the targeted project | From the workspace root |
|----------------------------------|----------------------------------------|
| `tool.pdm.dockerize` | the `pdm.lock` lock file |
| `tool.pdm.scripts` | the Python interpreter |
| the dependencies and the groups | the `tool.pdm.source` package indexes |
| the output directory | the `use_uv` setting |

**Nothing is inherited from the workspace root**: the `tool.pdm.dockerize` and
`tool.pdm.scripts` sections of the root are ignored when targeting a member,
and vice versa. A member `pdm.toml` is ignored too.

> [!NOTE]
> Selecting a group only defined in a member, using `--group`, is rejected:
> the workspace lock file only records the groups of the workspace root.

## Internals

This plugin works by subclassing some `pdm` classes to reuse the installation process:
Expand Down
83 changes: 58 additions & 25 deletions src/pdm_dockerize/commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@
from pdm.models.repositories.lock import LockedRepository
from pdm.models.requirements import Requirement

from .config import DockerizeSettings


def _split_combined_extras(req: Requirement, package_keys: set[str]) -> list[Requirement] | None:
"""Split a combined-extras requirement into single-extra requirements.
Expand Down Expand Up @@ -154,17 +156,35 @@ def get_locked_repository(*args, **kwargs) -> LockedRepository:


class DockerizeEnvironment(PythonLocalEnvironment):
"""An environment installaing into the dist/docker directory"""
"""An environment installing into the dist/docker directory

In a workspace, ``project`` is always the workspace **root**, so that the
interpreter, the sources, the caches and the ``${PROJECT_ROOT}`` expansion
keep resolving against it, exactly like ``pdm install`` does.
``member`` is the project the image is built for, and is only used for the
output directory and the ``tool.pdm.dockerize`` settings.
Outside a workspace, both are the same project.
"""

def __init__(
self, project: Project, *, target: str | None = None, python: str | None = None
self,
project: Project,
*,
member: Project | None = None,
target: str | None = None,
python: str | None = None,
) -> None:
super().__init__(project, python=python)
self.member = member if member is not None else project
self.target = Path(target) if target else None

@property
def packages_path(self) -> Path:
return self.target or self.project.root / "dist/docker"
return self.target or self.member.root / "dist/docker"

@property
def settings(self) -> DockerizeSettings:
return self.member.pyproject.settings.get("dockerize", {})


class DockerizeCommand(BaseCommand):
Expand All @@ -183,45 +203,55 @@ class DockerizeCommand(BaseCommand):
)

def handle(self, project: Project, options: argparse.Namespace) -> None:
if (workspace := project.workspace_project) is not None:
# Workspace members share the root lockfile and environment, so a
# single image is built from the workspace root, like `pdm install`.
raise PdmUsageError(
f"`pdm dockerize` can only be run from the workspace root: {workspace.root}"
# Workspace members share the root lock file, environment and sources,
# but provide their own dependencies, scripts, settings and output.
# Outside a workspace, both are the very same project.
root = project.workspace_project or project
member = project

check_project_file(member)
actions.check_lockfile(root)
selection = GroupSelection.from_options(member, options)
hooks = HookManager(member)
env = DockerizeEnvironment(root, member=member, target=options.target)

if member is not root and not member.is_distribution:
project.core.ui.warn(
f"Workspace member {member.name} is not a distribution: "
"it is not installed and its sources are not exposed."
)
check_project_file(project)
actions.check_lockfile(project)
selection = GroupSelection.from_options(project, options)
hooks = HookManager(project)
env = DockerizeEnvironment(project, target=options.target)

requirements = []
selection.validate()
for group in selection:
requirements.extend(project.get_dependencies(group))
if "default" in selection:
# Workspace members are implicit `default` dependencies of the
# workspace root: mirror `pdm.cli.actions.do_lock`.
requirements = project.with_workspace_dependencies(requirements)
requirements.extend(member.get_dependencies(group))
if "default" in selection and member.is_workspace_root:
# Only the workspace root implicitly depends on every member:
# mirror `pdm.cli.actions.do_lock`.
# `iter_workspace_dependencies()` called on a member would return
# every member of the workspace, installing unrelated siblings.
requirements = root.with_workspace_dependencies(requirements)
# Always use the resolvelib resolver to read from pdm.lock,
# even when uv is configured as the project resolver.
# UvResolver would run `uv lock` (a full re-resolution) which is
# inappropriate here — we only need to read existing locked candidates.
config = cast(collections.ChainMap, project.config)
# `Project.config` is a per-instance property, so the override has to be
# applied to the very project performing the resolution.
config = cast(collections.ChainMap, root.config)
config.maps.insert(0, {"use_uv": False})
try:
with _adapted_locked_repository(project):
requirements = _adapt_requirements_for_lockfile(requirements, project)
candidates = actions.resolve_candidates_from_lockfile(project, requirements)
with _adapted_locked_repository(root):
requirements = _adapt_requirements_for_lockfile(requirements, root)
candidates = actions.resolve_candidates_from_lockfile(root, requirements)
finally:
config.maps.pop(0)

use_uv = project.config.get("use_uv", False)
use_uv = root.config.get("use_uv", False)
if use_uv:
try:
project.core.uv_cmd # noqa: B018 — verify uv is available
synchronizer = DockerizeUvSynchronizer(
project,
root,
env,
candidates,
dry_run=options.dry_run,
Expand All @@ -246,13 +276,16 @@ def handle(self, project: Project, options: argparse.Namespace) -> None:

synchronizer.synchronize()

script = ProjectEntrypoint(project, hooks).as_script()
script = ProjectEntrypoint(member, hooks).as_script()

if options.dry_run:
project.core.ui.echo("Dry run: would write the following entrypoint script:")
project.core.ui.echo(script)
return

entrypoint = env.packages_path / "entrypoint"
# The output directory is only created by the installers,
# which do not run when there is nothing to install
entrypoint.parent.mkdir(parents=True, exist_ok=True)
entrypoint.write_text(script)
os.chmod(entrypoint, 0o555)
14 changes: 10 additions & 4 deletions src/pdm_dockerize/installer.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,11 @@ def __init__(
rename_pth: bool = False,
) -> None:
super().__init__(environment, use_install_cache=use_install_cache, rename_pth=rename_pth)
settings: DockerizeSettings = self.environment.project.pyproject.settings.get(
"dockerize", {}
)
# A `DockerizeEnvironment` sources the settings from the targeted
# workspace member, which is not always the project owning the lock file.
settings: DockerizeSettings = getattr(
environment, "settings", None
) or self.environment.project.pyproject.settings.get("dockerize", {})
self.include = filters.parse(settings, "include_bins")
self.exclude = filters.parse(settings, "exclude_bins")

Expand Down Expand Up @@ -119,6 +121,10 @@ class DockerizeUvSynchronizer:
This bypasses PDM's Synchronizer/InstallManager pipeline entirely
and invokes ``uv`` as a subprocess. After installation, scripts are
moved from ``lib/bin/`` to ``bin/`` with include/exclude filtering applied.

``project`` is the project owning the lock file: in a workspace, the root.
The settings are sourced from the environment, hence from the targeted
workspace member.
"""

def __init__(
Expand All @@ -133,7 +139,7 @@ def __init__(
self.environment = environment
self.candidates = candidates
self.dry_run = dry_run
settings: DockerizeSettings = project.pyproject.settings.get("dockerize", {})
settings: DockerizeSettings = environment.settings
self.include_bins = filters.parse(settings, "include_bins")
self.exclude_bins = filters.parse(settings, "exclude_bins")

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#!/usr/bin/env sh

set -eu

dirname=$(dirname "$0")
cmd=${1:-""}
[ "$cmd" ] && shift
cd "$dirname" > /dev/null

PYTHONPATH="$(pwd)/src":"$(pwd)/lib"
export PYTHONPATH
PATH="$(pwd)/bin":"$PATH"
export PATH
FROM_MEMBER="1"
export FROM_MEMBER

usage() {
echo "Available commands"
echo "=================="
echo "serve: echo serving"
}

case $cmd in
serve)
echo serving "$@"
;;
*)
usage
;;
esac
42 changes: 40 additions & 2 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

import subprocess
from pathlib import Path
from typing import TYPE_CHECKING, Protocol
from typing import TYPE_CHECKING, Any, Protocol

import pytest
import tomlkit
Expand Down Expand Up @@ -70,6 +70,8 @@ def __call__(
path: str | None = None,
dependencies: list[str] | None = None,
scripts: dict[str, str] | None = None,
pdm_scripts: dict[str, Any] | None = None,
dockerize: dict[str, Any] | None = None,
distribution: bool = True,
) -> Path: ...

Expand All @@ -81,6 +83,10 @@ def add_member(project: Project) -> AddMemberFixture:
Members must exist on disk: `Project.iter_members()` filters on the presence
of a `pyproject.toml` and `iter_workspace_dependencies()` requires the member
to be a named distribution.

`scripts` are `[project.scripts]` console entry points, while `pdm_scripts`
and `dockerize` are the member's `[tool.pdm.scripts]` and
`[tool.pdm.dockerize]` sections.
"""

def factory(
Expand All @@ -89,6 +95,8 @@ def factory(
path: str | None = None,
dependencies: list[str] | None = None,
scripts: dict[str, str] | None = None,
pdm_scripts: dict[str, Any] | None = None,
dockerize: dict[str, Any] | None = None,
distribution: bool = True,
) -> Path:
module = name.replace("-", "_")
Expand All @@ -112,8 +120,15 @@ def factory(
"project": metadata,
"build-system": {"requires": ["pdm-backend"], "build-backend": "pdm.backend"},
}
tool_pdm: dict = {}
if not distribution:
data["tool"] = {"pdm": {"distribution": False}}
tool_pdm["distribution"] = False
if pdm_scripts:
tool_pdm["scripts"] = pdm_scripts
if dockerize is not None:
tool_pdm["dockerize"] = dockerize
if tool_pdm:
data["tool"] = {"pdm": tool_pdm}
(member_dir / "pyproject.toml").write_text(tomlkit.dumps(data))
return member_dir

Expand All @@ -136,3 +151,26 @@ def workspace(project: Project, add_member: AddMemberFixture) -> Project:
project.pyproject.metadata["dependencies"] = []
project.pyproject.write()
return project


class MemberFixture(Protocol):
def __call__(self, name: str = ...) -> Project: ...


@pytest.fixture
def member(project: Project) -> MemberFixture:
"""Build a `Project` for a member of the current workspace.

The factory is called from the test body, once the workspace
`pyproject.toml` has been written: `Project.pyproject` and
`Project.workspace_project` are both cached properties.
"""

def factory(name: str = "member-a") -> Project:
member_project = project.core.create_project(project.root / "packages" / name)
# Guard against a fixture ordering regression silently degrading
# the tests to the plain, non-workspace code path
assert member_project.workspace_project is not None
return member_project

return factory
Loading