add terminal-based dev environment as alternative to devcontainers by CMonnin · Pull Request #361 · nf-neuro/modules

CMonnin · 2026-05-06T15:18:20Z

Describe your improvement

Adds a podman + apptainer dev environment as an editor-agnostic alternative to the existing VSCode devcontainers. This lets contributors who don't use VSCode (e.g. nvim, Emacs, etc.) get the same reproducible tooling without needing Docker or the devcontainer CLI.

Existing .devcontainer/ configs are unchanged.

Two flavors matching the existing devcontainer structure:

prototyping: scilus/scilus:2.0.2 base with nextflow, nf-test, poetry 1.8.5, apptainer, and gh CLI
devops: python:3.13-slim base with nextflow, nf-test, poetry 2.0.1, apptainer, gh CLI, node 20, act, actionlint, shellcheck, prettier, and editorconfig-checker

Launcher script (.dev/dev) builds and runs the selected flavor with podman, bind-mounting the repo at /workspace. Named volumes persist .venv, .cache, and .nextflow across sessions. Supports extra bind mounts via -v flags for working with external dirs.

Usage

.dev/dev                          # prototyping (default)
.dev/dev devops                   # devops flavor
.dev/dev -v /path:/path bash      # mount extra directories

Note: The dev launcher script is podman-specific (uses --userns=keep-id and :U volume flags). The Dockerfiles are standard and can be built and run with any container runtime
(Docker, podman, etc.).

Note: When running nf-test inside the container, use --profile apptainer,devcontainer instead of the default docker profile (e.g. nf-test test --profile
apptainer,devcontainer).

Type of improvement

Development container
Development tools

Describe how to test your improvement

Install podman on the host if not already present
Run .dev/dev from the repo root — it should build the prototyping image and drop into a bash shell
Inside the container, verify tools: nf-test version && nextflow -version && poetry --version && gh --version
Run .dev/dev devops and verify: nf-test version && nextflow -version && poetry --version && act --version && actionlint --version && node --version && prettier --version
Test apptainer execution: nf-test test <module_test_path> --profile apptainer,devcontainer
Test extra volume mounts: .dev/dev -v /tmp:/tmp and verify /tmp is accessible inside the container
Confirm .devcontainer/ is untouched and still works for VSCode users

Checklist before requesting a review

Ensure the syntax is correct (EditorConfig and Prettier must pass) — N/A, only Dockerfiles and shell script
Run the test suites if your changes affect any module — no module changes
Regenerate the Poetry lock file if you have updated the dependencies — no dependency changes
Ensure the documentation is up-to-date — usage documented in PR description

Tested and working on Void Linux (kernel 6.18 and kernel 7.03) with rootless podman. Other distros may need adjustments.

Editor-agnostic setup using rootless podman + apptainer, replacing VSCode devcontainers + Docker-in-Docker. Existing .devcontainer/ configs are unchanged. Two flavors matching the existing devcontainer structure: - prototyping: scilus/scilus:2.0.2 base with nextflow, nf-test, poetry 1.8.5, apptainer, and gh CLI - devops: python:3.13-slim base with nextflow, nf-test, poetry 2.0.1, apptainer, gh CLI, node 20, act, actionlint, shellcheck, prettier, and editorconfig-checker Launcher script (.dev/dev) builds and runs the selected flavor with podman, bind-mounting the repo at /workspace. Named volumes persist .venv, .cache, and .nextflow across sessions. Supports extra bind mounts via -v flags for working with external dirs. Usage: .dev/dev # prototyping (default) .dev/dev devops # devops flavor .dev/dev -v /path:/path bash # mount extra directories Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

CMonnin · 2026-05-06T15:18:57Z

not sure if there's any interest in this but i don't use vscode or devcontainers so this is an alternative

arnaudbore · 2026-05-11T15:00:27Z

@CMonnin

Please advise :)

$ ./.dev/dev
STEP 1/23: FROM scilus/scilus:2.0.2
Error: error creating build container: short-name "scilus/scilus:2.0.2" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"

Is there a way to get as much common building info as possible at the same place instead of duplicating code ?

CMonnin · 2026-05-11T16:13:26Z

@CMonnin

Please advise :)
$ ./.dev/dev
STEP 1/23: FROM scilus/scilus:2.0.2
Error: error creating build container: short-name "scilus/scilus:2.0.2" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"
Is there a way to get as much common building info as possible at the same place instead of duplicating code ?

do you mean between the devcontainers and these separate dockerfiles?

CMonnin · 2026-05-12T03:35:24Z

nf version needs to be moved to 25 to match CI

gdevenyi · 2026-05-14T14:12:12Z

I think this PR should include some documentation updates on how this tool is intended to be used for development.

AlexVCaron · 2026-05-14T14:14:42Z

I think this PR should include some documentation updates on how this tool is intended to be used for development.

Yes, once it works, some upgrades to the installation procedures on nf-neuro-documentation will need to be done.

gdevenyi · 2026-05-14T14:18:48Z

nf-test test fails in the container on every step:

  Command error:
    .command.run: line 298: docker: command not found

gdevenyi · 2026-05-14T14:36:01Z

Tests working with nf-test test --profile apptainer,devcontainer

gdevenyi

Needs docs on how to use.

gdevenyi · 2026-05-14T14:38:54Z

+        gnupg \
+        jq \
+        openjdk-17-jre \
+        pipx \


Swap for uv?

gdevenyi · 2026-05-14T14:39:04Z

+    && mv nf-test /home/${USERNAME}/.local/bin/nf-test \
+    && chmod +x /home/${USERNAME}/.local/bin/nf-test
+
+RUN pipx install poetry==${POETRY_VERSION}


uv tool install?

gdevenyi · 2026-05-14T14:39:19Z

+    && chmod 0755 /usr/local/bin/nextflow
+
+# pipx (system-wide) so the dev user can install poetry into ~/.local
+RUN python3 -m pip install --no-cache-dir pipx


Had to add pipx to apt-get install above in 2.2.2 for it to work, still prefer uv

gdevenyi · 2026-05-14T14:39:46Z

+RUN ARCH="$(dpkg --print-architecture)" \
+    && curl -fsSL -o /tmp/apptainer.deb \
+        "https://github.com/apptainer/apptainer/releases/download/v${APPTAINER_VERSION}/apptainer_${APPTAINER_VERSION}_${ARCH}.deb" \
+    && apt-get update \
+    && apt-get install -y --no-install-recommends /tmp/apptainer.deb \
+    && rm -rf /var/lib/apt/lists/* /tmp/apptainer.deb


apptainer has a PPA https://apptainer.org/docs/admin/latest/installation.html#install-ubuntu-packages

gdevenyi · 2026-05-14T14:40:46Z

+# Apptainer (replaces ghcr.io/devcontainers/features/docker-in-docker)
+RUN ARCH="$(dpkg --print-architecture)" \
+    && curl -fsSL -o /tmp/apptainer.deb \
+        "https://github.com/apptainer/apptainer/releases/download/v${APPTAINER_VERSION}/apptainer_${APPTAINER_VERSION}_${ARCH}.deb" \
+    && apt-get update \
+    && apt-get install -y --no-install-recommends /tmp/apptainer.deb \
+    && rm -rf /var/lib/apt/lists/* /tmp/apptainer.deb


apptainer has a ppa apptainer.org/docs/admin/latest/installation.html#install-ubuntu-packages

gdevenyi · 2026-05-14T14:42:06Z

@@ -0,0 +1,96 @@
+FROM docker.io/python:3.13-slim-bookworm


The most likely user for nf-neuro is ubuntu, it probably makes more sense to base the test environments on it?

gdevenyi · 2026-05-14T14:43:08Z

+    && mv nf-test /home/${USERNAME}/.local/bin/nf-test \
+    && chmod +x /home/${USERNAME}/.local/bin/nf-test
+
+RUN pipx install poetry==${POETRY_VERSION}


gdevenyi · 2026-05-14T15:08:06Z

@@ -0,0 +1,78 @@
+FROM docker.io/scilus/scilus:2.0.2


very out of date, should be 2.2.2

Swapping to 2.2.2 causes this build error, good reason for uv

--> f33d5b5fdabb STEP 14/23: RUN python3 -m pip install --no-cache-dir pipx /opt/venvs/scilpy/bin/python3: No module named pip Error: building at STEP "RUN python3 -m pip install --no-cache-dir pipx": while running runtime: exit status 1

Another bug on bump:

--> 65374466aed4 STEP 14/22: RUN groupadd --gid ${USER_GID} ${USERNAME} && useradd --uid ${USER_UID} --gid ${USER_GID} -m -s /bin/bash ${USERNAME} && echo "${USERNAME} ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/${USERNAME} && chmod 0440 /etc/sudoers.d/${USERNAME} groupadd: GID '1000' already exists Error: building at STEP "RUN groupadd --gid ${USER_GID} ${USERNAME} && useradd --uid ${USER_UID} --gid ${USER_GID} -m -s /bin/bash ${USERNAME} && echo "${USERNAME} ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/${USERNAME} && chmod 0440 /etc/sudoers.d/${USERNAME}": while running runtime: exit status 4

Default user is ubuntu and already present in the container now

gdevenyi · 2026-05-14T15:08:45Z

+        sudo \
+        tmux \
+        vim \
+        wget \


no need for both wget and curl

gdevenyi · 2026-05-14T15:09:05Z

+        shellcheck \
+        sudo \
+        tmux \
+        wget \


no need for both wget and curl

gdevenyi · 2026-05-14T15:16:03Z

+# Nextflow (was ghcr.io/robsyme/features/nextflow)
+RUN curl -fsSL https://get.nextflow.io | NXF_VER=${NEXTFLOW_VERSION} bash \
+    && mv nextflow /usr/local/bin/nextflow \
+    && chmod 0755 /usr/local/bin/nextflow


we are modern people, chmod has symbolic permissions mode

gdevenyi · 2026-05-14T15:27:48Z

+# Non-root user matching host UID (used with --userns=keep-id at runtime)
+RUN groupadd --gid ${USER_GID} ${USERNAME} \
+    && useradd --uid ${USER_UID} --gid ${USER_GID} -m -s /bin/bash ${USERNAME} \
+    && echo "${USERNAME} ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/${USERNAME} \
+    && chmod 0440 /etc/sudoers.d/${USERNAME}


Not needed for 2.2.2, just switch user to ubuntu

gdevenyi · 2026-05-14T15:27:59Z

+ARG NFTEST_VERSION=0.9.3
+ARG APPTAINER_VERSION=1.4.1
+ARG POETRY_VERSION=1.8.5
+ARG USERNAME=dev


switch to ubuntu for 2.2.2

- Base image scilus/scilus:2.0.2 → 2.2.2 - Use built-in ubuntu user (no more user creation) - Replace pipx/poetry with uv (COPY --from ghcr.io/astral-sh/uv) - Install apptainer via PPA instead of .deb download - Remove wget, python3-pip, python3-venv - Bump Nextflow 24.10.5 → 25.04.6 to match CI - Use symbolic chmod Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Base image python:3.13-slim-bookworm → ubuntu:24.04 - Use built-in ubuntu user (usermod for groups only) - Replace pipx/poetry with uv (COPY --from ghcr.io/astral-sh/uv) - Install apptainer via PPA instead of .deb download - Remove wget, use openjdk-17-jre-headless (smaller) - Bump Nextflow 24.10.5 → 25.04.6 to match CI - Use symbolic chmod Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

arnaudbore · 2026-05-15T19:40:06Z

It looks amazing. Great work !

Copilot

Pull request overview

Adds a terminal/editor-agnostic container dev environment under .dev/ as an alternative to the existing VS Code .devcontainer/ setup, aiming to provide reproducible tooling via Podman + Apptainer without requiring the devcontainer CLI.

Changes:

Added a Podman-based launcher script (.dev/dev) to build/run a selected dev “flavor” and persist caches/venvs via named volumes.
Added two new Dockerfile-based environments (.dev/prototyping/ and .dev/devops/) with Nextflow, nf-test, Apptainer, GitHub CLI, and additional CI tooling for devops.
Added .dev/README.md documenting usage and manual build steps.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
.dev/README.md	Documents the new container-based dev environment and how to use it.
.dev/prototyping/Dockerfile	Defines the “prototyping” container image with Nextflow/nf-test/Apptainer tooling.
.dev/devops/Dockerfile	Defines the “devops” container image with additional CI/dev tooling (node/act/actionlint/etc.).
.dev/dev	Podman launcher script to build/run the chosen flavor with persistent volumes and optional extra mounts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
+
+RUN usermod -aG nodeusers ubuntu \
+    && echo "ubuntu ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/ubuntu \
+    && chmod u=r,g=r /etc/sudoers.d/ubuntu \
+    && echo "prefix=/usr/local" >> /home/ubuntu/.npmrc
+
+ENV PATH="/home/ubuntu/.local/bin:${PATH}"
+ENV NXF_APPTAINER_CACHEDIR=/home/ubuntu/.cache/apptainer
+
+USER ubuntu
+WORKDIR /home/ubuntu
+


+volumes=()
+while [[ ${1:-} == -v ]]; do
+    shift
+    volumes+=(-v "$1")
+    shift
+done


+## Installing Dependencies
+
+```bash
+uv sync --no-install-project


+COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
+


+
+COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
+


+FROM docker.io/ubuntu:24.04
+
+ARG NODE_MAJOR=20
+ARG ACTIONLINT_VERSION=1.7.7
+ARG NEXTFLOW_VERSION=25.04.6
+ARG NFTEST_VERSION=0.9.3
+


gdevenyi · 2026-05-17T21:34:37Z

None of these co-pilot reviews are relevant, updated model works for me.

gdevenyi · 2026-05-28T14:31:59Z

Can this be merged? I'd like to use it to advance my registration too integration.

CMonnin marked this pull request as ready for review May 7, 2026 16:50

CMonnin requested a review from a team May 7, 2026 16:50

pull-request-management Bot requested review from AlexVCaron and arnaudbore May 7, 2026 16:50

adding container repo

f7c3b48

arnaudbore requested a review from gdevenyi May 14, 2026 13:09

gdevenyi suggested changes May 14, 2026

View reviewed changes

gdevenyi reviewed May 14, 2026

View reviewed changes

Comment thread .dev/devops/Dockerfile Outdated

gnupg \

jq \

openjdk-17-jre \

pipx \

Copy link
Copy Markdown

Contributor

gdevenyi May 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Swap for uv?

gdevenyi reviewed May 14, 2026

View reviewed changes

Comment thread .dev/prototyping/Dockerfile Outdated

sudo \

tmux \

vim \

wget \

Copy link
Copy Markdown

Contributor

gdevenyi May 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need for both wget and curl

gdevenyi reviewed May 14, 2026

View reviewed changes

Comment thread .dev/devops/Dockerfile Outdated

shellcheck \

sudo \

tmux \

wget \

Copy link
Copy Markdown

Contributor

gdevenyi May 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need for both wget and curl

gdevenyi reviewed May 14, 2026

View reviewed changes

CMonnin and others added 3 commits May 15, 2026 15:35

update dev script home path for ubuntu user

82a759e

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

add usage docs for container dev environment

e6928a4

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

CMonnin changed the title ~~add podman-based dev environment as alternative to devcontainers~~ add terminal-based dev environment as alternative to devcontainers May 15, 2026

Merge branch 'main' into feat/podman-dev-env

bd5d478

gdevenyi requested a review from Copilot May 16, 2026 17:42

Copilot started reviewing on behalf of gdevenyi May 16, 2026 17:42 View session

Copilot AI reviewed May 16, 2026

View reviewed changes

Merge branch 'main' into feat/podman-dev-env

f176291

arnaudbore enabled auto-merge May 28, 2026 14:34

arnaudbore approved these changes May 28, 2026

View reviewed changes

arnaudbore added this pull request to the merge queue May 28, 2026

Merged via the queue into nf-neuro:main with commit cc927b4 May 28, 2026
11 checks passed

		COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv

Uh oh!

Conversation

CMonnin commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Describe your improvement

Usage

Uh oh!

CMonnin commented May 6, 2026

Uh oh!

arnaudbore commented May 11, 2026

Uh oh!

CMonnin commented May 11, 2026

Uh oh!

CMonnin commented May 12, 2026

Uh oh!

gdevenyi commented May 14, 2026

Uh oh!

AlexVCaron commented May 14, 2026

Uh oh!

gdevenyi commented May 14, 2026

Uh oh!

gdevenyi commented May 14, 2026

Uh oh!

gdevenyi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arnaudbore commented May 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

gdevenyi commented May 17, 2026

Uh oh!

gdevenyi commented May 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

CMonnin commented May 6, 2026 •

edited

Loading