Podman for development environments: how to make containers secure?

[evertheylen]

I use podman to create (long-lived) containers for each project I do. As opposed to toolbx or distrobox, I want those containers to actually be secure, i.e. no blanket access to all my files or messing with the host. I wrote a tool probox for this, and also wrote an article documenting my approach.

Now I tried to read up on relevant documentation and discussion (in particular this one), but for security-related tools it's easy to miss a small hole somewhere. So my question: do you think this setup is secure? Here are the main podman flags I use:

--userns=keep-id
--security-opt label=disable  # required for mounting the volume
--pids-limit=-1               
--cap-add=NET_RAW             # required for ping
--device=/dev/fuse            # required for PINP
--volume ~/projects/foo:/home/evert/projects/foo
--volume ~/foo-ssh-agent:/home/evert/ssh-agent.socket
                              # I start an ssh-agent per project in the host
--network=pasta:-t,auto,-u,auto,-T,none,-U,none

What kind of protection does this give me? Thanks for any tips you can give me, they are very appreciated!

[baude]

@rhatdan wanna give your opinion ?

[rhatdan]

Not crazy about --security-opt label=disable # required for mounting the volume from a security point of view, we could solve some of this with.
--volume ~/projects/foo:/home/evert/projects/foo:Z
As long as foo directory is only used within one container. :z would be if it used in multiple.

The leaking of the ssh-agent-socket is more difficult, from an SELinux point of view. Although I would like to see the AVC's from this. Looks like it would only allow the access to a unconfined_t socket.

From a security point of view this socket is an issue, since it would allow access to user ssh keys potentially, allowing the containerized processes to attack other sites accessible via ssh.

You should be able to ping without CAP_NET_RAW.

$ podman run alpine ping 4.2.2.2
PING 4.2.2.2 (4.2.2.2): 56 data bytes
64 bytes from 4.2.2.2: seq=0 ttl=42 time=41.798 ms

[evertheylen]

Thanks for your insights @rhatdan !

The :Z trick is great, but without --security-opt label=disable it seems I need apply some extra rules and tags to make the ssh-agent socket work. I gave it a try last night but I will read up on SELinux first.

Regarding the ssh-agent itself, the containers are supposed to be used as development environments so they need some access to SSH keys. My current solution seems like it's a good tradeoff as I expect ssh-agent to be quite robust.

Regarding CAP_NET_RAW, I tried with multiple container distros without that capability I can't make non-root users use ping. In the container I run setcap cap_net_raw+p $(which ping) (but I tried a lot of variations).

[evertheylen]

It seems I jumped the gun on label=disable. While I was able to use :Z for both the project directory and even the ssh-agent socket, it does break podman-in-podman. @rhatdan the article you wrote specifies this command (which works for me):

podman run --security-opt label=disable --user podman --device /dev/fuse quay.io/podman/stable podman run alpine echo hello

But I really want to have "secure" containers in which one can do PINP. I tried to leave --security-opt label=disable out but I get this error (AVC added):

Error: crun: set propagation for `proc`: Permission denied: OCI permission denied
type=AVC msg=audit(1740058435.957:2179): avc:  denied  { mounton } for  pid=241121 comm="3" path="/home/podman/.local/share/containers/storage/overlay/dc8968230e729e98b046c459b0193ef0867cce50c0f8edc3c3c88a7d22c52b25/merged/proc" dev="proc" ino=1 scontext=system_u:system_r:container_t:s0:c167,c552 tcontext=system_u:object_r:proc_t:s0 tclass=dir permissive=0

After a little audit2allow and semodule -i, I get another error:

Error: crun: mount `devpts` to `dev/pts`: Permission denied: OCI permission denied
type=AVC msg=audit(1740058516.012:2282): avc:  denied  { mount } for  pid=241479 comm="3" name="/" dev="devpts" ino=1 scontext=system_u:system_r:container_t:s0:c337,c867 tcontext=system_u:object_r:devpts_t:s0 tclass=filesystem permissive=0

Another audit2allow later, similar but different error:

Error: crun: set propagation for `/dev/null`: Permission denied: OCI permission denied
type=AVC msg=audit(1740058603.380:2386): avc:  denied  { mounton } for  pid=241878 comm="3" path="/home/podman/.local/share/containers/storage/overlay/4eda878dbf67c7f38db71d32eb01bfc5e3125df4549a0fc120664d07ede1d0ac/merged/dev/null" dev="devtmpfs" ino=4 scontext=system_u:system_r:container_t:s0:c427,c680 tcontext=system_u:object_r:null_device_t:s0 tclass=chr_file permissive=0

I can keep going like this (zero, tty, random, urandom, proc/irq, ...). Clearly there's a problem with most/all stuff in /dev. Is PINP possible at all without disabling SELinux? This person seems to suggest it is, but I don't know what I'm doing differently.

podman info

host:
  arch: amd64
  buildahVersion: 1.38.1
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.12-3.fc41.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 98.38
    systemPercent: 0.65
    userPercent: 0.97
  cpus: 24
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: kinoite
    version: "41"
  eventLogger: journald
  freeLocks: 2048
  hostname: Orion
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 99000000
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 99000000
  kernel: 6.12.13-200.fc41.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 4976844800
  memTotal: 33561116672
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.14.0-1.fc41.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.14.0
    package: netavark-1.14.0-1.fc41.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.14.0
  ociRuntime:
    name: crun
    package: crun-1.20-2.fc41.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.20
      commit: 9c9a76ac11994701dd666c4f0b869ceffb599a66
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20250121.g4f2c8e7-2.fc41.x86_64
    version: |
      pasta 0^20250121.g4f2c8e7-2.fc41.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.3.1-1.fc41.x86_64
    version: |-
      slirp4netns version 1.3.1
      commit: e5e368c4f5db6ae75c2fce786e31eef9da6bf236
      libslirp: 4.8.0
      SLIRP_CONFIG_VERSION_MAX: 5
      libseccomp: 2.5.5
  swapFree: 8589144064
  swapTotal: 8589930496
  uptime: 6h 12m 33.00s (Approximately 0.25 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
store:
  configFile: /var/home/evert/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/evert/.local/share/containers/storage
  graphRootAllocated: 998483427328
  graphRootUsed: 114024947712
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 33
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /var/home/evert/.local/share/containers/storage/volumes
version:
  APIVersion: 5.3.2
  Built: 1737504000
  BuiltTime: Wed Jan 22 01:00:00 2025
  GitCommit: ""
  GoVersion: go1.23.4
  Os: linux
  OsArch: linux/amd64
  Version: 5.3.2

[rhatdan]

I was not aware of PINP, in which case the outer container would have to have SELinux disabled and some other security features diminished, but the inner container should be able to run in locked down mode.