ERROR: `snapper modify --default 2 2>&1` returned with error code 1.

[SharkPlush]

Title of the issue comes from the error that appears in the provided transactional.log file.

This is on Aeon.

Sorry if this isn't the correct place to report this I can't tell if this is an sdbootutil issue or a transactional-update issue.

Some systems don't seem to update "sudo snapper list" will show entries but they are always based off the 1st snapshot.

Talking with a user on Reddit this is what their boot entries looked like:

dad@localhost:~> ls /boot/efi/loader/entries
aeon-6.15.7-1-default-1.conf

And this is what their snapper list looked like:

dad@localhost:~> sudo snapper list
[sudo] password for dad: 
 # │ Type   │ Pre # │ Date                            │ User │ Used Space │ Cleanup │ Description           │ Userdata
───┼────────┼───────┼─────────────────────────────────┼──────┼────────────┼─────────┼───────────────────────┼─────────
0  │ single │       │                                 │ root │            │         │ current               │
1- │ single │       │ Fri 25 Jul 2025 02:52:03 PM EDT │ root │ 712.53 MiB │         │ first root filesystem │
2  │ single │       │ Sat 23 Aug 2025 12:14:45 AM EDT │ root │   3.31 MiB │ number  │ Snapshot Update of #1 │
3  │ single │       │ Sat 23 Aug 2025 12:16:11 AM EDT │ root │  27.95 MiB │ number  │ Snapshot Update of #2 │
4  │ single │       │ Sat 23 Aug 2025 01:51:41 PM EDT │ root │ 858.31 MiB │ number  │ Snapshot Update of #1 │
5  │ single │       │ Sun 24 Aug 2025 12:30:49 AM EDT │ root │   3.30 MiB │ number  │ Snapshot Update of #1 │
6  │ single │       │ Sun 24 Aug 2025 12:32:23 AM EDT │ root │  27.94 MiB │ number  │ Snapshot Update of #5 │
7  │ single │       │ Mon 25 Aug 2025 01:18:02 AM EDT │ root │   3.30 MiB │ number  │ Snapshot Update of #1 │
8+ │ single │       │ Mon 25 Aug 2025 01:19:27 AM EDT │ root │  27.93 MiB │ number  │ Snapshot Update of #7 │

Their system from what I gather is always in the first snapshot and never attempted to boot the newer entries which makes sense since their boot entries only show one entry.

I'm providing their transactional.log and their tik.log from Aeon (maybe something went wrong during install?)

transactional-update.log

tik.log

Also there is more information at: https://bugzilla.opensuse.org/show_bug.cgi?id=1248141

This is basically all the info I have and I can't figure out how to reproduce. :/

[SharkPlush]

Got more info I hope!

This is from a different user starting from snapshot 11 the same bug happens.

sudo snapper list
# │ Type   │ Pre # │ Date                         │ User │ Used Space │ Cleanup │ Description            │ Userdata
────┼────────┼───────┼──────────────────────────────┼──────┼────────────┼─────────┼────────────────────────┼──────────────
 0  │ single │       │                              │ root │            │         │ current                │
 1  │ single │       │ Mon 18 Aug 2025 18:36:18 BST │ root │   2.14 MiB │ number  │ first root filesystem  │ important=yes
 3  │ single │       │ Thu 21 Aug 2025 00:20:57 BST │ root │   3.61 MiB │ number  │ Snapshot Update of #2  │ important=yes
 5  │ single │       │ Fri 22 Aug 2025 01:44:45 BST │ root │  12.06 MiB │ number  │ Snapshot Update of #4  │ important=yes
 6  │ single │       │ Mon 25 Aug 2025 00:43:30 BST │ root │ 328.00 KiB │ number  │ Snapshot Update of #5  │
 7  │ single │       │ Mon 25 Aug 2025 00:44:21 BST │ root │ 416.00 KiB │ number  │ Snapshot Update of #6  │ important=yes
 8  │ single │       │ Tue 26 Aug 2025 01:36:59 BST │ root │ 472.00 KiB │ number  │ Snapshot Update of #7  │
 9  │ single │       │ Tue 26 Aug 2025 01:38:07 BST │ root │ 480.00 KiB │ number  │ Snapshot Update of #8  │ important=yes
10  │ single │       │ Wed 27 Aug 2025 00:39:38 BST │ root │ 388.00 KiB │ number  │ Snapshot Update of #9  │
11- │ single │       │ Wed 27 Aug 2025 00:39:50 BST │ root │ 444.00 KiB │ number  │ Snapshot Update of #10 │
12  │ single │       │ Thu 28 Aug 2025 10:41:52 BST │ root │ 376.00 KiB │ number  │ Snapshot Update of #11 │
13  │ single │       │ Thu 28 Aug 2025 10:42:41 BST │ root │ 420.00 KiB │ number  │ Snapshot Update of #12 │
14+ │ single │       │ Fri 29 Aug 2025 01:21:02 BST │ root │ 285.15 MiB │ number  │ Snapshot Update of #11 │

cat /etc/os-release 
NAME="Aeon"
# VERSION="20250825"

sudo cat /.snapshots/12/snapshot/etc/os-release
NAME="Aeon"
# VERSION="20250826"

sudo cat /.snapshots/13/snapshot/etc/os-release
NAME="Aeon"
# VERSION="20250826"

sudo cat /.snapshots/14/snapshot/etc/os-release
NAME="Aeon"
# VERSION="20250827"

ls /boot/efi/loader/entries
aeon-6.16.1-1-default-10.conf  aeon-6.16.1-1-default-3.conf  aeon-6.16.1-1-default-7.conf
aeon-6.16.1-1-default-11.conf  aeon-6.16.1-1-default-5.conf  aeon-6.16.1-1-default-8.conf
aeon-6.16.1-1-default-1.conf   aeon-6.16.1-1-default-6.conf  aeon-6.16.1-1-default-9.conf

Adding their trans-up log and sdbootutil log

sdbootutil.log
transactional-update.log

This user is willing to run tests and help out if you need more info.

[aplanas]

I think this is a bug in the chroot mount. From the logs I can see this:

+ 2025-08-29 10:58:38 sdbootutil:754:mount_chroot: read -r line
+ 2025-08-29 10:58:38 sdbootutil:755:mount_chroot: eval 'TARGET="/run/media/mouldychips/TIKINSTALL" FSTYPE="btrfs"'
++ 2025-08-29 10:58:38 sdbootutil:755:mount_chroot: TARGET=/run/media/mouldychips/TIKINSTALL
++ 2025-08-29 10:58:38 sdbootutil:755:mount_chroot: FSTYPE=btrfs
+ 2025-08-29 10:58:38 sdbootutil:757:mount_chroot: '[' btrfs = btrfs ']'
+ 2025-08-29 10:58:38 sdbootutil:758:mount_chroot: '[' /run/media/mouldychips/TIKINSTALL '!=' / ']'
+ 2025-08-29 10:58:38 sdbootutil:759:mount_chroot: '[' /run/media/mouldychips/TIKINSTALL = /etc ']'
+ 2025-08-29 10:58:38 sdbootutil:760:mount_chroot: [[ /run/media/mouldychips/TIKINSTALL != /.snapshots* ]]
+ 2025-08-29 10:58:38 sdbootutil:761:mount_chroot: mountpoint --quiet /.snapshots/15/snapshot/run/media/mouldychips/TIKINSTALL
+ 2025-08-29 10:58:38 sdbootutil:761:mount_chroot: mount --bind /run/media/mouldychips/TIKINSTALL /.snapshots/15/snapshot/run/media/mouldychips/TIKINSTALL
+ 2025-08-29 10:58:38 sdbootutil:1:mount_chroot: cleanup

The last mount seems to be failing, and I think this will be fixed by 4cd2eed

[aplanas]

So this should be the same issue from #265

[SharkPlush]

How would should current systems be fixed if they can't upgrade for when the patches come through?

[aplanas]

That is a good question. I do not know what is this /run/media/mouldychips/TIKINSTALL device. Can this be the USB device?

If this is the case, removing the USB device or unmounting any device form /run/media should allow the update.

[SharkPlush]

unmounting the device has fixed it but rebooting makes it re-appear but I think that's an Aeon issue from that point.

Everything seems to work now so feel free to close this bug if needed thank you for your help! (Though since they claim to never taken out the USB after install I wonder why the system updated fine for about a week then stopped working.)

[aplanas]

Great! This gives a path for update. I can close it when the fix land the main branch.

[PepperKnn]

I'm the original user SharkPlush was assisting.

The issue is once again rearing its head, but without any usb storage devices being attached. Perhaps I'll log a new issue with the later log files...

One of the major issues with troubleshooting this is my not being able to comprehend the sdbootutil.log file. It doesn't seem to record error results? Or am I mistaken?

[SharkPlush]

@PepperKnn

I'm the original user SharkPlush was assisting.

The issue is once again rearing its head, but without any usb storage devices being attached. Perhaps I'll log a new issue with the later log files...

One of the major issues with troubleshooting this is my not being able to comprehend the sdbootutil.log file. It doesn't seem to record error results? Or am I mistaken?

I can send the log files I still have them if you don't?

Sidenote: Sorry I randomly disappeared from DM's Reddit decided to ban me for some reason?

[aplanas]

there is no device or loopback in /run/media? Can you give me the mount output?

[SharkPlush]

sdbootutil.log

transactional-update.log

The issue in the trans log started around the 4th snapshot.

[PepperKnn]

@aplanas

~> sudo mount | grep run/media

~> sudo mount

/dev/mapper/aeon_root on / type btrfs (ro,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=481,subvol=/@/.snapshots/63/snapshot)
/dev/mapper/aeon_root on /etc type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=482,subvol=/@/.snapshots/63/snapshot/etc)
/dev/mapper/aeon_root on /root type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=261,subvol=/@/root,x-initrd.mount)
/dev/mapper/aeon_root on /var type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=263,subvol=/@/var,x-initrd.mount)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=32399408k,nr_inodes=8099852,mode=755,inode64)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate,memory_recursiveprot)
none on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime,seclabel)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,size=12978564k,nr_inodes=819200,mode=755,inode64)
tmpfs on /run/credentials/systemd-cryptsetup@aeon_root.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,nosuid,noexec,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19213)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime,seclabel)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,nosuid,nodev,relatime,seclabel,pagesize=2M)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,seclabel,size=32446408k,nr_inodes=1048576,inode64)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime,seclabel)
tmpfs on /run/credentials/systemd-journald.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
/dev/mapper/aeon_root on /.snapshots type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=257,subvol=/@/.snapshots)
/dev/mapper/aeon_root on /opt type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=260,subvol=/@/opt)
/dev/mapper/aeon_root on /srv type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=262,subvol=/@/srv)
/dev/mapper/aeon_root on /boot/writable type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=446,subvol=/@/boot/writable)
/dev/mapper/aeon_root on /usr/local type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=264,subvol=/@/usr/local)
/dev/mapper/aeon_root on /home type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=259,subvol=/@/home,x-systemd.growfs)
/dev/nvme1n1p1 on /boot/efi type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=6489280k,nr_inodes=1622320,mode=700,uid=1000,gid=1000,inode64)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
portal on /run/user/1000/doc type fuse.portal (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
tracefs on /sys/kernel/debug/tracing type tracefs (rw,nosuid,nodev,noexec,relatime,seclabel)
tmpfs on /run/credentials/serial-getty@ttyS0.service type tmpfs (ro,nosuid,nodev,noexec,relatime,nosymfollow,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap)

[aplanas]

I do not see anything in the sdbootutil logs ... The time that the error happens was (from t-u logs):

Cannot delete snapshot 64 since it is the next to be mounted snapshot.
2025-10-02 15:00:21 Found plugin "/usr/lib/tukit/plugins/10-sdbootutil.tukit"
ERROR: `snapper modify --default 64 2>&1` returned with error code 1.

But the lasts entries from sdbootutil are:

+ 2025-10-02 13:40:19 sdbootutil:86:cleanup: rm -rf /tmp/sdbootutil.IhGFXW
+ 2025-10-02 13:40:19 sdbootutil:88:cleanup: '[' -z '' ']'

Same day but different time. Would be possible to reproduce the error again?

[aplanas]

For completion. I see the same error happening much early (from t-u logs):

Cannot delete snapshot 4 since it is the next to be mounted snapshot.
2025-08-22 01:44:45 Found plugin "/usr/lib/tukit/plugins/10-sdbootutil.tukit"
ERROR: `snapper modify --default 4 2>&1` returned with error code 1.

This is happening only 4 minutes after the last log in sdbootutil, but still not captured by the logs.

[PepperKnn]

Yes, I've been examining the logs esp matching up the timestamps from the snapper delete error in trup with sdbootutil. Not much to see in the latter except the odd 'failed to find variant' sometimes. Otherwise it doesn't show much going wrong to my untrained eye.

There is a difference in behaviour between trup dup and trup.service that I've found. Trup dup always makes one snapshot. From my current 6.16.8-1 snap (#63) if I trup dup then I get one single failed snap.

If instead I let trup.service run, I get two snapshots. #64 will fail. However, #65 which bases itself on #64 will 'succeed'. By that I mean no snapper delete error (or other indication of process abort). The resultant snap #65 will sometimes boot once, but later will start thowing PCR register 15 missing (or corrupt) errors that prevent boot.

For instance, last night's #65 booted fine on a restart immediately after it was made. This morning, however, the system will not boot. It says something is missing. Something about a predictions file being missing I think.

To take this further I believe I need to know what exact sdbootutil commands trup is running, and then follow that rabbit hole as deep as I can.