ssh: add an ssh-dev feature and keep host keys on /var for wrynose

[jetm]

Problem

Getting a shell on a freshly flashed board took a bespoke sequence per board, and the image as built could not serve SSH at all even once openssh was installed.

Two independent gaps:

The host keys were unreachable. oe-core's sshd_config names HostKey under ${sysconfdir}/ssh, which on an Avocado image is the read-only rootfs. sshd_check_keys cannot create a key there, so the unit fails, sshd starts keyless, and every client is dropped at key exchange with kex_exchange_identification: Connection reset by peer - a symptom that reads like a network fault rather than a missing file.

There was no way to ask for SSH at build time. Enabling the networking feature group is not enough: a feature group populates the feed, so the openssh RPMs land in deploy/rpm and the rootfs still has no /usr/sbin/sshd. That is correct for a product - the base image is minimal by design and gains capability through extensions - and useless for a bench board, where the point is to have a shell before any extension has been installed.

Solution

kas/feature/ssh-dev.yml, composable onto any machine:

bakar build meta-avocado/kas/machine/<board>.yml:meta-avocado/kas/feature/ssh-dev.yml

plus an openssh bbappend that repoints HostKey at /var/lib/ssh, which is writable and persistent.

Key changes

• openssh_%.bbappend rewrites HostKey ${sysconfdir}/ssh/ to /var/lib/ssh/ in the shipped sshd_config
• kas/feature/ssh-dev.yml sets AVOCADO_DEV_ROOT_LOGIN, oe-core's allow-root-login/allow-empty-password/empty-root-password, and adds openssh to ROOTFS_IMAGE_EXTRA_INSTALL

Reviewer notes

A bbappend rather than a drop-in, deliberately. HostKey accumulates rather than replaces, so a drop-in in sshd_config.d would add the reachable path while leaving the unreachable ones in the list, and sshd_check_keys would keep failing on them.

Complementary to the existing SYSCONFDIR append, not a duplicate. avocado-image-rootfs.bb appends SYSCONFDIR=/var/lib/ssh to /etc/default/ssh, but SYSCONFDIR only decides where sshd_check_keys does its mkdir -p - the key paths themselves come from sshd -G reading HostKey. That is why the append alone still wrote keys to the read-only path. The gap has been invisible because the sshd-dev extension ships an entire sshd_config of its own with the right paths, so only an image carrying base-image openssh hits it.

Console and SSH need separate switches. AVOCADO_DEV_ROOT_LOGIN rewrites /etc/shadow's root:*: to root:: at package build; oe-core's allow-empty-password rewrites nullok_secure to nullok across pam.d at rootfs assembly. Without the second, an empty password authenticates on the console and is refused over SSH, because Linux-PAM 1.7.2 dropped nullok_secure while oe-core still ships it.

Dev only. This puts an sshd on the image that accepts root with an empty password. The feature file says so in its header; nothing enables it by default.

Verification

Built and booted on FRDM-IMX93 hardware (avocado-imx93-frdm, wrynose BSP).

Before: sshdgenkeys.service failed with chmod: cannot access '/etc/ssh/ssh_host_ecdsa_key.tmp': Read-only file system, /var/lib/ssh/ empty, SSH refused at kex.

After: sshdgenkeys.service active, HostKey /var/lib/ssh/ssh_host_ecdsa_key in the shipped config, key present, and ssh root@<board> returns a shell.

Reboot survival, which is the point of putting the key on /var:

	before reboot	after reboot
host key sha256	48ee1bf4…b290	48ee1bf4…b290
boot_id	95c47b29-…	9d9a29bb-…

The changed boot_id confirms a real reboot; the identical key confirms it persisted, so a client's known_hosts stays valid.

[jetm]

Same note as on #276, since this branch has the identical gap.

The host-key work here is necessary but not sufficient on its own — sshd still will not start, because there is no sshd privilege-separation account. avocado-users installs a fixed /etc/passwd and /etc/group over whatever the useradd class produces, so openssh's USERADD_PARAM:${PN}-sshd = "... --user-group sshd" is discarded. OpenSSH exits before writing a banner or a log line, so the symptom is indistinguishable from the missing-key one: connections dropped at kex_exchange_identification, and journalctl -u sshd@ completely empty.

Confirmed on an i.MX 93 FRDM running a wrynose build: with the key work in place the console reported Finished OpenSSH Key Generation and the key pair was present under /var/lib/ssh, and every connection was still dropped. Adding the account produced a running server.

#304 adds it for scarthgap; this branch needs the equivalent. The two changes are independent, so merge order does not matter, but sshd will not come up until both have landed — worth knowing when validating this one on its own.

[mobileoverlord]

Heads-up before rework: the host-key half of this is now superseded on wrynose. The secure-boot stack landed 6f321812d5, which deliberately removed the persist_ssh_host_keys image hook this PR re-adds and replaced it with OPENSSH_HOST_KEY_DIR_READONLY_CONFIG = "/var/lib/ssh" in avocado.conf — no image post-processing, no ordering dependence. What still stands here is the ssh-dev kas fragment and (if still needed against the new mechanism) the openssh HostKey rewrite. Worth rebasing onto current wrynose and re-scoping to just those. #276 on scarthgap is unaffected.

[jetm]

Rebased and re-scoped to just the kas fragment. Confirmed the host-key half is superseded: OPENSSH_HOST_KEY_DIR_READONLY_CONFIG = "/var/lib/ssh" is in avocado.conf on wrynose.

On whether the openssh HostKey rewrite is still needed — it isn't, and for a reason worth recording. The bbappend rewrote HostKey lines in sshd_config, but the recipe writes those from OPENSSH_HOST_KEY_DIR (10.3p1 lines 126-132) while the read-only path comes from OPENSSH_HOST_KEY_DIR_READONLY_CONFIG into sshd_config_readonly (137-143). This image sets read-only-rootfs, so sshd reads sshd_config_readonly. The bbappend was rewriting a config nothing here loads, so it was never doing the job it documented. Dropped both commits; what's left is the fragment.