Skip to content

add desec#7995

Merged
szaimen merged 21 commits into
mainfrom
copilot/improve-user-accessibility
Jun 23, 2026
Merged

add desec#7995
szaimen merged 21 commits into
mainfrom
copilot/improve-user-accessibility

Conversation

@szaimen

@szaimen szaimen commented Apr 24, 2026

Copy link
Copy Markdown
Collaborator
  • This PR is best reviewable commit by commit
  • The PR was tested and verified that it works locally
  • The PR was completely or partially created with AI

Summary

  • Integrates Desec into the AIO interface which simplifies the domain registration and domain setup.
  • You can either create a new account directly or use an existing account.
  • You have a button that you can open where the process will be done. After the process is done, the whole page reloads and you can continue with the setup.
  • Caddy and dnsmasq community containers will be activated during the process which will take over the cert generation and enable split-brain dns.
  • Community containers will work automatically for example Vaultwarden as the subdomains of the desec domain point automatically via a wildcard dns entry towards the main desec domain.
  • Playwright tests are added to ensure that the system works as expected in the future
  • the desec api is mocked via a small node js based file.
  • the qa and other docs were updated to mention desec

Comment thread php/src/Docker/DockerActionManager.php Outdated
Comment thread php/src/Docker/DockerActionManager.php Outdated
Comment thread php/src/Docker/DockerActionManager.php Outdated
Comment thread php/src/Docker/DockerActionManager.php Outdated
Comment thread php/src/Docker/DockerActionManager.php Outdated
Comment thread php/src/Docker/DockerActionManager.php Outdated
Comment thread php/src/Docker/DockerActionManager.php Outdated
Comment thread php/templates/containers.twig Outdated
Comment thread php/templates/containers.twig Outdated
Comment thread php/templates/containers.twig Outdated
Comment thread Containers/dnsmasq/Dockerfile Outdated
Comment thread community-containers/caddy/caddy.json
Comment thread php/src/Data/ConfigurationManager.php Outdated
szaimen and others added 3 commits June 18, 2026 13:19
- desec-mock.mjs: a dependency-free Node mock of the deSEC API endpoints the code
  uses, with the real status-code semantics (202 on account creation, 403 until
  verified then 200 + token, 201/409 on domain creation) plus /__control hooks to
  verify and reset state.
- desec-register.spec.js / desec-existing.spec.js drive the real AIO UI through
  the register -> verify -> domain flow and the existing-account login flow;
  desec-helpers.js holds the shared login helper. Each scenario ends by setting a
  domain, so they run as separate CI steps with a re-seed in between.
- seed-desec-mock-config.php points desec_api_base / desec_update_url at the mock
  (config key only, no env override, so production is unaffected) and seeds a
  known master password, since seeding configuration.json makes AIO consider
  itself already installed and /setup no longer shows a generated password.
- Both Playwright workflows start the mock, mount ./community-containers so the
  caddy/dnsmasq definitions enabled by the flow are present, seed the config, and
  run the two deSEC specs with a re-seed between them.

Co-Authored-By: szaimen <42591237+szaimen@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
desec-api-version-watch.yml is a scheduled workflow that watches the public
desec-stack "API Versions and Roadmap" table (no calls to the live deSEC API, so
it respects their ToS) and fails if it changes, prompting a review of the mock
and DesecManager against any new API version.

Co-Authored-By: szaimen <42591237+szaimen@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
Mention and recommend the built-in deSEC free domain registration across the
readme, reverse-proxy and local-instance guides, and add a deSEC step to the
new-instance QA checklist.

Co-Authored-By: szaimen <42591237+szaimen@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
@pabzm

pabzm commented Jun 19, 2026

Copy link
Copy Markdown
Member

Step 1

Initially the form looks like this, which needs UX love:

Screenshot 2026-06-19 at 10 03 15
  • the left input is unstyled,
  • the middle input placeholder isn't fully readable, therefore it's purpose is unclear,
  • the purpose of the right input is unclear

Step 2

After submitting the form the page reloads, thus jumps to the top, and only some details in the wall of text changed, which are hard to find. Especially that the desec.io password needs to be saved is easy to miss. Also I find it complicated to figure out what to do next (there's the note in the text that I should "click the button below" after verifying my email address, but actually I first have to copy the password and re-enter my chosen hostname "slug"; also I find it just too much text to be helpful).

Step 3

After submitting I see this error:

Warning: file_get_contents(/var/www/docker-aio/community-containers/dnsmasq/dnsmasq.json): Failed to open stream: No such file or directory in /var/www/docker-aio/php/src/ContainerDefinitionFetcher.php on line 59

Slim Application Error

A website error has occurred. Sorry for the temporary inconvenience.

Backtrace from container log:

NOTICE: PHP message: Slim Application Error
Type: JsonException
Code: 4
Message: Syntax error
File: /var/www/docker-aio/php/src/ContainerDefinitionFetcher.php
Line: 59
Trace: #0 /var/www/docker-aio/php/src/ContainerDefinitionFetcher.php(59): json_decode('', true, 512, 4194304)
#1 /var/www/docker-aio/php/src/ContainerDefinitionFetcher.php(392): AIO\ContainerDefinitionFetcher->GetDefinition()
#2 /var/www/docker-aio/php/public/index.php(142): AIO\ContainerDefinitionFetcher->FetchDefinition()

I'm stopping my test here, please let me know if this is fixed and has been tested.

pabzm
pabzm previously requested changes Jun 19, 2026

@pabzm pabzm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(see previous comment)

szaimen and others added 13 commits June 22, 2026 13:24
Move the deSEC "register a free domain" flow out of the inline containers
page form and into a modal backed by a dedicated /desec view loaded in an
iframe. The multi-step register -> verify -> domain process now re-renders
inside the modal, so the user can adjust the details and complete email
verification without reloading the whole page each step. Only once the
domain is fully registered does the view reload the parent containers page.

- add /desec route + desec.twig standalone view
- add desec-modal.js (open/close, backdrop + Escape) and desec-done.js
  (parent reload on completion)
- redirect register POST to the /desec view so steps stay in the modal
- drop the redundant <details> wrapper in desec-register.twig, add heading
- style the modal and let <button class="button"> pick up button styles
- drive the modal iframe in the Playwright specs and update the QA checklist

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
The email input was unstyled because input[type="email"] was missing from
the text/password input rules, and the deSEC form's inline-block inputs
flowed side by side. Add the email type to the input styling rules and lay
the deSEC modal form out as a vertical stack with full-width inputs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
Hide the "Your deSEC login credentials" block during the in-flight
register/verify steps — the credentials are still shown in the dedicated
"deSEC account credentials" section once the domain is registered.

Extract the deSEC domain info/dnsmasq-caution notice into a reusable include
and also show it at the top of the initial-install flow, directly above the
"download and start containers" explanation, while keeping it in the
Community Containers section where it already appeared.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
Signed-off-by: Simon L. <szaimen@e.mail.de>
Signed-off-by: Simon L. <szaimen@e.mail.de>
The deSEC entry <details> was moved up above the domain submit form, but the
#desec-modal container and desec-modal.js were left behind inside the
skip_domain_validation block. Move them up to sit directly after the entry
block so the open button, modal, and wiring script are always rendered
together (and present even when domain validation is skipped). This restores
the modal flow the Playwright tests drive via #desec-frame.

Signed-off-by: Simon L. <szaimen@e.mail.de>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
The deSEC registration entry point now renders regardless of
SKIP_DOMAIN_VALIDATION (it was moved out of the skip-validation guard), and
the deSEC flow already sets the domain with validation skipped internally.
Remove the now-unnecessary SKIP_DOMAIN_VALIDATION=false override from the
deSEC Playwright steps in both workflows and update the stale spec comment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
dnsmasq runs in host network mode and auto-detects the host's LAN IP. On
Docker Desktop (Windows/macOS) containers run inside a Linux VM isolated
from the real host LAN, so detection yields the wrong IP and local DNS
resolution does not work. Call this out in the dnsmasq readme and the
in-app deSEC domain notice.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
Signed-off-by: Simon L. <szaimen@e.mail.de>
Logging in with an existing deSEC account and entering a slug you already
own failed with "Domain limit exceeded": registerDomain() always tried to
POST /domains/, which deSEC rejects (403 when the account is at its quota,
400/409 when the name is taken) even though the domain belongs to you.

Now, for a user-specified slug, a 400/403/409 triggers an ownership check
(GET /domains/{name}/); if the account already owns the domain it is reused
and registration completes. Otherwise a clear message is shown (over-limit
vs. taken by someone else). Random-slug registration is unchanged.

Update the deSEC mock to mirror the real status codes (per-account ownership,
domain limit -> 403, taken name -> 400, GET /domains/{name}/) and add a spec
that exercises the over-limit ownership-recovery path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
Signed-off-by: Simon L. <szaimen@e.mail.de>
The containers page showed "Reveal deSEC password" for every deSEC domain,
but when the user logs in with their own existing account AIO stores an
empty password, so the reveal exposed nothing useful. Gate the reveal on a
stored password: AIO-created accounts show the generated credentials as
before; existing accounts now just say the domain is managed via that
account and to log in at desec.io with your own password.

Also clear the stored generated password in the awaiting-verification path
when the user logs in with their own password (the email already had an
account), so a non-empty stored password reliably means "AIO generated this
account" — and report the account as not-new so no wildcard CNAME is created
on a domain the user manages themselves.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
A slug typed into the deSEC form was lost when the form re-rendered after a
201 redirect (e.g. on the awaiting-verification step), forcing the user to
re-enter it. Persist the requested slug in the AIO config when register()
runs and pre-fill the slug input from it on every step, then clear it once a
domain is set. Stored in configuration.json rather than a URL parameter so it
survives the multi-step flow without leaking into the iframe address.

Signed-off-by: Simon L. <szaimen@e.mail.de>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@szaimen

szaimen commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator Author

@pabzm this is ready for review now 😊

@szaimen

szaimen commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator Author

@pabzm for testing best to compile the container from this branch locally and then testing it out. See https://github.com/nextcloud/all-in-one/blob/main/develop.md#how-to-locally-build-and-test-changes-to-mastercontainer

@pabzm pabzm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically this mostly works now, great!

There's more problem left, which is that caddy apparently can't get a certificate for the dyn hostname. It responds with status code 502, and writes this to the container log:

{"level":"error","ts":1782206806.7762344,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"my-nextcloud.dedyn.io","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:dns - DNS problem: NXDOMAIN looking up TXT for _acme-challenge.my-nextcloud.dedyn.io - check that a DNS record exists for this domain"}
{"level":"error","ts":1782206806.7764018,"logger":"tls.obtain","msg":"will retry","error":"[my-nextcloud.dedyn.io] Obtain: [my-nextcloud.dedyn.io] solving challenge: my-nextcloud.dedyn.io: [my-nextcloud.dedyn.io] authorization failed: HTTP 400 urn:ietf:params:acme:error:dns - DNS problem: NXDOMAIN looking up TXT for _acme-challenge.my-nextcloud.dedyn.io - check that a DNS record exists for this domain (ca=https://acme-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":12.806447381,"max_duration":2592000}

When checking manually via their web interface, desec.io also does not show this DNS record.

Additionally I'd like to raise two UX issues

  1. This caution text irritates me:
Image

I (as a user) didn't chose to run dnsmasq and now I'm told that it's going to be a problem. Also this sounds like I would need to turn it off. But how? (And isn't dnsmasq actually a requirement for this desec setup?)

  1. I'd suggest to use a different icon for warnings (as the first item is one, as far as I undertand) and notices. Or use the same prefix word, if both are warnings.

  2. Somewhere early before registering we should make it very clear that this only works if the server has a public IP address, or has certain ports forwarded to it.

Comment thread php/templates/includes/desec-domain-notice.twig Outdated

@pabzm pabzm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for debugging the certificate issues!

Here's my suggestion for the changed wording.

Comment thread php/templates/containers.twig Outdated
Comment thread php/templates/includes/desec-domain-notice.twig Outdated
Co-authored-by: Pablo Zmdl <57864086+pabzm@users.noreply.github.com>
Signed-off-by: Simon L. <szaimen@e.mail.de>
@szaimen

szaimen commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for debugging the certificate issues!

Here's my suggestion for the changed wording.

Thanks a lot!

Adjusted :)

Comment thread php/templates/includes/desec-domain-notice.twig
Signed-off-by: Simon L. <szaimen@e.mail.de>

@pabzm pabzm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Very nice feature! 🎉

@pabzm

pabzm commented Jun 23, 2026

Copy link
Copy Markdown
Member

🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3. to review Waiting for reviews enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants