fix(dotenv): save values as UTF-8 and escape the key before matching

[LHMQ878]

Summary

helpers/dotenv.py::save_dotenv_value persists usr/.env. Every credential the UI writes goes through it — AUTH_PASSWORD, ROOT_PASSWORD, RFC_PASSWORD, every API_KEY_*, and the CSRF ALLOWED_ORIGINS list. It had three defects; two lose user data.

1. Encoding — a non-ASCII password cannot be saved, and one non-ASCII byte in the file breaks every later save

Neither open() passed encoding=, so the file was read and written with the platform's preferred encoding — a legacy codepage on a stock Windows install (cp936 on a Chinese system, cp1252 on a Western one), not UTF-8.

Write side. Settings accepts any string as auth_password / root_password, so an accented or CJK password raised instead of being stored:

>>> save_dotenv_value("AUTH_PASSWORD", "pässwörd-密码")
UnicodeEncodeError: 'gbk' codec can't encode character '\xe4' in position 17

Read side, and this is the worse one. A non-ASCII byte already in the file made readlines() raise, so every later save failed too — including saves of purely ASCII values:

# usr/.env contains: "# café ☕ setup\nAUTH_LOGIN=admin\n"  (UTF-8)
>>> save_dotenv_value("AUTH_PASSWORD", "plain-ascii")   # value is pure ASCII
UnicodeDecodeError: 'gbk' codec can't decode byte 0x95 in position 10

The user never has to type a non-ASCII character for this to happen: helpers/secrets.py reads and writes this same file through files.read_file / files.write_file, which default to encoding="utf-8" (helpers/files.py:212, :465). So one file had two writers that disagreed about its encoding — content written by the secrets path could make the dotenv path fail.

2. The key is interpolated into a regex unescaped — saving one key can discard another's value

if re.match(rf"^\s*{key}\s*=", line):     # key is not escaped
    lines[i] = f"{key}={value}\n"          # ...and the matched line's NAME is rewritten

A key holding a regex metacharacter matches a different line, and because the replacement rewrites the whole line including its name, the entry that was there is gone:

# usr/.env: API_KEY_OPENAI=sk-real-openai
#           API_KEY_OPENROUTER=sk-real-openrouter
>>> save_dotenv_value("API_KEY_OPEN.I", "injected")
# usr/.env: API_KEY_OPEN.I=injected              ← the real OpenAI key is gone
#           API_KEY_OPENROUTER=sk-real-openrouter

This is caller-controlled, not hypothetical. Keys are built as API_KEY_{provider.upper()} from request payloads with no whitelist — plugins/_model_config/api/api_keys.py::_set_keys iterates the submitted keys dict directly, and helpers/settings.py::_write_sensitive_settings does the same over settings["api_keys"].

re.escape is already the established pattern for this in the repo: helpers/files.py:439, helpers/tailscale_tunnel.py:43, plugins/_browser/helpers/extension_manager.py:568, plugins/_office/hooks.py:380.

3. CRLF endings

Text mode translated the \n this function writes into \r\n on Windows. .env is also sourced by shells and read by Docker, both of which keep the CR, so a shell doing . .env bound AUTH_PASSWORD to "secret\r" and an auth comparison against it fails. (python-dotenv strips the CR, which is why this only bites the non-Python consumers.) newline="" writes the endings as-is.

Changes

helpers/dotenv.py — encoding="utf-8" on all three open() calls, newline="" on the write, and re.escape(key) in the pattern. The read and write are now separate calls instead of one r+ with seek(0)/truncate(); the in-place truncation window is unchanged (the original also truncated in place, and neither version takes a lock).

Formatting of the untouched functions in this file is left alone — ruff format already wants to reformat them on main, and that is out of scope here.

Validation

New tests/test_dotenv_save_value.py, 6 tests, isolated via the existing pattern (monkeypatch.setattr(dotenv, "get_dotenv_file_path", lambda: str(env_file)), as in tests/test_model_config_api_keys.py:58) and with load_dotenv stubbed so the real process env is untouched — per tests/AGENTS.md, no usr/ runtime state.

Test	Guards
test_non_ascii_value_is_saved_under_a_legacy_locale	write side
test_existing_non_ascii_content_does_not_break_an_ascii_save	read side; asserts the existing comment survives byte-identical
test_ascii_only_save_still_works_under_a_legacy_locale	control — the case the old code got right
test_key_with_regex_metacharacters_does_not_rewrite_another_entry	asserts API_KEY_OPENAI=sk-real-openai is still present
test_existing_key_is_updated_in_place	control for the escaping change — a normal key must still be replaced, not appended
test_saved_file_uses_lf_endings	CRLF

Reproducing a locale bug on a UTF-8 CI runner needs care. Monkeypatching locale.getpreferredencoding does not work — CPython reads the locale encoding at the C level, so open() ignores the patched function. A legacy_default_encoding context manager instead replaces open with a shim that supplies a codec in exactly the position CPython would supply the locale's — only when the caller passed no encoding:

def shim(file, mode="r", *args, **kwargs):
    if "b" not in mode and kwargs.get("encoding") is None and len(args) < 2:
        kwargs["encoding"] = codec
    return real_open(file, mode, *args, **kwargs)

Code that names its encoding is untouched by the shim, so these tests fail on a machine of any locale when the encoding= is missing and pass on a machine of any locale when it is present.

Control experiment — helpers/dotenv.py reverted, tests kept:

4 failed, 2 passed

FAILED test_non_ascii_value_is_saved_under_a_legacy_locale
       UnicodeEncodeError: 'charmap' codec can't encode characters in position 25-26
FAILED test_existing_non_ascii_content_does_not_break_an_ascii_save
FAILED test_key_with_regex_metacharacters_does_not_rewrite_another_entry
       assert 'API_KEY_OPENAI=sk-real-openai' in 'API_KEY_OPEN.I=injected\nAPI_KEY_OPENROUTER=...'
FAILED test_saved_file_uses_lf_endings
       assert b'\r\n' not in b'AUTH_LOGIN=admin\r\n\r\nAUTH_PASSWORD=secret\r\n'

The two that stay green are the controls. With the fix: 6 passed.

No regressions.

tree	result
this branch	933 passed, 102 failed, 2 skipped
unmodified main	933 passed, 102 failed, 2 skipped

Identical — the failing-test sets are the same, with an empty symmetric difference in both directions. I ran this on Windows, where a portion of the suite cannot run at all: 12 modules fail to collect on main (fcntl, resource, fastmcp.server.providers — POSIX-only or unavailable) and were excluded from both runs, and the 102 shared failures are environment-dependent (POSIX paths, symlinks, tunnel binaries). Since both runs are on the same machine with the same exclusions, the comparison is still valid for "this change adds no failures", but a maintainer run on Linux would be a better check of the absolute numbers.

Worth flagging: tests/test_timezone_regressions.py is the one existing module that exercises save_dotenv_value, and it cannot be collected on Windows (fcntl). Reading it, it monkeypatches save_dotenv_value wholesale and asserts only on the arguments it receives, so this change is outside what it covers — but I could not execute it, and say so rather than imply I did.

ruff check on both changed files → All checks passed.

I searched open and closed upstream PRs and issues before opening this (save_dotenv_value, dotenv in title, dotenv, UnicodeDecodeError, re.escape) — no existing or overlapping work. Targeting main, which is what 23 of the 25 currently-open PRs use.