fix(monitor_web): use full message content, not truncated session summary (#42)#106
Closed
Belugary wants to merge 1 commit into
Closed
fix(monitor_web): use full message content, not truncated session summary (#42)#106Belugary wants to merge 1 commit into
Belugary wants to merge 1 commit into
Conversation
…mary (ylytdeng#42) ## Problem In `--web` mode, messages longer than ~90 chars get truncated in the SSE feed. `SessionMonitor.check_updates()` pushes `SessionTable.summary` to clients, but `summary` is WeChat's own ~80-char preview kept for the client-side chat list — not the full message body. ## Fix `_lookup_latest_local_id` already hits `Msg_<md5(username)>` for the row at `(username, create_time)` to obtain `local_id` for ylytdeng#79's dedup. Extend the same query to also return `message_content` and `WCDB_CT_message_content`, and use it to replace `summary` when the DB body is longer: SELECT local_id, message_content, WCDB_CT_message_content FROM [Msg_<md5>] WHERE create_time = ? ORDER BY local_id DESC LIMIT 1 Same row, same query — zero additional IO vs. the prior `MAX(local_id)`. Renamed to `_lookup_latest_message` to reflect the new `(local_id, content)` return shape. zstd handling and `wxid_xxx:\n` group-prefix stripping mirror the existing `SessionTable.summary` logic in `check_updates`, so the SSE `content` field stays in the same format clients already render. Replacement is conservative — only swaps in `full_content` when strictly longer than `summary`. This never shortens existing behavior and degrades cleanly if the message-DB write hasn't landed yet (the SessionTable-vs-message_N.db timing race that ylytdeng#79 already documented). ## Scope - `monitor_web.py`: one helper extended + one call site adjusted. No schema change, no new dependency, no client/UI change. - `_check_hidden_messages` cold path is untouched — its same-second multi-message coverage still runs as before.
Owner
|
本地 rebase ff push 入 main,commit 保留 @Belugary 作者。 LGTM 点:
测试 185/185 通过 (monitor_web 主路径无单测覆盖,但 import + syntax 验证)。 Closes #42 |
Closed
ylytdeng
added a commit
that referenced
this pull request
May 17, 2026
## 根因 用户反馈实时消息延迟从亚秒级 → 8-125 秒。 后端 log: [perf] decrypt=576页/47.8ms, query=46.6ms ... 总耗时=10381.0ms / 44964.4ms / 125398.8ms ... 解密+查询只 95ms, 但总耗时 8-125 秒。源头: PR #106 (commit 1aa12c8, issue #42) 引入了 _lookup_latest_message, 在 check_updates 主循环里每个新消息都调: dec_path = self.db_cache.get(db_key) ^^^^^^^^^^^^^^^^^^^^^^^^ mtime 变化时同步 full_decrypt 整个 message_N.db (~10s) 微信写消息时 message_N.db mtime 跟着变 → get() 触发全量解密 → 主循环阻塞 10 秒。多个 session 同时更新就叠加成几十秒。 之前 db_cache.get 主要在 _check_hidden_messages (走 _hidden_executor 后台线程) 调用, 不阻塞主线程。PR #106 把它带到了主线程 hot path。 证据 (log 里清晰可见): [cache] message\message_0.db 全量解密 10551ms [18:23:31 延迟=14.0s] [...] 莫名感触: ... ← 实测消息延迟 14 秒 ## 修复 MonitorDBCache 加 peek(rel_key) 方法, **不触发**重新解密, 只读 当前已解密文件路径 (可能 stale 1 个 mtime 周期): def peek(self, rel_key): out_path = os.path.join(self.tmp_dir, out_name) return out_path if os.path.exists(out_path) else None _lookup_latest_message 把 self.db_cache.get(db_key) 改成 peek(db_key)。 ## Trade-off stale cache 可能让 _lookup_latest_message 查不到刚写入的 local_id, 返回 (None, None)。check_updates 会: - 跳过加 _shown_keys (issue #79 的去重保险) - 跳过用 full_content 替换 summary (issue #42 的扩展正文) 但**两个 fallback 路径都正常**: - 1 秒后 _check_hidden_messages (异步线程) 会用 db_cache.get 等待 解密完, 拿到 local_id 并 emit hidden 消息 (issue #79 不破坏) - 第一次推送仍用 SessionTable.summary 的 80 字短截断, 后续如果 user 开了详情自然加载完整 (issue #42 退化为"原始行为", 不影响主流程) 权衡: 用"偶发 80 字摘要" 换 "无 8-125 秒延迟"。 ## 实测预期 修复后主循环总耗时应回到 < 200ms (跟解密+查询 95ms 同量级)。 SessionTable 跑 hidden_executor 仍然能补抓密集消息 (issue #79 保留)。 ## 副作用 (无) - get() 行为不变, hidden 路径仍同步等解密 (它在后台线程, 不影响主线程) - peek() 是新加方法, 不影响现有调用方 - 测试 185/185 通过
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--web模式下消息超过 ~90 字符被截断_lookup_latest_local_id的查询(同行同表同时间戳),让它额外返回message_content,零额外 IOSessionTable.summary时才替换,保守降级,不阻塞 SSE 主路径Problem
SessionMonitor.check_updates()把SessionTable.summary推到 SSE,但summary是 WeChat 客户端聊天列表用的 ~80 字符短摘要,不是完整消息正文。Fix
_lookup_latest_local_id已经在Msg_<md5(username)>表里按(username, create_time)查一行拿local_id(issue #79 的去重)。把 SELECT 字段从MAX(local_id)改成同时拿local_id, message_content, WCDB_CT_message_content:```sql
SELECT local_id, message_content, WCDB_CT_message_content
FROM [Msg_]
WHERE create_time = ?
ORDER BY local_id DESC LIMIT 1
```
同行同 query — 相比原
MAX(local_id)零额外 IO。方法改名为
_lookup_latest_message,反映新的(local_id, content)返回。zstd 解压和wxid_xxx:\n群前缀剥离都沿用check_updates处理SessionTable.summary的现有 pattern,保证 SSEcontent字段格式跟前端已渲染的一致。替换条件保守 —— 只在
full_content严格长于summary时替换。永远不会变短;如果 message DB 写入滞后于 SessionTable(#79 已记录的时序竞争),自动回退到 summary。Scope
monitor_web.py:一个 helper 扩展 + 一处调用点调整_check_hidden_messages同秒多消息补抓路径未触碰Test plan
--web模式下用 ≥200 字文本消息验证 SSEcontent字段是完整正文content不带wxid_xxx:\n前缀(保持与现有 summary 渲染一致)_check_hidden_messages仍然补抓被覆盖的早期消息Closes #42