Skip to content

Stream Orchid - #773

Merged
ReagentX merged 143 commits into
releasefrom
develop
Jun 18, 2026
Merged

Stream Orchid#773
ReagentX merged 143 commits into
releasefrom
develop

Conversation

@ReagentX

Copy link
Copy Markdown
Owner
  • New Features
    • Inline sticker rendering for Support Jumbomoji #755 and Support inline stickers as emoji images #756
      • Render Memoji, Genmoji, and custom stickers inline with surrounding message text in HTML exports
      • Scale inline-only sticker and emoji messages like jumbomoji
      • Resolve inline sticker attachments by file-transfer GUID instead of attachment row order
    • Digital Touch rendering for Digital Touch Support #344
      • Support taps, sketches, kisses, heartbeats, heartbreaks, fireballs, photos, and videos
      • HTML exports now render static Digital Touch frames as embedded SVG on a black 4:5 canvas
      • Photo Digital Touch messages render the photo backdrop with drawing overlays
      • Video Digital Touch messages render as embedded video players
      • TXT exports describe each effect with useful stroke, point, or beat metadata
    • Business Chat messages for Support Business Form messages #769 and Support Business Quick reply messages #770
      • Render Quick Replies in HTML and TXT exports
      • Render Interactive Forms, including requests and submitted responses
      • Render List Pickers, including prompts and selected replies
    • Expanded Data Detector support
      • Parse structured currency amounts, postal addresses, package tracking numbers, flights, and unit/timezone conversions from DDScannerResult payloads
      • Preserve structured detector metadata for downstream crate consumers
  • Performance (≈+29%)
    • Optimize blob reads and database connection settings
    • Skip immediately discarded queries for tapbacks, poll votes, and poll updates
    • Cache resolved chat filenames
    • Increase base buffered-writer capacity
    • Pre-size date formatting buffers
  • Bug Fixes
    • Fix multi-attachment and inline-sticker pairing by matching body ranges to attachment rows by GUID
    • Preserve translated text, original text, and inline stickers in mixed HTML and TXT runs
    • Render edited Memoji / Genmoji history as inline images
    • Fix malformed empty-stroke handling in handwriting messages
    • Fix handwriting SVG file writes
    • Prevent Check In timestamp overflow
    • Add overflow handling to message parsing paths
    • Add plist UID recursion-limit handling
    • Fix index references in event parsing errors
    • Fix currency detection edge cases
    • Fix Digital Touch alpha conversion for SVG/CSS colors
    • Improve Digital Touch media kind matching in HTML exports
    • Remove CSS :has selector usage for broader HTML viewer compatibility
    • Improve resolved-contact percentage calculation
    • Correct busy-bar gating for attachment work
  • Enhancements
    • Refactor AttachmentManager into focused planning, target-resolution, mode, and execution modules
    • Reuse existing attachment outputs when rerunning exports against the same destination
    • Refactor temporary encrypted-backup database handling so cleanup happens after SQLite connections close
    • Add table and column existence checks for diagnostics on older database schemas
    • Centralize diagnostic count queries
    • Centralize URL/rich-link metadata handling behind shared helper traits
    • Centralize progress tracking logic
    • Migrate typedstream parsing to crabstep's foundation feature
    • Reorganize text-effect parsing into focused animation, style, and detected-data modules
    • Add dedicated Digital Touch models and SVG render pipeline
    • Add Business Chat parsing and rendering models behind the shared balloon-provider flow
  • API Changes
    • Breaking
      • Replace BubbleComponent::Text and BubbleComponent::Attachment with BubbleComponent::Run(Vec<AttributedRange>)
      • Replace TextAttributes with AttributedRange
        • Use AttributedRange::text(...), AttributedRange::attachment(...), or AttributedRange::inline_attachment(...)
        • Attachment ranges now carry attachment: Option<AttachmentMeta> and emoji_image
      • Move text-effect types into submodules:
        • message_types::text_effects::text_effect::TextEffect
        • message_types::text_effects::style::Style
        • message_types::text_effects::animation::Animation
        • message_types::text_effects::detected::unit::Unit
      • Replace digital_touch::from_payload with DigitalTouchMessage::from_payload
      • Replace the exported DigitalTouch alias with typed Digital Touch message models
      • Add guid to Attachment
      • Remove the unused typedstream error module
    • Ergonomic
      • Add BusinessMessage, QuickReply, FormRequest, FormResponse, ListPicker, and related model types
      • Add DigitalTouchMessage::render_svg() and DigitalTouchMessage::render_text()
      • Add DigitalTouchError
      • Add structured detected-data models:
        • DetectedAddress
        • DetectedCurrency
        • Flight
        • ShipmentTracking
        • Expanded Unit
      • Add ScannerResult and FromScannerResult helpers for Data Detector payload parsing
      • Add Attachment::is_animated_sticker()
      • Add HasUrl implementations for message types with URL-like payloads
      • Add translation helper methods used by shared rendering paths
  • Security
    • Add capped buffers for payload parsing
    • Reject oversized Digital Touch stroke counts before allocation
    • Add plist recursion-limit handling
    • Harden URL parsing logic
    • Prevent integer overflow in timestamp conversion paths
  • Miscellaneous
    • Update supported-features documentation
    • Update Digital Touch FAQ
    • Add substantial fixtures for Business Chat, Digital Touch, Data Detector, typedstream, inline sticker, translation, and edited-message cases
    • Significant new test coverage
    • Remove dead code
    • Clippy cleanups
    • Bump dependencies
    • Build with latest rustc

ReagentX and others added 30 commits May 27, 2026 14:56
… to preserve original text and inline stickers
@ReagentX ReagentX self-assigned this Jun 18, 2026
@ReagentX ReagentX added the release A release of `imessage-exporter` label Jun 18, 2026
@ReagentX
ReagentX requested a review from Copilot June 18, 2026 22:30

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR (“Stream Orchid”) significantly expands iMessage feature support and refactors core export/parsing paths to enable inline sticker rendering, full Digital Touch rendering, Business Chat balloons, and richer Data Detectors parsing, alongside performance and robustness improvements across DB querying and attachment handling.

Changes:

  • Add Business Chat parsing + HTML/TXT rendering (quick replies, forms, list pickers).
  • Implement/extend Digital Touch parsing and rendering (SVG/video for HTML; richer one-line text for TXT) and support inline stickers/jumbomoji sizing in HTML.
  • Refactor attachment handling (plan/execute/reuse), improve exporter streaming performance, and harden parsing/diagnostics (recursion limits, overflow handling, schema checks).

Reviewed changes

Copilot reviewed 128 out of 165 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
README.md Update supported versions/features list
imessage-exporter/src/exporters/txt/view_model.rs Add Business Chat TXT view models
imessage-exporter/src/exporters/txt/templates/balloons/digital_touch.txt Remove old Digital Touch TXT template
imessage-exporter/src/exporters/txt/templates/balloons/business_quick_reply.txt New Business quick reply TXT template
imessage-exporter/src/exporters/txt/templates/balloons/business_list_picker.txt New Business list picker TXT template
imessage-exporter/src/exporters/txt/templates/balloons/business_form_response.txt New Business form response TXT template
imessage-exporter/src/exporters/txt/templates/balloons/business_form_request.txt New Business form request TXT template
imessage-exporter/src/exporters/txt/balloons.rs Render Business + Digital Touch in TXT
imessage-exporter/src/exporters/shared/tapback.rs Unify tapback resolution error type
imessage-exporter/src/exporters/shared/message.rs Use RuntimeError in message context
imessage-exporter/src/exporters/shared/edited.rs Documentation tweaks for edit diffing
imessage-exporter/src/exporters/shared/driver.rs Perf: writer buffering, routing cache, skip discarded variants
imessage-exporter/src/exporters/shared/balloon.rs Business balloon dispatch + Digital Touch model switch
imessage-exporter/src/exporters/shared/attachment.rs Attachment manager plan/execute integration
imessage-exporter/src/exporters/html/text_effects.rs New text_effects module import paths + detected types
imessage-exporter/src/exporters/html/templates/message.html Add Digital Touch bubble class hook
imessage-exporter/src/exporters/html/templates/message_part.html Support bubble class + inline bubble segments
imessage-exporter/src/exporters/html/templates/balloons/placemark.html Trim placemark fields rendered in HTML
imessage-exporter/src/exporters/html/templates/balloons/digital_touch.html Remove old Digital Touch debug balloon template
imessage-exporter/src/exporters/html/templates/balloons/business_quick_reply.html New Business quick reply HTML template
imessage-exporter/src/exporters/html/templates/balloons/business_list_picker.html New Business list picker HTML template
imessage-exporter/src/exporters/html/templates/balloons/business_form_response.html New Business form response HTML template
imessage-exporter/src/exporters/html/templates/balloons/business_form_request.html New Business form request HTML template
imessage-exporter/src/exporters/html/templates/attachments/sticker_suffix.html Simplify sticker suffix template API
imessage-exporter/src/exporters/html/templates/attachments/sticker_inline.html New inline-sticker <img> template
imessage-exporter/src/exporters/html/safe.rs Allow extracting trusted HTML inner value
imessage-exporter/src/exporters/html/resources/style.css Styling for Digital Touch, inline stickers, jumbomoji, Business balloons
imessage-exporter/src/app/test_dir.rs Convert module docs to block form
imessage-exporter/src/app/progress.rs Progress bar rate formatting tweaks
imessage-exporter/src/app/export_type.rs Doc/comment cleanup for export type
imessage-exporter/src/app/error.rs Add PlistParseError → RuntimeError conversion
imessage-exporter/src/app/data_source.rs Temp decrypted DB cleanup via Drop wrapper
imessage-exporter/src/app/compatibility/models.rs Converter docs and naming cleanup
imessage-exporter/src/app/compatibility/error.rs Conversion error docs cleanup
imessage-exporter/src/app/compatibility/converters/video.rs Video converter docs cleanup
imessage-exporter/src/app/compatibility/converters/sticker.rs Sticker converter docs + behavior notes
imessage-exporter/src/app/compatibility/converters/mod.rs Converter module docs cleanup
imessage-exporter/src/app/compatibility/converters/common.rs Converter helper docs cleanup
imessage-exporter/src/app/compatibility/backup.rs Backup decrypt docs + error conversion
imessage-exporter/src/app/compatibility/attachment_manager/plan.rs New attachment plan model (skip/reuse/process)
imessage-exporter/src/app/compatibility/attachment_manager/mode.rs New attachment manager mode enum/module
imessage-exporter/src/app/compatibility/attachment_manager/mod.rs New attachment manager module wiring
imessage-database/src/util/union_find.rs Union-find doc cleanup
imessage-database/src/util/typedstream.rs Simplify typedstream helpers (keep type-length parsing)
imessage-database/src/util/streamtyped.rs Update legacy streamtyped docs/examples
imessage-database/src/util/size.rs Doc cleanup for size formatting
imessage-database/src/util/query_context.rs Doc cleanup + clarify inclusive bounds
imessage-database/src/util/platform.rs Clarify platform detection docs
imessage-database/src/util/mod.rs Add data_detected module export
imessage-database/src/util/dirs.rs Doc cleanup for path helpers
imessage-database/src/util/dates.rs Performance-focused date formatting + extra tests
imessage-database/src/util/data_detected.rs New DDScannerResult traversal helpers + marker filtering
imessage-database/src/util/bundle_id.rs Bundle ID helper docs cleanup
imessage-database/src/tables/messages/tests/variant.rs Add Business balloon variant test
imessage-database/src/tables/messages/tests/date_tests.rs Test comment cleanup
imessage-database/src/tables/messages/query_parts.rs Query template docs cleanup
imessage-database/src/tables/handle.rs Diagnostic hardening for missing schema columns + tests
imessage-database/src/tables/diagnostic.rs Add shared table/column/count helpers + tests
imessage-database/src/tables/chat.rs Chat table docs cleanup
imessage-database/src/tables/chat_handle.rs Chat-handle docs cleanup
imessage-database/src/message_types/url.rs Centralize URL resolution via HasUrl + safer array parsing
imessage-database/src/message_types/translation.rs Strip attachment placeholders from translated text + test
imessage-database/src/message_types/text_effects/text_effect.rs New TextEffect enum incl. detected entities
imessage-database/src/message_types/text_effects/style.rs New Style enum module
imessage-database/src/message_types/text_effects/mod.rs New text_effects module layout
imessage-database/src/message_types/text_effects/detected/unit.rs Parse unit conversions from DDScannerResult
imessage-database/src/message_types/text_effects/detected/shipment_tracking.rs Parse tracking numbers + carrier
imessage-database/src/message_types/text_effects/detected/mod.rs Detected-entity module docs/exports
imessage-database/src/message_types/text_effects/detected/flight.rs Parse flight info from DDScannerResult
imessage-database/src/message_types/text_effects/detected/currency.rs Parse currency symbols/amounts + tests
imessage-database/src/message_types/text_effects/detected/address.rs Parse structured postal addresses + tests
imessage-database/src/message_types/text_effects/animation.rs New Animation enum module
imessage-database/src/message_types/text_effects.rs Remove legacy monolithic text_effects module
imessage-database/src/message_types/sticker.rs Sticker docs cleanup
imessage-database/src/message_types/polls.rs Poll docs cleanup
imessage-database/src/message_types/placemark.rs Use shared rich-link metadata helper + HasUrl
imessage-database/src/message_types/music.rs Use shared rich-link metadata helper
imessage-database/src/message_types/mod.rs Export business_chat module
imessage-database/src/message_types/expressives.rs Clarify expressive_send_style_id semantics
imessage-database/src/message_types/digital_touch/tap.rs New Digital Touch tap effect model + SVG
imessage-database/src/message_types/digital_touch/svg.rs New shared SVG canvas builder
imessage-database/src/message_types/digital_touch/sketch.rs New Digital Touch sketch effect model + SVG
imessage-database/src/message_types/digital_touch/mod.rs Digital Touch module API overhaul + exports
imessage-database/src/message_types/digital_touch/media.rs New Digital Touch media (photo/video) + overlay decoding
imessage-database/src/message_types/digital_touch/kiss.rs New Digital Touch kiss effect model + SVG
imessage-database/src/message_types/digital_touch/heartbeat.rs New Digital Touch heartbeat/heartbreak model + SVG
imessage-database/src/message_types/digital_touch/fireball.rs New Digital Touch fireball model + SVG
imessage-database/src/message_types/digital_touch/digital_touch.proto Extend proto docs + add media kind
imessage-database/src/message_types/collaboration.rs Use shared rich-link metadata helper + HasUrl
imessage-database/src/message_types/business_chat/quick_reply.rs Parse business quick reply JSON + tests
imessage-database/src/message_types/business_chat/mod.rs Business Chat module exports + schema keys
imessage-database/src/message_types/business_chat/list_picker.rs Parse list picker JSON + tests
imessage-database/src/message_types/business_chat/form.rs Parse interactive form request/response JSON + tests
imessage-database/src/message_types/business_chat/business.rs Business payload classification + tests
imessage-database/src/message_types/app.rs Prevent timestamp overflow in check-in parsing + test
imessage-database/src/message_types/app_store.rs Use shared rich-link metadata helper + HasUrl + tests
imessage-database/src/error/typedstream.rs Remove unused legacy typedstream error module
imessage-database/src/error/streamtyped.rs Remove unused StreamTypedError variant
imessage-database/src/error/plist.rs Add DigitalTouchError payload + recursion limit error
imessage-database/src/error/mod.rs Export Digital Touch error; drop typedstream error
imessage-database/src/error/handwriting.rs Add explicit empty-stroke error
imessage-database/src/error/digital_touch.rs New Digital Touch parsing error type
imessage-database/Cargo.toml Bump crabstep + enable foundation feature
docs/features.md Document new supported features (business, detectors, inline stickers, Digital Touch)
docs/faq.md Update performance numbers
Cargo.lock Dependency lock updates

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread imessage-database/src/util/data_detected.rs

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 128 out of 166 changed files in this pull request and generated 1 comment.

Comment thread imessage-exporter/src/exporters/shared/attachment.rs
@ReagentX
ReagentX merged commit f4021a1 into release Jun 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release A release of `imessage-exporter`

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants