fix: replace UTF-8 em dashes with ASCII dashes in partition CSV files#133
Merged
Keralots merged 1 commit intoJun 29, 2026
Merged
Conversation
PlatformIO's espressif32 platform reads partition CSV files using Python's default encoding, which is GBK on Chinese Windows systems (and CP932/CP949 on Japanese/Korean Windows). The UTF-8 em dash character (U+2014, encoded as 0xE2 0x80 0x94) in comment lines cannot be decoded by these codecs, causing a UnicodeDecodeError during the build process: UnicodeDecodeError: 'gbk' codec can't decode byte 0x94 This error occurs at the checkprogsize step after successful linking, making it particularly frustrating as the build appears to succeed until the final verification stage. Fix: Replace all UTF-8 em dashes (—) with ASCII double dashes (--) in partition CSV comments, making these files pure ASCII. This ensures they can be read under any locale encoding without errors, and the comment meaning remains intact. Files changed: - partitions_4mb.csv: 1 occurrence - partitions_16mb.csv: 2 occurrences - partitions_8mb.csv: no change needed (already pure ASCII)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PlatformIO's espressif32 platform reads partition CSV files using Python's default encoding, which is GBK on Chinese Windows systems (and CP932/CP949 on Japanese/Korean Windows). The UTF-8 em dash character (U+2014, encoded as 0xE2 0x80 0x94) in comment lines cannot be decoded by these codecs, causing a UnicodeDecodeError during the build process:
UnicodeDecodeError: 'gbk' codec can't decode byte 0x94
This error occurs at the checkprogsize step after successful linking, making it particularly frustrating as the build appears to succeed until the final verification stage.
Fix: Replace all UTF-8 em dashes (—) with ASCII double dashes (--) in partition CSV comments, making these files pure ASCII. This ensures they can be read under any locale encoding without errors, and the comment meaning remains intact.
Files changed: