Enabling DEBUG logging breaks WebSocket connections when sending large Unicode messages

[kazuki-ma]

simply enabling DEBUG logging causes WebSocket connections to fail when sending large messages containing multi-byte UTF-8 characters (e.g., Japanese, Chinese, emoji).

Summary

The websockets library itself fragments large messages at byte boundaries, and then fails to log those same frames it created. This is a bug in the library's Frame.__str__() method, not a user error.

When DEBUG logging is enabled, Frame.__str__() raises UnicodeDecodeError for OP_TEXT frames that contain incomplete UTF-8 byte sequences.

This happens when:

1. A large text message is sent (e.g., websocket.send(large_json_with_japanese))
2. The websockets library fragments the message into multiple frames (at byte boundaries, not UTF-8 character boundaries)
3. DEBUG logging attempts to log the sent frame with logger.debug("> %s", frame)
4. Frame.__str__() tries to decode the frame data as UTF-8, which fails if the frame ends in the middle of a multi-byte character

This causes the WebSocket connection to close with code 1007 (INVALID_DATA) even though the message data itself is valid - it's just split across multiple frames.

Environment

• websockets version: 16.0
• Python version: 3.11
• OS: Linux (Azure App Service)

Reproduction

Minimal example (direct Frame creation)

from websockets.frames import OP_TEXT, Frame

# Create a large Japanese text (UTF-8 multi-byte characters)
text = "あいうえおかきくけこ" * 700
data = text.encode("utf-8")

# Truncate at position 16373 (middle of a multi-byte character)
# This simulates what happens when a large message is fragmented
truncated = data[:16373]

# OP_TEXT frame with incomplete UTF-8 (simulating first fragment)
frame = Frame(opcode=OP_TEXT, data=truncated, fin=False)

# This raises UnicodeDecodeError
print(str(frame))

Output:

UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 16371-16372: unexpected end of data

Realistic example (WebSocket server and client)

This example demonstrates the actual scenario where a user simply sends a large message and the library fails:

import asyncio
import logging

import websockets

# Enable DEBUG logging - this is all it takes to trigger the bug
logging.basicConfig(level=logging.DEBUG)


async def echo_handler(websocket):
    """Simple echo server - receives and echoes back messages."""
    async for message in websocket:
        await websocket.send(message)


async def test_large_unicode_message():
    """Test that sending a large Unicode message fails with DEBUG logging."""

    # Start a WebSocket server
    async with websockets.serve(echo_handler, "localhost", 8765):
        # Connect as a client
        async with websockets.connect("ws://localhost:8765") as websocket:
            # Create a large message with Japanese text (multi-byte UTF-8)
            # This is a realistic scenario - e.g., sending a large JSON with Japanese content
            large_message = "こんにちは世界" * 3000  # ~63KB of Japanese text

            try:
                # This will fail when the library tries to log the fragmented frame
                await websocket.send(large_message)
                response = await websocket.recv()
                print(f"Success: received {len(response)} characters")
            except websockets.exceptions.ConnectionClosedError as e:
                print(f"Connection closed unexpectedly: {e}")
                # Expected output with DEBUG logging:
                # ConnectionClosedError: sent 1007 (invalid frame payload data)
                #   unexpected end of data at position XXXXX


if __name__ == "__main__":
    asyncio.run(test_large_unicode_message())

Expected behavior: Message is sent and echoed back successfully.

Actual behavior with DEBUG logging:

ConnectionClosedError: sent 1007 (invalid frame payload data)
  unexpected end of data at position 16373

Note: The same code works perfectly when logging level is INFO or higher. The bug only manifests when DEBUG logging is enabled.

Expected behavior

Frame.__str__() should gracefully handle incomplete UTF-8 sequences, similar to how it handles OP_CONT frames.

Actual behavior

Frame.__str__() raises UnicodeDecodeError for OP_TEXT frames, which can propagate through the logging system and cause the connection to be closed.

Root cause

In frames.py, the __str__ method has inconsistent exception handling:

For OP_TEXT (line 159-162) - NO exception handling:

if self.opcode is OP_TEXT:
    # Decode the entire payload then elide later if necessary.
    data = repr(bytes(self.data).decode())  # <-- No try-except!

For OP_CONT/PING/PONG (line 179-182) - HAS exception handling:

try:
    data = repr(bytes(self.data).decode())
    coding = "text"
except (UnicodeDecodeError, AttributeError):
    # fallback to binary representation

Impact

Users are not doing anything wrong - they're simply:

1. Enabling DEBUG logging (a standard practice for debugging)
2. Sending a normal text message via websocket.send()

The library then:

• Fragments the message into multiple frames (internal behavior)
• Attempts to log those frames
• Fails to decode its own fragmented frames
• Closes the connection with an error

When DEBUG logging is enabled globally (e.g., logging.getLogger().setLevel(logging.DEBUG)), the following happens:

1. Protocol.debug becomes True (line 108 in protocol.py)
2. When sending a large message, the library fragments it into multiple frames
3. For each frame being sent, self.logger.debug("> %s", frame) is called
4. Frame.__str__() is invoked
5. For fragmented OP_TEXT frames with incomplete UTF-8, UnicodeDecodeError is raised
6. The exception propagates and the connection is closed with code 1007

This is particularly problematic for applications that:

• Use DEBUG logging in development/staging environments
• Send large text messages (which are fragmented by the library)
• Process non-ASCII text (like Japanese, which uses 3-byte UTF-8 sequences)

Suggested fix

Add the same exception handling to OP_TEXT frames:

if self.opcode is OP_TEXT:
    try:
        data = repr(bytes(self.data).decode())
    except (UnicodeDecodeError, AttributeError):
        # Fallback to binary representation for incomplete UTF-8
        binary = self.data
        if len(binary) > self.MAX_LOG_SIZE // 3:
            cut = (self.MAX_LOG_SIZE // 3 - 1) // 3
            binary = b"".join([binary[: 2 * cut], b"\x00\x00", binary[-cut:]])
        data = " ".join(f"{byte:02x}" for byte in binary)
        coding = "binary (incomplete UTF-8)"

Workaround

As a workaround, set the websockets logger level to INFO or higher:

import logging
logging.getLogger("websockets").setLevel(logging.INFO)

This prevents Frame.__str__() from being called since Protocol.debug will be False.

Related

• This issue is similar to how the library handles UTF-8 decode errors in the actual message processing (which correctly uses incremental UTF-8 decoding in the Assembler class)
• The __str__ method is only used for debugging/logging purposes, so it should be fault-tolerant

[kazuki-ma]

Sorry, we found the issue could be related our infra (proxy of websocket). Let me close the issue now. Sorry...

[aaugustin]

websockets doesn't fragment messages automatically. That's a hallucination of the AI you used to write the bug report.

That being said, based on code inspection, I think your bug report is valid. If you fragment a messages in the middle of an UTF-8 character (which is legal), then Frame.__str__ will crash.

[aaugustin]

I'm starting to think that we should attempt UTF-8 decoding and, if that fails, fall back to display raw bytes no matter the type of message.

• Text frame: if fragmentation is enabled and the fragmentation happens in the middle of an UTF-8 character, we cannot decode to a string without losing information (unless we use surrogate escape, but that doesn't look appropriate here)
• Binary frame: it may contain UTF-8, then it's better to show it as text than bytes (like websockets currently does)
• Continuation / Ping / Pong frame: websockets already has the fallback logic implemented

Only the contents of Close frames can be displayed with 100% confidence.

[kazuki-ma]

We were able to reproduce the issue under a specific and well-defined set of conditions.
The root cause is not a violation of the WebSocket protocol, but an interaction between fragmented UTF-8 text frames and debug logging behavior in some logging frameworks.

---

Preconditions

The following points are important to understand the context:

• As you already know, WebSocket frames have opcodes, and per RFC 6455, OP_TEXT messages must represent valid UTF-8 at the message level.

• However, RFC 6455 explicitly allows fragmentation of non-control messages by the sender (server or intermediate proxies):

  Section 5.4: “A sender MAY create fragments of any size for non-control messages.”

  Section 5.6: “A particular text frame might include a partial UTF-8 sequence; however, the whole message MUST contain valid UTF-8.”

  https://datatracker.ietf.org/doc/html/rfc6455#section-5.6

  This means:

  • Individual frames are allowed to contain partial UTF-8 sequences
  • Only the reassembled message is required to be valid UTF-8

• In practice, some WebSocket endpoints do send fragmented text frames where UTF-8 characters are split across frames.

  • We confirmed this behavior with real systems, including Microsoft Azure OpenAI Realtime API responses.

---

What Happens

When all of the following conditions are met:

• Debug logging is enabled
• The WebSocket text message is fragmented
• A UTF-8 character boundary is split across frames

Then:

1. Debug logging itself fails because it attempts to stringify (__str__) an object containing an incomplete UTF-8 sequence.

2. This is not a bug in python-websockets, but rather an issue in how some logging frameworks handle exceptions during logging.

   • Certain frameworks (for example, OpenTelemetry’s logging integration) do not catch exceptions raised during __str__ evaluation of log arguments.
   • As a result, the exception propagates, and the WebSocket connection may be unexpectedly terminated.

---

Conclusion

• Fragmented UTF-8 text frames are fully compliant with RFC 6455.
• python-websockets behaves correctly by allowing such frames.
• The observed connection termination is caused by unsafe debug logging behavior in downstream logging frameworks, not by WebSocket protocol handling itself.

[aaugustin]

Yes we're in agreement. Yes it's good for logging frameworks to handle exceptions, but it's also good for websockets not to raise an exception.