grpc: add configurable maxInboundMessageSize by bryce-anderson · Pull Request #3564 · apple/servicetalk

bryce-anderson · 2026-07-09T22:39:38Z

Motivation

The gRPC deframer reads a message length prefix from the peer (and inflates
compressed frames) with no upper bound, so a large or highly compressible message
can cause excessive memory allocation and risk heap exhaustion before it is
deserialized. grpc-java caps inbound messages at 4 MiB by default; ServiceTalk had
no equivalent for gRPC. The HTTP maxAggregatedPayloadSize does not cover this: it
bounds only aggregated messages, not the streaming deframing path, and is not
coordinated with a gRPC-level limit.

Modifications

Add maxInboundMessageSize(int) to GrpcClientBuilder / GrpcServerBuilder,
defaulting to 4 MiB (matching grpc-java): 0 disables the limit, > 0 enforces it,
and -1 enables warn-only mode (a rate-limited log instead of rejecting, as a
rollout aid). The default can be overridden globally via the temporary
io.servicetalk.grpc.netty.temporaryDefaultMaxInboundMessageSize system property.
Enforce the limit at the gRPC deframer against the declared length prefix before
any bytes are buffered toward it, so oversized frames are rejected without
accumulating memory (covers both single-buffer and split delivery). Applied on
the receiving side of every paradigm: a server bounds requests, a client bounds
responses. Oversized messages are rejected with RESOURCE_EXHAUSTED (matching
grpc-java) via MaxMessageSizeExceededException.
For the aggregated (unary) paradigms, coordinate the underlying HTTP
maxAggregatedPayloadSize to the same bound so oversized messages are rejected
before the whole body is buffered, and map PayloadTooLargeException to
RESOURCE_EXHAUSTED.
For compressed messages, also bound the decompressed size. When enforcing, the
decoder is re-capped to abort mid-inflate at the limit for built-in codecs via a
new best-effort BufferDecoder#withMaxDecompressedBytes(int); otherwise the
codec's own decompressed-bytes cap applies (a custom codec that cannot be
re-capped logs a one-time warning). NettyCompressionSerializer throws
MaxMessageSizeExceededException on cap overflow.
Pass the configuration as objects rather than positional parameters:
GrpcServiceFactory#bind takes a GrpcMessageConfig and GrpcClientCallFactory#from
takes a GrpcClientCallConfig (which adds the client-only default timeout); both
use the repo's Config + Builder idiom. The pre-existing no-limit overloads are
deprecated.
Map MaxMessageSizeExceededException (and PayloadTooLargeException) to
RESOURCE_EXHAUSTED in GrpcStatusException, ordered before the
SerializationException branch since the former is a subtype.

Result

An inbound gRPC message larger than the limit -- compressed or not -- is rejected
up front with RESOURCE_EXHAUSTED rather than buffering toward an OOM, matching
grpc-java's default behavior.

Behavior change

This adds a default 4 MiB inbound message-size limit to a previously-unbounded
gRPC path. Services or clients that legitimately exchange larger messages must
raise the limit, disable it (0), or select warn-only mode (-1) via
maxInboundMessageSize(int) or the temporary system property.

idelpivnitskiy · 2026-07-10T01:15:42Z

+*decoder's* own decompressed-bytes cap (`ZipCompressionBuilder#maxDecompressedBytes(long)`, *64 MiB* by default), which
+fails fast mid-inflate via `DecompressedByteLimitHandler`. This cap is independent of `maxInboundMessageSize`.
+
+To bound decompression at (or near) the gRPC message-size limit rather than the 64 MiB default, either lower the global


We should enhance ProtocolCompatibilityTest in grpc-netty to demonstrate that we are compatible in behavior with grpc-java for all cases with/without compression

The gRPC deframer reads a message length prefix from the peer (and inflates compressed frames) with no upper bound, so a large or highly compressible message can cause excessive memory allocation and risk heap exhaustion before it is deserialized. grpc-java caps inbound messages at 4 MiB by default; ServiceTalk had no equivalent for gRPC. The HTTP maxAggregatedPayloadSize does not cover this: it bounds only aggregated messages, not the streaming deframing path, and is not coordinated with a gRPC-level limit. - Add maxInboundMessageSize(int) to GrpcClientBuilder / GrpcServerBuilder, defaulting to 4 MiB (matching grpc-java): 0 disables the limit, > 0 enforces it, and -1 enables warn-only mode (a rate-limited log instead of rejecting, as a rollout aid). The default can be overridden globally via the temporary io.servicetalk.grpc.netty.temporaryDefaultMaxInboundMessageSize system property. - Enforce the limit at the gRPC deframer against the declared length prefix before any bytes are buffered toward it, so oversized frames are rejected without accumulating memory (covers both single-buffer and split delivery). Applied on the receiving side of every paradigm: a server bounds requests, a client bounds responses. Oversized messages are rejected with RESOURCE_EXHAUSTED (matching grpc-java) via MaxMessageSizeExceededException. - For the aggregated (unary) paradigms, coordinate the underlying HTTP maxAggregatedPayloadSize to the same bound so oversized messages are rejected before the whole body is buffered, and map PayloadTooLargeException to RESOURCE_EXHAUSTED. - For compressed messages, also bound the decompressed size. When enforcing, the decoder is re-capped to abort mid-inflate at the limit for built-in codecs via a new best-effort BufferDecoder#withMaxDecompressedBytes(int); otherwise the codec's own decompressed-bytes cap applies (a custom codec that cannot be re-capped logs a one-time warning). NettyCompressionSerializer throws MaxMessageSizeExceededException on cap overflow. - Pass the configuration as objects rather than positional parameters: GrpcServiceFactory#bind takes a GrpcMessageConfig and GrpcClientCallFactory#from takes a GrpcClientCallConfig (which adds the client-only default timeout); both use the repo's Config + Builder idiom. The pre-existing no-limit overloads are deprecated. - Map MaxMessageSizeExceededException (and PayloadTooLargeException) to RESOURCE_EXHAUSTED in GrpcStatusException, ordered before the SerializationException branch since the former is a subtype. An inbound gRPC message larger than the limit -- compressed or not -- is rejected up front with RESOURCE_EXHAUSTED rather than buffering toward an OOM, matching grpc-java's default behavior. This adds a default 4 MiB inbound message-size limit to a previously-unbounded gRPC path. Services or clients that legitimately exchange larger messages must raise the limit, disable it (0), or select warn-only mode (-1) via maxInboundMessageSize(int) or the temporary system property.

daschl · 2026-07-14T13:41:47Z

            @Override
            protected RuntimeException newException(final long maxBytes) {
-                return new BufferEncodingException(
+                return new MaxMessageSizeExceededException(


I think I'm ok with this since it makes sense semantically - I just want to flag it since this is not a subclass, do we need a better callout on the commit message for this breaking change?

daschl · 2026-07-14T13:43:42Z

change lgtm, I think we should merge the mentioned PR and rebase this one on top to enable the tests before merge.

…actory" This reverts commit 0d3a9c4.

…-limits # Conflicts: # servicetalk-grpc-netty/src/test/java/io/servicetalk/grpc/netty/GrpcLargeMessageTest.java

bryce-anderson · 2026-07-14T15:19:18Z

change lgtm, I think we should merge the mentioned PR and rebase this one on top to enable the tests before merge.

Done.

bryce-anderson · 2026-07-15T01:04:39Z

+
+    @ParameterizedTest(name = "{displayName} [{index}]: client={0} server={1}")
+    @MethodSource("stackMatrixParams")
+    void serverEnforcesDefaultMaxInboundMessageSize(final Stack clientStack, final Stack serverStack) throws Exception {


This test is failing because the java-grpc server behaves differently than the servicetalk server. Specifically, the java-grpc server doesn't abort the inbound stream, it just sends a response with stream end. That makes the ST client stall because it's waiting for the write to finish before it cleans up but that never happens. It does pass with the ST server because the ST server drains the payload.

I've reduced the test matrix and added a TODO to fix it since it's not related to this PR.

bryce-anderson force-pushed the bl_anderson/grpc-size-limits branch from ab8dd38 to 6a21859 Compare July 9, 2026 23:23

idelpivnitskiy mentioned this pull request Jul 10, 2026

serializer-utils: bound frame size in length-prefixed streaming serializers #3563

Merged

idelpivnitskiy reviewed Jul 10, 2026

View reviewed changes

bryce-anderson force-pushed the bl_anderson/grpc-size-limits branch 4 times, most recently from a12713b to 93cae35 Compare July 13, 2026 22:32

bryce-anderson force-pushed the bl_anderson/grpc-size-limits branch from 93cae35 to 4b12ec0 Compare July 13, 2026 23:05

Disable tests for now until we can fix DefaultGrpcClientCallFactory

0d3a9c4

bryce-anderson commented Jul 14, 2026

View reviewed changes

Comment thread servicetalk-grpc-netty/src/test/java/io/servicetalk/grpc/netty/GrpcLargeMessageTest.java Outdated

bryce-anderson commented Jul 14, 2026

View reviewed changes

Comment thread servicetalk-grpc-netty/src/test/java/io/servicetalk/grpc/netty/GrpcLargeMessageTest.java Outdated

bryce-anderson requested review from daschl and idelpivnitskiy July 14, 2026 00:32

bryce-anderson marked this pull request as ready for review July 14, 2026 00:32

daschl approved these changes Jul 14, 2026

View reviewed changes

bryce-anderson added 2 commits July 14, 2026 09:05

Revert "Disable tests for now until we can fix DefaultGrpcClientCallF…

56297d7

…actory" This reverts commit 0d3a9c4.

Merge remote-tracking branch 'origin/main' into bl_anderson/grpc-size…

a30e59e

…-limits # Conflicts: # servicetalk-grpc-netty/src/test/java/io/servicetalk/grpc/netty/GrpcLargeMessageTest.java

bryce-anderson added 2 commits July 14, 2026 16:25

Revert coupling framing from compression limits

ad6ef39

Add tests to ProtocolCompatibilityTest

3d57a05

bryce-anderson commented Jul 15, 2026

View reviewed changes

Work around the stalled client

32b15c7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

grpc: add configurable maxInboundMessageSize#3564

grpc: add configurable maxInboundMessageSize#3564
bryce-anderson wants to merge 7 commits into
apple:mainfrom
bryce-anderson:bl_anderson/grpc-size-limits

bryce-anderson commented Jul 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

idelpivnitskiy Jul 10, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

daschl Jul 14, 2026

Uh oh!

daschl commented Jul 14, 2026

Uh oh!

bryce-anderson commented Jul 14, 2026

Uh oh!

bryce-anderson Jul 15, 2026

Uh oh!

bryce-anderson Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

bryce-anderson commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Result

Behavior change

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

idelpivnitskiy Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

daschl Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

daschl commented Jul 14, 2026

Uh oh!

bryce-anderson commented Jul 14, 2026

Uh oh!

bryce-anderson Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

bryce-anderson Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bryce-anderson commented Jul 9, 2026 •

edited

Loading