Repository navigation
Conversation
SslSocket passes at most 16384 plaintext bytes to each SSL_write, so Envoy always sends full-size TLS records. Add Network::TransportSocket::setTlsDataChunkSendLimit() and Network::Connection::setTransportSocketDataChunkSendLimit() so the per-write plaintext, and with it the record size, can be lowered. SslSocket uses the limit in place of the hard-coded 16384. StartTLS passes it to both of its sockets until it switches to TLS, passthrough sockets forward it, the happy eyeballs connection replays it on later attempts, and sockets that do not send TLS records ignore it. Expose it to dynamic module network filters as envoy_dynamic_module_callback_network_filter_set_transport_socket_data_chunk_send_limit, which accepts 1 through 16384 and returns false otherwise, and in the Rust SDK as EnvoyNetworkFilter::set_transport_socket_data_chunk_send_limit. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Dev Narula <devnarula2003@gmail.com>
…imit test With TLS 1.3 the client's encrypted Certificate, CertificateVerify and Finished flight is sent as an application data record, and the data chunk send limit does not apply to it. Start recording record lengths once the handshake completes, so the test only measures data writes. Also apply the formatter's line wrapping to two of the new tests. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Dev Narula <devnarula2003@gmail.com>
ggreenway
left a comment
There was a problem hiding this comment.
This adds niche-feature-specific methods to common interfaces, so it's pretty messy.
It also doesn't restrict the length of handshake records, such as if there's a long cert chain.
I think it would be better to just add a config knob on the TLS transport socket config, and not make this configurable from a filter at all. That makes most of the diff go away.
/wait
| bytes_to_retry_ = 0; | ||
| } else { | ||
| bytes_to_write = std::min(write_buffer.length(), static_cast<uint64_t>(16384)); | ||
| bytes_to_write = std::min(write_buffer.length(), data_chunk_send_limit_); |
There was a problem hiding this comment.
Instead of this, you should call SSL_CTX_set_max_send_fragment.
There was a problem hiding this comment.
updated to use SSL_CTX_set_max_send_fragment and added the field to the proto
…parameter Per review, drop the Connection and TransportSocket setters, the dynamic module network filter callback and the Rust SDK method, and add an optional max_send_fragment to TlsParameters instead. When it is set, the TLS context calls SSL_CTX_set_max_send_fragment, so BoringSSL caps the plaintext of every record it sends, handshake messages included. The value must be 512 through 16384, the range BoringSSL accepts. It is rejected in certificate-level tls_params, where it would not be applied, and has no effect on QUIC. The OpenSSL compatibility layer now exposes SSL_CTX_set_max_send_fragment. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Dev Narula <devnarula2003@gmail.com>
|
CC @envoyproxy/api-shepherds: Your approval is needed for changes made to |
Thanks for the feedback, instead of adding the byte limit directly, I have added the parameter max_send_fragment to TlsParameters and the tls record should default to 16kb (as the original functionality) if this field is missing. |
|
/retest |
agrawroh
left a comment
There was a problem hiding this comment.
💚 Approved. It still needs an API shepherd's approval, one from another company and a re-review from the reviewer who asked for this redesign.
- ❓ Should
max_send_fragmentlive onCommonTlsContext, besidealpn_protocolsandkey_log? - 💡 The rendered docs never show the 512 floor, "not supported" reads as ignored while the code rejects it, and the doc does not say what a small value costs.
- 💡 The record test runs only at 512, BoringSSL's floor, so a change that passed 512 for every configured value still passes every new test.
- 💡 The description still describes the removed connection and dynamic module setters, names a release note file the diff does not add and says integration tests were added. Could we rewrite it for the TLS parameter, with the API considerations and AI-use note the pull request and contribution guides ask for, since it can become the squash commit?
|
|
||
| // The maximum plaintext size, in bytes, of each TLS record sent, handshake records included. | ||
| // Defaults to 16384. Not supported in certificate-level ``tls_params`` and ignored by QUIC. | ||
| google.protobuf.UInt32Value max_send_fragment = 7 |
There was a problem hiding this comment.
❓ Should max_send_fragment live on CommonTlsContext, beside alpn_protocols and key_log? It is the first TlsParameters field a server certificate cannot override, so here it needs per-certificate support through SSL_set_max_send_fragment or this reject and a carve-out in the certificate tls_params doc, and once it ships it can only be deprecated, never moved. Either placement works for me.
There was a problem hiding this comment.
+1. Should we put it in the ComonTlsContext directly rather than put it at there and reject certificates level API?
There was a problem hiding this comment.
I see, yeah it makes sense to move the field to CommonTlsContext since it applies to the entire context. I have moved it and updated the pr.
| // The maximum plaintext size, in bytes, of each TLS record sent, handshake records included. | ||
| // Defaults to 16384. Not supported in certificate-level ``tls_params`` and ignored by QUIC. |
There was a problem hiding this comment.
💡 The rendered docs never show the 512 floor, "not supported" reads as ignored while the code rejects it, and the doc does not say what a small value costs. Could we state all three?
| // The maximum plaintext size, in bytes, of each TLS record sent, handshake records included. | |
| // Defaults to 16384. Not supported in certificate-level ``tls_params`` and ignored by QUIC. | |
| // The maximum plaintext size, in bytes, of each TLS record sent, handshake records included. | |
| // Must be 512 to 16384 inclusive, and defaults to 16384. Smaller values send more records, which | |
| // costs CPU, socket writes and bytes on the wire. QUIC ignores it, and setting it in a | |
| // certificate's ``tls_params`` is rejected. |
| // max_send_fragment caps every TLS record both sides send, handshake records included, and the data | ||
| // still arrives intact. | ||
| BORINGSSL_TEST_P(SslReadBufferLimitTest, MaxSendFragment) { | ||
| constexpr uint32_t max_send_fragment = 512; |
There was a problem hiding this comment.
💡 The record test runs only at 512, BoringSSL's floor, so a change that passed 512 for every configured value still passes every new test. Could it also run at 8192 and check that its largest record is exactly 8192 plus the record overhead?
There was a problem hiding this comment.
thanks, i added a test with 8192 byte cap
|
/wait |
max_send_fragment applies to the whole TLS context: connections are created from the first certificate's SSL_CTX and SSL_new snapshots the fragment size, so a certificate cannot override it. Every TlsParameters field can be overridden per certificate, so the field moves to CommonTlsContext beside alpn_protocols and key_log, and the certificate-level rejection goes away. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Dev Narula <devnarula2003@gmail.com>
The record test only ran at 512, BoringSSL's floor, so a change that applied 512 for every configured value would still pass. Run the exchange at 8192 and check that the largest record fills to 8192 bytes of plaintext without exceeding it. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Dev Narula <devnarula2003@gmail.com>
Commit Message: dynamic_modules: let a network filter cap the downstream TLS record size
The goal of this PR is to let a dynamic module network filter choose a smaller downstream TLS record size than the 16384-byte maximum.
Additional Description:
SslSocket::doWritepasses at most 16384 plaintext bytes to eachSSL_write, so Envoy always sends full-size TLS records, and nothing can ask for smaller ones. Some clients do not cope with 16 KiB records. For example, asynchronous libpq clients can hang on them, so a PostgreSQL proxy written as a dynamic module network filter needs to keep downstream records at 8192 bytes.This adds:
Network::TransportSocket::setTlsDataChunkSendLimit()andNetwork::Connection::setTransportSocketDataChunkSendLimit().SslSocketuses the limit in place of the hard-coded 16384, so the limit caps the plaintext of each record. The StartTLS socket passes the limit to both its clear-text and TLS sockets until it switches, so a limit set while the connection is still clear-text is not lost. Passthrough-based sockets forward it, and the happy eyeballs connection replays it on later attempts. The raw buffer, ALTS and dynamic module transport sockets, QUIC, and the API listener's synthetic connection ignore it, since none of them sends TLS records.envoy_dynamic_module_callback_network_filter_set_transport_socket_data_chunk_send_limit, which accepts 1 through 16384 and returns false for anything else. The Rust SDK exposes this asEnvoyNetworkFilter::set_transport_socket_data_chunk_send_limit.Risk Level: Low. The default stays 16384, and nothing changes unless a module calls the new callback. The two interface methods are new pure virtuals, so out-of-tree
ConnectionandTransportSocketimplementations need to add them.Testing: Unit and integration tests.
Docs Changes: N/A (ABI header and Rust SDK docs updated)
Release Notes: Added
changelogs/current/new_features/dynamic_modules__network-filter-tls-data-chunk-send-limit.rst.Platform Specific Features: N/A