load_aware_locality: add out-of-band ORCA reporting

api/envoy/extensions/load_balancing_policies/load_aware_locality/v3/BUILD

@@ -7,6 +7,7 @@ licenses(["notice"])  # Apache 2
 api_proto_package(
     deps = [
         "//envoy/config/cluster/v3:pkg",
+        "//envoy/extensions/load_balancing_policies/common/v3:pkg",
         "@xds//udpa/annotations:pkg",
     ],
 )

api/envoy/extensions/load_balancing_policies/load_aware_locality/v3/load_aware_locality.proto

@@ -3,6 +3,7 @@ syntax = "proto3";
 package envoy.extensions.load_balancing_policies.load_aware_locality.v3;
 
 import "envoy/config/cluster/v3/cluster.proto";
+import "envoy/extensions/load_balancing_policies/common/v3/common.proto";
 
 import "google/protobuf/duration.proto";
 import "google/protobuf/wrappers.proto";
@@ -21,7 +22,7 @@ option (udpa.annotations.file_status).package_version_status = ACTIVE;
 
 // Configuration for the load_aware_locality LB policy which uses ORCA utilization data
 // to route traffic between localities based on available headroom.
-// [#next-free-field: 10]
+// [#next-free-field: 11]
 message LoadAwareLocality {
   // The child LB policy to create for endpoint-picking within each locality.
   config.cluster.v3.LoadBalancingPolicy endpoint_picking_policy = 1
@@ -95,4 +96,9 @@ message LoadAwareLocality {
   // server may not provide reports as frequently as the client requests.
   // Used only when enable_oob_load_report is true. Default is 10 seconds.
   google.protobuf.Duration oob_reporting_period = 9;
+
+  // Optional overrides for the OOB reporting connection (alternative port,
+  // ``:authority``, transport socket selection). Honored only when
+  // ``enable_oob_load_report`` is true.
+  common.v3.OrcaOobReportingConfig oob_reporting_config = 10;
 }

changelogs/current/new_features/load_balancing__load-aware-locality-lb-policy.rst

@@ -1,6 +1,7 @@
 load_balancing: implemented the
 :ref:`envoy.load_balancing_policies.load_aware_locality
 <envoy_v3_api_msg_extensions.load_balancing_policies.load_aware_locality.v3.LoadAwareLocality>`
-locality-picking load balancer. It weights localities by ORCA-derived utilization headroom,
-consumes in-band ORCA reporting, and applies at all priority levels. The extension is
-work-in-progress and not intended for production use.
+locality-picking load balancer. It weights localities by ORCA-derived utilization headroom
+and applies at all priority levels. ORCA data may be consumed in-band, out-of-band (setting
+``enable_oob_load_report`` opens a per-host reporting stream, with optional connection
+overrides via ``oob_reporting_config``), or both.

docs/root/intro/arch_overview/upstream/load_balancing/load_aware_locality.rst

@@ -3,11 +3,6 @@
 Load-aware locality load balancing
 -----------------------------------
 
-.. attention::
-
-  This extension is **work-in-progress**. Functionality is incomplete and it
-  is not intended for production use.
-
 The load-aware locality LB policy
 (:ref:`envoy.load_balancing_policies.load_aware_locality
 <envoy_v3_api_msg_extensions.load_balancing_policies.load_aware_locality.v3.LoadAwareLocality>`)
@@ -115,10 +110,13 @@ The policy is implemented as a ``ThreadAwareLoadBalancer``:
 ORCA data flow
 """"""""""""""
 
-Upstream endpoints must report ORCA utilization in-band: ORCA reports are
-returned on the response headers or trailers of upstream responses. Sample
-rate is tied to the request rate to each host, so probing
-(``remote_probe_fraction``) is required to keep remote-locality data fresh.
+Upstream endpoints report ORCA utilization in-band -- on the response headers
+or trailers of upstream responses -- and/or out-of-band (OOB): with
+``enable_oob_load_report`` the policy opens a gRPC stream per host and
+receives reports every ``oob_reporting_period``, independent of request
+traffic. In-band sample rate is tied to the request rate to each host, so
+without OOB reporting, probing (``remote_probe_fraction``) is required to
+keep remote-locality data fresh.
 
 Reports land in per-host ``HostLbPolicyData`` slots, which feed weight
 computation.
@@ -130,6 +128,26 @@ locality selection by aggregate headroom, endpoint selection by
 per-endpoint capacity. Each consumer attaches independent
 ``HostLbPolicyData`` entries, so the two policies do not interfere.
 
+Report delivery is shared: every ORCA report received for a host -- in-band
+or OOB, whichever policy's stream it arrived on -- is delivered to every
+ORCA consumer attached to that host. Enabling OOB reporting at one level is
+therefore sufficient to feed both policies, and is the recommended
+configuration.
+
+Because delivery is shared, the reports must carry the fields every attached
+consumer needs. This policy only needs a utilization signal; CSWRR
+additionally requires ``rps_fractional`` greater than 0 and rejects any
+report that lacks it. A rejection does not fail the stream --
+``report_errors`` counts per-recipient rejection, not stream failure -- so
+pairing this policy's OOB reporting with a CSWRR child that only receives
+utilization data permanently increments ``report_errors`` and logs a
+periodic error for the host, even though the stream itself stays healthy.
+
+If both levels enable OOB, each opens its own independent stream per host
+honoring its own ``oob_reporting_period`` and ``oob_reporting_config``; this
+is harmless for correctness, but doubles the per-host connection and stream
+count, and the cluster-scoped ``lb_orca_oob.*`` stats sum across the two.
+
 Utilization is derived from each host's ORCA report using the same
 extraction as CSWRR, which takes the first source whose value is greater
 than 0. The runtime flag
@@ -359,8 +377,9 @@ Configuration parameters
      - 0.03
      - Minimum fraction of traffic sent to non-local localities to keep ORCA
        data fresh in all-local mode. The deficit is redistributed
-       proportionally to host count. Set to 0 to disable (safe only when
-       cross-zone traffic must be strictly avoided). Range: [0, 1). See
+       proportionally to host count. Set to 0 to disable (safe when ORCA
+       reports arrive out-of-band, or when cross-zone traffic must be
+       strictly avoided). Range: [0, 1). See
        :ref:`Caveats <load_aware_locality_caveats>` for scaling notes.
    * - ``weight_expiration_period``
      - 3 minutes
@@ -371,6 +390,21 @@ Configuration parameters
        falls back to host-count-proportional weighting. Tune higher to
        tolerate longer reporting gaps; tune lower to prune draining
        backends faster. Set to 0 s to disable expiration.
+   * - ``enable_oob_load_report``
+     - false
+     - Open a per-host out-of-band ORCA reporting stream and consume its
+       reports in addition to any in-band reports. See
+       :ref:`ORCA data flow <load_aware_locality_orca_data_flow>`.
+   * - ``oob_reporting_period``
+     - 10 s
+     - Reporting interval requested from the server on each OOB stream. The
+       server may report less frequently. Only used when
+       ``enable_oob_load_report`` is true.
+   * - ``oob_reporting_config``
+     - (unset)
+     - Optional overrides for the OOB reporting connection (alternative
+       port, ``:authority``, transport socket selection). Only honored when
+       ``enable_oob_load_report`` is true.
 
 Priority support
 ^^^^^^^^^^^^^^^^
@@ -401,13 +435,11 @@ weight, computed from the same per-host ORCA data in a single tick pass.
 Caveats and known limitations
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-- **Out-of-band ORCA reporting is not yet supported.** Only in-band
-  (per-response) ORCA reports are consumed. ``enable_oob_load_report`` and
-  ``oob_reporting_period`` are accepted but currently have no effect.
-- **Probing is required.** A locality only produces fresh ORCA samples
-  when it receives traffic, so ``remote_probe_fraction`` must stay above 0
-  to keep remote localities reporting. Set it to 0 only when cross-zone
-  traffic must be strictly avoided.
+- **Probing is required for in-band reporting.** Without OOB reporting, a
+  locality only produces fresh ORCA samples when it receives traffic, so
+  ``remote_probe_fraction`` must stay above 0 to keep remote localities
+  reporting. With ``enable_oob_load_report``, samples arrive independent of
+  traffic and the probe floor can safely be set to 0.
 - **Cold start snaps local.** Until the first ORCA reports arrive, every
   locality reads as utilization 0, so the stage-4 local-preference check
   routes ~100% of traffic to the local locality (minus
@@ -471,6 +503,12 @@ Caveats and known limitations
   every few ticks. To avoid this, either reduce locality count, raise
   ``remote_probe_fraction``, or raise ``weight_expiration_period`` to
   tolerate longer gaps.
+- **OOB resource cost.** Each policy that sets ``enable_oob_load_report``
+  opens one connection, one HTTP/2 stream, and one watchdog timer per host,
+  all serviced by the main-thread dispatcher. This scales with cluster size:
+  the 100-remote-localities-by-10-hosts scenario in the table above means up
+  to 1000 concurrent OOB streams for a single enabling policy. When both
+  this policy and its CSWRR child enable OOB, that per-host cost doubles.
 - **Variance-threshold oscillation.** Workloads sitting near the
   ``utilization_variance_threshold`` boundary can theoretically oscillate
   between snap-to-local and spillover modes across consecutive ticks.
@@ -527,6 +565,41 @@ every priority and each of the three host subsets (healthy, degraded,
 all-hosts) the policy weighs independently; ``stale_locality_total`` is
 derived from the all-hosts subset.
 
+When ``enable_oob_load_report`` is set -- on this policy, its CSWRR child, or
+both -- the OOB manager additionally emits stats under
+``cluster.<cluster_name>.lb_orca_oob.*``:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 30 70
+
+   * - Stat
+     - Increments when / value
+   * - ``reports_received`` (Counter)
+     - An OOB report is decoded from a host's stream.
+   * - ``report_errors`` (Counter)
+     - A decoded report has no attached ORCA consumer, or any attached
+       consumer rejects it (for example CSWRR when ``rps_fractional`` is not
+       greater than 0). See :ref:`ORCA data flow
+       <load_aware_locality_orca_data_flow>`.
+   * - ``stream_failures`` (Counter)
+     - A per-host OOB connection or stream fails transiently; the session
+       reconnects with backoff.
+   * - ``stream_terminated`` (Counter)
+     - A per-host OOB session hits a terminal condition (for example the
+       server does not implement the ORCA OOB service) and stops retrying
+       that host until cluster membership changes.
+   * - ``active_sessions`` (Gauge)
+     - Number of hosts with an OOB session tracked, including sessions that
+       are disconnected and reconnecting after a transient failure. Use
+       ``stream_failures`` to detect sessions that are not currently
+       delivering reports.
+
+These stats are only emitted when OOB is enabled. When both this policy and
+its CSWRR child enable it, each opens its own OOB manager against the same
+cluster stats scope, so the values compose: the counters sum and the
+delta-updated ``active_sessions`` gauge reflects the combined session count.
+
 Migrating from zone-aware routing? The per-request zone routing counters
 (``lb_zone_routing_all_directly``, ``lb_zone_routing_sampled``,
 ``lb_zone_routing_cross_zone``) are still emitted with equivalent semantics,

source/extensions/extensions_metadata.yaml

@@ -2192,7 +2192,7 @@ envoy.load_balancing_policies.load_aware_locality:
   categories:
   - envoy.load_balancing_policies
   security_posture: unknown
-  status: wip
+  status: alpha
   type_urls:
   - envoy.extensions.load_balancing_policies.load_aware_locality.v3.LoadAwareLocality
 envoy.load_balancing_policies.random:

source/extensions/load_balancing_policies/common/orca_oob_manager.cc

@@ -77,12 +77,13 @@ OrcaOobManager::OrcaOobManager(OrcaOobManagerConfig config,
       priority_set_(priority_set), oob_stats_(generateOrcaOobStats(stats_scope)) {}
 
 OrcaOobManager::~OrcaOobManager() {
+  const size_t session_count = oob_sessions_.size();
   for (auto& [host, session] : oob_sessions_) {
     session->disarm();
     dispatcher_.deferredDelete(std::move(session));
   }
   oob_sessions_.clear();
-  oob_stats_.active_sessions_.set(0);
+  oob_stats_.active_sessions_.sub(session_count);
 }
 
 absl::Status OrcaOobManager::initialize() {
@@ -115,8 +116,8 @@ void OrcaOobManager::onHostsAdded(const Upstream::HostVector& hosts) {
     const std::chrono::milliseconds initial_delay(random_.random() % period_ms);
     it->second = std::make_unique<OobSession>(*this, host, initial_delay);
   }
-  if (oob_sessions_.size() != prior_size) {
-    oob_stats_.active_sessions_.set(oob_sessions_.size());
+  if (oob_sessions_.size() > prior_size) {
+    oob_stats_.active_sessions_.add(oob_sessions_.size() - prior_size);
   }
 }
 
@@ -131,8 +132,8 @@ void OrcaOobManager::onHostsRemoved(const Upstream::HostVector& hosts) {
     dispatcher_.deferredDelete(std::move(it->second));
     oob_sessions_.erase(it);
   }
-  if (oob_sessions_.size() != prior_size) {
-    oob_stats_.active_sessions_.set(oob_sessions_.size());
+  if (oob_sessions_.size() < prior_size) {
+    oob_stats_.active_sessions_.sub(prior_size - oob_sessions_.size());
   }
 }
 
@@ -141,7 +142,7 @@ void OrcaOobManager::onSessionTerminated(OobSession* session) {
   ASSERT(it != oob_sessions_.end() && it->second.get() == session);
   dispatcher_.deferredDelete(std::move(it->second));
   oob_sessions_.erase(it);
-  oob_stats_.active_sessions_.set(oob_sessions_.size());
+  oob_stats_.active_sessions_.dec();
 }
 
 OrcaOobManager::OobSession::OobSession(OrcaOobManager& parent, Upstream::HostConstSharedPtr host,

source/extensions/load_balancing_policies/load_aware_locality/BUILD

@@ -38,6 +38,7 @@ envoy_cc_library(
         "//source/common/upstream:load_balancer_context_base_lib",
         "//source/common/upstream:upstream_includes",
         "//source/extensions/load_balancing_policies/common:load_balancer_lib",
+        "//source/extensions/load_balancing_policies/common:orca_oob_manager_lib",
         "//source/extensions/load_balancing_policies/common:orca_weight_manager_lib",
         "@abseil-cpp//absl/container:flat_hash_map",
         "@abseil-cpp//absl/container:flat_hash_set",

source/extensions/load_balancing_policies/load_aware_locality/config.cc

@@ -59,6 +59,14 @@ Factory::loadConfig(Server::Configuration::ServerFactoryContext& context,
   std::vector<std::string> metric_names(lb_config.metric_names_for_computing_utilization().begin(),
                                         lb_config.metric_names_for_computing_utilization().end());
 
+  const bool enable_oob_load_report = lb_config.enable_oob_load_report().value();
+  Common::OrcaOobManagerConfig oob_manager_config;
+  oob_manager_config.reporting_period = std::chrono::milliseconds(PROTOBUF_GET_MS_OR_DEFAULT(
+      lb_config, oob_reporting_period, Common::kDefaultOobReportingPeriodMs));
+  if (lb_config.has_oob_reporting_config()) {
+    Common::applyOrcaOobConnectionOverrides(lb_config.oob_reporting_config(), oob_manager_config);
+  }
+
   // Resolve the endpoint-picking child policy.
   Upstream::TypedLoadBalancerFactory* endpoint_picking_policy_factory = nullptr;
   for (const auto& endpoint_picking_policy : lb_config.endpoint_picking_policy().policies()) {
@@ -87,8 +95,8 @@ Factory::loadConfig(Server::Configuration::ServerFactoryContext& context,
           *endpoint_picking_policy_factory,
           LoadBalancerConfigSharedPtr(std::move(lb_config_or_error.value())), weight_update_period,
           utilization_variance_threshold, ewma_alpha, remote_probe_fraction,
-          weight_expiration_period, std::move(metric_names), context.mainThreadDispatcher(),
-          context.threadLocal());
+          weight_expiration_period, std::move(metric_names), enable_oob_load_report,
+          std::move(oob_manager_config), context.mainThreadDispatcher(), context.threadLocal());
     }
   }
 

source/extensions/load_balancing_policies/load_aware_locality/load_aware_locality_lb.cc

@@ -66,6 +66,12 @@ LoadAwareLocalityLoadBalancer::LoadAwareLocalityLoadBalancer(
 
   weight_update_timer_ = typed_config->mainThreadDispatcher().createTimer(
       [this]() { computeLocalityRoutingWeights(); });
+
+  if (typed_config->enableOobLoadReport()) {
+    orca_oob_manager_ = std::make_unique<Common::ProdOrcaOobManager>(
+        typed_config->oobManagerConfig(), priority_set, typed_config->mainThreadDispatcher(),
+        random, cluster_info.statsScope());
+  }
 }
 
 LoadAwareLocalityLoadBalancer::~LoadAwareLocalityLoadBalancer() = default;
@@ -97,6 +103,10 @@ absl::Status LoadAwareLocalityLoadBalancer::initialize() {
       });
 
   computeLocalityRoutingWeights();
+
+  if (orca_oob_manager_ != nullptr) {
+    RETURN_IF_NOT_OK(orca_oob_manager_->initialize());
+  }
   return absl::OkStatus();
 }
 

source/extensions/load_balancing_policies/load_aware_locality/load_aware_locality_lb.h

@@ -22,6 +22,7 @@
 #include "source/common/common/logger.h"
 #include "source/common/upstream/upstream_impl.h"
 #include "source/extensions/load_balancing_policies/common/load_balancer_impl.h"
+#include "source/extensions/load_balancing_policies/common/orca_oob_manager.h"
 
 #include "absl/container/flat_hash_map.h"
 #include "absl/status/status.h"
@@ -97,6 +98,8 @@ class LoadAwareLocalityLbConfig : public Upstream::LoadBalancerConfig {
                             double remote_probe_fraction,
                             std::chrono::milliseconds weight_expiration_period,
                             std::vector<std::string> metric_names_for_computing_utilization,
+                            bool enable_oob_load_report,
+                            Common::OrcaOobManagerConfig oob_manager_config,
                             Event::Dispatcher& main_thread_dispatcher,
                             ThreadLocal::SlotAllocator& tls_slot_allocator)
       : endpoint_picking_policy_factory_(endpoint_picking_policy_factory),
@@ -106,6 +109,8 @@ class LoadAwareLocalityLbConfig : public Upstream::LoadBalancerConfig {
         remote_probe_fraction_(remote_probe_fraction),
         weight_expiration_period_(weight_expiration_period),
         metric_names_for_computing_utilization_(std::move(metric_names_for_computing_utilization)),
+        enable_oob_load_report_(enable_oob_load_report),
+        oob_manager_config_(std::move(oob_manager_config)),
         main_thread_dispatcher_(main_thread_dispatcher), tls_slot_allocator_(tls_slot_allocator) {}
 
   Upstream::TypedLoadBalancerFactory& endpointPickingPolicyFactory() const {
@@ -123,6 +128,8 @@ class LoadAwareLocalityLbConfig : public Upstream::LoadBalancerConfig {
   const std::vector<std::string>& metricNamesForComputingUtilization() const {
     return metric_names_for_computing_utilization_;
   }
+  bool enableOobLoadReport() const { return enable_oob_load_report_; }
+  const Common::OrcaOobManagerConfig& oobManagerConfig() const { return oob_manager_config_; }
   Event::Dispatcher& mainThreadDispatcher() const { return main_thread_dispatcher_; }
   ThreadLocal::SlotAllocator& tlsSlotAllocator() const { return tls_slot_allocator_; }
   absl::Status validateEndpoints(const Upstream::PriorityState& priorities) const override {
@@ -140,6 +147,8 @@ class LoadAwareLocalityLbConfig : public Upstream::LoadBalancerConfig {
   const double remote_probe_fraction_;
   const std::chrono::milliseconds weight_expiration_period_;
   const std::vector<std::string> metric_names_for_computing_utilization_;
+  const bool enable_oob_load_report_;
+  const Common::OrcaOobManagerConfig oob_manager_config_;
   Event::Dispatcher& main_thread_dispatcher_;
   ThreadLocal::SlotAllocator& tls_slot_allocator_;
 };
@@ -391,6 +400,9 @@ class LoadAwareLocalityLoadBalancer : public Upstream::ThreadAwareLoadBalancer,
   Upstream::ThreadAwareLoadBalancerPtr child_thread_aware_lb_;
   std::shared_ptr<WorkerLocalLbFactory> factory_;
   Envoy::Common::CallbackHandlePtr priority_update_cb_;
+  // Opens one OOB ORCA stream per host when enable_oob_load_report is set; initialized last so
+  // per-host LocalityLbHostData slots exist before the first report decodes.
+  std::unique_ptr<Common::OrcaOobManager> orca_oob_manager_;
 };
 
 } // namespace LoadAwareLocality