[EXPORTER] Add custom HTTP client

[ltowarek]

Fixes #2084

Changes

Adds a way to provide a custom HTTP client instead of the default libcurl-based client.

The main motivation are embedded and cross-compiled targets where libcurl is unavailable. A working example for ESP32S3 using ESP-IDF's esp_http_client as the transport backend is at ltowarek/esp-opentelemetry-cpp#30.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: ltowarek / name: Lukasz Towarek (a323792)
• ✅ login: ThomsonTan / name: Tom Tan (07d6483, 8b2112a, a5264df)

[owent]

Could you please add a test to use a simple custom http client?

[lalitb]

Thanks @ltowarek. Couple of review comments as of now -

1. As @owent mentioned, could we also add a minimal example for this option, even if the transport is just a no-op custom HttpClient / HttpClientSync? The important part is showing where the user provides HttpClientFactory::Create() and CreateSync(), and how their target is linked with the OTLP HTTP exporter.

2. Also, I believe there are still a few CMake paths that bring in curl directly. For example, Zipkin, Elasticsearch, HTTP examples, and the W3C tracecontext HTTP test server still link opentelemetry_http_client_curl / CURL::libcurl, while WITH_CUSTOM_HTTP_CLIENT=ON disables the curl client target globally. Can we either make this option explicitly incompatible with those curl-only paths, or update them to follow the same custom-client path?

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.51%. Comparing base (503010a) to head (9c2035d).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4071      +/-   ##
==========================================
+ Coverage   82.00%   82.51%   +0.52%     
==========================================
  Files         385      387       +2     
  Lines       16030    16435     +405     
==========================================
+ Hits        13143    13559     +416     
+ Misses       2887     2876      -11     

Files with missing lines	Coverage Δ
...y/exporters/elasticsearch/es_log_record_exporter.h	100.00% <ø> (+100.00%)	⬆️
...orters/elasticsearch/src/es_log_record_exporter.cc	12.22% <100.00%> (+12.22%)	⬆️
...de/opentelemetry/exporters/otlp/otlp_http_client.h	100.00% <ø> (ø)
exporters/otlp/src/otlp_http_client.cc	67.50% <100.00%> (-0.06%)	⬇️
exporters/otlp/src/otlp_http_exporter.cc	99.56% <100.00%> (+0.51%)	⬆️
exporters/otlp/src/otlp_http_exporter_factory.cc	78.95% <100.00%> (+15.32%)	⬆️
...xporters/otlp/src/otlp_http_log_record_exporter.cc	99.12% <100.00%> (+0.99%)	⬆️
.../otlp/src/otlp_http_log_record_exporter_factory.cc	78.95% <100.00%> (+15.32%)	⬆️
exporters/otlp/src/otlp_http_metric_exporter.cc	99.14% <100.00%> (+0.96%)	⬆️
...ters/otlp/src/otlp_http_metric_exporter_factory.cc	78.95% <100.00%> (+15.32%)	⬆️
... and 4 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[dbarker]

Thanks for the PR. Please see the question below.

[ltowarek]

Thanks for the feedback! I think I've covered everything in the new patch set:

• Added opentelemetry_http_client interface library to easily inject different implementations.
• Added tests based on http_client::nosend::HttpClient.
• Added an example app.
• Used WITH_HTTP_CLIENT_CURL=OFF instead of WITH_CUSTOM_HTTP_CLIENT.

[marcalff]

Thanks for the contribution.

Please consider to use:

• an abstract class HttpClientFactory, with virtual methods for Create and CreateSync
• A child class HttpCurlClientFactory, which implement this for CURL
• a HttpCurlClientFactory::singleton

Everywhere the static method HttpClientFactory::Create() is used today, pass a HttpClientFactory instance instead, taken as input parameter.

The user of an HTTP client can then decide which implementation to use:

• pass HttpCurlClientFactory::singleton to use CURL
• pass an instance of MyOwnHttpClientFactory for alternate implementations.

It should be possible to mix CURL and non CURL HTTP clients in the same binary, for example:

• send traces using HTTP + CURL to endpoint A
• send metrics using HTTP + alternate to endpoint B

[ltowarek]

Thanks for the contribution.

Please consider to use:

* an abstract class `HttpClientFactory`, with virtual methods for `Create` and `CreateSync`

* A child class `HttpCurlClientFactory`, which implement this for CURL

* a HttpCurlClientFactory::singleton

Everywhere the static method HttpClientFactory::Create() is used today, pass a HttpClientFactory instance instead, taken as input parameter.

The user of an HTTP client can then decide which implementation to use:

* pass HttpCurlClientFactory::singleton to use CURL

* pass an instance of MyOwnHttpClientFactory for alternate implementations.

It should be possible to mix CURL and non CURL HTTP clients in the same binary, for example:

* send traces using HTTP + CURL to endpoint A

* send metrics using HTTP + alternate to endpoint B

@marcalff, it looks like a good idea.

My main goal with the current PR is build-time configuration - to enable opentelemetry-cpp for embedded systems as they simply can't build
libcurl. The binary for such systems will always contain only one HTTP client.

I think runtime changes should be a part of a separate PR which I'm open to working on.

[ltowarek]

@marcalff I've added runtime changes to this PR. Please let me know if b8eb45d fits the needs.

[lalitb]

Thanks for adding the runtime factory path. I think this is moving in the right direction, but can we make the injection story consistent across all HTTP exporters?

Right now OtlpHttpExporterFactory has overloads that accept a custom HttpClientFactory / HttpClient, but the OTLP HTTP log and metric exporter factories still only expose the default-client path. That means a user can inject a custom client for traces, but not for logs/metrics through the same public factory API.

Also, after making HttpClientFactory::Create*() virtual instance methods, there still seem to be a few static call sites left, for example the HTTP example and test_ext_http_curl.cc. Those should probably go through GetDefaultHttpClientFactory()->Create*() or a concrete curl factory.

[ltowarek]

@lalitb thanks for the feedback! I've addressed your comments in the latest commits.

Need to review again after changes.

[lalitb]

Can we keep the custom-client path independent from the default-client path?

If a user explicitly passes a custom HttpClientFactory or HttpClient, that should not require the default curl factory symbol to exist. The default factory should only be needed when a default constructor/factory overload is actually used.

Right now the exporter object files still contain default-client constructors like this one that reference GetDefaultHttpClientFactory(). In a no-curl/custom-only build, that can still create a link-time dependency on a symbol the user did not intend to use. Could we guard the default-client overloads when no default backend exists, or provide a no-curl fallback that fails clearly only if the default path is used?