ESP-IDF heap: malloc returns 4-byte-aligned memory despite alignof(std::max_align_t)==8 on Xtensa

[ltowarek]

Fix lives in: ESP-IDF heap component (components/heap/multi_heap.c)

What the workaround does

We provide an alignas(8)-guaranteed initial block when constructing the protobuf Arena inside OtlpHttpExporter::Export(). Because the initial_block is the first block the arena touches, TaggedAllocationPolicyPtr always operates on an 8-byte-aligned base, avoiding the misaligned pointer crash:

// src/workarounds/otlp_http_exporter_esp.cc (replaces otlp_http_exporter.cc
// via set_property(SOURCES))
alignas(8) char initial_buf[1024];
google::protobuf::ArenaOptions arena_options;
arena_options.initial_block      = initial_buf;
arena_options.initial_block_size = sizeof(initial_buf);
arena_options.max_block_size     = 65536;
google::protobuf::Arena arena{arena_options};

The replacement file is wired in through CMakeLists.txt using set_property(TARGET opentelemetry_http_client_curl PROPERTY SOURCES ...) so no submodule file is modified. This is a targeted fix: it guarantees 8-byte alignment for the single arena constructed by the exporter without changing global allocator behaviour.

Root cause

ESP-IDF's TLSF-backed heap uses a 4-byte alignment granularity on 32-bit Xtensa:

// components/heap/multi_heap.c
#define ALIGN(X) ((X) & ~(sizeof(void *)-1))   // sizeof(void*)==4 → 4-byte alignment

The C++ standard requires operator new to return memory aligned to
alignof(std::max_align_t). The ESP32-S3 Xtensa toolchain reports
alignof(std::max_align_t)==8 (empirically confirmed via logging at startup).
The heap's 4-byte granularity therefore violates the standard's alignment
guarantee: two out of eight consecutive operator new(256) calls returned a
4-byte-aligned (not 8-byte-aligned) address during testing.

The practical crash comes from protobuf's TaggedAllocationPolicyPtr
(google/protobuf/arena_allocation_policy.h). It stores three flag bits in the
low bits of an AllocationPolicy* pointer using kPtrMask = ~7, which requires
the stored pointer to be 8-byte aligned. The crash sequence when
ArenaOptions with non-default max_block_size are used:

1. InitializeWithPolicy places an AllocationPolicy at ptr() of the first
   arena block. ptr() starts at block_base + kBlockHeaderSize (8). If
   block_base is only 4-byte aligned, ptr() is also only 4-byte aligned.
2. alloc_policy_.set_policy(p) stores the 4-byte-aligned pointer p.
3. alloc_policy_.get() computes p & ~7 = p - 4, reading 4 bytes before
   the AllocationPolicy struct.
4. get()->block_alloc therefore reads from p + 4, which is the
   max_block_size field — 0x00010000 when max_block_size = 65536.
5. When the arena needs a second block it calls block_alloc(size), jumping to
   address 0x00010000 → InstrFetchProhibited exception.

The OTLP HTTP exporter (otlp_http_exporter.cc) creates an arena with
max_block_size = 65536 on every Export() call, triggering the crash
consistently after the first span export.

Note that protobuf's existing ABSL_DCHECK_EQ(0u, ptr & 3) only verifies
4-byte alignment, masking the bug on platforms where the two overlap.

Proposed fix

ESP-IDF's ALIGN macro in components/heap/multi_heap.c should use
alignof(max_align_t) instead of sizeof(void*) as the alignment granularity
on targets where the two differ:

// components/heap/multi_heap.c
#include <stdalign.h>
#define ALIGN(X) ((X) & ~(alignof(max_align_t) - 1))

This is the conforming minimum: malloc and operator new must return memory
suitably aligned for any fundamental type. No existing upstream bug report was
found for this mismatch.

What we delete once fixed

• src/workarounds/otlp_http_exporter_esp.cc
• The set_property(TARGET opentelemetry_http_client_curl PROPERTY SOURCES ...) override in CMakeLists.txt

[ltowarek]

Update: workaround approach changed.

The previous approach (src/workarounds/otlp_http_exporter_esp.cc with an alignas(8) initial_buf passed as the Arena's initial_block) was never wired into the build — no set_property(TARGET ... PROPERTY SOURCES) call existed for it in CMakeLists.txt.

Replaced with the global operator new override from PR #42, now placed at src/workarounds/esp_heap_align.cpp:

void* operator new(std::size_t size) {
    void* p = heap_caps_aligned_alloc(
        alignof(std::max_align_t), size, MALLOC_CAP_DEFAULT);
    if (p == nullptr) throw std::bad_alloc();
    return p;
}
// + operator new[], operator delete, operator delete[] variants

Forced onto the link line via -Wl,--undefined=_Znwj (Itanium ABI mangled name for operator new(unsigned int) on 32-bit targets). Fixes alignment for every C++ allocation, not just the OTLP exporter's arena.