Skip to content

Opaque-struct fallback silently corrupts a fully-specified struct's layout on native Windows (mismatched/MSVC libclang) #3442

Description

@Magnelo1

Summary

On native Windows, bindgen can silently fall back to an opaque single-byte
placeholder (pub struct X { pub _address: u8 }) for a fully-specified C
struct, while its own layout_tests output still asserts the struct's
real (correctly-computed) size — producing a compile-time
arithmetic-overflow panic (attempt to compute N_usize - correct_usize, which would overflow) instead of either (a) correct bindings or (b) a
clear diagnostic that layout resolution failed.

This reproduces in two distinct ways depending on the Windows
toolchain/libclang pairing in use, both against the same real-world
struct (OQS_SIG from liboqs, via the oqs-sys crate,
open-quantum-safe/liboqs-rust):

  1. MSVC target (x86_64-pc-windows-msvc), with LIBCLANG_PATH
    pointing at a standard LLVM.org Windows release: fails with
    error[E0080]: attempt to compute 1_usize - 88_usize, which would overflow in oqs-sys's generated sig_bindings.rs. (General class
    covered by Telling libclang where to look for system includes on Windows #975, Bindgen tries to use MSVC headers when using x86_64-pc-windows-gnu target #1760, Windows keeps using MSVC, even when CC and CXX are set explicitly and empty PATH #2879, C++ ABI in MSVC and function returning non-POD type #2865 — libclang not reliably
    resolving/matching MSVC's real headers and ABI.)
  2. GNU/MinGW target (x86_64-pc-windows-gnu), with a mismatched
    libclang — MinGW's gcc.exe/g++.exe as CC/CXX, but
    LIBCLANG_PATH pointing at an unrelated LLVM.org clang release rather
    than a libclang built to match MinGW's own toolchain: identical
    symptom
    , pub struct OQS_SIG { pub _address: u8 } generated instead
    of the real 88-byte layout, reproducing the exact same E0080
    assertion via a completely different toolchain path. Once headers
    resolve cleanly (via BINDGEN_EXTRA_CLANG_ARGS pointed at MinGW's own
    include dirs) libclang's own layout computation is demonstrably
    correct — bindgen's Rust-codegen step is what falls back to the opaque
    stub, not libclang's parse.

Workaround found (not a bindgen fix): installing a matched GCC +
libclang pair from a single distributor (WinLibs'
BrechtSanders.WinLibs.POSIX.MSVCRT.LLVM package, which ships a real
libclang.dll built against the same MinGW headers as its gcc.exe)
resolves case 2 completely. This strongly suggests the underlying bug is
in how bindgen's Rust-codegen step decides whether it has "enough"
libclang-reported layout information to emit a concrete struct, when
libclang's own header/ABI resolution is itself internally consistent but
drawn from a different toolchain family than the one actually compiling
the Rust/C code.

Related, not duplicate: #1683 ("Bad bindings for empty C struct on
windows") shows the same _address: u8 opaque-fallback shape, but for a
struct that is actually empty in the source. This report is a
different, more concerning case: the fallback triggers for a
fully-specified, well-formed struct (2 pointers, a uint8_t, 3 bools,
3 size_ts, 5 function pointers — nothing exotic), and does so silently
except for a layout_tests size-assertion that then fails at compile
time with a confusing arithmetic-overflow message rather than a clear
"bindgen couldn't determine this struct's layout" error.

The struct in question

From liboqs's src/sig/sig.h (as vendored by oqs-sys 0.11.0 /
liboqs 0.13.0), unconditionally defined — no #ifdef/feature gating at
all, so this isn't a config-dependent struct shape:

typedef struct OQS_SIG {
    const char *method_name;
    const char *alg_version;
    uint8_t claimed_nist_level;
    bool euf_cma;
    bool suf_cma;
    bool sig_with_ctx_support;
    size_t length_public_key;
    size_t length_secret_key;
    size_t length_signature;
    OQS_STATUS (*keypair)(uint8_t *public_key, uint8_t *secret_key);
    OQS_STATUS (*sign)(uint8_t *signature, size_t *signature_len,
                        const uint8_t *message, size_t message_len,
                        const uint8_t *secret_key);
    OQS_STATUS (*sign_with_ctx_str)(uint8_t *signature, size_t *signature_len,
                        const uint8_t *message, size_t message_len,
                        const uint8_t *ctx_str, size_t ctx_str_len,
                        const uint8_t *secret_key);
    OQS_STATUS (*verify)(const uint8_t *message, size_t message_len,
                        const uint8_t *signature, size_t signature_len,
                        const uint8_t *public_key);
    OQS_STATUS (*verify_with_ctx_str)(const uint8_t *message, size_t message_len,
                        const uint8_t *signature, size_t signature_len,
                        const uint8_t *ctx_str, size_t ctx_str_len,
                        const uint8_t *public_key);
} OQS_SIG;

2 pointers + 1 uint8_t + 3 bool + 3 size_t + 5 function pointers,
standard x86-64 alignment, sums to exactly 88 bytes by hand — matching
what bindgen's own layout_tests asserts as the "real" size, even in the
runs where it emits the opaque 1-byte stub instead of the real struct.
That internal inconsistency (assert the real size, but emit the wrong
type) is the actual bug being reported here, distinct from "libclang
couldn't resolve this struct at all."

oqs-sys's bindgen invocation (identical across all platforms — the

divergence is not in this call)

bindgen::Builder::default()
    .clang_arg(format!("-I{}", includedir.display()))
    .header(includedir.join("oqs").join(format!("{headerfile}.h")).to_str().unwrap())
    .default_enum_style(bindgen::EnumVariation::Rust { non_exhaustive: false })
    .size_t_is_usize(true)
    .generate_comments(cfg!(feature = "docs"))
    .allowlist_recursively(false)
    .allowlist_type(allow_filter)      // "OQS_SIG.*"
    .allowlist_function(allow_filter)
    .allowlist_var(allow_filter)
    .blocklist_type(block_filter)      // "OQS_SIG_STFL.*"
    .blocklist_function(block_filter)
    .blocklist_var(block_filter)
    .use_core()
    .ctypes_prefix("::libc")
    .generate()
    .expect("Unable to generate bindings")
    .write_to_file(out_path.join(format!("{headerfile}_bindings.rs")))
    .expect("Couldn't write bindings!");

Same call, same allowlist regex, on every OS. The same source builds
cleanly on Linux/WSL2 in every run — ruling out a project-level
misconfiguration or an oqs-sys-specific bug in this invocation.

Reproduction

  • Target: x86_64-pc-windows-msvc (case 1) or x86_64-pc-windows-gnu
    with mismatched libclang (case 2, see above).
  • Crate: oqs-sys 0.11.0 (liboqs 0.13.0 vendored), via
    open-quantum-safe/liboqs-rust, with any feature set that enables at
    least one OQS_ENABLE_SIG_* build (confirmed: this is not
    feature-set-dependent — the struct has no #ifdef gating, and the
    failure reproduces identically whether zero or multiple SIG algorithms
    are enabled at CMake level).
  • cargo build --release --target x86_64-pc-windows-msvc (or the
    mismatched-libclang GNU case) →
    error[E0080]: attempt to compute 1_usize - 88_usize, which would overflow in the generated sig_bindings.rs's layout_tests module.
  • The identical source, same feature set, builds clean under WSL2/Linux
    in the same run.

Environment (case 1, MSVC)

  • Rust 1.88.0, x86_64-pc-windows-msvc host
  • LIBCLANG_PATH → a standalone LLVM.org Windows release
  • oqs-sys 0.11.0 / liboqs 0.13.0

Environment (case 2, mismatched GNU)

  • Rust 1.88.0 (stable-x86_64-pc-windows-gnu / pinned
    1.88.0-x86_64-pc-windows-gnu)
  • CC/CXX → MinGW-w64's own gcc.exe/g++.exe
  • LIBCLANG_PATH → an unrelated LLVM.org Windows clang release (not
    matched to the MinGW toolchain)
  • BINDGEN_EXTRA_CLANG_ARGS--target=x86_64-w64-mingw32 -I<mingw>\x86_64-w64-mingw32\include (headers resolve cleanly with this
    in place — the failure is not a header-not-found error at this point)
  • Fixed by switching LIBCLANG_PATH to WinLibs'
    BrechtSanders.WinLibs.POSIX.MSVCRT.LLVM package's bin\ (a matched
    GCC + libclang pair from one distributor, shipping a real
    libclang.dll) — after which the real 88-byte OQS_SIG layout is
    generated correctly, no code changes needed on the oqs-sys or
    consuming-crate side.

Why this is worth fixing upstream (vs. just documenting the workaround)

The matched-toolchain fix works, but the failure mode itself — silently
emitting a wrong, opaque struct while still asserting the correct size in
the generated test — is what turns a toolchain-mismatch problem into a
confusing, hard-to-diagnose E0080 arithmetic panic. A clearer failure
(e.g. bindgen erroring out at generation time when it can't determine a
struct's real layout, instead of emitting an opaque stub paired with a
size assertion for the layout it couldn't determine) would have
surfaced the actual problem (toolchain/libclang mismatch) immediately,
rather than requiring the multi-day root-cause investigation that
produced this report.

Related issues (same general class: libclang not reliably

resolving/matching a Windows target's real headers/ABI)

A structurally similar symptom (same attempt to compute X_usize - Y_usize class of panic, different underlying cause — glibc-specific
types leaking into an MSVC build) was also reported independently against
a different bindgen-based crate: Dimillian/CodexMonitor#599
(whisper-rs-sys).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions