[bp/1.37] Fix hyperscan build when using non-hermetic LLVM toolchain (#43479)

[krinkinmu]

Commit Message:

When using non-hermetic LLVM toolchain the path to the nm and objcopy tools provided via environment variables to the hyperscan build script are not correct.

When using non-hermetic toolchain, make C++ toolchain make variables NM and OBJCOPY will have absolute paths to the host tools, so prefixing them actually turns correct paths to incorrect onces.

So one change that this PR does is to check if we are using a non-hermetic toolchain and if so, just pass in $(NM) and $(OBJCOPY) without modifications.

The other part of this PR modifies the hyperscan slightly to make build_wrapper.sh fail when one of the commands in the script fails.

The reason for this change is that currently, if this script cannot find nm and objcopy tools (i.e., when you don't have them installed - which could happen if you use non-hermetic compiler or, before this PR, because we added a prefix to the host tool paths that wasn't needed) it does not fail, but still does not produce quite correct result.

What you will have in the end is a failure during envoy-contrib linking due to linker failing to find definitions for a bunch of the symbols.

Let me try to explain what is going on there...

Hyperscan, specifically when we use fat runtime option, does something quite clever and quite terrible - it builds the same library source code several times enabling different optimizations and links all those together into the final hyperscan library.

It will build a version of the library assuming that AVX512 instruction set is available, another version of the library that assumes that AVC512VBMI instruction set is available and so on. Thus we will have multiple versions of the same library built with different optimizations.

Then it takes each of these versions and using nm and objcopy modifies the names of exported symbols adding a prefix to them to avoid name collisions later when it will link them together (remember all of them are built from the same source).

Finally, it links all those versions together into one library and provides a dispatcher function - this dispatcher function during runtime detects what instruction sets are actually available on the machine and calls an appropriate implementation for that instruction set.

The
build_wrapper.sh script is what takes the compiled object files and renames symbols in it to avoid name collision.

The build_wrapper.sh is written in such a way that when nm or objcopy are not available - it does not fail. You can do a simple experiment for yourself and run the following script:

blahblah > /tmp/test.file
if test -s /tmp/test.file
then
        blahblah-again
fi

Even though neither blahblah nor blahblah-again command exists (and you will see an error about it in the output) the overall script status code when you run it will be 0 (e.g. echo $? will return 0).

A similar thing happens in build_wrapper.sh when it cannot find the objcopy or nm tool - the script does not fail, but it does not produce the object file we expect it to produce.

When later CMake links libhs.a out of the available object files it links everything that exists and produces libhs.a, which miss a bunch of symbol definitions, but otherwise is still a valid static library.

Because libhs.a build didn't fail properly, we proceed to eventually linking envoy-contrib (that's where hyperscan is used) and at that point linker discovers that we don't have all the symbol definitions available.

Why just not use hermetic toolchain?

I'd be happy to, but available published LLVM toolchains are quite limited and only support a few OS (note when it comes to hyperscan, it's specifically limited to x86 architecture, so architecture is not the issue here).

So if you're buildin on Linux other that RedHat or Ubuntu - you're basically out of luck at the moment unfortunately.

Many companies and communities publish custom built LLVM toolchains, but when they do, it's still typically done in format of a package for whatever is the package manager of the platform (e.g., deb, rpm, etc) and toolchains_llvm that we use to download hermetic LLVM toolchain does not support those.

Additional Description:

I'd like to backport a few non-hermetic build fixes to the 1.37 release branch to fix builds from the release branch as well.
The full list of the PRs I want to backport is:

• Fix hyperscan build when using non-hermetic LLVM toolchain #43479 (this is what this PR backports)
• Don't use sysroot when building using non-hermetic toolchain #43681
• Declare BAZEL_USE_HOST_SYSROOT environment variable in the envoy repo… #43731

NOTE: There was also #43636, but the issue that PR affects only happens when the host system has boost library installed already which is quite rare, so I don't think we have to backport that.

Risk Level: Low
Testing: Manually that envoy-contrib builds successfully and that if nm and objcopy aren't found, the build will fail early; +ci Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: n/a

[repokitteh-read-only]

CC @envoyproxy/dependency-shepherds: Your approval is needed for changes made to (bazel/.*repos.*\.bzl)|(bazel/dependency_imports\.bzl)|(api/bazel/.*\.bzl)|(.*/requirements\.txt)|(.*\.patch).
envoyproxy/dependency-shepherds assignee is @moderation

🐱

Caused by: #43763 was opened by krinkinmu.

see: more, trace.

[moderation]

/lgtm deps

[phlax]

lgtm, thanks @krinkinmu

[phlax]

if you have multiple commits to backport - you can open a pr with all of them and ill rebase onto the branch

[krinkinmu]

Thank you, @phlax! I will open a single PR for the last two, because those would have been a single PR if I didn't mixed things up.