[build] Ship the clang builtin headers and prefer them at runtime

[aaronj0]

This is the second part of the patch that ships the required runtime clang headers. With this change, a build/install of cppjit is self-contained and with the LLVM that is statically linked into libClangCppInterOp.so, we drop the LLVM dependency on a target machine. The wheels build PR will exercise this on clean images without LLVM.

As settled in review: the bundle is produced by the CMake install rules from the LLVM the build was pointed at, so it is identical to the headers libClangCppInterOp was compiled against under every install channel (pip, conda recipes, distro tooling, bare cmake --install) and for development builds against any LLVM_DIR. Only include/ ships; the JIT reads nothing else from a resource directory. Setups without a bundle such as raw build trees, or distributions that strip vendored copies, fall back to the existing CppInterOp DetectResourceDir probing, which was the previous default behaviour (unchanged). This increases wheel size by 3-5MB

[github-actions]

Test Results

Configuration	Result
macos-26-intel-llvm21-py3.14-cxx20	= 437 passed, 44 skipped, 85 xfailed, 32 xpassed, 100 warnings in 87.94s (0:01:27) =
macos-26-llvm21-py3.14-cxx20	==== 438 passed, 37 skipped, 93 xfailed, 30 xpassed, 100 warnings in 39.01s ====
ubuntu-24.04-llvm21-py3.14-cxx20-vg	====== 553 passed, 25 skipped, 18 xfailed, 2 xpassed in 93.29s (0:01:33) =======
ubuntu-24.04-llvm22-py3.14-cxx20	====== 553 passed, 25 skipped, 19 xfailed, 1 xpassed in 108.79s (0:01:48) ======
ubuntu-24.04-llvm22-py3.14-cxx20-cling	====== 543 passed, 25 skipped, 27 xfailed, 3 xpassed in 92.77s (0:01:32) =======
ubuntu-24.04-llvm22-py3.14-cxx23	====== 574 passed, 3 skipped, 20 xfailed, 1 xpassed in 107.94s (0:01:47) =======

[Vipul-Cariappa]

Can we prioritize the system's resource directory.
First, check clang-version, then the one we include, then simple clang.

[aaronj0]

No that would not work. If we build a binary for manylinux or OS X based on llvm22 and a user installs it on their system which happens to have a lower LLVM version (apple always has clang) that would crash the interpreter. The point is that cppjit should not depend on system LLVM.

[aaronj0]

That is exactly the problem I described in last weeks meeting.

[vgvassilev]

No that would not work. If we build a binary for manylinux or OS X based on llvm22 and a user installs it on their system which happens to have a lower LLVM version (apple always has clang) that would crash the interpreter. The point is that cppjit should not depend on system LLVM.

That would fail the match of the resource dir folder name which incorporates the version. We can also protect against older clangs such a check...

[aaronj0]

That would fail the match of the resource dir folder name which incorporates the version. We can also protect against older clangs such a check...

Can you point me to which match fail you are referring to? Yes we can incorporate a check if the major version matches but just to clarify, we decided to bundle the headers so that we don't run into this problem and introduce mechanisms to ensure that system or package manager LLVM's agree with what cppjit needs.

If we always prioritize system-installed LLVM, that goes against the very solution you proposed (and we agreed on) to bundle the headers, leading to a self contained binary and not depend on an arbitrary LLVM picked up at runtime which can differ based on user environments

[Vipul-Cariappa]

What I am proposing here is that if the system LLVM version matches our LLVM version, we use the system's resource directory; otherwise, the one we ship. Similar to what is being done with the ROOT built-ins.
Look at the code you have now. You first check std::string resourceDir = Paths.ClangIncludeDir; if (resourceDir.empty() ..., then Cpp::DetectResourceDir("clang"), then Cpp::DetectResourceDir("clang-" CPPJIT_CLANG_MAJOR).

Can you point me to which match fail you are referring to?

https://github.com/compiler-research/CppInterOp/blob/1cfe7ee0da5e7abeb80982b9e67270806884aa52/lib/CppInterOp/CppInterOp.cpp#L5419-L5421

[aaronj0]

What I am proposing here is that if the system LLVM version matches our LLVM version, we use the system's resource directory; otherwise, the one we ship. Similar to what is being done with the ROOT built-ins. Look at the code you have now. You first check std::string resourceDir = Paths.ClangIncludeDir; if (resourceDir.empty() ..., then Cpp::DetectResourceDir("clang"), then Cpp::DetectResourceDir("clang-" CPPJIT_CLANG_MAJOR).

This is a different case as the ROOT builtins, no ROOT libraries built with some builtin library headers is going to then prefer the ones found on the system at runtime. (e.g A ROOT binary's lib A built with package B headers do not select system package B headers instead of the ones bundled in the binary). It only chooses to build the built-in version of the library if it does not find an existing installation (which is build-time). For the Python package (and any install), ROOT bundles all of the required clang headers, and the interpreter uses only those headers. This is for the same reason and does not prefer the ones that may exist on the system.

The logic you are talking about exists when running a source build of cppjit. If you clone the repo and try to pip install, we do find_package to look for a compatible system LLVM to build the package. This saves disk space and build time since we don't need a package manager installation or externalproject_add. But once the package is built/installed with an LLVM source and we have already bundled the headers from the same source, you guarantee that it will always work.

With that approach, the benefit of preferring system headers is also non-existent. The bundled headers exist anyway, and we do not save build time (as this is a runtime probe) or disk space.

https://github.com/compiler-research/CppInterOp/blob/1cfe7ee0da5e7abeb80982b9e67270806884aa52/lib/CppInterOp/CppInterOp.cpp#L5419-L5421

Based on the check you have pointed to, I only need to do Cpp::DetectResourceDir("clang-" CPPJIT_CLANG_MAJOR). I can correct that, thanks.

[Vipul-Cariappa]

Could you check whether, when using the headers packages by CppJIT, you can run OpenMP code? If yes, then there is no problem with defaulting to what we provide. If not, we will need to think about what the correct default is. I am talking about the case where the system LLVM matches CppJIT's LLVM version.

[aaronj0]

Yes, with the current system, omp.h lands in the resource dir's include/ when the LLVM build is openmp enabled.

Of course, you do need the openmp runtime which needs to be present on the system, which is standard behavior for third-party libraries. If I build a wheel and install it on a system with no LLVM, the OpenMP code still compiles and works. The wheel carries omp.h and runs as soon as the runtime is loaded.

Here's how I verified this: built a wheel from this branch against an openmp-enabled LLVM 21 which ships cppjit_backend/lib/clang/21/include/omp.h. I then installed it in a bare ubuntu:24.04 container with only g++, libomp-dev and python3 (no LLVM):

import cppjit
cppjit.load_library("libomp.so.5")
cppjit.cppdef('''
#include <omp.h>
int maxthreads() { return omp_get_max_threads(); }
long parsum(int n) {
  long s = 0;
  #pragma omp parallel for reduction(+:s)
  for (int i = 1; i <= n; ++i) s += i;
  return s;
}
''')
print(cppjit.gbl.maxthreads())    # 16
print(cppjit.gbl.parsum(100000))  # 5000050000

Here we need to pass -fopenmp in CPPINTEROP_EXTRA_INTERPRETER_ARGS as usual. We can either LD_PRELOAD or load_library libomp

[vgvassilev]

Well, the openmp is a separate problem. The one we have is the one shipped with llvm and we need to be careful because last time I checked the intel implementation had some problems to be used. If the user tries to use the system omp (which gcc and clang allow for) we have no way to do that. I guess we need to make sure we have some level of diagnostics in that case.