GH-37848: [C++][Gandiva] Migrate LLVM JIT engine from MCJIT to ORC v2/LLJIT#39098

Merged

kou merged 28 commits into

apache:mainfrom

niyue:feature/orc-v2-jit

Jan 4, 2024

Commits on Dec 20, 2023

Switch Gandiva JIT engine from MCJIT to ORC v2 with LLJIT API.
niyue
committed
Store target machine instead of target ir analysi in the engine to avoid crash.
niyue
committed
Use a new configuration flag to indicate Gandiva engine to dump IR.
niyue
committed
Enhance module api of gandiva engine to ensure it is not called after FinalizeModule.
niyue
committed
Add optional object cache for gandiva engine creation API.
niyue
committed
Add back the SetLLVMObjectCache API to addObjectFile to LLJIT instance.
niyue
committed
Set generator for LLJIT so that current process symbol can be found.
niyue
committed
Set module data layout correctly and add back the module parameter wehn creating function so that symbols can be found.
niyue
committed
Use reinterpret_cast instead of C-style static cast for converting function pointer.
niyue
committed
Change LLVMGenerator and Engine's Make factory method to return arrow Result instead of using out parameter so that it is simpler and optional parameter can be used as well.
niyue
committed
Make CompileFunction to return arrow Result to handle error better, and resolve the LLVM 15 symbol getAddress API change issue, and fix the ExecutorSymbolDef issue for LLVM 17.
niyue
committed
Include inttypes.h to avoid PRIxPTR not defined issue in CI.
niyue
committed
Hide LLJIT into implementation to try to address the compilation issue.
niyue
committed
Remove LLVM IR API from Gandiva's pyarrow binding since the configuration is not exposed so LLVM IR cannot be called.
niyue
committed
Use full qualified API to avoid compilation issue.
niyue
committed
Troubleshoot CI failure.
niyue
committed
Rename function name to be different with the module name used by LLJIT, so that LLVM 10 won't return a 0-address function pointer when doing lookup.
niyue
committed
Minor tweaks for llvm generator and engine to simplify the code.
niyue
committed
Add a minimum atexit symbol lookup test for troubleshooting.
niyue
committed
Remove the lookup test case used for troubleshooting.
niyue
committed
Set several variables to be const.
niyue
committed
Add atexit symbol if it doesn't exist so that ASAN won't complain.
niyue
committed
Switch to use JITLink for LLVM 9+ to see if it addresses the ASAN issue.
niyue
committed
Make JITLink defaults to off, and add a benchmark for expression compilation.
niyue
committed
Rename configuration to `dump_ir` and return Status for the SetLLVMObjectCache API.
niyue
committed
Add a gandiva Configuration class for the python binding so that dumping IR can be done.
niyue
committed
Remove WIN64 macro since WIN32 could be enough to skip Windows.
niyue
committed

Commits on Dec 22, 2023

Require error_context for AsArrowResult API.
niyue
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

GH-37848: [C++][Gandiva] Migrate LLVM JIT engine from MCJIT to ORC v2/LLJIT#39098

GH-37848: [C++][Gandiva] Migrate LLVM JIT engine from MCJIT to ORC v2/LLJIT#39098
kou merged 28 commits into
apache:mainfrom
niyue:feature/orc-v2-jit

Commits on Dec 20, 2023

Switch Gandiva JIT engine from MCJIT to ORC v2 with LLJIT API.

Store target machine instead of target ir analysi in the engine to avoid crash.

Use a new configuration flag to indicate Gandiva engine to dump IR.

Enhance module api of gandiva engine to ensure it is not called after FinalizeModule.

Add optional object cache for gandiva engine creation API.

Add back the SetLLVMObjectCache API to addObjectFile to LLJIT instance.

Set generator for LLJIT so that current process symbol can be found.

Set module data layout correctly and add back the module parameter wehn creating function so that symbols can be found.

Use reinterpret_cast instead of C-style static cast for converting function pointer.

Change LLVMGenerator and Engine's Make factory method to return arrow Result instead of using out parameter so that it is simpler and optional parameter can be used as well.

Make CompileFunction to return arrow Result to handle error better, and resolve the LLVM 15 symbol getAddress API change issue, and fix the ExecutorSymbolDef issue for LLVM 17.

Include inttypes.h to avoid PRIxPTR not defined issue in CI.

Hide LLJIT into implementation to try to address the compilation issue.

Remove LLVM IR API from Gandiva's pyarrow binding since the configuration is not exposed so LLVM IR cannot be called.

Use full qualified API to avoid compilation issue.

Troubleshoot CI failure.

Rename function name to be different with the module name used by LLJIT, so that LLVM 10 won't return a 0-address function pointer when doing lookup.

Minor tweaks for llvm generator and engine to simplify the code.

Add a minimum atexit symbol lookup test for troubleshooting.

Remove the lookup test case used for troubleshooting.

Set several variables to be const.

Add atexit symbol if it doesn't exist so that ASAN won't complain.

Switch to use JITLink for LLVM 9+ to see if it addresses the ASAN issue.

Make JITLink defaults to off, and add a benchmark for expression compilation.

Rename configuration to `dump_ir` and return Status for the SetLLVMObjectCache API.

Add a gandiva Configuration class for the python binding so that dumping IR can be done.

Remove WIN64 macro since WIN32 could be enough to skip Windows.

Commits on Dec 22, 2023

Require error_context for AsArrowResult API.