Skip to content

[Draft, for CI] Take Julia's LLVM passes from JuliaIR.jl, as an opt-in - #1026

Draft
maleadt wants to merge 17 commits into
mainfrom
tb/juliair-ports
Draft

maleadt wants to merge 17 commits into
mainfrom
tb/juliair-ports

Conversation

@maleadt

@maleadt maleadt commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Draft, not for merging yet. This needs LLVM.jl 10.3 (released) and JuliaIR.jl, which
is not registered yet: the test environment takes its main branch from [sources], and
the commit titled [temp] makes CI work with that; it will be dropped once JuliaIR.jl is
registered.

GPUCompiler's optimization pipeline runs fourteen of the LLVM passes of Julia's code
generator (AllocOpt, LateLowerGCFrame, PropagateJuliaAddrspaces, ...), which only exist
inside Julia's own libLLVM. This lets a compilation take them from
JuliaIR.jl instead, which ports them to Julia, as
the first step towards running GPUCompiler's middle end on a newer LLVM than Julia's.

  • CompilerConfig gets a julia_passes field, :cpp (Julia's passes, as today) or :ports.
    Its default comes from the julia_passes preference. It is part of the configuration, so
    cached artifacts are keyed on it; inference caches are not.
  • JuliaIR.jl is a weak dependency. Its extension registers the ports and maps each of Julia's
    passes to its port; loading it changes nothing by itself, and selecting :ports without it
    is an error before compilation starts. When the preference selects the ports, loading
    GPUCompiler also loads JuliaIR.jl if the environment provides it, so that packages
    building on GPUCompiler (and their test suites) get the ports without loading it
    themselves.
  • The pipeline in optim.jl gets each Julia pass from julia_pass(job, :Name).
  • The ports need opaque pointers. On Julia 1.10 and 1.11, reflection then compiles in an
    opaque-pointer context, JuliaContext() creates one when the preference selects the
    ports, and compiling a :ports job in a typed context is an error.
  • Precompilation workloads of packages that do not load JuliaIR.jl keep using the C++ passes
    (and cache under that configuration).
  • The cached bitcode of runtime functions records whether it uses opaque pointers, and is
    recompiled when read in the other kind of context (on 1.10 and 1.11, reading opaque
    bitcode in a typed context failed; a pre-existing issue that mixing the modes exposes).
  • classify_arguments checks that the number of parameters matches the signature, which a
    Type{T} argument not keyed as Core.TypeEgal would silently break.

Tests: test/julia_passes.jl covers the configuration, the providers, the requirements,
loading JuliaIR.jl through the preference, a precompilation workload under the ports
preference, that the ports actually run (JuliaIR.jl's ledger), and a differential: a corpus
of kernels compiled for PTX, GCN, SPIR-V, Metal, native and BPF with both providers must
give identical IR. On 1.10 and 1.11 it also compares typed with opaque pointers using the
C++ passes (unless Julia itself runs with --opaque-pointers). The whole suite passes with
the preference set to either value.

CI on this PR (temporary)

  • GitHub Actions: Pkg.test can't resolve the unregistered JuliaIR.jl on Julia 1.10, which
    ignores [sources], so every job sets up test/ (.ci/juliair.jl test) and runs
    test/runtests.jl. The matrix has a julia_passes axis (cpp, ports); the ports jobs
    also check that the preference was applied and that JuliaIR.jl's ledger shows the core
    ports ran. Coverage is not collected meanwhile.
  • Buildkite: the back-end steps (CUDA.jl, oneAPI.jl, AMDGPU.jl, Metal.jl, OpenCL.jl) are
    unchanged, and each has a "(ports)" variant that sets the julia_passes = "ports"
    preference, adds JuliaIR.jl to the back-end's test environment, runs its whole test
    suite, and fails unless the ledger shows the ports ran.
  • Back-end steps are skipped on draft PRs unless the last commit message selects them:
    [only ports] runs the five ports steps, [only cuda] (etc.) runs both steps of that
    back-end, [only cuda, amdgpu, metal, oneapi, opencl] runs all ten; [skip ports] skips
    the ports steps.

To start the back-end steps on the draft PR, the last pushed commit's message has to carry
the tag, e.g. an empty commit git commit --allow-empty -m "[only ports]" (or
[only cuda, amdgpu, metal, oneapi, opencl] for all ten steps).

`CompilerConfig` gains `julia_passes`: `:cpp` for the passes of Julia's
code generator (wrapped by `LLVM.Interop`), `:ports` for their ports in
JuliaIR.jl. The default comes from the `julia_passes` preference. Being a
config field, it is part of the identity of every cached LLVM artifact,
but not of the inference caches.

JuliaIR.jl is a weak dependency: its extension tells GPUCompiler how to
register the ports and which of Julia's passes they replace. Loading it
alone changes nothing; selecting `:ports` without it is an error before
compilation starts. Until JuliaIR.jl is registered, the test environment takes
it from `[sources]`.
`optimize!` now asks `julia_pass` for each of Julia's passes, which
returns the C++ pass or, with `julia_passes=:ports`, the name of the
port, registered on the pass builder. Passes JuliaIR.jl does not port
yet (FinalLowerGC, JuliaLICM, LowerPTLS, RemoveJuliaAddrspaces) run as
the C++ pass in both modes; the extension lists them explicitly.
…sses

JuliaIR.jl's passes only support opaque pointers, while Julia 1.10 and
1.11 use typed pointers by default. With `julia_passes=:ports`, the
reflection entry points create opaque contexts, and `JuliaContext()`
does so too when the preference selects the ports, so that the many
callers that create their own context follow the preference. Compiling
a `:ports` job in a typed context is an error. Without the preference,
`JuliaContext()` keeps LLVM's default.

The precompilation workload pins the C++ passes, as JuliaIR.jl is not
loaded while GPUCompiler precompiles.
On Julia 1.10 and 1.11, contexts may use typed or opaque pointers, and
compiling with the ports of Julia's passes makes opaque ones common
there. The bitcode of runtime functions is cached per configuration, so a
function compiled in one kind of context was read back in the other,
which fails for opaque bitcode in a typed context. Recompile it instead.
Running the suite with the `julia_passes` preference set to "ports" needs
JuliaIR.jl loaded, also in the processes some tests spawn. The test of
Metal's lowering of atomics on thread-private objects pins the C++
passes: JuliaIR.jl's AllocOpt uses the escape analysis of Julia 1.13 on
every version, which removes the object, so the IR it lowers would not
occur.
Covers the configuration, the providers and the passes left to C++, the
requirements (JuliaIR.jl loaded, opaque pointers), and, through
JuliaIR.jl's ledger, that the ports run. A differential compiles a corpus
of kernels (allocations, `@simd` loops, Float16, exceptions, Int128 and
aggregate arguments, closures) for every target with both providers in
opaque-pointer contexts and requires identical IR, modulo name counters.

On Julia 1.10 and 1.11, a second set judges the switch from typed to
opaque pointers that the ports need, with the C++ passes: PTX code is
identical, other targets compile or fail alike.
`classify_arguments` treats `Type{T}` arguments as ghosts, which matches
codegen only when the signature keys them as `Core.TypeEgal`, as
GPUCompiler's `methodinstance` does. A job built from another
MethodInstance could get a boxed `Type{T}` parameter, misaligning every
later argument without an error. Before optimization the number of
parameters is known, so check it.
With the preference selecting the ports, precompilation workloads of
packages that don't depend on JuliaIR.jl (GPUCompiler's own, or a
back-end's) would fail, as they compile in a process without it. There,
configurations now default to the C++ passes. That is part of the
configuration, so whatever such a workload caches is not used for
configurations selecting the ports.
Explains the `julia_passes` setting and its preference, what the ports require, and how to run the test suite with them.
With `--opaque-pointers` on Julia 1.10 and 1.11, Julia's code generator does not
support typed-pointer contexts, so the code generated in one differs and the
comparison of typed with opaque pointers fails. Julia then never generates typed IR,
so there is nothing to compare.
Packages building on GPUCompiler do not load JuliaIR.jl, so with the julia_passes
preference set to "ports", their compilations failed for want of the ports unless the
user loaded it. When the preference selects the ports, GPUCompiler now loads JuliaIR.jl
at initialization if the environment provides it (not while precompiling), and the
extension registers the ports once GPUCompiler has loaded. This also makes the
preference usable in a package's test suite, by adding JuliaIR.jl to its test
environment.
JuliaIR.jl now ports RemoveJuliaAddrspaces, FinalLowerGC, LowerPTLS and JuliaLICM too, so
with julia_passes=:ports every one of Julia's passes in the pipeline is a port and none
falls back to the C++ pass. The ledger test now also expects JuliaLICM and
RemoveJuliaAddrspaces, and FinalLowerGC and LowerPTLS for a target that uses Julia's
runtime.

[only cuda oneapi amdgpu metal opencl]
Both test/native/precompile.jl and the test of precompilation without JuliaIR.jl
created a package named NativeCompiler. When one ran after the other on the same
worker, the cache of the second test's package depended on a NativeCompiler other
than the loaded one, so it was loaded from source in the test process: the workload
then reported the ports, and native/precompile found results and inferred code that
should only exist in the precompilation process.
On Julia 1.10 and 1.11 on AArch64, where Julia aligns Int128 to 16 bytes, the PTX
of the aggregate kernel taking its argument by reference differs between typed and
opaque pointers: with opaque pointers the argument's padding is copied too. Only the
padding bytes of the stored value differ, so there only require the code to be
generated.
Julia's LLVM is built without the AMDGPU target on macOS, so GCNCompilerTarget has no
datalayout there and Julia emits allocas in addrspace(0). Since LLVM 23 the verifier
rejects those for an amdgcn triple, aborting the julia_passes test on macOS with Julia
nightly. The GCN test suite itself is skipped on macOS already.
From 1.14, `Base.@cancel_check` emits `julia.cancellation_point()` calls, which
Julia lowers with CancellationLowering right before LateLowerGCFrame
(JuliaLang/julia#62604). GPUCompiler never ran that pass, so native jobs using
Julia's runtime kept the calls unlowered. Run it there, as the port with
`julia_passes=:ports`, or else by its pipeline name since LLVM.jl has no
constructor for it. Other targets have no task to deliver a cancellation to.
The ports come from JuliaIR.jl, which is not registered yet. .ci/juliair.jl
sets up GPUCompiler's test environment, or a downstream package's
environment, with JuliaIR.jl from the `[sources]` entry in test/Project.toml,
which Julia 1.10 ignores.

GitHub Actions tests with julia_passes set to cpp and to ports. Buildkite runs
every back-end as before and, in a "(ports)" step, with the ports; that step
fails unless JuliaIR.jl's ledger shows that the ports ran. The commit message
tag that selects only those steps is documented in downstream.yml. Drop this
once JuliaIR.jl is registered.

[only cuda, amdgpu, metal, oneapi, opencl]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant