Explain what JuliaIR is for, and remove stale statements from the docs

README.md

@@ -10,19 +10,23 @@ that does not contain Julia's own passes.
 
 "IR" in the package name refers to that dialect, not to Julia's own SSA IR.
 
+Julia's own passes are compiled into Julia's libLLVM and exist nowhere else. The ports
+let code that processes Julia's IR with another LLVM, mainly GPUCompiler on a standalone,
+newer libLLVM, run the same passes, with the behaviour of the Julia version that produced
+the IR.
+
 ## Status
 
-Work in progress. Ported so far: RemoveNI (`jl-remove-ni`), AllocOpt (`jl-alloc-opt`),
-PropagateJuliaAddrspaces (`jl-propagate-addrspaces`), DemoteFloat16 (`jl-demote-float16`),
-GCInvariantVerifier (`jl-gc-invariant-verifier`), LowerExcHandlers
-(`jl-lower-exc-handlers`), and the cleanup part of LateLowerGCFrame for functions without
-a GC frame, as on GPUs (`jl-late-gc-cleanup`).
+Every pass of Julia's optimization pipelines, Julia 1.10 to 1.15, is ported (see the
+table on the documentation's home page), and the pipeline itself is rebuilt from the
+ports for each Julia version. The package is not registered yet, and needs an unreleased
+version of LLVM.jl, which `Project.toml` takes from a branch.
 
 ## What is supported
 
 - **The IR of the running Julia.** JuliaIR.jl processes the IR that the Julia version it
   runs on generates: Julia 1.12 processes Julia 1.12's IR, not that of an older or newer
-  Julia. Every Julia version from 1.10 up to the current development version is
+  Julia. Every Julia version from 1.10 up to the current development version (1.15) is
   supported this way.
 - **Julia's LLVM, or a newer one.** The passes work with LLVM.jl on the LLVM library that
   Julia bundles, and on newer LLVM versions, e.g. a separately built library like
@@ -57,6 +61,13 @@ or construct a single pass: `run!(JuliaIR.RemoveNIPass(), mod)`. The pass
 names start with `jl-`, so they never clash with Julia's own C++ passes, which are also
 available when running on Julia's LLVM.
 
+Julia's whole optimization pipeline is available as `JuliaIR.optimize!(mod; level=2)`,
+or `JuliaIR.build_pipeline!(pb; level=2)` to add it to a pass builder.
+
+GPUCompiler runs the ports instead of Julia's C++ passes with `julia_passes = :ports` in
+its `CompilerConfig` (or its `julia_passes` preference set to `"ports"`) and JuliaIR.jl
+loaded (JuliaGPU/GPUCompiler.jl#1026, not merged yet).
+
 ## Testing
 
 `Pkg.test()` runs the package's own tests, including a [lit](https://llvm.org/docs/CommandGuide/lit.html)

docs/src/index.md

@@ -1,28 +1,95 @@
 # JuliaIR.jl
 
-Julia implementations of the LLVM passes that operate on Julia's LLVM IR dialect.
+JuliaIR.jl provides Julia implementations of the LLVM passes that operate on *Julia's
+LLVM IR dialect*: the IR that Julia's code generator emits, with GC-tracked pointers in
+address spaces 10 to 13, intrinsics like `julia.gc_alloc_obj` and
+`julia.gc_preserve_begin`, and Julia-specific metadata and operand bundles. "IR" in the
+package name refers to that dialect, not to Julia's own SSA IR.
 
-The passes are ports of the C++ passes in Julia's `src/llvm-*.cpp`, written against
+The passes are ports of the C++ passes in Julia's `src/llvm-*.cpp` (and of the pipeline
+in `src/pipeline.cpp` that runs them), written against
 [LLVM.jl](https://github.com/JuliaLLVM/LLVM.jl)'s custom-pass API.
 
-What is supported:
+## Why
 
-- The passes process the IR that the running Julia generates: a Julia version handles
-  its own version's IR, not that of an older or newer Julia. Julia 1.10 up to the current
-  development version are supported.
-- They work with LLVM.jl on the LLVM library that Julia bundles, and on newer LLVM
-  versions (e.g. a separately built library like libLLVMGPU).
-- Only IR with opaque pointers is supported.
+Julia's own passes are compiled into Julia's copy of libLLVM, so they only exist in that
+library. Code that processes Julia's IR with another LLVM can't use them. The main case is
+GPUCompiler, which can load a standalone, newer libLLVM next to Julia's (e.g. to target a
+GPU that Julia's LLVM doesn't support yet), and still has to lower Julia's GC intrinsics,
+address spaces and so on before handing the IR to the back-end. The ports run on any
+LLVM library that LLVM.jl can load, so they work in either configuration, and they behave
+like the C++ passes of the Julia version that produced the IR.
 
-What the IR of each Julia version looks like is described by `JuliaIR.Dialect`, and the
-passes use the running Julia's (`JuliaIR.Dialect(VERSION)`). Their options cover what
-the version doesn't determine, like DemoteFloat16's `fp16` and `bf16` policies or the
-GC write-barrier configuration Julia was built with (`gc-barrier`). Facts about Julia's
-runtime that some passes need (see below) are taken from the running Julia too; IR for
-GPUs doesn't need them.
+## What is supported
 
-A required pass also runs on `optnone` functions, which LLVM's pass managers skip for
-the others; each port is required exactly when Julia's C++ pass is.
+- **The IR of the running Julia.** The passes process the IR that the running Julia
+  generates: Julia 1.12 processes Julia 1.12's IR, not that of an older or newer Julia.
+  Julia 1.10 up to the current development version (1.15) are supported. What the IR of
+  each Julia version looks like is described by its *dialect*, `JuliaIR.Dialect`, and the
+  passes use the running Julia's (`JuliaIR.Dialect(VERSION)`).
+- **Julia's LLVM, or a newer one.** The passes work with LLVM.jl on the LLVM library that
+  Julia bundles, and on newer LLVM versions (e.g. a separately built library like
+  libLLVMGPU). They are tested on both: on the LLVM of each supported Julia version, and
+  with each version's IR on the newest LLVM (a standalone LLVM 23).
+- **Opaque pointers only.** Julia 1.10 emits typed pointers by default; its IR is
+  supported when parsed into a context with opaque pointers
+  (`Context(; opaque_pointers=true)`), which upgrades it.
+
+JuliaIR.jl is not registered yet. It needs LLVM.jl 10.3 or later.
+
+## Using the passes
+
+Each pass is an LLVM.jl custom pass named after Julia's pass with a `jl-` prefix (see
+[the table below](#Passes)), so that it never clashes with Julia's C++ pass, which is also
+available when running on Julia's LLVM. Register them with a pass builder and use them by
+name, optionally with options:
+
+```julia
+using LLVM, LLVM.IR, LLVM.Passes, JuliaIR
+
+@dispose ctx=Context(; opaque_pointers=true) pb=PassBuilder() begin
+    mod = parse(LLVM.Module, ir)
+    JuliaIR.register!(pb)
+    add!(pb, "jl-alloc-opt,jl-late-lower-gc-frame,jl-final-lower-gc")
+    run!(pb, mod)
+end
+```
+
+or run a single pass: `run!(JuliaIR.RemoveNIPass(), mod)`. Julia's whole optimization
+pipeline, rebuilt from the ports for the running Julia's version, is available too (see
+[Julia's optimization pipeline](pipeline.md)):
+
+```julia
+JuliaIR.optimize!(mod; level=2)           # optimize a module like Julia does
+JuliaIR.build_pipeline!(pb; level=2)      # or add the pipeline to a PassBuilder
+```
+
+From the command line, `jlopt` is an `opt`-compatible driver that runs the ports, and
+maps Julia's pass names (`-passes=AllocOpt`, `-passes='julia<level=2>'`) to them; see
+[`jlopt`](testing.md#jlopt) for how to run or install it.
+
+GPUCompiler uses the ports instead of Julia's C++ passes when its `CompilerConfig` has
+`julia_passes = :ports` (or GPUCompiler's `julia_passes` preference is `"ports"`) and
+JuliaIR.jl is loaded; the compilation then needs a context with opaque pointers, which
+GPUCompiler's `JuliaContext()` creates for that setting. This
+is not merged in GPUCompiler yet (JuliaGPU/GPUCompiler.jl#1026).
+
+## Terms
+
+- *Port*: the Julia implementation of one of Julia's C++ passes, e.g. `jl-alloc-opt` for
+  `AllocOpt`. A port does what the C++ pass of the running Julia's version does, except
+  for the deviations listed on this page and on the passes' own pages.
+- *Dialect*: what the IR of a Julia version looks like, and which passes and behaviours
+  that version has (`JuliaIR.Dialect`). A pass's options cover what the version doesn't
+  determine, like DemoteFloat16's `fp16` and `bf16` policies or the GC write-barrier
+  configuration Julia was built with (`gc-barrier`).
+- *Required*: a required pass also runs on `optnone` functions, which LLVM's pass
+  managers skip for the others. Each port is required exactly when Julia's C++ pass is.
+- *Runtime facts*: facts about the Julia runtime that the IR doesn't state, like the
+  offset of a field in the current task, which some passes need (see
+  [below](#Facts-about-the-Julia-runtime)). IR for GPUs doesn't need them.
+
+## Passes
 
 | Pass | Julia's name | Required |
 |------|--------------|----------|
@@ -33,9 +100,12 @@ the others; each port is required exactly when Julia's C++ pass is.
 | `jl-gc-invariant-verifier` | `GCInvariantVerifier` | yes |
 | `jl-combine-muladd` | `CombineMulAdd` (Julia 1.10 and 1.11) | no |
 | `jl-late-lower-gc-frame` | `LateLowerGCFrame` | yes |
+| `jl-final-lower-gc` | `FinalLowerGC` (a module pass on Julia 1.10, a function pass later, see [FinalLowerGC](final_lower_gc.md)) | yes |
 | `jl-late-gc-cleanup` | (`LateLowerGCFrame` on functions without a GC frame, for GPUs) | yes |
 | `jl-lower-exc-handlers` | `LowerExcHandlers` (Julia 1.10 to 1.12) | yes |
 | `jl-lower-simdloop` | `LowerSIMDLoop` (a loop pass) | no |
+| `jl-licm` | `JuliaLICM` (a loop pass, see [JuliaLICM](julia_licm.md)) | no |
+| `jl-lower-ptls` | `LowerPTLSPass` (a module pass, see [LowerPTLS](lower_ptls.md)) | yes |
 | `jl-expand-atomic-modify` | `ExpandAtomicModify` (Julia 1.13 and later) | no |
 | `jl-cancellation-lowering` | `CancellationLowering` (Julia 1.14 and later) | yes |
 | `jl-remove-addrspaces` | `RemoveJuliaAddrspaces` | yes |
@@ -157,3 +227,6 @@ Differences with Julia's C++ passes:
   metadata of global variable declarations, the types of `inalloca` and `preallocated`
   attributes, and indirect functions, which the C++ passes drop, leave unmapped or
   corrupt; see [RemoveJuliaAddrspaces](@ref).
+- FinalLowerGC, LowerPTLS, JuliaLICM and JuliaMultiVersioning record their differences on
+  their own pages: [FinalLowerGC](final_lower_gc.md), [LowerPTLS](lower_ptls.md),
+  [JuliaLICM](julia_licm.md) and [JuliaMultiVersioning](multiversioning.md).

docs/src/testing.md

@@ -118,10 +118,14 @@ Every file in the checkout must be in the manifest, as one of:
   in each test's ledger that every port in its pipelines ran;
 - `skip`: it will run once the ports listed under `missing` exist (the harness fails once
   they do, so that it is enabled);
-- `exclude`: it tests something this package does not port (CPU-only passes not ported
-  yet, Julia's code generator);
+- `exclude`: it tests something this package does not port (Julia's code generator,
+  or Julia's pipeline as a whole on code that Julia compiles itself);
 - `override`: a derived copy runs, with a reviewed patch from `test/upstream/derived`
-  applied. The harness fails if the pristine fixture passes too.
+  applied. The harness fails if the pristine fixture passes too;
+- `derived`: a copy runs that tests a port instead of what the fixture tests, with a
+  patch from `test/upstream/derived` applied, e.g. `pipeline-o*.jl`, which run Julia's
+  pipeline, rebuilt from the ports, in `jlopt` instead of in Julia's JIT. The copy must
+  run a port.
 
 RUN lines that use typed pointers (`--opaque-pointers=0`) are always skipped; legacy pass
 manager lines with opaque pointers (Julia 1.10) are translated.
@@ -199,8 +203,8 @@ the testsets, the lit feature `julia-llvm` in RUN lines).
 Two things differ from mode A, the default run of the previous section:
 
 - A standalone LLVM doesn't have Julia's C++ passes. RUN lines whose pipeline names one
-  of them that isn't ported (`FinalLowerGC`, `JuliaMultiVersioning`, `julia`, ...) are
-  skipped, and the harness lists them. Mode B skips them on Julia's own LLVM too, so
+  of them that isn't ported would be skipped, and the harness lists them; every pass of
+  Julia's pipelines is ported now, so this only applies to one that a newer Julia adds. Mode B skips them on Julia's own LLVM too, so
   that a run there (a dry run of mode B) tests the same lines. A pass name that no LLVM
   knows is not skipped: the line fails. Julia's pipeline (`julia<...>`) is rebuilt from
   the ports, but it also runs LLVM's own passes, which optimize differently on another