refactor(runtime)!: adopt the modern Infini stack

.github/ci_config_nvidia.yaml

@@ -0,0 +1,74 @@
+repo:
+  url: https://github.com/InfiniTensor/InfiniLM.git
+  branch: main
+
+github:
+  status_context_prefix: "ci/infinilm"
+
+platforms:
+  nvidia:
+    image:
+      dockerfile: images/nvidia/
+      build_args:
+        BASE_IMAGE: nvcr.io/nvidia/pytorch:25.12-py3
+        CUDA_ARCH: sm_80,sm_86,sm_89,sm_90
+        APT_MIRROR: https://mirrors.tuna.tsinghua.edu.cn/ubuntu
+        PIP_INDEX_URL: https://pypi.org/simple
+        InfiniCore_BRANCH: __Branch_Name__
+    docker_args:
+      - "--user=root"
+      - "--network=host"
+      - "--privileged"
+      - "--cap-add=ALL"
+      - "--pid=host"
+      - "--ipc=host"
+      - "--workdir=/workspace"
+    volumes:
+      - /data:/data
+      - /data-aisoft:/data-aisoft
+      - /data-aisoft/artifacts/CI_nvidia_test/__WORKSPACE__:/artifacts
+    setup: pip install .[dev] --no-build-isolation
+    jobs:
+      gpu_inferencetest:
+        type: inferencetest
+        resources:
+          ngpus: [1, 4]
+          gpu_style: nvidia
+          shm_size: 64g
+          timeout: 3600
+        stages:
+          - name: test
+            run: python InfiniLM/examples/test_infer.py --device nvidia --model=/data-aisoft/mechdancer/models/9g_8b_thinking/
+      gpu_benchtest:
+        type: benchtest
+        resources:
+          gpu_style: nvidia
+          shm_size: 64g
+          timeout: 3600
+        env:
+          TEST_PARAM: ['default']
+        stages:
+          - name: test
+            run: python InfiniLM/examples/bench.py --device nvidia --model=/data-aisoft/mechdancer/models/9g_8b_thinking/ --input-len=256,1024 --output-len=256,1024 --batch-size=8 <TEST_PARAM>
+      gpu_accuracytest:
+        type: accuracytest
+        resources:
+          gpu_style: nvidia
+          shm_size: 64g
+          timeout: 3600
+        env:
+          TEST_PARAM: ['--bench mmlu']
+        stages:
+          - name: test
+            run: python InfiniLM/test/bench/test_benchmark.py --device nvidia --model /data-aisoft/mechdancer/models/9g_8b_thinking/ --bench mmlu --backend cpp --max-new-tokens 5 --cache-dir /data-aisoft/pepe/datasets/ --split=val <TEST_PARAM>
+      gpu_servicetest:
+        type: servicetest
+        resources:
+          shm_size: 64g
+        env:
+          MODEL_LIST: 9g_8b_thinking
+          ENGINE: InfiniLM
+          TEST_PARAM: ['default']
+        stages:
+          - name: test
+            run: python InfiniLM/scripts/test_perf.py --verbose

.github/workflows/ci_test.yml

@@ -43,9 +43,9 @@ jobs:
   ci:
     if: github.event_name == 'workflow_dispatch'
     needs: check-format
-    uses: InfiniTensor/ci/.github/workflows/infinilm-ci.yml@infiniCore_ci
+    uses: InfiniTensor/ci/.github/workflows/infinilm-ci.yml@refactor/adopt-modern-infini-stack
     with:
-      config_path: .github/ci_config.yaml
-      ci_ref: infiniCore_ci
+      config_path: .github/ci_config_nvidia.yaml
+      ci_ref: refactor/adopt-modern-infini-stack
       infinicore_branch: ${{ github.event.inputs.infinicore_branch || 'main' }}
     secrets: inherit

CONTRIBUTING.md

@@ -58,7 +58,29 @@ Existing branch names may use the legacy format `issue/###`, followed by a suffi
 
 # Development Guide
 
-Refer to [ReadMe](README.md) and [Adapt New Models](MODELS.md)
+Refer to [ReadMe](README.md) and [Adapt New Models](MODELS.md).
+
+Run the migrated stack-builder unit tests with:
+
+```shell
+python -m unittest test/scripts/test_build_infini_stack.py -v
+```
+
+Check the Core-backed build commands without creating build output:
+
+```shell
+python scripts/build_infini_stack.py --infinicore-root ../InfiniCore --dry-run --jobs 1 --cuda-arch sm_80
+```
+
+Run the static migration contracts with:
+
+```shell
+python -m unittest discover -s test/static -p "test_*.py" -v
+```
+
+Native stack changes must be validated in the owning component repository
+first. After validation, update the corresponding submodule pin in InfiniCore
+before validating InfiniLM.
 
 ## Troubleshooting
 

README.md

@@ -1,177 +1,98 @@
 # InfiniLM
 
-![star](https://atomgit.com/InfiniTensor/InfiniLM/star/badge.svg)
-
-本项目是基于 [`InfiniCore`](https://github.com/InfiniTensor/InfiniCore) 的推理引擎。
-
-## 使用方式
-#### 一、编译并安装 `InfiniCore`
-编译并安装 `InfiniCore`， 详情见 InfiniCore的 [`README`](https://github.com/InfiniTensor/InfiniCore) :
-
-- 注意根据提示设置好 `INFINI_ROOT` 环境变量（默认为 `$HOME/.infini`）
-- 根据硬件平台，选择 xmake 构建配置
-- 编译安装InfiniCore
-- 安装 C++ 库
-- 安装 Python 包
-
-
-#### 二、编译并安装  `InfiniLM`
-  - 克隆项目
-
-    由于仓库中含有子模块，所以在克隆时请添加 `--recursive` 或 `--recurse-submodules`，如：
-
-    ```shell
-    git clone --recursive https://github.com/InfiniTensor/InfiniLM.git
-    ```
-
-    或者在普通克隆后进行更新：
-
-    ```shell
-    git submodule update --init --recursive
-    ```
-
-  - 安装 InfiniLM Python 包
-    ```bash
-      pip install -e .
-    ```
-
-  - 单次推理测试
-    - llama示例
-    ```bash
-    python examples/test_infer.py --device [cpu | nvidia | qy | metax | moore | iluvatar | ali | cambricon | hygon] --model=<path/to/model_dir>
-    ```
-    - 例如：
-    ```bash
-    python examples/test_infer.py --device=nvidia --model=/models/TinyLlama-1.1B-Chat-v1.0
-    ```
-  - 分布式推理测试
-      - 9g示例
-      ```bash
-    python examples/test_infer.py [-- device nvidia] --model=<path/to/model> --backend=cpp --tp=NDEV --batch-size=MAX_BATCH
-    ```
-
-    - 例如： 9G7B模型，cpp后端，batch_size为16，4卡分布式
-    ```bash
-    python examples/test_infer.py --device nvidia --model=/models/9G7B_MHA/ --backend=cpp --tp=4 --batch-size=16
-    ```
-
-    - PP=2 示例：
-
-      在两个终端中分别启动 stage 0 和 stage 1。两个进程的模型、并行和缓存参数必须保持一致。
-
-      ```bash
-      # Terminal 1: stage 0 / coordinator (--node-rank=0)
-      CUDA_VISIBLE_DEVICES=0 python examples/test_infer.py --device=nvidia --model=<path/to/model> --tp=1 --pp=2 --node-rank=0 --master-addr=127.0.0.1 --master-port=29500 --enable-paged-attn --attn=flash-attn --num-blocks=128
-
-      # Terminal 2: stage 1 / worker
-      CUDA_VISIBLE_DEVICES=1 python examples/test_infer.py --device=nvidia --model=<path/to/model> --tp=1 --pp=2 --node-rank=1 --master-addr=127.0.0.1 --master-port=29500 --enable-paged-attn --attn=flash-attn --num-blocks=128
-      ```
-
-      跨节点运行时，每个节点的命令中的 `--master-addr` 和 `--master-port` 设置为 stage 0 节点的 IP 地址和通信端口。
-
-
-  - 推理服务测试
-    - 启动推理服务
-      ```bash
-      python python/infinilm/server/inference_server.py --device [cpu | nvidia | qy | metax | moore | iluvatar | ali | cambricon | hygon] --model=<path/to/model-dir> --max-new-tokens=MAX_TOKENS --max-batch-size=MAX_BATCH --tp=NDEV --temperature=TEMP --top-p=TOP_P --top-k=TOP_K --host=HOST --port=PORT
-      ```
-
-    - 单卡示例：
-      ```bash
-      CUDA_VISIBLE_DEVICES=0 python python/infinilm/server/inference_server.py --device nvidia --model=/models/9G7B_MHA/ --max-new-tokens=100 --max-batch-size=32 --tp=1 --temperature=1.0 --top-p=0.8 --top-k=1
-      ```
-
-    - 多卡分布式示例：
-      ```bash
-      CUDA_VISIBLE_DEVICES=0,1,2,3 python python/infinilm/server/inference_server.py --device nvidia --model=/models/9G7B_MHA/ --max-new-tokens=100 --max-batch-size=32 --tp=4 --temperature=1.0 --top-p=0.8 --top-k=1
-      ```
-
-    - 使用paged attention, flash attention后端，cuda graph等功能：
-      ```bash
-      CUDA_VISIBLE_DEVICES=0,1,2,3 python python/infinilm/server/inference_server.py --device nvidia --model=/models/9G7B_MHA/ --enable-paged-attn --attn=flash-attn --enable-graph
-      ```
-
-    - PP=2 推理服务示例：
-
-      只有 stage 0 启动 HTTP 服务。两个进程使用相同的 PP rendezvous 地址和模型配置。
-
-      ```bash
-      # Terminal 1: stage 0 / coordinator and HTTP server
-      python python/infinilm/server/inference_server.py --device=nvidia --model=<path/to/model> --tp=1 --pp=2 --node-rank=0 --master-addr=<HOST.IP> --master-port=29500 --enable-paged-attn --attn=flash-attn --num-blocks=128 --max-batch-size=32 --port=8000
-
-      # Terminal 2: stage 1 / worker
-      python python/infinilm/server/inference_server.py --device=nvidia --model=<path/to/model> --tp=1 --pp=2 --node-rank=1 --master-addr=<HOST.IP> --master-port=29500 --enable-paged-attn --attn=flash-attn --num-blocks=128 --max-batch-size=32 --port=8000
-      ```
-
-    - 测试推理服务性能：
-      ```bash
-      python scripts/test_perf.py --verbose
-      ```
-
-    - 单请求推理服务测试
-      ```bash
-      python test/service/request.py --content="text:Image 1:" --content="image_url:xxx.jpg" --content="text:Image 2:" --content="image_url:xxxx.jpg" --content="text:Compare the 2 images."
-      ```
-
-  - 运行推理基准测试（C-Eval/MMLU）
-
-    ```bash
-    python test/bench/test_benchmark.py --device [cpu | nvidia | qy | metax | moore | iluvatar | ali | cambricon | hygon] --model <path/to/model_dir> --bench {ceval|mmlu} [--backend cpp] [--tp N] [--subject SUBJECT] [--num-samples N] [--max-new-tokens N] [--output-csv PATH] [--cache-dir PATH]
-    ```
-
-    - 参数说明：
-      - `--subject`: 指定科目，支持单个科目、多个科目（逗号分隔）或 `all`（默认值，加载全部科目）
-      - `--output-csv`: 可选，指定CSV输出文件路径。如未指定则不生成CSV文件。CSV包含每个科目的结果和总体结果
-      - `--cache-dir`: 可选，指定数据集缓存目录的父目录。应指向包含 `ceval___ceval-exam` 和 `cais___mmlu` 等数据集子目录的父目录（例如 `~/.cache/huggingface/datasets/`）。设置后脚本优先使用本地 CSV（`pandas.read_csv`）离线加载数据，避免 `load_dataset` 的网络请求
-
-    - C-Eval示例：
-      - 单个科目：
-        ```bash
-        python test/bench/test_benchmark.py --device nvidia /models/9G7B_MHA --bench ceval --subject middle_school_mathematics --num-samples 100 --backend cpp --tp 1
-        ```
-      - 多个科目（逗号分隔）：
-        ```bash
-        python test/bench/test_benchmark.py --device nvidia /models/9G7B_MHA --bench ceval --subject middle_school_mathematics,high_school_physics --backend cpp --tp 1 --output-csv results.csv
-        ```
-      - 全部科目并输出CSV：
-        ```bash
-        python test/bench/test_benchmark.py --device nvidia /models/9G7B_MHA --bench ceval --subject all --backend cpp --tp 1 --output-csv results.csv
-        ```
-      - 使用缓存目录加速加载：
-        ```bash
-        python test/bench/test_benchmark.py --device nvidia /models/9G7B_MHA --bench ceval --subject middle_school_mathematics --backend cpp --tp 1 --cache-dir ~/.cache/huggingface/datasets/
-        ```
-        > 注意：`--cache-dir` 应指向包含 `ceval___ceval-exam` 和 `cais___mmlu` 等数据集子目录的父目录，而不是直接指向这些子目录
-
-    - MMLU示例：
-      - 单个科目：
-        ```bash
-        python test/bench/test_benchmark.py --device nvidia /models/9G7B_MHA --bench mmlu --subject abstract_algebra --backend cpp --tp 1
-        ```
-      - 多个科目（逗号分隔）：
-        ```bash
-        python test/bench/test_benchmark.py --device nvidia /models/9G7B_MHA --bench mmlu --subject abstract_algebra,anatomy,astronomy --backend cpp --tp 1 --output-csv results.csv
-        ```
-      - 使用缓存目录加速加载：
-        ```bash
-        python test/bench/test_benchmark.py --device nvidia /models/9G7B_MHA --bench mmlu --subject abstract_algebra --backend cpp --tp 1 --cache-dir ~/.cache/huggingface/datasets/
-        ```
-        > 注意：`--cache-dir` 应指向包含 `ceval___ceval-exam` 和 `cais___mmlu` 等数据集子目录的父目录，而不是直接指向这些子目录
-
-  - 试验中功能
-    - Warm Up
-      ```bash
-      python examples/bench.py --device nvidia --model=<model-path> --warmup
-      ```
-    - Paged Attention
-      ```bash
-      python examples/bench.py --device nvidia --model=<model-path> --enable-paged-attn
-      ```
-    - CUDA Graph
-      ```bash
-      python examples/bench.py --device nvidia --model=<model-path> --enable-paged-attn --enable-graph
-      ```
-    - 选择attention后端 (使用flash attention后端需要先在InfiniCore完成相关配置和编译)
-      ```bash
-      python examples/bench.py --device nvidia --model=<model-path> --enable-paged-attn [--attn=default | --attn=flash-attn]
-      ```
+InfiniLM is the high-level inference engine in the InfiniTensor stack. It owns
+model execution, runtime management, tensor abstractions, and the Python API.
+The lower-level runtime, operator, and collective APIs are provided by
+[InfiniRT](https://github.com/InfiniTensor/InfiniRT),
+[InfiniOps](https://github.com/InfiniTensor/InfiniOps), and
+[InfiniCCL](https://github.com/InfiniTensor/InfiniCCL), respectively.
+
+[InfiniCore](https://github.com/InfiniTensor/InfiniCore) pins those three
+projects as submodules. InfiniLM no longer consumes runtime or Python-package
+artifacts built by that separate repository; it builds and packages its own
+`infinicore` Python module.
+
+## Build
+
+Clone both repositories with their submodules:
+
+```shell
+git clone --recurse-submodules https://github.com/InfiniTensor/InfiniCore.git
+git clone --recurse-submodules https://github.com/InfiniTensor/InfiniLM.git
+```
+
+From InfiniLM, build the NVIDIA dependency stack pinned by the InfiniCore
+checkout. The default operator set is the set required by InfiniLM:
+
+```shell
+cd InfiniLM
+python3 scripts/build_infini_stack.py \
+  --infinicore-root ../InfiniCore \
+  --cuda-arch sm_80 \
+  --jobs 16 \
+  --test
+export INFINI_ROOT="$PWD/build/integration/nvidia/prefix"
+export LD_LIBRARY_PATH="$INFINI_ROOT/lib:${LD_LIBRARY_PATH:-}"
+```
+
+Then build and install InfiniLM:
+
+```shell
+python3 -m pip install . --no-build-isolation
+```
+
+Current migration validation is limited to NVIDIA A100 and dense,
+non-quantized Qwen3 configurations without linear bias. Qwen3-0.6B has passed
+static and paged attention, eager and graph execution, single-request and
+batch-2 inference, greedy and non-greedy sampling, TP2, PP2, and combined
+TP2+PP2. Paged attention was validated with the default 256-token block size.
+
+Only `qwen3` can be instantiated by the modern model factory. Other model
+families, quantized models, and biased Qwen3 configurations remain gated.
+Other platforms and custom paged-cache block sizes have not yet been
+validated.
+
+## Inference
+
+Run a single-model smoke test:
+
+```shell
+python examples/test_infer.py --device nvidia --model=/path/to/model
+```
+
+For tensor-parallel inference:
+
+```shell
+python examples/test_infer.py --device nvidia --model=/path/to/model --tp=2 --batch-size=2
+```
+
+Start the OpenAI-compatible server:
+
+```shell
+python python/infinilm/server/inference_server.py --device nvidia --model=/path/to/model --tp=1
+```
+
+Paged attention and graph execution are selected by InfiniLM arguments and are
+built as part of InfiniLM:
+
+```shell
+python examples/bench.py --device nvidia --model=/path/to/model --enable-paged-attn --enable-graph
+```
+
+## Development
+
+Format staged files with the repository formatter:
+
+```shell
+python scripts/format.py --staged
+```
+
+Run the static migration contracts with:
+
+```shell
+python -m unittest discover -s test/static -p "test_*.py"
+```
+
+## License
+
+InfiniLM is licensed under the MIT License. See [LICENSE](LICENSE).