diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml index 9611ce1..98eda55 100644 --- a/.github/workflows/CI.yml +++ b/.github/workflows/CI.yml @@ -3,7 +3,7 @@ on: pull_request: push: branches: - - '**' + - main tags: - '*' workflow_dispatch: diff --git a/Cargo.toml b/Cargo.toml index 50e4ac4..8cba2d7 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "forust-ml" -version = "0.5.0" +version = "0.7.0" edition = "2021" authors = ["James Inlow "] homepage = "https://github.com/jinlow/forust" diff --git a/README.md b/README.md index 1c86083..dab9152 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ pip install forust To use in a rust project add the following to your Cargo.toml file. ```toml -forust-ml = "0.5.0" +forust-ml = "0.7.0" ``` ## Usage diff --git a/agent/feat/goss/plan.md b/agent/feat/goss/plan.md deleted file mode 100644 index a33d264..0000000 --- a/agent/feat/goss/plan.md +++ /dev/null @@ -1,271 +0,0 @@ -# Forust GOSS Implementation Plan - -Date: 2026-10-06. Branch: `feat/goss` (based on `feat/optimize`). - -Goal: make Gradient-based One-Side Sampling (GOSS) a supported, documented, -user-selectable setting, and show with benchmarks that it keeps eval logloss -close to the no-sampling result while reducing training time. - -## Starting point - -GOSS is already partly implemented, but not documented or tested: - -- `SampleMethod::Goss` and `GossSampler` in `src/sampler.rs`. -- `top_rate` and `other_rate` on `GradientBooster`, wired through - `sample_index` in `src/gradientbooster.rs`. -- Python: `GradientBooster(sample_method="goss", top_rate=..., other_rate=...)`. - -Problems with the current version: - -| # | Problem | Where | -| --- | --- | --- | -| 1 | Default rates are swapped: booster and Python use `top_rate=0.1, other_rate=0.2`; LightGBM and `GossSampler::default` use `0.2 / 0.1`. | `default_top_rate`, `default_other_rate`, `GradientBooster::default`, `py-forust/forust/__init__.py` | -| 2 | No warm-up: LightGBM skips GOSS for the first `int(1 / learning_rate)` iterations. | `fit_trees` | -| 3 | Rows are ranked by `\|g\|`; LightGBM ranks by `\|g * h\|`. | `GossSampler::sample` | -| 4 | Full `O(n log n)` sort of every row each iteration. | `GossSampler::sample` | -| 5 | Small-gradient rows are sampled independently at random, so the count varies; LightGBM samples an exact count. | `GossSampler::sample` | -| 6 | No validation of `top_rate` / `other_rate`; Python silently turns unknown `sample_method` strings into `"Random"`. | `validate_parameters`, `SAMPLE_METHODS.get(..., "Random")` | -| 7 | The only test is `assert True`. | `test_goss_sampling_method` | -| 8 | Not mentioned in the README or docs. | `README.md`, `py-forust/docs` | - -## Decisions (agreed) - -- Match LightGBM: warm-up, `|g * h|` ranking, defaults `top_rate=0.2`, - `other_rate=0.1`, exact-count sampling, multiplier `(n - top_k) / other_k`. -- Invalid settings raise errors (Rust `ForustError`, Python `ValueError`). -- Benchmarks use synthetic data only (`make_perf_data.py`), one dataset per size. -- Accuracy is measured with eval logloss only (no AUC), matching XGBoost and - LightGBM reporting. -- LightGBM (with and without GOSS) and XGBoost `hist` are reference columns. -- **Baseline is random row sampling with `subsample=0.8`** (`random80`), the - common practical setting; no sampling is reported for information only. - XGBoost uses `subsample=0.8` and LightGBM `bagging_fraction=0.8`, - `bagging_freq=1`; LightGBM GOSS is compared with LightGBM bagging. -- Sampled runs use 5 model seeds (3 for the 1M early-stopping and 1-thread runs). - -## Library changes - -### 1. Sampler (`src/sampler.rs`) - -Rewrite `GossSampler::sample` to follow LightGBM's `GOSSStrategy::Helper`: - -1. Score each row by `|grad[i] * hess[i]|`. -2. `top_k = max(1, floor(n * top_rate))`, `other_k = floor(n * other_rate)`. -3. Find the `top_k`-th largest score with `select_nth_unstable_by` on a copy of - the scores (`O(n)`, no full sort). Rows with score `>= threshold` are kept - unchanged. -4. Walk the remaining rows in order and keep each with probability - `rest_need / rest_all`, which yields exactly `other_k` rows. Scale the kept - rows' `grad` and `hess` by `(n - top_k) / other_k`. -5. Return the chosen row indices in ascending order, so `tree.fit` sorts cheaply. - -Fix the index handling: the sampler must return values from `index`, not -positions in it. (They are equal today because `data.index` is `0..rows`.) - -### 2. Booster (`src/gradientbooster.rs`) - -- Defaults: `default_top_rate() = 0.2`, `default_other_rate() = 0.1`, and the - same in `GradientBooster::default()`. -- Warm-up: in `fit_trees`, use the full index for iterations - `i < (1.0 / learning_rate) as usize` when `sample_method == Goss`; pass - `SampleMethod::None` to `tree.fit` for those iterations so it skips the sort. -- Validation in `validate_parameters`, when `sample_method == Goss`: - - `0 < top_rate`, `0 < other_rate`, `top_rate + other_rate <= 1`; - - `subsample == 1.0`, matching LightGBM's "Cannot use bagging in GOSS". -- Rust doc comments for `sample_method`, `top_rate`, `other_rate` updated. - -Saved models are unaffected: they store `top_rate` and `other_rate` -explicitly. The changed serde defaults only apply to JSON from versions -before those fields existed, and they only matter if that model is refit. - -### 3. Python (`py-forust`) - -- Defaults `top_rate=0.2`, `other_rate=0.1` in `GradientBooster.__init__`. -- Unknown `sample_method` raises `ValueError` listing valid options. -- Keep the existing rule that `subsample < 1` with no `sample_method` means - `"random"`; `sample_method="goss"` with `subsample < 1` raises `ValueError`. -- Docstrings for `sample_method`, `top_rate`, `other_rate`, explaining the - warm-up and that GOSS ignores `subsample`. -- Update `.pyi` stubs if they list these parameters. - -### 4. Docs - -- `README.md` and `py-forust/README.md`: short GOSS section with an example. -- `py-forust/docs`: parameter descriptions picked up from the docstrings. - -### 5. Contiguous row subset (added after first benchmarks) - -As LightGBM does when `top_rate + other_rate <= 0.5` (and for bagging), copy the -sampled rows' binned values, gradients, and hessians into reused contiguous -buffers (`RowSubset` in `src/sampler.rs`) and fit the tree on rows `0..m`. -Applies to GOSS and random sampling whenever at most 50% of rows are sampled. -Trees are byte-identical to fitting on the full data with the sampled index. - -Measured at 1M x 200, 8 threads, GOSS 0.2/0.1, steady state per tree: tree -111.1 ms -> 89.2 ms, plus an 11.6 ms copy (bandwidth-bound). Row selection -(15 ms, serial) is now the largest sampling cost. - -## Tests - -Rust (`src/sampler.rs`, `src/gradientbooster.rs`): - -- Sampler returns exactly `top_k + other_k` rows for distinct scores. -- All `top_k` highest-`|g * h|` rows are chosen and left unscaled. -- Sampled small-gradient rows are scaled by `(n - top_k) / other_k`; unsampled - rows are untouched. -- Returned indices are sorted and come from `index`. -- Warm-up: the first `int(1 / learning_rate)` trees train on all rows (check - root node cover/hessian sum equals the full-data value). -- Same seed gives identical models; different seeds differ. -- Validation errors for bad rates and for `subsample < 1` with GOSS. - -Python (`py-forust/tests/test_booster.py`): - -- Replace `assert True` with a check that GOSS eval logloss is within a loose - tolerance of no sampling on the Titanic data. -- `ValueError` for an unknown `sample_method`, bad rates, and GOSS with - `subsample < 1`. -- JSON save/load round-trip keeps `sample_method`, `top_rate`, `other_rate`. -- Full Python suite, including XGBoost parity checks, still passes. The default - is `sample_method=None`, so parity tests should be unaffected. - -## Benchmark - -### Tooling changes - -- `examples/perf_phases.rs` - - New flags: `--sample-method none|random|goss`, `--top-rate`, - `--other-rate`, `--subsample`, `--seed` (both `fit` and `phases` modes). - - `phases` mode calls the sampler (with the same warm-up rule as `fit`) and - records a new `sample` time in `phases_s`, plus rows used per tree. - - Report eval logloss in both modes (already present). -- `scripts/perf_sweep.py`: no structural change; GOSS flags pass through after - `--`, and `--label` tags each sampling setting. -- `scripts/perf_reference.py` (generalizes `perf_xgboost.py`): - `--library xgboost|lightgbm`, `--goss`, `--top-rate`, `--other-rate`, same - data, threads, depth, bins, learning rate and iterations; reports fit time - and eval logloss. LightGBM uses `data_sample_strategy="goss"`. - Keep `perf_xgboost.py` as a thin wrapper, or replace it if nothing else uses it. -- `scripts/perf_goss.py`: drives the runs below, writes - `agent/feat/goss/results.jsonl`, and prints the pass/fail table. - -### Environment - -The previously used environment -`/home/azureuser/localfiles/uv-environments/forust` no longer exists. Recreate -it with `uv`, install `py-forust` (maturin, release), `xgboost`, `lightgbm`, -`scikit-learn`, `numpy`. - -### Data - -```sh -PY=/home/azureuser/localfiles/uv-environments/forust/bin/python -$PY scripts/make_perf_data.py --out /tmp/forust-perf/w200 --rows 250000 --eval-rows 62500 --cols 200 -$PY scripts/make_perf_data.py --out /tmp/forust-perf/w500 --rows 100000 --eval-rows 25000 --cols 500 -$PY scripts/make_perf_data.py --out /tmp/forust-perf/w200-1m --rows 1000000 --eval-rows 250000 --cols 200 -``` - -### Sampling settings compared - -| Label | Setting | -| --- | --- | -| `none` | `sample_method=None` (information only) | -| `random80` | `random`, `subsample=0.8` (baseline) | -| `random30` | `random`, `subsample=0.3` (same row budget as default GOSS) | -| `goss` | `goss`, `top_rate=0.2`, `other_rate=0.1` | -| `goss-small` | `goss`, `top_rate=0.1`, `other_rate=0.1` | - -`random30` shows whether choosing rows by gradient beats choosing them at -random with the same number of rows. - -### Runs - -1. **Per-tree speed (phases):** `perf_sweep.py` on w200 (25k, 100k, 250k - rows), w500 (100k), w200-1m (1M); depth 5 and 8; 8 threads; 3 repeats, - median reported. Shows tree, sample, predict, eval and grad/hess time per - setting. - - ```sh - cargo build --release --example perf_phases - for s in "none" "random --subsample 0.3" "goss" "goss --top-rate 0.1 --other-rate 0.1"; do - python3 scripts/perf_sweep.py --data /tmp/forust-perf/w200 \ - --rows 25000 100000 250000 --threads 8 --repeats 3 --label "$s" \ - --out agent/feat/goss/results.jsonl -- --sample-method $s - done - ``` - -2. **Full fit, fixed iterations:** `perf_phases --mode fit`, 1 eval set, - learning rate 0.1, 100 iterations (300 at 100k for a longer run), 8 threads, - seeds 0-4 for sampled settings. Records total fit time and eval logloss. -3. **Full fit, early stopping:** same as run 2 with `early_stopping_rounds=20` - and up to 1,000 iterations. Records iterations used, time, and best eval - logloss. GOSS may need more trees, so this checks total time to reach the - same quality. -4. **References:** `perf_reference.py` with XGBoost `hist`, LightGBM `gbdt`, - and LightGBM GOSS on the same data, threads, and settings as run 2. -5. **Thread check:** run 2 on w200-1m at 1 and 8 threads, to see whether the - GOSS gain differs between serial and parallel training. -6. **Profile:** release build with debug info, `perf record` of - `perf_phases` on w200 100k for `none` and `goss`, to confirm histogram time - drops and the sampler stays a small share. - - ```sh - CARGO_PROFILE_RELEASE_DEBUG=true cargo build --release --example perf_phases - for m in none goss; do - RAYON_NUM_THREADS=8 perf record -F 999 -o /tmp/forust-perf/goss-$m.data -- \ - target/release/examples/perf_phases --rows 100000 --iterations 60 --sample-method $m - perf report -i /tmp/forust-perf/goss-$m.data --stdio --no-children | head -60 - done - ``` - -7. **Sampler micro-benchmark:** add a Criterion case to - `benches/forust_benchmarks.rs` for `GossSampler::sample` at 100k and 1M rows. - -### Pass/fail rule - -For default GOSS (`top_rate=0.2`, `other_rate=0.1`) against `random80`, same -configuration: - -- **Accuracy:** mean eval logloss over 5 seeds is no more than 1% above the - baseline's mean, on every dataset, in both fixed-iteration and early-stopping runs. -- **Speed:** full `fit` (with an eval set) on w200-1m at 8 threads is at least - 1.5x faster at a fixed number of iterations. - -Also reported, not gated: speedups at smaller sizes, `goss-small`, `random30`, -and how forust's GOSS gain compares with LightGBM's. - -Expected limits: GOSS only shrinks per-row tree work. Binning, gradients, -training-set prediction updates, and eval scoring still cover all rows, and the -first `1 / learning_rate` trees use all rows. Gains should be largest at 1M rows -and deeper trees, and small at 25k rows with 8 threads. - -### Outputs - -- `agent/feat/goss/results.jsonl`: raw records. -- `agent/feat/goss/results.md`: tables for speed, logloss (mean and standard - deviation over seeds), references, profile summary, and pass/fail. - -## Order of work - -1. Recreate the Python environment; build; run the current Python and Rust - tests to confirm a clean baseline. -2. Benchmark tooling (`perf_phases` flags, `perf_reference.py`, - `perf_goss.py`), and record the current, unfixed GOSS as a reference point. -3. Sampler rewrite and booster changes, with Rust tests. -4. Python changes, tests, and docs. -5. Full benchmark runs and `results.md`. -6. Final check: `cargo test`, `cargo clippy`, full Python suite. - -Each step is a separate commit on `feat/goss`. - -## Open points for review - -- **Ties at the threshold:** LightGBM keeps every row with score `>= threshold`, - so ties can push the count slightly above `top_k`. Plan: do the same. -- **Parallel sampling (done):** like LightGBM, rows are sampled in blocks, each - with its own threshold and random stream. Blocks have a fixed target size - (32,768 rows) rather than one per thread, so models don't depend on the thread - count. 1M x 200, 8 threads: 15-17 ms -> 4.8 ms per tree. -- **Changed GOSS results:** the new sampler uses random numbers differently, so - existing GOSS users get different models after upgrading. GOSS was - undocumented, so this seems acceptable, but it should be in the release notes. diff --git a/agent/feat/goss/results.jsonl b/agent/feat/goss/results.jsonl deleted file mode 100644 index 21c6cd3..0000000 --- a/agent/feat/goss/results.jsonl +++ /dev/null @@ -1,262 +0,0 @@ -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 3.593605522, "fit_s": 3.593605522, "eval_logloss": 0.3279430393472949, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 3.46668076, "fit_s": 3.46668076, "eval_logloss": 0.3279430393472949, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 3.336159626, "fit_s": 3.336159626, "eval_logloss": 0.3279430393472949, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 3.210413023, "fit_s": 3.210413023, "eval_logloss": 0.3250719230959926, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 3.214293144, "fit_s": 3.214293144, "eval_logloss": 0.3239106125967988, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 3.099117459, "fit_s": 3.099117459, "eval_logloss": 0.32370341939737757, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.997139528, "fit_s": 2.997139528, "eval_logloss": 0.32279502061223336, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 3.337804389, "fit_s": 3.337804389, "eval_logloss": 0.32445591377946015, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.136096094, "fit_s": 2.136096094, "eval_logloss": 0.31517956027489585, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.236246528, "fit_s": 2.236246528, "eval_logloss": 0.3196099507342123, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.426775998, "fit_s": 2.426775998, "eval_logloss": 0.3177323611700004, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.293265885, "fit_s": 2.293265885, "eval_logloss": 0.3179977966965715, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.164323266, "fit_s": 2.164323266, "eval_logloss": 0.3175757356432682, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.278009526, "fit_s": 2.278009526, "eval_logloss": 0.32349541186951414, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.372516426, "fit_s": 2.372516426, "eval_logloss": 0.3244407370751807, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.306061478, "fit_s": 2.306061478, "eval_logloss": 0.3250261358288657, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.331753388, "fit_s": 2.331753388, "eval_logloss": 0.3254595078457059, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.235513636, "fit_s": 2.235513636, "eval_logloss": 0.32460559370333175, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.07890026, "fit_s": 2.07890026, "eval_logloss": 0.3246034065158998, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.063504204, "fit_s": 2.063504204, "eval_logloss": 0.3260894837431421, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.035589836, "fit_s": 2.035589836, "eval_logloss": 0.32519906641560536, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 1.981193532, "fit_s": 1.981193532, "eval_logloss": 0.3254079115097443, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 2.122651039, "fit_s": 2.122651039, "eval_logloss": 0.3266858774532134, "trees": 100} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.24042098700010683, "fit_s": 2.6870486239995444, "trees": 100, "eval_logloss": 0.3253629614713043, "total_s": 2.927469610999651} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.2407604950003588, "fit_s": 2.5579295190000266, "trees": 100, "eval_logloss": 0.32630142824102193, "total_s": 2.7986900140003854} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.2330286510004953, "fit_s": 2.2105740730003163, "trees": 100, "eval_logloss": 0.32609352653361856, "total_s": 2.4436027240008116} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.2412541189996773, "fit_s": 2.613948498999889, "trees": 100, "eval_logloss": 0.3248995416274294, "total_s": 2.8552026179995664} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.23342226000022492, "fit_s": 2.552980409999691, "trees": 100, "eval_logloss": 0.3276741140280664, "total_s": 2.786402669999916} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 1.007391772000119, "fit_s": 1.4095075440000073, "trees": 100, "eval_logloss": 0.32083052942492063, "total_s": 2.4168993160001264} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.9726882670001942, "fit_s": 1.6301238280002508, "trees": 100, "eval_logloss": 0.3203793542488186, "total_s": 2.602812095000445} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.8893470740003977, "fit_s": 1.5750842140005261, "trees": 100, "eval_logloss": 0.3266078277680473, "total_s": 2.464431288000924} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.9186651720001464, "fit_s": 1.2096716859996377, "trees": 100, "eval_logloss": 0.32399914582391104, "total_s": 2.128336857999784} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.9380579660000876, "fit_s": 1.4679353349993107, "trees": 100, "eval_logloss": 0.3256932657083108, "total_s": 2.4059933009993983} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.9562564360003307, "fit_s": 1.170508449999943, "trees": 100, "eval_logloss": 0.32151599015735005, "total_s": 2.1267648860002737} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.9501961539999684, "fit_s": 0.847603975000311, "trees": 100, "eval_logloss": 0.32146290066673466, "total_s": 1.7978001290002794} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.9392554329997438, "fit_s": 1.218491090000498, "trees": 100, "eval_logloss": 0.32573009340034753, "total_s": 2.1577465230002417} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.9921164040006261, "fit_s": 1.0851809890000368, "trees": 100, "eval_logloss": 0.32199885724058375, "total_s": 2.077297393000663} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 1.002787420000459, "fit_s": 1.0693695750005645, "trees": 100, "eval_logloss": 0.3258253700494844, "total_s": 2.0721569950010235} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 7.93662816, "fit_s": 7.93662816, "eval_logloss": 0.3271226778537963, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 7.434188917, "fit_s": 7.434188917, "eval_logloss": 0.3271226778537963, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 7.913627773, "fit_s": 7.913627773, "eval_logloss": 0.3271226778537963, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 7.129227043, "fit_s": 7.129227043, "eval_logloss": 0.3233034940191527, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 6.948304863, "fit_s": 6.948304863, "eval_logloss": 0.325543884404929, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 6.891115721, "fit_s": 6.891115721, "eval_logloss": 0.32407862785277697, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 9.720273247, "fit_s": 9.720273247, "eval_logloss": 0.32615818150866965, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 7.168206587, "fit_s": 7.168206587, "eval_logloss": 0.3233504115234871, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.203027105, "fit_s": 4.203027105, "eval_logloss": 0.31516087068797366, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.350348081, "fit_s": 4.350348081, "eval_logloss": 0.3182334917435293, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.224542214, "fit_s": 4.224542214, "eval_logloss": 0.3207262836789986, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.110253477, "fit_s": 4.110253477, "eval_logloss": 0.31817014145277084, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.46998997, "fit_s": 4.46998997, "eval_logloss": 0.3190479440783559, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.369379109, "fit_s": 4.369379109, "eval_logloss": 0.323063130871686, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.372423132, "fit_s": 4.372423132, "eval_logloss": 0.32559758433538283, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.673573495, "fit_s": 4.673573495, "eval_logloss": 0.321961914675259, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.420581131, "fit_s": 4.420581131, "eval_logloss": 0.3219676164547349, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 5.069178998, "fit_s": 5.069178998, "eval_logloss": 0.32088884452601046, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.140527511, "fit_s": 4.140527511, "eval_logloss": 0.3205967812012385, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.311403868, "fit_s": 4.311403868, "eval_logloss": 0.32034569629285964, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 3.938277586, "fit_s": 3.938277586, "eval_logloss": 0.3186410121771903, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 3.900361079, "fit_s": 3.900361079, "eval_logloss": 0.3208476471624061, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 3.944793845, "fit_s": 3.944793845, "eval_logloss": 0.3208644204342995, "trees": 100} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.5781811880005989, "fit_s": 5.210112167000261, "trees": 100, "eval_logloss": 0.32384573768068853, "total_s": 5.78829335500086} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.5705833399997573, "fit_s": 4.9892924069999935, "trees": 100, "eval_logloss": 0.32557090800949184, "total_s": 5.559875746999751} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.5311234000000695, "fit_s": 5.2395363800005725, "trees": 100, "eval_logloss": 0.3224499665875137, "total_s": 5.770659780000642} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.5847138279996216, "fit_s": 5.178630957000678, "trees": 100, "eval_logloss": 0.3262126185927168, "total_s": 5.7633447850002995} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.5658206460002475, "fit_s": 5.293224432999523, "trees": 100, "eval_logloss": 0.3249362667952329, "total_s": 5.85904507899977} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.001307262999944, "fit_s": 3.291304536000098, "trees": 100, "eval_logloss": 0.3264300459713206, "total_s": 5.292611799000042} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.1987791420006033, "fit_s": 3.1278743200000463, "trees": 100, "eval_logloss": 0.32370068214622005, "total_s": 5.32665346200065} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.1485966589998498, "fit_s": 3.264858500000628, "trees": 100, "eval_logloss": 0.3239496838906831, "total_s": 5.413455159000478} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.0633529059996363, "fit_s": 3.3980047220002234, "trees": 100, "eval_logloss": 0.32581856567032574, "total_s": 5.46135762799986} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.062892687000385, "fit_s": 3.324718786999256, "trees": 100, "eval_logloss": 0.32221689161284384, "total_s": 5.387611473999641} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.2339999830001034, "fit_s": 2.0935470509994047, "trees": 100, "eval_logloss": 0.32135287318881767, "total_s": 4.327547033999508} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.002670812999895, "fit_s": 2.394806946999779, "trees": 100, "eval_logloss": 0.3190089301276555, "total_s": 4.397477759999674} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.185778025000218, "fit_s": 2.340053447000173, "trees": 100, "eval_logloss": 0.3245975712260753, "total_s": 4.525831472000391} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.377188497000134, "fit_s": 2.1718878279998535, "trees": 100, "eval_logloss": 0.3242531197712971, "total_s": 4.549076324999987} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.0590007399996466, "fit_s": 2.343909801999871, "trees": 100, "eval_logloss": 0.32072044941738503, "total_s": 4.402910541999518} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 7.7107849040000005, "fit_s": 7.7107849040000005, "eval_logloss": 0.36343754917898585, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 7.922348519, "fit_s": 7.922348519, "eval_logloss": 0.36343754917898585, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 8.424828211, "fit_s": 8.424828211, "eval_logloss": 0.36343754917898585, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 7.170869195, "fit_s": 7.170869195, "eval_logloss": 0.3622866741086639, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 7.688866753, "fit_s": 7.688866753, "eval_logloss": 0.3653813035099335, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 7.309070303, "fit_s": 7.309070303, "eval_logloss": 0.3612907411897185, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 7.195519702, "fit_s": 7.195519702, "eval_logloss": 0.3639312153243123, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 7.198334745, "fit_s": 7.198334745, "eval_logloss": 0.3646783865735538, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.624187054, "fit_s": 4.624187054, "eval_logloss": 0.3609852136608519, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.9511052, "fit_s": 4.9511052, "eval_logloss": 0.3636081132729814, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.985609432, "fit_s": 4.985609432, "eval_logloss": 0.3592825231484779, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 5.299777423, "fit_s": 5.299777423, "eval_logloss": 0.36007076435176333, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.7942414079999995, "fit_s": 4.7942414079999995, "eval_logloss": 0.36147633298648946, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 5.117295285, "fit_s": 5.117295285, "eval_logloss": 0.3616094241108412, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.758365569, "fit_s": 4.758365569, "eval_logloss": 0.3646106372783539, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.880078969, "fit_s": 4.880078969, "eval_logloss": 0.36511046594590063, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.8078137850000005, "fit_s": 4.8078137850000005, "eval_logloss": 0.36159509062656786, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.906774819, "fit_s": 4.906774819, "eval_logloss": 0.36351516327481426, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.703559646, "fit_s": 4.703559646, "eval_logloss": 0.3760214833071641, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.467952351, "fit_s": 4.467952351, "eval_logloss": 0.37630932388764865, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.225029516, "fit_s": 4.225029516, "eval_logloss": 0.3778638884385997, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.593003458, "fit_s": 4.593003458, "eval_logloss": 0.3751184136108542, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 4.2196187179999995, "fit_s": 4.2196187179999995, "eval_logloss": 0.3762480064900631, "trees": 100} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.5798680139996577, "fit_s": 7.214540799000133, "trees": 100, "eval_logloss": 0.3620581007976085, "total_s": 7.794408812999791} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.5782740770000601, "fit_s": 7.364346542000021, "trees": 100, "eval_logloss": 0.3626524955233932, "total_s": 7.942620619000081} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.5547754529998201, "fit_s": 6.860984310000276, "trees": 100, "eval_logloss": 0.36317447029836475, "total_s": 7.415759763000096} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.5096094890004679, "fit_s": 6.960733826000251, "trees": 100, "eval_logloss": 0.3620776344587654, "total_s": 7.470343315000719} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 0.5953174170008424, "fit_s": 7.6892255350003325, "trees": 100, "eval_logloss": 0.36546729167729614, "total_s": 8.284542952001175} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.732605971000339, "fit_s": 3.4060784599996623, "trees": 100, "eval_logloss": 0.36166820891374957, "total_s": 6.1386844310000015} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.3162701569999626, "fit_s": 3.5740105389995733, "trees": 100, "eval_logloss": 0.3613254067092303, "total_s": 5.890280695999536} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.4438762970003154, "fit_s": 3.8070311940000465, "trees": 100, "eval_logloss": 0.3643269235720807, "total_s": 6.250907491000362} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.4004841669993766, "fit_s": 3.3783016600000337, "trees": 100, "eval_logloss": 0.3634053241568907, "total_s": 5.77878582699941} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.2501654659999986, "fit_s": 3.32407914399937, "trees": 100, "eval_logloss": 0.3651219888490811, "total_s": 5.5742446099993685} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.4368822169999476, "fit_s": 2.0783229750004466, "trees": 100, "eval_logloss": 0.36159283188379276, "total_s": 4.515205192000394} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.2637090260004697, "fit_s": 2.2431876159998865, "trees": 100, "eval_logloss": 0.36287022127215124, "total_s": 4.506896642000356} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.4229147460000604, "fit_s": 2.2885190329998295, "trees": 100, "eval_logloss": 0.3636529047592121, "total_s": 4.71143377899989} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.3926114460000463, "fit_s": 2.0646372190003603, "trees": 100, "eval_logloss": 0.3623269041575737, "total_s": 4.457248665000407} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.3958078109999406, "fit_s": 2.219008828999904, "trees": 100, "eval_logloss": 0.36221347429793416, "total_s": 4.6148166399998445} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 26.123446054, "fit_s": 26.123446054, "eval_logloss": 0.3363535089725551, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 26.737220657, "fit_s": 26.737220657, "eval_logloss": 0.3363535089725551, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 26.835458003, "fit_s": 26.835458003, "eval_logloss": 0.3363535089725551, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 24.101287326, "fit_s": 24.101287326, "eval_logloss": 0.3357783701509772, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 23.797783207, "fit_s": 23.797783207, "eval_logloss": 0.3351172331354733, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 23.374187447, "fit_s": 23.374187447, "eval_logloss": 0.33591420345085704, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 24.305714387, "fit_s": 24.305714387, "eval_logloss": 0.33664636129984254, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 24.719235650999998, "fit_s": 24.719235650999998, "eval_logloss": 0.3375857529974142, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 14.428165783, "fit_s": 14.428165783, "eval_logloss": 0.3308684601234352, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 14.391695589, "fit_s": 14.391695589, "eval_logloss": 0.33411006269161714, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 14.807845429, "fit_s": 14.807845429, "eval_logloss": 0.33212817150493684, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 14.927669322, "fit_s": 14.927669322, "eval_logloss": 0.33117607846624875, "trees": 100} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 15.557784133, "fit_s": 15.557784133, "eval_logloss": 0.3332984541985553, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 15.192571244, "fit_s": 15.192571244, "eval_logloss": 0.335759070475801, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 18.23989082, "fit_s": 18.23989082, "eval_logloss": 0.3353976828778981, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 16.161137672, "fit_s": 16.161137672, "eval_logloss": 0.3353008510444272, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 15.346937359, "fit_s": 15.346937359, "eval_logloss": 0.3349725406482815, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 15.338017702, "fit_s": 15.338017702, "eval_logloss": 0.3361738754091186, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "total_s": 12.872274782, "fit_s": 12.872274782, "eval_logloss": 0.3319110869888643, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "total_s": 13.079447336, "fit_s": 13.079447336, "eval_logloss": 0.3300436844285364, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "total_s": 13.419614425, "fit_s": 13.419614425, "eval_logloss": 0.3314027271171573, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "total_s": 12.826445816, "fit_s": 12.826445816, "eval_logloss": 0.33103828495023163, "trees": 100} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "total_s": 12.57496972, "fit_s": 12.57496972, "eval_logloss": 0.33245731309936544, "trees": 100} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.237310247000096, "fit_s": 19.141623049999907, "trees": 100, "eval_logloss": 0.3357418310296703, "total_s": 21.378933297000003} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.204860311999255, "fit_s": 17.89846103399941, "trees": 100, "eval_logloss": 0.33677365045706, "total_s": 20.103321345998665} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.2713880100000097, "fit_s": 18.0420560320008, "trees": 100, "eval_logloss": 0.33628788940311594, "total_s": 20.31344404200081} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.1054694300000847, "fit_s": 17.84844877999967, "trees": 100, "eval_logloss": 0.33530538693842477, "total_s": 19.953918209999756} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 2.168983417000163, "fit_s": 18.856449187999715, "trees": 100, "eval_logloss": 0.33618378100251967, "total_s": 21.025432604999878} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 4.388843810999788, "fit_s": 12.022380281000551, "trees": 100, "eval_logloss": 0.33675389223707713, "total_s": 16.41122409200034} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 4.073749508999754, "fit_s": 12.203135918000044, "trees": 100, "eval_logloss": 0.3364298606534307, "total_s": 16.276885426999797} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 4.181726793000053, "fit_s": 12.31989072200031, "trees": 100, "eval_logloss": 0.335348904688963, "total_s": 16.501617515000362} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 4.0720905079997465, "fit_s": 11.992687961000229, "trees": 100, "eval_logloss": 0.3370946376716768, "total_s": 16.064778468999975} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 4.245047237000108, "fit_s": 11.627587805000076, "trees": 100, "eval_logloss": 0.3355269870996891, "total_s": 15.872635042000184} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 4.233482384999661, "fit_s": 7.897043513999961, "trees": 100, "eval_logloss": 0.3334609793107755, "total_s": 12.130525898999622} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 4.3399454529999275, "fit_s": 8.486859482, "trees": 100, "eval_logloss": 0.33468372384770495, "total_s": 12.826804934999927} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 4.049837259999549, "fit_s": 8.29018647599969, "trees": 100, "eval_logloss": 0.3339390513321975, "total_s": 12.340023735999239} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 3, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 4.979138444000455, "fit_s": 7.622369362999962, "trees": 100, "eval_logloss": 0.33515666748545553, "total_s": 12.601507807000417} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 4, "label": "fixed", "commit": "c3bf3a7", "dataset_s": 4.1855178069999965, "fit_s": 8.85051741299958, "trees": 100, "eval_logloss": 0.3345706390230095, "total_s": 13.036035219999576} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 25.626365764, "fit_s": 25.626365764, "eval_logloss": 0.24046569327021625, "trees": 771} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 27.842411245, "fit_s": 27.842411245, "eval_logloss": 0.2348103975178169, "trees": 890} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 28.750766193, "fit_s": 28.750766193, "eval_logloss": 0.2345264764533108, "trees": 891} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 30.737093569, "fit_s": 30.737093569, "eval_logloss": 0.23565965460396873, "trees": 1000} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 23.560258187, "fit_s": 23.560258187, "eval_logloss": 0.2406796892004917, "trees": 704} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 25.400509777, "fit_s": 25.400509777, "eval_logloss": 0.23505048985939236, "trees": 817} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 15.42797377, "fit_s": 15.42797377, "eval_logloss": 0.24636546323696296, "trees": 664} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 14.376308975, "fit_s": 14.376308975, "eval_logloss": 0.2456334818237112, "trees": 658} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 13.729147711, "fit_s": 13.729147711, "eval_logloss": 0.24641856086955904, "trees": 608} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 11.483294924, "fit_s": 11.483294924, "eval_logloss": 0.24837403809939845, "trees": 538} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 13.246947982, "fit_s": 13.246947982, "eval_logloss": 0.24799510373191336, "trees": 604} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 18.06039778, "fit_s": 18.06039778, "eval_logloss": 0.23995623151206114, "trees": 901} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 19.68964447, "fit_s": 19.68964447, "eval_logloss": 0.23709673529325903, "trees": 856} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 19.615412703, "fit_s": 19.615412703, "eval_logloss": 0.23896342310796106, "trees": 865} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 21.868805181, "fit_s": 21.868805181, "eval_logloss": 0.2379074101277915, "trees": 998} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 20.751762455, "fit_s": 20.751762455, "eval_logloss": 0.23832756009419315, "trees": 985} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 20.478384228, "fit_s": 20.478384228, "eval_logloss": 0.2516805058810973, "trees": 898} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 12.206834227, "fit_s": 12.206834227, "eval_logloss": 0.2578907623142078, "trees": 607} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 10.131693575, "fit_s": 10.131693575, "eval_logloss": 0.26664589737928224, "trees": 470} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 7.605564023, "fit_s": 7.605564023, "eval_logloss": 0.27158558595155463, "trees": 371} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 14.490714427, "fit_s": 14.490714427, "eval_logloss": 0.2557068283224003, "trees": 796} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.20776019400000223, "fit_s": 19.912506922999455, "trees": 911, "eval_logloss": 0.2358272611334169, "total_s": 20.120267116999457} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.23221848599951045, "fit_s": 14.806207482999525, "trees": 654, "eval_logloss": 0.2442208056548983, "total_s": 15.038425968999036} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.2616913829997429, "fit_s": 20.822942064999552, "trees": 992, "eval_logloss": 0.23445731072478374, "total_s": 21.084633447999295} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.22038495099877764, "fit_s": 15.536071053000342, "trees": 722, "eval_logloss": 0.23842595311405487, "total_s": 15.75645600399912} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.2381603100002394, "fit_s": 19.744836781999766, "trees": 915, "eval_logloss": 0.23364364459195727, "total_s": 19.982997092000005} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.9754080590009835, "fit_s": 10.753337627998917, "trees": 722, "eval_logloss": 0.23930275493019326, "total_s": 11.7287456869999} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.9692074960003083, "fit_s": 13.330730756999401, "trees": 871, "eval_logloss": 0.2330329563515818, "total_s": 14.29993825299971} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.9126419580006768, "fit_s": 13.816275445000429, "trees": 936, "eval_logloss": 0.23769643572262283, "total_s": 14.728917403001105} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.9833429720001732, "fit_s": 14.871090303000528, "trees": 1000, "eval_logloss": 0.2359709585152218, "total_s": 15.854433275000702} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.9843131639991043, "fit_s": 13.987811524999415, "trees": 969, "eval_logloss": 0.23389556160624903, "total_s": 14.97212468899852} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 1.0587028589998226, "fit_s": 8.053390738999951, "trees": 798, "eval_logloss": 0.23824735652433995, "total_s": 9.112093597999774} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.9272761489992263, "fit_s": 9.964712447999773, "trees": 988, "eval_logloss": 0.2382392821390285, "total_s": 10.891988596999} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.9319690100001026, "fit_s": 7.869472391999807, "trees": 811, "eval_logloss": 0.24279422925166377, "total_s": 8.80144140199991} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 1.058575008999469, "fit_s": 9.158542981998835, "trees": 891, "eval_logloss": 0.2403281723969063, "total_s": 10.217117990998304} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.9784070360001351, "fit_s": 9.517981904000408, "trees": 1000, "eval_logloss": 0.23653982611622598, "total_s": 10.496388940000543} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 67.185543753, "fit_s": 67.185543753, "eval_logloss": 0.22053839446408632, "trees": 984} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 65.246894479, "fit_s": 65.246894479, "eval_logloss": 0.218032731573198, "trees": 999} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 62.646512075, "fit_s": 62.646512075, "eval_logloss": 0.21824351886547397, "trees": 1000} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 64.652815804, "fit_s": 64.652815804, "eval_logloss": 0.2189240292457814, "trees": 995} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 65.470850887, "fit_s": 65.470850887, "eval_logloss": 0.21748381149888832, "trees": 999} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 65.840512172, "fit_s": 65.840512172, "eval_logloss": 0.21874299672982864, "trees": 1000} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 36.880835419, "fit_s": 36.880835419, "eval_logloss": 0.2205285870254582, "trees": 926} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 41.325913077, "fit_s": 41.325913077, "eval_logloss": 0.21965428948774335, "trees": 1000} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 41.422857165, "fit_s": 41.422857165, "eval_logloss": 0.21963181454931735, "trees": 974} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 31.470362944, "fit_s": 31.470362944, "eval_logloss": 0.2237809581955796, "trees": 796} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 30.572742985, "fit_s": 30.572742985, "eval_logloss": 0.22465595000749378, "trees": 797} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 35.086242422, "fit_s": 35.086242422, "eval_logloss": 0.22106483784204883, "trees": 1000} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 34.903562734, "fit_s": 34.903562734, "eval_logloss": 0.22384161005260872, "trees": 998} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 34.48442732, "fit_s": 34.48442732, "eval_logloss": 0.22232838086601656, "trees": 1000} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 36.598654133, "fit_s": 36.598654133, "eval_logloss": 0.22071884941908948, "trees": 1000} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 36.645292274, "fit_s": 36.645292274, "eval_logloss": 0.22127669239100364, "trees": 1000} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 31.008893034, "fit_s": 31.008893034, "eval_logloss": 0.2312277627771354, "trees": 997} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 27.136632042, "fit_s": 27.136632042, "eval_logloss": 0.2312341530525923, "trees": 892} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 28.436592949, "fit_s": 28.436592949, "eval_logloss": 0.2320429017916541, "trees": 925} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 27.691863245, "fit_s": 27.691863245, "eval_logloss": 0.23175107623706323, "trees": 904} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 31.626432622, "fit_s": 31.626432622, "eval_logloss": 0.23191766596731614, "trees": 1000} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.5665403579987469, "fit_s": 37.36321439600033, "trees": 998, "eval_logloss": 0.21886881963558605, "total_s": 37.92975475399908} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.5008870319998096, "fit_s": 37.77567917699889, "trees": 1000, "eval_logloss": 0.21865965529741951, "total_s": 38.2765662089987} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.5685398299992812, "fit_s": 37.243277679999665, "trees": 1000, "eval_logloss": 0.22092531321445327, "total_s": 37.811817509998946} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.5453400320002402, "fit_s": 37.89585757399982, "trees": 995, "eval_logloss": 0.21951482988178392, "total_s": 38.44119760600006} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.5032200690002355, "fit_s": 39.377229766998425, "trees": 997, "eval_logloss": 0.21888959862096508, "total_s": 39.88044983599866} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.1251390440011164, "fit_s": 31.576964003001194, "trees": 1000, "eval_logloss": 0.21935560730260356, "total_s": 33.70210304700231} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.0582779919986933, "fit_s": 31.697198969000965, "trees": 999, "eval_logloss": 0.21963853835101294, "total_s": 33.75547696099966} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.00087291699856, "fit_s": 31.595869765000316, "trees": 1000, "eval_logloss": 0.2183632967997897, "total_s": 33.596742681998876} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 1.9915755279998848, "fit_s": 31.241449654000462, "trees": 997, "eval_logloss": 0.21904871206107188, "total_s": 33.23302518200035} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.0191847780006356, "fit_s": 32.14149391399951, "trees": 997, "eval_logloss": 0.21774156225977212, "total_s": 34.16067869200015} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.1235951159997057, "fit_s": 19.25630336299946, "trees": 1000, "eval_logloss": 0.22273828186571554, "total_s": 21.379898478999166} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.0569523430003755, "fit_s": 18.940938331001234, "trees": 1000, "eval_logloss": 0.22231230330229404, "total_s": 20.99789067400161} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.0341062629995577, "fit_s": 18.81532326100023, "trees": 1000, "eval_logloss": 0.22096134344841953, "total_s": 20.849429523999788} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 1.9792460589997063, "fit_s": 19.399411439999312, "trees": 1000, "eval_logloss": 0.22169329333630888, "total_s": 21.37865749899902} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 1.9684796639994602, "fit_s": 19.30395688599856, "trees": 1000, "eval_logloss": 0.22125196760260138, "total_s": 21.27243654999802} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 68.540172723, "fit_s": 68.540172723, "eval_logloss": 0.25669154736790856, "trees": 993} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 66.215012806, "fit_s": 66.215012806, "eval_logloss": 0.25078400654234295, "trees": 1000} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 65.669766291, "fit_s": 65.669766291, "eval_logloss": 0.25007595285098794, "trees": 1000} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 67.96098441, "fit_s": 67.96098441, "eval_logloss": 0.24862333619057467, "trees": 998} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 63.770384542, "fit_s": 63.770384542, "eval_logloss": 0.25359609598132943, "trees": 995} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 65.016595808, "fit_s": 65.016595808, "eval_logloss": 0.2505314368396296, "trees": 1000} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 42.334751205, "fit_s": 42.334751205, "eval_logloss": 0.25768390912795913, "trees": 921} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 29.794673893, "fit_s": 29.794673893, "eval_logloss": 0.26711081381022383, "trees": 694} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 39.879322105, "fit_s": 39.879322105, "eval_logloss": 0.2577010941687969, "trees": 919} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 31.81408915, "fit_s": 31.81408915, "eval_logloss": 0.26298564970903904, "trees": 681} -{"setting": "random30", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 42.923688091, "fit_s": 42.923688091, "eval_logloss": 0.25834586812355764, "trees": 1000} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 43.106697212, "fit_s": 43.106697212, "eval_logloss": 0.24901707549887708, "trees": 997} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 41.888849034, "fit_s": 41.888849034, "eval_logloss": 0.24873512411264037, "trees": 999} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 41.36881436, "fit_s": 41.36881436, "eval_logloss": 0.2516367882071785, "trees": 1000} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 39.95604905, "fit_s": 39.95604905, "eval_logloss": 0.25280621965588007, "trees": 999} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 40.85319358, "fit_s": 40.85319358, "eval_logloss": 0.250086074932131, "trees": 1000} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 36.930593455, "fit_s": 36.930593455, "eval_logloss": 0.27090161966317544, "trees": 998} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 30.373890055, "fit_s": 30.373890055, "eval_logloss": 0.27805901278646444, "trees": 845} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 34.755714299, "fit_s": 34.755714299, "eval_logloss": 0.26947816215931997, "trees": 1000} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 35.229564907, "fit_s": 35.229564907, "eval_logloss": 0.27042948085785806, "trees": 998} -{"setting": "goss-small", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 35.252321725, "fit_s": 35.252321725, "eval_logloss": 0.2712011647739133, "trees": 1000} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.5630704830000468, "fit_s": 57.798942726998575, "trees": 1000, "eval_logloss": 0.2480839162227558, "total_s": 58.36201320999862} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.5273373899999569, "fit_s": 58.298325801999454, "trees": 1000, "eval_logloss": 0.2513352610988298, "total_s": 58.82566319199941} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.5812317349991645, "fit_s": 58.413421589999416, "trees": 999, "eval_logloss": 0.2489214203938935, "total_s": 58.99465332499858} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.5827979059995414, "fit_s": 61.15694192599949, "trees": 1000, "eval_logloss": 0.24924270718358457, "total_s": 61.73973983199903} -{"setting": "xgboost-sub80", "library": "xgboost", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 0.5643782100014505, "fit_s": 58.00827035600014, "trees": 999, "eval_logloss": 0.25360945471504065, "total_s": 58.57264856600159} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.6879711719993793, "fit_s": 33.18761895999887, "trees": 999, "eval_logloss": 0.25263782424058306, "total_s": 35.87559013199825} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.5653849909995188, "fit_s": 34.55788431399924, "trees": 999, "eval_logloss": 0.24906121663662156, "total_s": 37.12326930499876} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.2605531209992478, "fit_s": 35.01572387399938, "trees": 997, "eval_logloss": 0.2516625920361714, "total_s": 37.276276994998625} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.4284972170007677, "fit_s": 33.947803381000995, "trees": 990, "eval_logloss": 0.25249901366485267, "total_s": 36.37630059800176} -{"setting": "lightgbm-bag80", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.4733770589991764, "fit_s": 34.4003282900012, "trees": 1000, "eval_logloss": 0.2509968090440238, "total_s": 36.873705349000375} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.406209658998705, "fit_s": 20.40725073700014, "trees": 1000, "eval_logloss": 0.24928873541364316, "total_s": 22.813460395998845} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.4936150269986683, "fit_s": 20.206396485998994, "trees": 993, "eval_logloss": 0.24777349912116298, "total_s": 22.700011512997662} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 2, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.3923557279995293, "fit_s": 20.73570882199965, "trees": 998, "eval_logloss": 0.2507573215963794, "total_s": 23.12806454999918} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 3, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.407497477999641, "fit_s": 20.823623861999295, "trees": 1000, "eval_logloss": 0.2531126745333114, "total_s": 23.231121339998936} -{"setting": "lightgbm-goss", "library": "lightgbm", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 4, "label": "early-stopping", "commit": "c3bf3a7", "dataset_s": 2.5427018720001797, "fit_s": 16.467643156998747, "trees": 783, "eval_logloss": 0.25900644608594325, "total_s": 19.010345028998927} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 0, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 211.450685476, "fit_s": 211.450685476, "eval_logloss": 0.21207469141462787, "trees": 1000} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 1000, "max_depth": 5, "early_stopping_rounds": 20, "seed": 1, "label": "early-stopping", "commit": "c3bf3a7", "total_s": 209.193742196, "fit_s": 209.193742196, "eval_logloss": 0.21224243315643251, "trees": 998} diff --git a/agent/feat/goss/results.md b/agent/feat/goss/results.md deleted file mode 100644 index dbbaa49..0000000 --- a/agent/feat/goss/results.md +++ /dev/null @@ -1,158 +0,0 @@ -# Forust GOSS Results - -Date: 2026-10-06. Branch: `feat/goss`. Machine: 8 physical cores (16 vCPU), 62 GB. -Raw records: `results.jsonl` (generated by `scripts/perf_goss.py run`; tables by -`scripts/perf_goss.py report`). - -## Summary - -- **Baseline:** random row sampling with `subsample=0.8` (`random80`). No sampling - (`none`) is shown for information. -- **Speed:** on 1M x 200 at 8 threads, a 100-iteration fit with default GOSS - (`top_rate=0.2`, `other_rate=0.1`) takes 15.35 s versus 24.10 s for `random80`, - **1.57x faster** (target 1.5x, pass). Across datasets GOSS is 1.39-1.86x faster. -- **Accuracy, fixed 100 iterations:** GOSS is within -0.55% to +0.19% of the - baseline's eval logloss on every dataset (pass). -- **Accuracy, early stopping:** GOSS is +0.98% (w200 100k, pass), -0.11% (w500 - 100k, pass), and **+1.63% (w200 250k, fail)**. LightGBM's GOSS loses a similar - amount against LightGBM bagging on the same data (+1.38%, +0.25%, +1.35%). -- **GOSS versus random sampling with the same row budget (`random30`):** at a fixed - 100 iterations, `random30` is as fast and slightly better, because neither model - has converged and smaller samples regularise. Under early stopping, GOSS is - clearly better on w200 100k (+0.98% vs +4.58%) and w500 (-0.11% vs +4.01%), and - tied on w200 250k (+1.63% vs +1.54%). -- **`goss-small` (0.1/0.1) is not recommended:** up to +10% logloss under early stopping. -- **Forust versus LightGBM GOSS:** similar accuracy; LightGBM trains faster - (its fit excludes dataset construction, shown as the gap between Total and Train). - -## Optimisations measured with `perf_phases` (1M x 200, 8 threads, steady state per tree) - -| Version | Tree | Row selection | Subset copy | Total | -| --- | ---: | ---: | ---: | ---: | -| No sampling | 220.5 ms | - | - | 220.5 ms | -| GOSS, serial selection, no subset | 111.1 ms | 17.6 ms | - | 128.7 ms | -| + contiguous row subset | 89.2 ms | 15.2 ms | 11.6 ms | 116.0 ms | -| + parallel block selection | 85.5 ms | 4.8 ms | 11.1 ms | 101.4 ms | - -The subset copy reads almost the whole binned matrix, so it is bandwidth-bound. -Subset trees are byte-identical to full-data trees (unit tested); block selection -gives identical models with `parallel` on or off. - -## Caveats - -- **The 1,000-tree cap was usually reached.** With learning rate 0.1 most - "early stopping" runs stopped at or near 1,000 trees, so they are closer to - 1,000 fixed iterations than to converged models. The w200 250k failure is in - this regime. A larger cap or a higher learning rate would test convergence properly. -- **Runs stopped early by request:** the 1M early-stopping comparison (only 2 - `random80` runs finished, no GOSS), the 1-thread check, and the per-tree - `perf_sweep` runs were not completed. The 1M early-stopping table is omitted. -- Totals include each library's dataset construction. XGBoost's DMatrix doesn't bin, - so its Train column is the comparable figure. -- Synthetic data only (`make_perf_data.py`, seed 0); sampled settings use seeds 0-4. - -## Results - -### w200, 100,000 rows, 8 threads, depth 5, early stopping 20, max 1000 iterations (early-stopping) - -| Setting | Runs | Total s | Train s | Speedup | Trees | Eval logloss | vs baseline | -| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| none | 1 | 25.63 | 25.63 | 1.09x | 771 | 0.24047 | +1.83% | -| **random80** (baseline) | 5 | 27.84 | 27.84 | 1.00x | 860 | 0.23615 ± 0.00257 | +0.00% | -| random30 | 5 | 13.73 | 13.73 | 2.03x | 614 | 0.24696 ± 0.00117 | +4.58% | -| goss | 5 | 19.69 | 19.69 | 1.41x | 921 | 0.23845 ± 0.00108 | +0.98% | -| goss-small | 5 | 12.21 | 12.21 | 2.28x | 628 | 0.26070 ± 0.00819 | +10.40% | -| xgboost-sub80 | 5 | 19.98 | 19.74 | - | 839 | 0.23731 ± 0.00427 | - | -| lightgbm-bag80 | 5 | 14.73 | 13.82 | - | 900 | 0.23598 ± 0.00260 | - | -| lightgbm-goss | 5 | 10.22 | 9.16 | 1.44x | 898 | 0.23923 ± 0.00240 | +1.38% | - -### w200, 250,000 rows, 8 threads, depth 5, early stopping 20, max 1000 iterations (early-stopping) - -| Setting | Runs | Total s | Train s | Speedup | Trees | Eval logloss | vs baseline | -| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| none | 1 | 67.19 | 67.19 | 0.97x | 984 | 0.22054 | +1.03% | -| **random80** (baseline) | 5 | 65.25 | 65.25 | 1.00x | 999 | 0.21829 ± 0.00058 | +0.00% | -| random30 | 5 | 36.88 | 36.88 | 1.77x | 899 | 0.22165 ± 0.00239 | +1.54% | -| goss | 5 | 35.09 | 35.09 | 1.86x | 1000 | 0.22185 ± 0.00127 | +1.63% | -| goss-small | 5 | 28.44 | 28.44 | 2.29x | 944 | 0.23163 ± 0.00038 | +6.12% | -| xgboost-sub80 | 5 | 38.28 | 37.78 | - | 998 | 0.21937 ± 0.00093 | - | -| lightgbm-bag80 | 5 | 33.70 | 31.60 | - | 999 | 0.21883 ± 0.00077 | - | -| lightgbm-goss | 5 | 21.27 | 19.26 | 1.58x | 1000 | 0.22179 ± 0.00073 | +1.35% | - -### w500, 100,000 rows, 8 threads, depth 5, early stopping 20, max 1000 iterations (early-stopping) - -| Setting | Runs | Total s | Train s | Speedup | Trees | Eval logloss | vs baseline | -| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| none | 1 | 68.54 | 68.54 | 0.96x | 993 | 0.25669 | +2.38% | -| **random80** (baseline) | 5 | 65.67 | 65.67 | 1.00x | 999 | 0.25072 ± 0.00181 | +0.00% | -| random30 | 5 | 39.88 | 39.88 | 1.65x | 843 | 0.26077 ± 0.00418 | +4.01% | -| goss | 5 | 41.37 | 41.37 | 1.59x | 999 | 0.25046 ± 0.00174 | -0.11% | -| goss-small | 5 | 35.23 | 35.23 | 1.86x | 968 | 0.27201 ± 0.00344 | +8.49% | -| xgboost-sub80 | 5 | 58.83 | 58.30 | - | 1000 | 0.25024 ± 0.00223 | - | -| lightgbm-bag80 | 5 | 36.87 | 34.40 | - | 997 | 0.25137 ± 0.00145 | - | -| lightgbm-goss | 5 | 22.81 | 20.41 | 1.62x | 955 | 0.25199 ± 0.00439 | +0.25% | - -### w200, 100,000 rows, 8 threads, depth 5, 100 iterations (fixed) - -| Setting | Runs | Total s | Train s | Speedup | Trees | Eval logloss | vs baseline | -| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| none | 3 | 3.47 | 3.47 | 0.93x | 100 | 0.32794 ± 0.00000 | +1.22% | -| **random80** (baseline) | 5 | 3.21 | 3.21 | 1.00x | 100 | 0.32399 ± 0.00085 | +0.00% | -| random30 | 5 | 2.24 | 2.24 | 1.44x | 100 | 0.31762 ± 0.00159 | -1.97% | -| goss | 5 | 2.31 | 2.31 | 1.39x | 100 | 0.32461 ± 0.00074 | +0.19% | -| goss-small | 5 | 2.06 | 2.06 | 1.56x | 100 | 0.32560 ± 0.00081 | +0.50% | -| xgboost-sub80 | 5 | 2.80 | 2.56 | - | 100 | 0.32607 ± 0.00106 | - | -| lightgbm-bag80 | 5 | 2.42 | 1.47 | - | 100 | 0.32350 ± 0.00281 | - | -| lightgbm-goss | 5 | 2.08 | 1.09 | 1.16x | 100 | 0.32331 ± 0.00227 | -0.06% | - -### w200, 250,000 rows, 8 threads, depth 5, 100 iterations (fixed) - -| Setting | Runs | Total s | Train s | Speedup | Trees | Eval logloss | vs baseline | -| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| none | 3 | 7.91 | 7.91 | 0.90x | 100 | 0.32712 ± 0.00000 | +0.81% | -| **random80** (baseline) | 5 | 7.13 | 7.13 | 1.00x | 100 | 0.32449 ± 0.00130 | +0.00% | -| random30 | 5 | 4.22 | 4.22 | 1.69x | 100 | 0.31827 ± 0.00202 | -1.92% | -| goss | 5 | 4.42 | 4.42 | 1.61x | 100 | 0.32270 ± 0.00180 | -0.55% | -| goss-small | 5 | 3.94 | 3.94 | 1.81x | 100 | 0.32026 ± 0.00093 | -1.30% | -| xgboost-sub80 | 5 | 5.77 | 5.21 | - | 100 | 0.32460 ± 0.00149 | - | -| lightgbm-bag80 | 5 | 5.39 | 3.29 | - | 100 | 0.32442 ± 0.00170 | - | -| lightgbm-goss | 5 | 4.40 | 2.34 | 1.22x | 100 | 0.32199 ± 0.00239 | -0.75% | - -### w200-1m, 1,000,000 rows, 8 threads, depth 5, 100 iterations (fixed) - -| Setting | Runs | Total s | Train s | Speedup | Trees | Eval logloss | vs baseline | -| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| none | 3 | 26.74 | 26.74 | 0.90x | 100 | 0.33635 ± 0.00000 | +0.04% | -| **random80** (baseline) | 5 | 24.10 | 24.10 | 1.00x | 100 | 0.33621 ± 0.00094 | +0.00% | -| random30 | 5 | 14.81 | 14.81 | 1.63x | 100 | 0.33232 ± 0.00138 | -1.16% | -| goss | 5 | 15.35 | 15.35 | 1.57x | 100 | 0.33552 ± 0.00046 | -0.20% | -| goss-small | 5 | 12.87 | 12.87 | 1.87x | 100 | 0.33137 ± 0.00091 | -1.44% | -| xgboost-sub80 | 5 | 20.31 | 18.04 | - | 100 | 0.33606 ± 0.00056 | - | -| lightgbm-bag80 | 5 | 16.28 | 12.02 | - | 100 | 0.33623 ± 0.00076 | - | -| lightgbm-goss | 5 | 12.60 | 8.29 | 1.29x | 100 | 0.33436 ± 0.00067 | -0.56% | - -### w500, 100,000 rows, 8 threads, depth 5, 100 iterations (fixed) - -| Setting | Runs | Total s | Train s | Speedup | Trees | Eval logloss | vs baseline | -| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| none | 3 | 7.92 | 7.92 | 0.91x | 100 | 0.36344 ± 0.00000 | -0.02% | -| **random80** (baseline) | 5 | 7.20 | 7.20 | 1.00x | 100 | 0.36351 ± 0.00169 | +0.00% | -| random30 | 5 | 4.95 | 4.95 | 1.45x | 100 | 0.36108 ± 0.00164 | -0.67% | -| goss | 5 | 4.88 | 4.88 | 1.48x | 100 | 0.36329 ± 0.00164 | -0.06% | -| goss-small | 5 | 4.47 | 4.47 | 1.61x | 100 | 0.37631 ± 0.00099 | +3.52% | -| xgboost-sub80 | 5 | 7.79 | 7.21 | - | 100 | 0.36309 ± 0.00141 | - | -| lightgbm-bag80 | 5 | 5.89 | 3.41 | - | 100 | 0.36317 ± 0.00165 | - | -| lightgbm-goss | 5 | 4.52 | 2.22 | 1.30x | 100 | 0.36253 ± 0.00077 | -0.18% | - -### Pass/fail against random80 (logloss within 1%, speedup at least 1.5x) - -| Check | Value | Result | -| --- | ---: | --- | -| logloss: w200 100,000 rows, 8 thr, depth 5, early stopping 20, max 1000 iterations | +0.98% | pass | -| logloss: w200 250,000 rows, 8 thr, depth 5, early stopping 20, max 1000 iterations | +1.63% | FAIL | -| logloss: w500 100,000 rows, 8 thr, depth 5, early stopping 20, max 1000 iterations | -0.11% | pass | -| logloss: w200 100,000 rows, 8 thr, depth 5, 100 iterations | +0.19% | pass | -| logloss: w200 250,000 rows, 8 thr, depth 5, 100 iterations | -0.55% | pass | -| logloss: w200-1m 1,000,000 rows, 8 thr, depth 5, 100 iterations | -0.20% | pass | -| speed: w200-1m 1,000,000 rows, 8 thr, depth 5, 100 iterations | 1.57x | pass | -| logloss: w500 100,000 rows, 8 thr, depth 5, 100 iterations | -0.06% | pass | diff --git a/agent/feat/hist-perf/fit.jsonl b/agent/feat/hist-perf/fit.jsonl deleted file mode 100644 index 6ba159d..0000000 --- a/agent/feat/hist-perf/fit.jsonl +++ /dev/null @@ -1,36 +0,0 @@ -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "b2a8687", "total_s": 2.652335826, "fit_s": 2.652335826, "eval_logloss": 0.3279430393472949, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "b2a8687", "total_s": 2.491133769, "fit_s": 2.491133769, "eval_logloss": 0.3279430393472949, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "b2a8687", "total_s": 2.442100583, "fit_s": 2.442100583, "eval_logloss": 0.3279430393472949, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "b2a8687", "total_s": 2.340952219, "fit_s": 2.340952219, "eval_logloss": 0.3250719230959926, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "b2a8687", "total_s": 2.172317999, "fit_s": 2.172317999, "eval_logloss": 0.3239106125967988, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "b2a8687", "total_s": 2.300397405, "fit_s": 2.300397405, "eval_logloss": 0.32370341939737757, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "b2a8687", "total_s": 1.938085966, "fit_s": 1.938085966, "eval_logloss": 0.32349541186951414, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "b2a8687", "total_s": 1.861371835, "fit_s": 1.861371835, "eval_logloss": 0.3244407370751807, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "b2a8687", "total_s": 1.994381747, "fit_s": 1.994381747, "eval_logloss": 0.3250261358288657, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "b2a8687", "total_s": 5.604127666, "fit_s": 5.604127666, "eval_logloss": 0.3271226778537963, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "b2a8687", "total_s": 6.658129628, "fit_s": 6.658129628, "eval_logloss": 0.3271226778537963, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "b2a8687", "total_s": 5.548895799, "fit_s": 5.548895799, "eval_logloss": 0.3271226778537963, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "b2a8687", "total_s": 5.195690632, "fit_s": 5.195690632, "eval_logloss": 0.3233034940191527, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "b2a8687", "total_s": 5.025468101, "fit_s": 5.025468101, "eval_logloss": 0.325543884404929, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "b2a8687", "total_s": 5.178431711, "fit_s": 5.178431711, "eval_logloss": 0.32407862785277697, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "b2a8687", "total_s": 3.403949393, "fit_s": 3.403949393, "eval_logloss": 0.323063130871686, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "b2a8687", "total_s": 3.462424382, "fit_s": 3.462424382, "eval_logloss": 0.32559758433538283, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200", "rows": 250000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "b2a8687", "total_s": 3.534857227, "fit_s": 3.534857227, "eval_logloss": 0.321961914675259, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "b2a8687", "total_s": 5.960050718, "fit_s": 5.960050718, "eval_logloss": 0.36343754917898585, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "b2a8687", "total_s": 5.761254784, "fit_s": 5.761254784, "eval_logloss": 0.36343754917898585, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "b2a8687", "total_s": 5.370648572, "fit_s": 5.370648572, "eval_logloss": 0.36343754917898585, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "b2a8687", "total_s": 4.97597853, "fit_s": 4.97597853, "eval_logloss": 0.3622866741086639, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "b2a8687", "total_s": 5.079038635, "fit_s": 5.079038635, "eval_logloss": 0.3653813035099335, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "b2a8687", "total_s": 4.961095665, "fit_s": 4.961095665, "eval_logloss": 0.3612907411897185, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "b2a8687", "total_s": 3.570373536, "fit_s": 3.570373536, "eval_logloss": 0.3616094241108412, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "b2a8687", "total_s": 3.754869388, "fit_s": 3.754869388, "eval_logloss": 0.3646106372783539, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w500", "rows": 100000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "b2a8687", "total_s": 3.493066827, "fit_s": 3.493066827, "eval_logloss": 0.36511046594590063, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "b2a8687", "total_s": 19.218695322, "fit_s": 19.218695322, "eval_logloss": 0.3363535089725551, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "b2a8687", "total_s": 19.632377002, "fit_s": 19.632377002, "eval_logloss": 0.3363535089725551, "trees": 100} -{"setting": "none", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "b2a8687", "total_s": 19.527649096, "fit_s": 19.527649096, "eval_logloss": 0.3363535089725551, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "b2a8687", "total_s": 18.353958157, "fit_s": 18.353958157, "eval_logloss": 0.3357783701509772, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "b2a8687", "total_s": 18.948585015, "fit_s": 18.948585015, "eval_logloss": 0.3351172331354733, "trees": 100} -{"setting": "random80", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "b2a8687", "total_s": 18.153341674, "fit_s": 18.153341674, "eval_logloss": 0.33591420345085704, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 0, "label": "fixed", "commit": "b2a8687", "total_s": 12.622745092, "fit_s": 12.622745092, "eval_logloss": 0.335759070475801, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 1, "label": "fixed", "commit": "b2a8687", "total_s": 13.115549821, "fit_s": 13.115549821, "eval_logloss": 0.3353976828778981, "trees": 100} -{"setting": "goss", "library": "forust", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "threads": 8, "iterations": 100, "max_depth": 5, "early_stopping_rounds": null, "seed": 2, "label": "fixed", "commit": "b2a8687", "total_s": 12.920877933, "fit_s": 12.920877933, "eval_logloss": 0.3353008510444272, "trees": 100} diff --git a/agent/feat/hist-perf/results.jsonl b/agent/feat/hist-perf/results.jsonl deleted file mode 100644 index d4cedd7..0000000 --- a/agent/feat/hist-perf/results.jsonl +++ /dev/null @@ -1,44 +0,0 @@ -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 14.90815216, "sample_ms": 0.02416654, "subset_ms": 0.000108, "tree_rows": 25000, "bin_s": 0.044305898, "per_iteration_ms": {"eval_predict_metric": 0.3347551166666668, "grad_hess": 0.19534388333333333, "sample": 0.026080533333333322, "subset": 0.0001616833333333333, "train_predict": 0.4083083166666668, "tree": 15.075280583333335}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [14.90815216, 12.39811144, 18.153842], "bin_s_samples": [0.044305898, 0.045114115, 0.042072052], "label": "baseline-none-d5", "commit": "8383fbd-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 30.08671614, "sample_ms": 0.14713098, "subset_ms": 0.00018202, "tree_rows": 100000, "bin_s": 0.169409236, "per_iteration_ms": {"eval_predict_metric": 1.1007940666666665, "grad_hess": 0.7958910666666665, "sample": 0.17304189999999997, "subset": 0.00019001666666666671, "train_predict": 1.2696207500000003, "tree": 38.25696070000001}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [37.8878742, 30.08671614, 29.02076868], "bin_s_samples": [0.177066691, 0.166943685, 0.169409236], "label": "baseline-none-d5", "commit": "8383fbd-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 67.47062118, "sample_ms": 0.42148246, "subset_ms": 0.000174, "tree_rows": 250000, "bin_s": 0.482230583, "per_iteration_ms": {"eval_predict_metric": 2.38075325, "grad_hess": 1.9134005833333336, "sample": 0.4329686999999999, "subset": 0.00013333333333333329, "train_predict": 2.42434585, "tree": 64.4458937}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [63.216846079999996, 71.76056968, 67.47062118], "bin_s_samples": [0.477575086, 0.498392208, 0.482230583], "label": "baseline-none-d5", "commit": "8383fbd-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 48.5267956, "sample_ms": 0.03030055, "subset_ms": 0.00016, "tree_rows": 25000, "bin_s": 0.041939851, "per_iteration_ms": {"eval_predict_metric": 0.4843098, "grad_hess": 0.21122099999999996, "sample": 0.037954066666666675, "subset": 0.00015666666666666663, "train_predict": 0.9606928666666668, "tree": 43.967409}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [36.53896655, 49.11530195, 48.5267956], "bin_s_samples": [0.042773468, 0.039193496, 0.041939851], "label": "baseline-none-d8", "commit": "8383fbd-dirty", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 90.99650315000001, "sample_ms": 0.1427829, "subset_ms": 0.00016999999999999999, "tree_rows": 100000, "bin_s": 0.169698658, "per_iteration_ms": {"eval_predict_metric": 1.63881, "grad_hess": 0.8601308666666666, "sample": 0.14785630000000002, "subset": 0.00019333333333333323, "train_predict": 3.3538981333333333, "tree": 95.54538386666665}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [90.99650315000001, 85.55728355000001, 100.33387925], "bin_s_samples": [0.168653526, 0.178391423, 0.169698658], "label": "baseline-none-d8", "commit": "8383fbd-dirty", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 15.74812892, "sample_ms": 0.9834679399999999, "subset_ms": 1.04572934, "tree_rows": 29993, "bin_s": 0.165663476, "per_iteration_ms": {"eval_predict_metric": 1.0936656500000004, "grad_hess": 0.7633005666666667, "sample": 0.9864283500000002, "subset": 1.1706822500000003, "train_predict": 1.4051786333333338, "tree": 16.36639341666667}, "eval_logloss": 0.3461969207536007, "tree_ms_samples": [16.458881299999998, 15.74812892, 15.152870779999999], "bin_s_samples": [0.166809498, 0.159700854, 0.165663476], "label": "baseline-random30-d5", "commit": "8383fbd-dirty", "extra": ["--sample-method", "random", "--subsample", "0.3"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 25.5268786, "sample_ms": 2.50351526, "subset_ms": 2.4980231600000002, "tree_rows": 75170, "bin_s": 0.498805513, "per_iteration_ms": {"eval_predict_metric": 2.402430666666667, "grad_hess": 1.9483346666666665, "sample": 2.539241816666667, "subset": 2.739042383333333, "train_predict": 2.692228266666667, "tree": 26.321774733333328}, "eval_logloss": 0.3461480650012045, "tree_ms_samples": [25.38829738, 25.559622859999998, 25.5268786], "bin_s_samples": [0.49851586, 0.512082236, 0.498805513], "label": "baseline-random30-d5", "commit": "8383fbd-dirty", "extra": ["--sample-method", "random", "--subsample", "0.3"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 255, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 13.0651492, "sample_ms": 0.02509846, "subset_ms": 0.000118, "tree_rows": 25000, "bin_s": 0.041506048, "per_iteration_ms": {"eval_predict_metric": 0.3262035166666667, "grad_hess": 0.18913870000000005, "sample": 0.02608888333333333, "subset": 0.00011666666666666658, "train_predict": 0.36688080000000006, "tree": 13.597688400000001}, "eval_logloss": 0.3699417432836186, "tree_ms_samples": [12.808112240000002, 13.9172888, 13.0651492], "bin_s_samples": [0.047180166, 0.041064941, 0.041506048], "label": "baseline-none-d5-nbins255", "commit": "8383fbd-dirty", "extra": ["--nbins", "255"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 255, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 30.34861874, "sample_ms": 0.14085684, "subset_ms": 0.000174, "tree_rows": 100000, "bin_s": 0.154405658, "per_iteration_ms": {"eval_predict_metric": 1.0232793666666669, "grad_hess": 0.7744340666666667, "sample": 0.14249455, "subset": 0.00014499999999999997, "train_predict": 1.0535348166666663, "tree": 29.757470366666656}, "eval_logloss": 0.35535923373212674, "tree_ms_samples": [29.56330076, 30.34861874, 30.43231162], "bin_s_samples": [0.17012188, 0.152221813, 0.154405658], "label": "baseline-none-d5-nbins255", "commit": "8383fbd-dirty", "extra": ["--nbins", "255"]} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 224.73409518, "sample_ms": 1.62598694, "subset_ms": 0.00030003999999999995, "tree_rows": 1000000, "bin_s": 1.761653978, "per_iteration_ms": {"eval_predict_metric": 9.005727949999999, "grad_hess": 7.95532, "sample": 2.6734677500000004, "subset": 0.00023338333333333339, "train_predict": 8.763013050000003, "tree": 227.6215985}, "eval_logloss": 0.361499131954832, "tree_ms_samples": [226.65162306, 224.73409518, 219.06511152], "bin_s_samples": [1.761653978, 1.7298712840000001, 1.763460218], "label": "baseline-none-d5", "commit": "8383fbd-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 255, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 219.81581934, "sample_ms": 3.01170318, "subset_ms": 0.00022799999999999999, "tree_rows": 1000000, "bin_s": 1.808381072, "per_iteration_ms": {"eval_predict_metric": 8.940188333333335, "grad_hess": 8.126750033333336, "sample": 3.3159573166666667, "subset": 0.00020666666666666658, "train_predict": 8.635652099999998, "tree": 222.72219283333334}, "eval_logloss": 0.36205314097031416, "tree_ms_samples": [219.81581934, 216.37893603999999, 220.3548088], "bin_s_samples": [1.845796896, 1.746011544, 1.808381072], "label": "baseline-none-d5-nbins255", "commit": "8383fbd-dirty", "extra": ["--nbins", "255"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 10.95286662, "sample_ms": 0.023954399999999997, "subset_ms": 0.00011999999999999999, "tree_rows": 25000, "bin_s": 0.044233791, "per_iteration_ms": {"eval_predict_metric": 0.3268731833333333, "grad_hess": 0.18979720000000005, "sample": 0.026265516666666672, "subset": 0.00013333333333333329, "train_predict": 0.34167361666666646, "tree": 11.199893900000003}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [10.95286662, 9.40797946, 11.23261468], "bin_s_samples": [0.044414194, 0.044233791, 0.042805464], "label": "8a-none-d5", "commit": "8383fbd-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 20.26717494, "sample_ms": 0.146965, "subset_ms": 0.000196, "tree_rows": 100000, "bin_s": 0.169198814, "per_iteration_ms": {"eval_predict_metric": 1.0256574166666672, "grad_hess": 0.7552669500000005, "sample": 0.15989150000000005, "subset": 0.000145, "train_predict": 1.0613263666666668, "tree": 20.115464266666663}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [19.708463700000003, 20.26717494, 22.883439919999997], "bin_s_samples": [0.169198814, 0.177993892, 0.167327177], "label": "8a-none-d5", "commit": "8383fbd-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 43.80125432, "sample_ms": 0.49162668, "subset_ms": 0.00016, "tree_rows": 250000, "bin_s": 0.495738987, "per_iteration_ms": {"eval_predict_metric": 2.389738566666667, "grad_hess": 1.9373189999999998, "sample": 0.5096035499999999, "subset": 0.00016, "train_predict": 2.3724431666666663, "tree": 44.34737803333333}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [43.80125432, 43.40637014, 44.4265667], "bin_s_samples": [0.487101995, 0.495738987, 0.501709583], "label": "8a-none-d5", "commit": "8383fbd-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 37.50314049999999, "sample_ms": 0.03235075, "subset_ms": 0.000135, "tree_rows": 25000, "bin_s": 0.042860638, "per_iteration_ms": {"eval_predict_metric": 0.46411676666666657, "grad_hess": 0.19165413333333328, "sample": 0.036157566666666675, "subset": 0.0001333333333333333, "train_predict": 0.908353333333333, "tree": 39.5974101}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [36.57716075, 37.50314049999999, 39.90683655], "bin_s_samples": [0.042860638, 0.042815537, 0.043513934], "label": "8a-none-d8", "commit": "8383fbd-dirty", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 82.2643453, "sample_ms": 0.1391879, "subset_ms": 0.000125, "tree_rows": 100000, "bin_s": 0.15980363, "per_iteration_ms": {"eval_predict_metric": 1.6033210666666664, "grad_hess": 0.7674297999999999, "sample": 0.1513665666666667, "subset": 0.00011999999999999992, "train_predict": 3.3838559333333325, "tree": 81.38285013333332}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [82.82575779999999, 82.2643453, 79.9722873], "bin_s_samples": [0.155944747, 0.15980363, 0.163024438], "label": "8a-none-d8", "commit": "8383fbd-dirty", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 11.54145234, "sample_ms": 0.92693304, "subset_ms": 1.06333972, "tree_rows": 29993, "bin_s": 0.170967401, "per_iteration_ms": {"eval_predict_metric": 1.0537965500000002, "grad_hess": 0.7549004166666665, "sample": 0.9243789833333336, "subset": 1.1292681000000002, "train_predict": 1.2622158833333335, "tree": 11.506888933333332}, "eval_logloss": 0.3461969207536007, "tree_ms_samples": [11.33210404, 11.54145234, 12.23002458], "bin_s_samples": [0.170967401, 0.160335383, 0.173331494], "label": "8a-random30-d5", "commit": "8383fbd-dirty", "extra": ["--sample-method", "random", "--subsample", "0.3"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 21.32094046, "sample_ms": 2.30950322, "subset_ms": 2.83064428, "tree_rows": 75170, "bin_s": 0.463541476, "per_iteration_ms": {"eval_predict_metric": 2.5259238333333327, "grad_hess": 1.9992393999999996, "sample": 2.279811816666667, "subset": 2.941187666666666, "train_predict": 2.8722295000000004, "tree": 20.92357096666667}, "eval_logloss": 0.3461480650012045, "tree_ms_samples": [21.32094046, 23.61567338, 19.24605414], "bin_s_samples": [0.461462534, 0.469159598, 0.463541476], "label": "8a-random30-d5", "commit": "8383fbd-dirty", "extra": ["--sample-method", "random", "--subsample", "0.3"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 255, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 10.0072611, "sample_ms": 0.02503268, "subset_ms": 0.00010999999999999999, "tree_rows": 25000, "bin_s": 0.038997231, "per_iteration_ms": {"eval_predict_metric": 0.32159016666666673, "grad_hess": 0.19778085000000006, "sample": 0.026545716666666677, "subset": 0.00010999999999999992, "train_predict": 0.3503875000000001, "tree": 10.016268266666668}, "eval_logloss": 0.3699417432836186, "tree_ms_samples": [10.0072611, 10.69495396, 9.86044002], "bin_s_samples": [0.043960237, 0.038997231, 0.03897233], "label": "8a-none-d5-nbins255", "commit": "8383fbd-dirty", "extra": ["--nbins", "255"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 255, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 23.62998514, "sample_ms": 0.1578001, "subset_ms": 0.00015199999999999998, "tree_rows": 100000, "bin_s": 0.166528094, "per_iteration_ms": {"eval_predict_metric": 1.1440875499999998, "grad_hess": 0.78986045, "sample": 0.16048913333333342, "subset": 0.00015166666666666668, "train_predict": 1.352823983333334, "tree": 29.620636983333345}, "eval_logloss": 0.35535923373212674, "tree_ms_samples": [30.49897886, 23.62998514, 23.36794618], "bin_s_samples": [0.169355008, 0.162307283, 0.166528094], "label": "8a-none-d5-nbins255", "commit": "8383fbd-dirty", "extra": ["--nbins", "255"]} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 157.97694496, "sample_ms": 1.5995328599999998, "subset_ms": 0.00023801999999999997, "tree_rows": 1000000, "bin_s": 1.855348363, "per_iteration_ms": {"eval_predict_metric": 8.879162783333332, "grad_hess": 7.567392483333335, "sample": 1.6476174666666663, "subset": 0.0002483500000000001, "train_predict": 8.505492583333334, "tree": 154.7512352}, "eval_logloss": 0.361499131954832, "tree_ms_samples": [151.69181286, 164.670179, 157.97694496], "bin_s_samples": [1.774894804, 1.855348363, 1.894365717], "label": "8a-none-d5", "commit": "8383fbd-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 255, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 154.62110335999998, "sample_ms": 1.61613234, "subset_ms": 0.00023, "tree_rows": 1000000, "bin_s": 1.770569385, "per_iteration_ms": {"eval_predict_metric": 9.162306566666667, "grad_hess": 7.850876916666668, "sample": 1.7675052999999996, "subset": 0.00023166666666666664, "train_predict": 8.417385049999998, "tree": 157.12517185}, "eval_logloss": 0.36205314097031416, "tree_ms_samples": [154.62110335999998, 163.75918302, 153.4800265], "bin_s_samples": [1.7568485489999999, 1.782043534, 1.770569385], "label": "8a-none-d5-nbins255", "commit": "8383fbd-dirty", "extra": ["--nbins", "255"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 8.67794198, "sample_ms": 0.026550439999999998, "subset_ms": 0.00010999999999999999, "tree_rows": 25000, "bin_s": 0.043096275, "per_iteration_ms": {"eval_predict_metric": 0.36221405, "grad_hess": 0.19223221666666665, "sample": 0.04612099999999998, "subset": 0.00010999999999999991, "train_predict": 0.4510407833333335, "tree": 11.646086216666667}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [11.3731987, 8.67794198, 8.62532496], "bin_s_samples": [0.044733308, 0.043096275, 0.040998132], "label": "8b-none-d5", "commit": "6639d83-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 20.07539122, "sample_ms": 0.14748113999999998, "subset_ms": 0.000172, "tree_rows": 100000, "bin_s": 0.166584079, "per_iteration_ms": {"eval_predict_metric": 1.0567580499999998, "grad_hess": 0.7832592999999997, "sample": 0.1505613833333333, "subset": 0.00017499999999999992, "train_predict": 1.1675836833333335, "tree": 20.6638175}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [20.07539122, 21.68574176, 19.63393172], "bin_s_samples": [0.172431197, 0.166584079, 0.157401224], "label": "8b-none-d5", "commit": "6639d83-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 41.33623002, "sample_ms": 0.35154698, "subset_ms": 0.00015199999999999998, "tree_rows": 250000, "bin_s": 0.483868333, "per_iteration_ms": {"eval_predict_metric": 2.4280464999999998, "grad_hess": 1.9277867166666665, "sample": 0.3705023333333334, "subset": 0.00015, "train_predict": 2.293532149999999, "tree": 41.54507453333335}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [40.5415066, 42.78471616, 41.33623002], "bin_s_samples": [0.505613135, 0.463898507, 0.483868333], "label": "8b-none-d5", "commit": "6639d83-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 31.5404761, "sample_ms": 0.02763565, "subset_ms": 0.00014505, "tree_rows": 25000, "bin_s": 0.045419024, "per_iteration_ms": {"eval_predict_metric": 0.4523324999999999, "grad_hess": 0.1911004, "sample": 0.03088073333333333, "subset": 0.00013003333333333326, "train_predict": 0.8903447999999999, "tree": 35.9595876}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [31.5404761, 33.7023055, 30.268645600000003], "bin_s_samples": [0.0442846, 0.045419024, 0.046981155], "label": "8b-none-d8", "commit": "6639d83-dirty", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 73.0915052, "sample_ms": 0.14844795, "subset_ms": 0.00016499999999999997, "tree_rows": 100000, "bin_s": 0.172792914, "per_iteration_ms": {"eval_predict_metric": 1.5803686000000001, "grad_hess": 0.7948129666666667, "sample": 0.16657336666666664, "subset": 0.0001666666666666666, "train_predict": 3.385578933333333, "tree": 75.79412153333332}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [73.39429265, 73.0915052, 67.5389407], "bin_s_samples": [0.171731493, 0.172792914, 0.175501014], "label": "8b-none-d8", "commit": "6639d83-dirty", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 9.65416166, "sample_ms": 0.9835542799999999, "subset_ms": 1.0527316999999998, "tree_rows": 29993, "bin_s": 0.170314907, "per_iteration_ms": {"eval_predict_metric": 1.0434964499999997, "grad_hess": 0.849965816666667, "sample": 0.9611298166666665, "subset": 1.0792138999999994, "train_predict": 1.1006493499999999, "tree": 9.727515266666671}, "eval_logloss": 0.3461969207536007, "tree_ms_samples": [9.60819182, 9.65416166, 10.85292074], "bin_s_samples": [0.170314907, 0.163482367, 0.173148985], "label": "8b-random30-d5", "commit": "6639d83-dirty", "extra": ["--sample-method", "random", "--subsample", "0.3"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 17.60385256, "sample_ms": 2.4084799, "subset_ms": 2.61638574, "tree_rows": 75170, "bin_s": 0.471011652, "per_iteration_ms": {"eval_predict_metric": 2.443325616666667, "grad_hess": 1.9543988000000008, "sample": 2.5836368666666667, "subset": 2.8468922333333344, "train_predict": 2.780467483333333, "tree": 17.583617483333335}, "eval_logloss": 0.3461480650012045, "tree_ms_samples": [17.60385256, 18.0690602, 16.552560059999998], "bin_s_samples": [0.480608352, 0.471011652, 0.462433301], "label": "8b-random30-d5", "commit": "6639d83-dirty", "extra": ["--sample-method", "random", "--subsample", "0.3"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 255, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 8.491796520000001, "sample_ms": 0.03619674, "subset_ms": 9.601999999999999e-05, "tree_rows": 25000, "bin_s": 0.043638396, "per_iteration_ms": {"eval_predict_metric": 0.3163048333333334, "grad_hess": 0.19102061666666667, "sample": 0.024195499999999998, "subset": 9.833333333333326e-05, "train_predict": 0.3531438833333334, "tree": 8.5668327}, "eval_logloss": 0.3699417432836186, "tree_ms_samples": [8.491796520000001, 9.0641782, 8.1895162], "bin_s_samples": [0.043638396, 0.044124207, 0.041448052], "label": "8b-none-d5-nbins255", "commit": "6639d83-dirty", "extra": ["--nbins", "255"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 255, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 19.13946804, "sample_ms": 0.13975488, "subset_ms": 0.00016, "tree_rows": 100000, "bin_s": 0.172283982, "per_iteration_ms": {"eval_predict_metric": 1.0414449, "grad_hess": 0.7617256833333331, "sample": 0.15100645000000001, "subset": 0.00016500000000000003, "train_predict": 1.1090796, "tree": 19.767063116666673}, "eval_logloss": 0.35535923373212674, "tree_ms_samples": [19.06247204, 19.648262640000002, 19.13946804], "bin_s_samples": [0.18786386, 0.165093033, 0.172283982], "label": "8b-none-d5-nbins255", "commit": "6639d83-dirty", "extra": ["--nbins", "255"]} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 152.81576962000003, "sample_ms": 1.59407276, "subset_ms": 0.00015199999999999998, "tree_rows": 1000000, "bin_s": 1.8292819740000001, "per_iteration_ms": {"eval_predict_metric": 8.967785616666667, "grad_hess": 7.605584383333336, "sample": 1.6459837166666667, "subset": 0.00013999999999999993, "train_predict": 8.316120566666667, "tree": 154.60135626666664}, "eval_logloss": 0.361499131954832, "tree_ms_samples": [153.81002516, 152.81576962000003, 150.69120338], "bin_s_samples": [1.8292819740000001, 1.788062134, 1.829420114], "label": "8b-none-d5", "commit": "6639d83-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 255, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 155.65956226, "sample_ms": 1.57977576, "subset_ms": 0.00016999999999999999, "tree_rows": 1000000, "bin_s": 1.793212591, "per_iteration_ms": {"eval_predict_metric": 9.009074133333334, "grad_hess": 7.647316283333333, "sample": 1.6381934333333334, "subset": 0.0001649999999999999, "train_predict": 8.263323850000003, "tree": 156.1266067666666}, "eval_logloss": 0.36205314097031416, "tree_ms_samples": [155.65956226, 150.32418414, 156.18679082], "bin_s_samples": [1.793212591, 1.769582252, 1.9540599379999999], "label": "8b-none-d5-nbins255", "commit": "6639d83-dirty", "extra": ["--nbins", "255"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 9.57813848, "sample_ms": 0.0286206, "subset_ms": 0.00012200000000000001, "tree_rows": 25000, "bin_s": 0.043693042, "per_iteration_ms": {"eval_predict_metric": 0.31665356666666666, "grad_hess": 0.2062993666666667, "sample": 0.02702889999999999, "subset": 9.666666666666659e-05, "train_predict": 0.33510058333333326, "tree": 8.72128465}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [8.45889836, 9.57813848, 11.57561348], "bin_s_samples": [0.043693042, 0.041930903, 0.048634648], "label": "8c-inplace-none-d5", "commit": "b2a8687-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 21.441563220000003, "sample_ms": 0.1613114, "subset_ms": 0.000166, "tree_rows": 100000, "bin_s": 0.169179947, "per_iteration_ms": {"eval_predict_metric": 1.0553676833333332, "grad_hess": 0.7800935833333337, "sample": 0.1622917333333334, "subset": 0.00016666666666666663, "train_predict": 1.0940152833333328, "tree": 21.9972592}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [21.1021729, 21.441563220000003, 22.98880282], "bin_s_samples": [0.166178883, 0.169179947, 0.177343978], "label": "8c-inplace-none-d5", "commit": "b2a8687-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 46.59344308, "sample_ms": 0.37720784, "subset_ms": 0.000144, "tree_rows": 250000, "bin_s": 0.477163498, "per_iteration_ms": {"eval_predict_metric": 2.74068015, "grad_hess": 2.1122388500000002, "sample": 0.4549477166666665, "subset": 0.00031666666666666686, "train_predict": 2.9436411166666665, "tree": 56.344588733333346}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [56.05950482, 45.29156216, 46.59344308], "bin_s_samples": [0.477163498, 0.501387943, 0.451761719], "label": "8c-inplace-none-d5", "commit": "b2a8687-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 30.109531200000003, "sample_ms": 0.027055549999999998, "subset_ms": 0.00012505, "tree_rows": 25000, "bin_s": 0.043533798, "per_iteration_ms": {"eval_predict_metric": 0.4420958333333334, "grad_hess": 0.19114393333333327, "sample": 0.031903900000000006, "subset": 0.00011003333333333325, "train_predict": 0.8659210999999997, "tree": 32.87289823333334}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [30.109531200000003, 33.26219605, 29.51464875], "bin_s_samples": [0.047055671, 0.042726281, 0.043533798], "label": "8c-inplace-none-d8", "commit": "b2a8687-dirty", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 67.67695264999999, "sample_ms": 0.13739795, "subset_ms": 0.000175, "tree_rows": 100000, "bin_s": 0.173601081, "per_iteration_ms": {"eval_predict_metric": 1.5791857666666662, "grad_hess": 0.7667191333333333, "sample": 0.1416296333333333, "subset": 0.0001633333333333333, "train_predict": 3.266717366666667, "tree": 75.94016086666669}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [67.67695264999999, 68.6589494, 67.0955724], "bin_s_samples": [0.173601081, 0.168144839, 0.179179865], "label": "8c-inplace-none-d8", "commit": "b2a8687-dirty", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 10.467236640000001, "sample_ms": 0.9150747, "subset_ms": 1.03089714, "tree_rows": 29993, "bin_s": 0.172606139, "per_iteration_ms": {"eval_predict_metric": 1.131874783333334, "grad_hess": 0.8574176333333331, "sample": 0.913528666666667, "subset": 1.1671356333333331, "train_predict": 1.1799475666666666, "tree": 10.784302733333334}, "eval_logloss": 0.3461969207536007, "tree_ms_samples": [10.6076355, 10.467236640000001, 10.25555828], "bin_s_samples": [0.190017786, 0.158073941, 0.172606139], "label": "8c-inplace-random30-d5", "commit": "b2a8687-dirty", "extra": ["--sample-method", "random", "--subsample", "0.3"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 17.704476439999997, "sample_ms": 2.2702905, "subset_ms": 2.70701352, "tree_rows": 75170, "bin_s": 0.470283641, "per_iteration_ms": {"eval_predict_metric": 2.5128831833333325, "grad_hess": 1.9195093000000003, "sample": 2.239862666666667, "subset": 2.8177777333333327, "train_predict": 2.694558633333334, "tree": 17.9682017}, "eval_logloss": 0.3461480650012045, "tree_ms_samples": [17.975752460000002, 17.266449480000002, 17.704476439999997], "bin_s_samples": [0.471029357, 0.470283641, 0.448831289], "label": "8c-inplace-random30-d5", "commit": "b2a8687-dirty", "extra": ["--sample-method", "random", "--subsample", "0.3"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 255, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 8.248697, "sample_ms": 0.02191628, "subset_ms": 0.000106, "tree_rows": 25000, "bin_s": 0.040459828, "per_iteration_ms": {"eval_predict_metric": 0.3108596833333333, "grad_hess": 0.1910556666666667, "sample": 0.025040349999999996, "subset": 0.00011334999999999991, "train_predict": 0.41871351666666684, "tree": 8.409182283333333}, "eval_logloss": 0.3699417432836186, "tree_ms_samples": [8.248697, 8.95715336, 7.33983096], "bin_s_samples": [0.040227824, 0.042612972, 0.040459828], "label": "8c-inplace-none-d5-nbins255", "commit": "b2a8687-dirty", "extra": ["--nbins", "255"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 255, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 20.77190132, "sample_ms": 0.13668492, "subset_ms": 0.000174, "tree_rows": 100000, "bin_s": 0.167390196, "per_iteration_ms": {"eval_predict_metric": 1.0841870333333334, "grad_hess": 0.7640071833333335, "sample": 0.1333610166666667, "subset": 0.00018166666666666667, "train_predict": 1.1132009000000003, "tree": 21.734149266666666}, "eval_logloss": 0.35535923373212674, "tree_ms_samples": [21.227040679999998, 18.67819288, 20.77190132], "bin_s_samples": [0.156579677, 0.167390196, 0.171915888], "label": "8c-inplace-none-d5-nbins255", "commit": "b2a8687-dirty", "extra": ["--nbins", "255"]} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 168.38434736, "sample_ms": 1.5837002199999999, "subset_ms": 0.000194, "tree_rows": 1000000, "bin_s": 1.787713178, "per_iteration_ms": {"eval_predict_metric": 9.073341066666668, "grad_hess": 7.573325983333331, "sample": 1.6336812500000006, "subset": 0.00019333333333333347, "train_predict": 8.29470548333333, "tree": 163.14628489999998}, "eval_logloss": 0.361499131954832, "tree_ms_samples": [162.05946609999998, 168.38434736, 172.93713748], "bin_s_samples": [1.756253521, 1.787713178, 1.8038619329999999], "label": "8c-inplace-none-d5", "commit": "b2a8687-dirty", "extra": []} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 255, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 168.13818268, "sample_ms": 1.60364838, "subset_ms": 0.00024601999999999995, "tree_rows": 1000000, "bin_s": 1.796918132, "per_iteration_ms": {"eval_predict_metric": 9.385940733333333, "grad_hess": 7.657112566666665, "sample": 1.6539599666666664, "subset": 0.00020999999999999998, "train_predict": 8.608619950000001, "tree": 169.06943383333328}, "eval_logloss": 0.36205314097031416, "tree_ms_samples": [167.59268656, 170.5478896, 168.13818268], "bin_s_samples": [1.918634259, 1.796918132, 1.784605262], "label": "8c-inplace-none-d5-nbins255", "commit": "b2a8687-dirty", "extra": ["--nbins", "255"]} diff --git a/agent/feat/hist-perf/results.md b/agent/feat/hist-perf/results.md deleted file mode 100644 index 9f1fb86..0000000 --- a/agent/feat/hist-perf/results.md +++ /dev/null @@ -1,78 +0,0 @@ -# Histogram Performance Results (ideas 8 and 9) - -Date: 2026-10-06. Branch: `feat/hist-perf` (off `feat/goss`). Ideas from -`agent/perf-ideas.md`. Machine: 8 physical cores (16 vCPU). All timings at 8 -threads on `make_perf_data.py` data. Every kept change leaves models -byte-identical (`scripts/perf_golden.py`, 10 configurations including sampling, -LossGuide, missing branch and `nbins` 255/64), and the Python suite passes -(121 tests). - -## Outcome - -| Idea | Result | -| --- | --- | -| 8a. Skip histograms for children at `max_depth` | **Kept** (`6639d83`). 10-35% less tree time. | -| 8b. Drop `cut_value` from `Bin` | **Kept** (`b2a8687`). Up to 21% less tree time on small nodes and deep trees. | -| 8c. Reuse histogram buffers | **Dropped.** In-place sibling subtraction, with or without a buffer pool, was slower at depth 5 (250k: +6%, 1M: +7%) and not clearly better at depth 8. | -| 8d. Reuse the per-column `sums` buffer | **Dropped.** A thread-local buffer was slower (250k: +6-30%, 100k depth 8: +17%). | -| 9. `u8` bins | **Skipped.** Microbenchmark below: modest gain, only when `nbins <= 255`, and the generic plumbing is large. | -| 9. `u32` row index | **Rejected.** No gain in the microbenchmark. | - -## End-to-end fit, 100 iterations, one evaluation set - -Before is `feat/goss` (`agent/feat/goss/results.jsonl`, label `fixed`); after -is this branch (`fit.jsonl`). Median total seconds; GOSS and `random80` over -seeds 0-2. Eval logloss is identical before and after. - -| Data | Rows | Setting | Before s | After s | Change | -| --- | ---: | --- | ---: | ---: | ---: | -| w200 | 100k | none | 3.47 | 2.49 | -28.1% | -| w200 | 100k | random80 | 3.21 | 2.30 | -28.3% | -| w200 | 100k | goss | 2.31 | 1.94 | -16.0% | -| w200 | 250k | none | 7.91 | 5.60 | -29.2% | -| w200 | 250k | random80 | 7.13 | 5.18 | -27.4% | -| w200 | 250k | goss | 4.42 | 3.46 | -21.7% | -| w500 | 100k | none | 7.92 | 5.76 | -27.3% | -| w500 | 100k | random80 | 7.20 | 4.98 | -30.9% | -| w500 | 100k | goss | 4.88 | 3.57 | -26.8% | -| w200-1m | 1M | none | 26.74 | 19.53 | -27.0% | -| w200-1m | 1M | random80 | 24.10 | 18.35 | -23.8% | -| w200-1m | 1M | goss | 15.35 | 12.92 | -15.8% | - -For reference, LightGBM with 80% bagging took 2.42 s (100k), 5.39 s (250k) and -16.28 s (1M) on the same data in the GOSS benchmark. - -## Steady-state tree time per tree (`perf_sweep.py`, ms) - -Raw records in `results.jsonl`, labels `baseline`, `8a`, `8b`. - -| Configuration | Baseline | 8a | 8b | -| --- | ---: | ---: | ---: | -| w200 25k, depth 5 | 14.9 | 11.0 | 8.7 | -| w200 100k, depth 5 | 30.1 | 20.3 | 20.1 | -| w200 250k, depth 5 | 67.5 | 43.8 | 41.3 | -| w200 25k, depth 8 | 48.5 | 37.5 | 31.5 | -| w200 100k, depth 8 | 91.0 | 82.3 | 73.1 | -| w200 100k, random30 | 15.7 | 11.5 | 9.7 | -| w200 250k, random30 | 25.5 | 21.3 | 17.6 | -| w200-1m 1M, depth 5 | 224.7 | 158.0 | 152.8 | - -Single runs vary by about 5-10%; 8c and 8d were judged with interleaved A/B runs -(`/tmp/forust-perf/ab.sh`), 3-4 rounds each. - -## Item 9 microbenchmark - -Histogram fill over 200 columns, parallel over columns, 8 threads, best of 7. -"30%" is a sorted index of 30% of rows. - -| Rows | Index | u16 / usize (now) | u8 / usize | u16 / u32 | u8 / u32 | -| ---: | --- | ---: | ---: | ---: | ---: | -| 100k | full | 3.80 ms | 3.47 ms | 3.75 ms | 3.49 ms | -| 100k | 30% | 1.39 ms | 1.11 ms | 1.16 ms | 0.99 ms | -| 1M | full | 31.03 ms | 28.95 ms | 33.23 ms | 30.99 ms | -| 1M | 30% | 12.27 ms | 9.86 ms | 11.95 ms | 9.71 ms | - -`u8` bins make the fill 7-20% faster, which is likely 4-10% of tree time, and -only applies when every column has at most 255 non-missing bins (`nbins <= 255` -or low-cardinality data). The default `nbins=256` needs `u16`: bin 0 is missing -and bins 1-256 hold values. diff --git a/agent/feat/optimize/plan.md b/agent/feat/optimize/plan.md deleted file mode 100644 index 0553684..0000000 --- a/agent/feat/optimize/plan.md +++ /dev/null @@ -1,376 +0,0 @@ -# Forust Performance Implementation Plan - -Date: 2026-10-05 - -This plan implements the four recommendations in `recommendations.md`. It -also defines the speed and correctness checks for each one. - -## Ground rules - -- **One change per commit.** Each item, and each sub-item of item 2, is a - separate commit. Every commit gets its own before/after measurement, so a - regression can be traced to one change. -- **Models must be bit-identical.** None of these changes alter what the - algorithm computes, so the saved model JSON must be byte-for-byte the same - before and after, for both serial and parallel training. This is stricter - than the Python suite's XGBoost tolerance checks, and catches subtle - ordering bugs. -- **`parallel=false` stays fully single-threaded.** Every new parallel path is - gated on the existing `parallel` flag. -- **Every commit passes:** `cargo test --all-targets` and the Python suite - (111 tests, including the XGBoost parity checks). - -## Step 0: Baseline and tooling (no library changes) - -1. **Harness flags** (`examples/perf_phases.rs`): - - `--save-model PATH` writes the trained booster with `save_booster`. - - `--missing-branch true` trains with `create_missing_branch`, so the second - splitter (`MissingBranchSplitter`) is covered. -2. **Golden models:** with the current code, save models for these - configurations to `/tmp/forust-perf/golden/`: - - | Data | Rows | Depth | Iterations | Mode | - | --- | ---: | ---: | ---: | --- | - | `w200` | 100k | 5 | 50 | serial and 8 threads | - | `w200` | 25k | 8 | 20 | 8 threads | - | `w500` | 25k | 5 | 20 | 8 threads | - | `w200` | 25k | 5 | 20 | 8 threads, missing branch | - - After each change, retrain and compare with `cmp`. First, confirm that - today's serial and parallel models already match each other. If they don't, - find out why before changing anything. -3. **Determinism test** (Rust, in `src/gradientbooster.rs` tests): train on the - Titanic resource data with `parallel` true and false, for both splitters, - and assert `json_dump()` is identical. This runs in CI and protects all - four items. -4. **Sweep tooling:** - - `scripts/perf_sweep.py`: add `--repeats 3` (report the median) and - `--label` to tag result rows, such as `baseline` or `item1`. - - New `scripts/perf_compare.py`: prints a before/after table by - configuration from two labels. -5. **Baseline run of the core grid** (median of 3 repeats, about 8 minutes): - - | Grid | Data | Rows | Depth | Iterations | Modes | What it isolates | - | --- | --- | --- | ---: | ---: | --- | --- | - | G1 | `w200` | 25k, 100k, 250k | 5 | 60 | serial, 8 threads | Fixed vs per-row cost | - | G2 | `w200` | 25k, 100k | 8 | 30 | serial, 8 threads | Many small nodes | - | G3 | `w500` | 25k, 100k | 5 | 30 | serial, 8 threads | Wide data | - | G4 | `w200-1m` | 1M | 5 | 20 | 8 threads | Final confirmation only | - - Also record the **main-thread share** of the tree phase from a `perf` - profile at 100k x 200, depth 5, 8 threads (`perf report -s pid`). The - baseline is about 50%. This is the most direct measure of serial work - remaining. - -## Item 1: Parallelize the split search across features - -**Change** (`src/splitter.rs`): - -- Add a `parallel` argument to `Splitter::best_split`; `split_node` passes its - existing `parallel` value. -- Parallel branch: scan features with `par_iter`, keep only splits with gain - > 0, and reduce to the highest gain, breaking ties by the lowest feature - position. The serial branch is unchanged. -- Add `Sync` as a supertrait of `Splitter`. Both splitters only hold - thread-safe data, so this needs no other changes. -- Batch features per task (`with_min_len`). Start at 8 and tune between 1, 8, - and 32 using G2 at 25k rows, which is most sensitive to scheduling - overhead. - -**Why the model can't change:** each feature's best split is computed by the -same function as today. Today's loop keeps the first feature with a strictly -higher gain. The reduction reproduces that exactly: highest gain wins, and on -a tie the lowest feature position wins. - -**Correctness:** a new unit test builds random histograms, including forced -gain ties between features, and checks that parallel and serial -`best_split` pick the same feature, bin, and gain. Then golden models and the -determinism test. - -**Speed testing:** - -| Check | Expected result | -| --- | --- | -| G1-G3, 8 threads | Fixed per-tree cost falls; 25k rows and depth 8 improve most | -| G1-G3, serial | Unchanged within noise (control) | -| Per-row cost (slope of G1) | Unchanged | -| Main-thread profile | `evaluate_split` moves to the worker threads; main-thread share falls from ~50% toward ~30% | - -Targets (estimates; record actuals): 100k x 200 depth 5 on 8 threads from -69 ms to 55 ms or less per tree; 100k depth 8 from 349 ms to 260 ms or less. - -## Item 2: Move the remaining per-node work into the parallel section - -Four sub-items, each a separate commit (`src/histogram.rs`, `src/splitter.rs`): - -- **2a. Stop copying gradients/Hessians when they don't need reordering.** - `HistogramMatrix::new` copies the full arrays with `to_vec()` when - `sort == false`; borrow them instead (`Cow<[f32]>`). When a reorder is - needed, gather in parallel, preserving order. -- **2b. Write histograms in place.** Allocate the node's full histogram once. - Split it into one slice per column and fill the slices in parallel. This - replaces the parallel `flat_map`, which allocates a vector per column and - then concatenates. Each column keeps the same f64 accumulation and f32 - conversion, so bins are identical. -- **2c. Parallel histogram subtraction.** `from_parent_child` and - `from_parent_two_children` subtract element by element. Run them in - parallel in large chunks when `parallel` is set; this needs a new argument - at 7 call sites across both splitters. -- **2d. Small-node threshold (optional).** When a node's rows x columns is - below a threshold, build its histogram serially even in parallel mode, - because scheduling costs more than the work. Tune with G2 at 25k rows, - trying thresholds of 0, 50k, 200k, and 1M elements. - -**Not now:** row partitioning (`pivot_on_split`, about 3.5% of serial time). -Revisit only if it becomes the top item in the main-thread profile. - -**Correctness:** a new unit test checks that serial and parallel -`HistogramMatrix::new` and subtraction give identical bins. Then golden -models and the determinism test. - -**Speed testing:** - -| Check | Expected result | -| --- | --- | -| G1-G3 after each of 2a-2d | Each step improves or is neutral; any step that regresses is dropped | -| Main-thread profile | Gather, subtraction, `memmove`, and `free` leave the main thread | -| Peak memory (`/usr/bin/time -v`, 100k x 200 depth 8) | No increase; 2b should reduce allocations | - -Targets: 8-thread speedup at 100k x 200 depth 5 of 3x or more (from 2.2x); -depth 8 of 2.5x or more (from 1.5x); no regression versus serial at 25k rows. - -## Item 3: Parallelize binning and sort each column once - -**Change** (`src/binning.rs`): - -- **3a. One sort per column.** Today `percentiles_or_value` sorts and - deduplicates a copy of the column, then `percentiles` sorts an index array. - Instead, sort the index array once, with the same comparator as today. Count - distinct values while walking it. If there are at most `nbins + 1`, return - them; otherwise run the existing percentile walk on the same sorted index. - Same sort on the same input, so cuts are identical. The only possible - difference is which of `-0.0` or `0.0` is kept, and the two compare equal, - so bins don't change. -- **3a' (optional).** Sort (value, weight) pairs directly instead of an index - array, which is more cache-friendly and faster. Tied values may be added in - a different order, so with non-uniform weights a cut could differ in the - last digit in rare cases. Measure it, and only adopt it with your approval - and if golden models and weighted tests are unchanged. -- **3b. Parallel per column.** Compute each column's cuts in parallel - (`into_par_iter().map().collect()` keeps column order), then assemble the - cut matrix serially. Assign bins in parallel per column, which also removes - the per-element `i / rows` division. -- **API:** `bin_matrix` is public. Option A: add a `parallel: bool` argument - (breaks direct crate users; about 15 internal call sites, mostly tests and - benches). Option B: keep the signature and add a parallel variant. - `GradientBooster` passes `self.parallel`. - -**Correctness:** the existing `test_bin_data`, plus a new test that serial and -parallel cuts and binned data are identical. Then golden models and the -weighted Python tests. - -**Speed testing:** - -| Check | Expected result | -| --- | --- | -| Binning time from `perf_phases` at 100k, 250k, 1M; serial and 8 threads | 3a: serial faster by roughly a third; 3b: 5-7x faster at 8 threads | -| 1M x 200, 100-iteration end-to-end fit | Total time falls by roughly the binning savings | - -Target: 1M x 200 binning from 24 s to 5 s or less on 8 threads. - -Also replace the toy `wide-200-column` Criterion group in -`benches/forust_benchmarks.rs`. It can load the generated data when an -environment variable points to it (skipping otherwise), or be removed in -favor of `perf_phases`. - -## Item 4: Thread count control - -**Change:** - -- Add `num_threads: Option` to `GradientBooster`. Use a serde default - of `None` so older saved models still load, and add a `set_num_threads` - setter. -- When it is set and `parallel` is true, build a local Rayon thread pool and - run `fit`, `predict`, `predict_contributions`, and partial dependence inside - it (`pool.install`). `None` keeps today's behavior (the global pool). -- Python: add `num_threads: int | None = None` to `GradientBooster.__init__` - (`py-forust/forust/__init__.py`), the pyo3 constructor and `get_params` - (`py-forust/src/lib.rs`), and the docstrings. - -**Correctness:** a Rust test that `num_threads` of 1, 2, and 4 give identical -models. A Python test that the parameter survives `get_params` and -save/load, and that predictions match. - -**Speed testing:** - -| Check | Expected result | -| --- | --- | -| G1, G2 with `num_threads` of 1, 2, 4, 8, 16 | Matches the same `RAYON_NUM_THREADS` runs within noise, confirming the wiring | -| `predict` on 1k rows x 200 columns, 1,000 calls, `num_threads=8` vs unset | Pool creation adds under ~5% per call; otherwise cache the pool on the booster | - -## Final confirmation - -After all four items: - -- Run the full G1-G4 grid and the XGBoost comparison with the same scripts. -- Repeat the 1,000-iteration drift run on 100k x 200. -- Run the Python suite. -- Update `recommendations.md` with a before/after table. - -## Decisions needed from you - -1. **Bit-identical models** as the acceptance bar for items 1-3 (recommended)? - Answer: Yes -2. **`bin_matrix` API:** add a `parallel` argument (Option A) or a separate - parallel function (Option B)? - Answer: No one uses that function API breaking is OK, add a prallel argument. -3. **`num_threads`:** is the name OK, and should the default stay `None` - (today's behavior)? Defaulting to physical cores would need a new - dependency or a heuristic, and should first be measured on your production - machines. - answer: I think its fine. -4. **3a':** try the faster pair sort, accepting a small risk of last-digit cut - differences with non-uniform weights, or skip it? - ansewr: We could test 3a last - -## Item 5: Parallel row partitioning (not started) - -Added after items 1-4 were done. This section is a self-contained hand-off: -items 1-4 are implemented and measured (see `results.md`), and the ground -rules above still apply. Byte-identical trees are required. - -### Why - -At 1M rows x 200 columns, depth 8, 8 threads, Forust takes 633.7 ms per tree -versus 454.6 ms for XGBoost (39% slower). From depth 5 to 8, Forust's tree -time grows 177% at 1M rows versus 22% for XGBoost. After every split, the -node's rows are reordered on one thread, so each tree level makes a serial pass -over all rows. It was the largest named serial item in the main-thread profile -after item 2c (`Splitter::handle_split_info`, which inlines the partition). - -**First, confirm this with a profile** of 1M x 200, depth 8, 8 threads. If -partitioning isn't a large share there, stop and re-plan: - -```sh -CARGO_PROFILE_RELEASE_DEBUG=true cargo build --release --example perf_phases -RAYON_NUM_THREADS=8 perf record -F 999 --delay 25000 -o /tmp/forust-perf/rows.data -- \ - target/release/examples/perf_phases --data /tmp/forust-perf/w200-1m \ - --rows 1000000 --max-depth 8 --iterations 10 -perf report -i /tmp/forust-perf/rows.data --stdio --no-children -s symbol --percent-limit 1 -``` - -The delay skips data loading and binning (roughly 20-25 s at 1M rows; adjust if -the profile shows `read_f64` or `bin_matrix`). Look for `pivot_on_split`, -`pivot_on_split_exclude_missing`, or `handle_split_info`. - -### Where the code is - -- `src/utils.rs`: `pivot_on_split` (two-way split, used by - `MissingImputerSplitter`) and `pivot_on_split_exclude_missing` (three-way: - missing, left, right; used by `MissingBranchSplitter`). -- `src/splitter.rs`: both `handle_split_info` implementations call these on - `&mut index[node.start_idx..node.stop_idx]`, then build the smaller child's - histogram from its slice of `index`. Both receive the `parallel` flag. -- Existing tests: `test_pivot` and `test_pivot_missing` in `src/utils.rs`. - -### The order requirement - -Each child's histogram is built by summing gradients in the order of its rows -in `index` (f64 sums, then rounded to f32). Changing that order can change the -last bits of the sums, then split gains and tie-breaks, so trees stop being -byte-identical. A parallel version must produce **exactly the same `index` -order** as today's functions, not just the same set of rows on each side. - -`pivot_on_split` is a Hoare-style partition. Its output order has a closed -form. A Python port of the loop matched this closed form on 200,000 random -inputs (lengths 1-40, random bins, missing rows, both `missing_right` -settings); still confirm it with Rust tests: -- `low` scans forward and stops at each row that belongs on the right; - `high` scans backward and stops at each row that belongs on the left; the - two are swapped. -- So the k-th right-side row from the front is swapped with the k-th left-side - row from the back, for k = 1..m, where m is the number of misplaced pairs. -- The result is computable without the sequential loop: - 1. Let L be the number of rows that go left. The returned split index is L, - **except** when every row goes left (L = n), where it returns n - 1. - That case never happens in training (both sides of a split have rows), - but match it anyway. - 2. Misplaced right rows are the right-side rows in positions `[0, L)`; - misplaced left rows are the left-side rows in positions `[L, n)`. There - are equally many, m. - 3. The k-th misplaced right row (counting forward from the start) swaps - places with the k-th misplaced left row counting backward from the end. - Every other row stays where it is. - -This is parallelizable: -1. Classify every row and count left rows per chunk (parallel); a prefix sum - gives L. -2. In parallel per chunk, number the misplaced right rows in `[0, L)` from the - front and the misplaced left rows in `[L, n)` from the back (chunk counts - plus prefix sums give each its rank). -3. Write the swaps into a copy of the slice, or swap in place since each swap - pair is disjoint. - -`pivot_on_split_exclude_missing` is harder: it also moves missing rows to the -front with extra swaps inside the same loop, so its output order interleaves -two processes. Either derive its closed form the same way and test it -exhaustively, or keep it serial and parallelize only `pivot_on_split` first. -`MissingImputerSplitter` is the default (`create_missing_branch=False`), so -`pivot_on_split` covers the common case. - -Edge cases to preserve: -- `pivot_on_split` handles missing rows through `missing_compare` and the - `missing_right` flag. -- Both functions index `index.len() - 1`, so they assume a non-empty slice. -- The returned split index (and the missing index for the three-way version) - must match exactly. - -### Implementation steps - -1. **Reference tests first.** Add property tests in `src/utils.rs` that - compare a new `pivot_on_split_parallel` against `pivot_on_split` on random - inputs: lengths 1 to ~10,000, many bin values, values equal to the split - value, all-left, all-right, `missing_right` both ways, and many missing - rows. Assert the returned index **and the full `index` order** are equal. -2. **Implement the closed form serially** and make those tests pass. This - proves the reasoning before adding threads. -3. **Parallelize it** with Rayon (chunks of a few thousand rows, chunk counts, - prefix sums). Keep the property tests passing. -4. **Use it only for large nodes:** call the parallel version in - `handle_split_info` when `parallel` is set and the node has more than a - threshold of rows (start at 65,536 and tune). Small nodes keep the serial - function, where Rayon overhead would dominate. -5. **Optional:** repeat for `pivot_on_split_exclude_missing` if the profile - shows `MissingBranchSplitter` matters for your workloads. - -### Correctness - -- New property tests (above), plus `cargo test --all-targets`. -- `python scripts/perf_golden.py check`: all 5 golden models `same`. The - `missing_branch` golden model covers the three-way function. -- The Python suite (`uv run pytest` in `py-forust`). - -### Speed testing - -Add a G5 grid to `scripts/perf_grid.py` for the case that motivated this item: - -```python -"G5": [("w200-1m", 1_000_000, 8, 20, False)], -``` - -Regenerate the 1M data if needed: - -```sh -python scripts/make_perf_data.py --out /tmp/forust-perf/w200-1m --rows 1000000 --eval-rows 250000 --cols 200 -``` - -| Check | Expected result | -| --- | --- | -| G5 (1M x 200, depth 8, 8 threads), before and after | The main target; baseline 633.7 ms/tree | -| G4 (1M x 200, depth 5) | Improves, less than G5 | -| G1-G3 | Unchanged or better; small nodes shouldn't regress (threshold) | -| Threshold sweep: 16k, 65k, 262k rows | Pick the fastest across G1, G2, G5 | -| `scripts/perf_xgboost.py --data /tmp/forust-perf/w200-1m --rows 1000000 --threads 8 --iterations 20 --max-depth 8` | Rerun XGBoost (454.6 ms) for comparison | - -Record results with `--label item5` and add a step-log entry and summary -column to `results.md`, as for items 1-4. diff --git a/agent/feat/optimize/recommendations.md b/agent/feat/optimize/recommendations.md deleted file mode 100644 index 81997ba..0000000 --- a/agent/feat/optimize/recommendations.md +++ /dev/null @@ -1,427 +0,0 @@ -# Forust Performance Recommendations - -Date: 2026-10-05, updated 2026-10-06 - -Status: the four recommended changes are implemented on branch -`feat/optimize` (local commits, not pushed). Per-change measurements are in -`results.md`. The original analysis follows the final recommendations below. - -Target workload: binary classification, 200+ columns, up to 1,000+ iterations, -and one evaluation set scored on every iteration. - -## Final results - -All changes leave trained models byte-identical (checked after every commit), -and the Python suite passes (112 tests, including the XGBoost parity checks). - -| Workload | Before | After | -| --- | ---: | ---: | -| 100-iteration `fit`, 1M x 200, 8 threads | 52.7 s | 26.1 s (2.0x) | -| 1,000 iterations, 100k x 200, default threads | 80.8 s | 37.2 s (2.2x) | -| Tree time, 8 threads, 25k-250k rows | | 39-81% less | -| Binning, 8 threads | | 91-93% less (1M x 200: 24.7 s to 1.8 s) | -| 8-thread speedup over serial | 1.0-3.1x | 4.6-5.2x | - -Versus XGBoost `hist` at 8 threads, Forust went from 1.6-6x slower to: -- within 1-30% on small and deep cases; -- faster on 100k x 500 (79.5 vs 107.1 ms) and 1M x 200 at depth 5 - (228.9 vs 373.2 ms); -- but 39% slower on 1M x 200 at depth 8 (633.7 vs 454.6 ms). - -A matched 1M x 200, depth 6, 20-iteration `LossGuide`/`lossguide` fit with -one evaluation set was effectively tied: Forust 8.07 s median versus XGBoost -8.15 s. Forust's corrected tree-only phase was 310.0 ms versus XGBoost's -407.3 ms, but -the full-fit comparison is the meaningful one because it includes predictions -and evaluation in both libraries. - -| Change | Commit | Main effect | -| --- | --- | --- | -| Parallel split search | `c111e36` | 17-38% faster at 8 threads | -| Borrow gradients instead of copying | `763e861` | 3-10% faster | -| Write histograms in place | `18744aa` | 15-33% faster at 8 threads | -| Parallel histogram subtraction | `aefa101` | Up to 28% faster at 8 threads | -| Parallel binning, one sort per column | `86a77b3` | Binning 11-14x faster | -| `num_threads` setting | `fcbc1c0` | Thread-count control | -| Run parallel work on a pool thread | `c1baff6` | 4-40% faster at 8 threads | - -Tried and dropped: building small nodes serially (slower at every threshold), -and a faster pair sort for weighted binning (~2% gain, small risk of -last-digit cut changes). - -## Recommendations - -1. **Merge the branch.** Every change is model-neutral and measured. API - changes to review: - - `bin_matrix` takes a `parallel` argument (approved). - - The `Splitter` trait now requires `Sync`, and `best_split` takes a - `parallel` argument; this only affects code implementing its own splitter. - - New optional `num_threads` setting in Rust and Python. Older saved models - still load. - - The branch also includes the analysis tooling (`scripts/`, - `examples/perf_phases.rs`), the Python packaging fix, and your `polars` - removal. -2. **Set `num_threads` to the number of physical cores.** On this VM - (8 cores, 16 vCPU), the default of 16 threads is 2-18% slower than 8 at - depth 5 and 25-66% slower at depth 8. Before changing the default, measure - on your production machines; detecting physical cores would need a new - dependency or a heuristic. -3. **Next optimization candidates, if you want more.** - - **Row partitioning, first.** After each split, `pivot_on_split` reorders - the node's rows on one thread, so every tree level makes a serial pass - over all rows. It was the top remaining serial item after item 2c, and - is the likely reason Forust is 39% slower than XGBoost at 1M x 200, - depth 8. Profile that case first to confirm. - - In the final profile (25k x 200, depth 8, 8 threads), the remaining time - is mostly: - - the split search itself, ~40% (`evaluate_split` and its loop): it - computes weights and gains for every bin of every feature at every node; - skipping bins that fail `min_leaf_weight` before computing gains would - cut work without changing results; - - histogram accumulation, ~17%: already parallel; fusing it with the split - scan per column (one pass while the histogram is in cache) is the - follow-up from the plan; - - Rayon scheduling, ~15%: tasks are small with many tiny nodes at depth 8. - - This is where the remaining gap to XGBoost on deep, small-row trees comes - from. The gains here are likely smaller than what's already done. -4. **Keep the tooling as a regression check.** `scripts/perf_golden.py` proves - models are unchanged; `scripts/perf_grid.py` and `scripts/perf_compare.py` - reproduce the timing tables. The serial-versus-parallel determinism test - already runs in `cargo test`. The toy `wide-200-column` Criterion group was - removed; `examples/perf_phases.rs` on generated data replaces it. -5. **Read small serial differences as noise.** Serial timings moved by up to - ±6% between builds because a hot loop's position in memory changed, even - with byte-identical instructions (see item 2c in `results.md`). - -Housekeeping: `perf.data` in the repository root is the raw capture from your -flamegraph run and can be deleted. The benchmark datasets in `/tmp/forust-perf` -may not survive the VM being deallocated, but `scripts/make_perf_data.py` -regenerates them exactly. - ---- - -# Original analysis (2026-10-05) - -This is the analysis that led to the plan, before any library change. - -## Summary - -1. **Tree building is ~93% of training time.** In a 1,000-iteration run - (100k rows x 200 columns), the other phases are small: - - | Phase | Share | - | --- | ---: | - | Binning | 2.4% | - | Training-set predictions | 1.7% | - | Evaluation predictions + metric | 1.6% | - | Gradients/Hessians | 1.0% | - - Per-tree cost is stable across the run, falling from 84 ms early to about - 75 ms. -2. **Single-threaded, Forust is about as fast as XGBoost `hist`.** On the same - data and settings, Forust's serial tree time is 0.64-1.5x XGBoost's. -3. **The gap is parallel scaling.** At 8 threads: - - XGBoost speeds up 3.6-4.7x. - - Forust speeds up 1.0-3.0x up to 250k rows, and 4.5x at 1M rows. -4. **Why Forust scales poorly:** only histogram accumulation runs in parallel. - The rest of the per-node work stays serial on the main thread: - - the split search - - histogram subtraction - - gathering gradients/Hessians - - concatenating per-column histograms - - row partitioning - - This serial work is about half of the tree-building time at 100k x 200 with - 8 threads. It grows with columns x bins x nodes, so deep trees suffer most: - at depth 8, Forust gets 1.0-1.5x versus XGBoost's 3.6-3.8x. -5. **Binning is serial and slow at large row counts.** At 1M x 200 it takes - 24 s, versus 1.5 s for XGBoost's data preparation (`DMatrix`) on 8 threads. - That is about 8% of a 1,000-iteration fit but almost half of a - 100-iteration fit. -6. **Thread count:** 8 threads (one per physical core) is best for both - libraries. 16 threads (with hyperthreading) is 4-19% slower for Forust and - 55-380% slower for XGBoost. -7. **Several earlier findings were artifacts of the toy benchmark and are - withdrawn** (details below): "parallel is slower than serial", "binning is - the top priority", and "evaluation costs 61%". - -## Method - -- **Data** (`scripts/make_perf_data.py`): scikit-learn `make_classification` - with 20% informative and 20% redundant features, 3 clusters per class, 5% - label noise, and about 17% positives. Then, to mimic real data: - - 25% of columns reduced to 2-50 levels - - 15% zero-inflated "amount" columns - - 3% near-constant columns - - missing values in 30% of columns, at 1-30% rates - - 56 of 200 columns end up with 256 or fewer distinct values. Forust reaches - about 0.90 validation AUC (100k rows, 200 iterations). -- **Datasets** (local disk, `/tmp/forust-perf/`; the script is seeded, so they - can be regenerated): - - | Name | Training rows | Evaluation rows | Columns | - | --- | ---: | ---: | ---: | - | `w200` | 250k | 62.5k | 200 | - | `w500` | 100k | 25k | 500 | - | `w200-1m` | 1M | 250k | 200 | - - Rows are shuffled, so the first N rows are a random sample. -- **Harness** (`examples/perf_phases.rs`): `--mode phases` replays the - `GradientBooster::fit` loop with a timer on each phase; `--mode fit` times - the real `fit`. Both modes give identical evaluation log-loss and total - times within 1%. -- **Sweeps** (`scripts/perf_sweep.py`): steady-state ms per tree (skipping the - first 10 iterations), serial versus `RAYON_NUM_THREADS` values. -- **XGBoost** (`scripts/perf_xgboost.py`): xgboost 1.7.6 `hist` with the same - depth, bins, learning rate, `lambda`, and `min_child_weight`, and one - evaluation set. XGBoost times are full iterations (tree, predictions, and - evaluation); Forust's are tree-only, which understates Forust's iteration - time by 3-10%. -- **Settings** unless noted: learning rate 0.1, depth 5, 256 bins, evaluation - rows = training rows / 4. -- **Host:** 16 vCPU (8 physical cores with hyperthreading), Xeon 8272CL, - 62 GB RAM, Azure VM. Repeat runs vary by about 5%. -- **Profiles:** `perf` at 999-1999 Hz on a release build with debug info, - split by thread ID. - -## Results - -### Where the time goes over 1,000 iterations - -100k x 200, depth 5, default 16 Rayon threads, 80.8 s total: - -| Phase | Time | Share | -| --- | ---: | ---: | -| Tree building | 75.4 s | 93.3% | -| Binning | 2.0 s | 2.4% | -| Training-set predictions | 1.3 s | 1.7% | -| Evaluation predictions + metric | 1.3 s | 1.6% | -| Gradients/Hessians | 0.8 s | 1.0% | - -| Iterations | Mean ms per tree | Mean nodes per tree | -| --- | ---: | ---: | -| 0-10 | 84.5 | 62.8 | -| 10-100 | 79.1 | 62.3 | -| 100-300 | 75.0 | 59.5 | -| 300-600 | 75.4 | 59.5 | -| 600-1000 | 74.6 | 58.8 | - -Trees stay nearly full (63 nodes is the maximum at depth 5), so 50-60 -steady-state iterations are a valid stand-in for a full 1,000-iteration fit. - -### Thread scaling - -Milliseconds per tree, 200 columns, depth 5: - -| Rows | Forust serial | 2 threads | 4 threads | 8 threads | 16 threads | XGBoost serial | XGBoost 8 threads | XGBoost 16 threads | -| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 25k | 63.8 | 55.1 | 48.1 | 52.6 | 60.6 | 49.1 | 12.9 | 62.5 | -| 50k | 92.2 | 72.4 | 59.5 | 56.7 | 67.3 | - | - | - | -| 100k | 152.0 | 98.9 | 74.7 | 68.9 | 77.3 | 123.6 | 27.1 | 42.0 | -| 250k | 305.4 | 186.2 | 124.0 | 100.5 | 104.2 | 268.8 | 56.6 | 95.3 | -| 1M | 1146.4 | - | - | 257.2 | - | 1787.3 | 404.4 | - | - -At 1M rows, a full Forust iteration on 8 threads is about 285 ms (259 ms -tree, plus about 9 ms each for training predictions, evaluation, and -gradients), versus 404 ms for XGBoost. - -### Depth and width - -Milliseconds per tree: - -| Case | Forust serial | Forust 8 threads | Speedup | XGBoost serial | XGBoost 8 threads | Speedup | -| --- | ---: | ---: | ---: | ---: | ---: | ---: | -| 25k x 200, depth 8 | 230.1 | 227.5 | 1.01x | 178.8 | 50.0 | 3.6x | -| 100k x 200, depth 8 | 514.7 | 348.6 | 1.48x | 341.2 | 90.4 | 3.8x | -| 25k x 500, depth 5 | 181.4 | 123.9 | 1.46x | 177.2 | 40.4 | 4.4x | -| 100k x 500, depth 5 | 404.6 | 165.5 | 2.44x | 495.0 | 105.7 | 4.7x | - -### Fixed versus per-row cost - -Reducing bins from 256 to 64 cut about 30 ms per tree at both 25k rows -(63.8 to 31.3 ms) and 250k rows (305.4 to 277.4 ms), so part of each tree's -cost depends on bins rather than rows. Fitting the 25k-250k results: - -| Mode | Fixed cost per tree | Per-row cost | -| --- | ---: | ---: | -| Serial | ~40 ms | ~1.07 ms per 1k rows | -| 8 threads | ~47 ms | ~0.21 ms per 1k rows | - -The per-row part (histogram accumulation) already speeds up about 5x at -8 threads. The fixed part (proportional to columns x bins x nodes) does not -speed up at all. At 100k rows with 8 threads it is about 68% of each tree's -time. With 64 bins, the 8-thread speedup at 250k rows rises from 3.0x to -3.8x. - -### Profiles - -**Serial, 100k x 200, 60 iterations** (share of all samples): - -| Function | Share | -| --- | ---: | -| Histogram accumulation (`create_feature_histogram`) | 52% | -| Binning (`bin_matrix` + percentile sorts) | ~14% | -| `evaluate_split` | 6.6% | -| Concatenating per-column histograms (`flat_map`) | 3.5% | -| Tree prediction | 3.1% | -| Gradient/Hessian gather (`HistogramMatrix::new`) | 1.5% | -| Row partitioning (`handle_split_info`) | 1.4% | -| Histogram subtraction (`from_parent_child`) | 1.3% | - -**8 threads, 100k x 200, 60 iterations** (6.45 s total, 4.3 s tree phase): -the main thread logged about 5.4 s of samples and each of the 8 Rayon workers -about 1.3 s. The main thread only runs serial code, because it blocks without -sampling while parallel tasks run. - -| Serial work on the main thread | Time | -| --- | ---: | -| Binning and its sorts | ~1.9 s (outside the tree phase) | -| Split search (`evaluate_split` + loops inlined into the caller) | ~1.36 s | -| Allocator, memory copies, kernel (mostly loading the input) | ~0.75 s | -| Gradient/Hessian gather | ~0.25 s | -| Histogram subtraction | ~0.22 s | -| Row partitioning | ~0.15 s | - -Roughly half of the 4.3 s tree phase is serial. `strace` logged 259,714 -`sched_yield` calls (Rayon workers spinning idle while the main thread works) -but only 50 `mmap` calls, so histogram allocations are not going to the -kernel. - -### Withdrawn earlier findings - -| Earlier finding | Why it was wrong | -| --- | --- | -| Parallel training is slower than serial | Measured on 20k rows with an unlearnable target. On realistic data, 2-8 threads always win. | -| Binning is the top priority | Measured with 10 iterations and a profile that ran Criterion in test mode. At 1,000 iterations, binning is 2-8%. | -| One evaluation set adds 61% | Measured on 5 columns. Prediction cost doesn't grow with columns; at 200 columns it is 1.6%. | -| Flamegraphs at `/tmp/forust-*.svg` | Captured in Criterion test mode plus setup code; don't use them. | - -## Recommendations (ranked) - -None of these change what the algorithm computes, so models should be -identical before and after (tie-breaking is noted where it matters). - -### 1. Parallelize the split search across features - -`Splitter::best_split` (`src/splitter.rs`) scans every feature's histogram -serially. It is the largest serial block: about 1.36 s of the 4.3 s tree phase -above, or about 23 ms of the ~47 ms fixed cost per tree. Each feature's scan is -independent, so the scans can run in parallel and then be reduced by gain, -breaking ties by the lowest feature index. That reproduces the current serial -choice exactly: today the first feature with the strictly highest gain wins. - -- Expected gain: at 100k x 200 on 8 threads, roughly 69 ms to 50 ms per tree - (estimate). Larger at depth 8 and with small nodes. -- Risk: low. No floating-point order changes. -- Follow-up option: fuse histogram building and the split scan into one - parallel task per column, so each histogram is scanned while still in cache. - -### 2. Move the remaining per-node work into the parallel section - -These run serially on every node: - -- `HistogramMatrix::new` gathers gradients/Hessians for the node's rows, and - copies the full arrays when no gather is needed. -- `from_parent_child` / `from_parent_two_children` subtract histograms for the - sibling node. -- The parallel `flat_map` allocates one vector per column and then - concatenates them. - -Options: subtract in parallel per column (or inside the per-column task), -write each column's histogram straight into a preallocated buffer, and -parallelize the gather. - -- Risk: low. Subtraction is element-wise, so results are unchanged. - -### 3. Parallelize binning and sort each column once - -`bin_matrix` processes columns one at a time. `percentiles_or_value` sorts and -deduplicates each column, then `percentiles` sorts an index array a second -time. Binning columns in parallel and sorting once should cut 24 s at 1M x 200 -to a few seconds (estimate, not measured). - -- Matters most for large data with fewer iterations: almost half of a - 100-iteration fit at 1M rows, about 8% of a 1,000-iteration fit. -- Risk: low. Each column is computed the same way, so bin boundaries are - unchanged. - -### 4. Thread count control - -16 hyperthreaded threads were 4-19% slower than 8 threads for Forust (and -55-380% slower for XGBoost). Options: add a `num_threads` parameter (like -XGBoost's `nthread`) using a local Rayon pool, or document -`RAYON_NUM_THREADS`. Low priority, since users can already set the environment -variable. - -### Not recommended now - -- **Evaluation changes:** 1.6% of time. -- **Row-parallel histograms (XGBoost-style):** the per-row part already scales - about 5x at 8 threads; it is not the bottleneck. -- **Lowering the default `nbins`:** faster, but changes models. - -### Expected ceiling - -If the fixed per-tree cost scaled like the per-row part (~5x at 8 threads), -100k x 200 at depth 5 would drop from about 69 ms per tree to about 30 ms, -comparable to XGBoost's 27 ms. This is a rough estimate to verify after items -1 and 2. - -## Validation plan for implementation - -For each change: - -1. Run the Python suite (111 tests, including the XGBoost parity checks). -2. Check the model is unchanged: `perf_phases --mode fit` should report an - identical evaluation log-loss before and after. -3. Re-measure with the same grid: `w200` at 25k/100k/250k rows, serial and - 8 threads, depth 5 and 8, and `w500`; then confirm on `w200-1m`. - -The Criterion `wide-200-column` group in `benches/forust_benchmarks.rs` still -uses the old unlearnable data. It should be switched to the generated data or -removed; `perf_phases` is the better tool for this analysis. - -## Reproducing - -```sh -PY=/home/azureuser/localfiles/uv-environments/forust/bin/python -$PY scripts/make_perf_data.py --out /tmp/forust-perf/w200 --rows 250000 --eval-rows 62500 --cols 200 -$PY scripts/make_perf_data.py --out /tmp/forust-perf/w500 --rows 100000 --eval-rows 25000 --cols 500 -$PY scripts/make_perf_data.py --out /tmp/forust-perf/w200-1m --rows 1000000 --eval-rows 250000 --cols 200 - -cargo build --release --example perf_phases -python3 scripts/perf_sweep.py --data /tmp/forust-perf/w200 \ - --rows 25000 100000 250000 --threads 8 --out /tmp/forust-perf/results/sweep.jsonl -python3 scripts/perf_sweep.py --data /tmp/forust-perf/w200 \ - --rows 25000 100000 --threads 8 --iterations 30 \ - --out /tmp/forust-perf/results/sweep_depth8.jsonl -- --max-depth 8 -$PY scripts/perf_xgboost.py --data /tmp/forust-perf/w200 --rows 100000 --threads 1 8 - -# Profile (release build with debug info) -CARGO_PROFILE_RELEASE_DEBUG=true cargo build --release --example perf_phases -RAYON_NUM_THREADS=8 perf record -F 999 -o /tmp/forust-perf/par.data -- \ - target/release/examples/perf_phases --rows 100000 --iterations 60 -perf report -i /tmp/forust-perf/par.data --stdio -s pid -``` - -Raw results are in `/tmp/forust-perf/results/`. `/tmp` is local disk and may -not survive the VM being deallocated; everything can be regenerated with the -commands above. - -## Working tree - -- New: `scripts/make_perf_data.py`, `scripts/perf_sweep.py`, - `scripts/perf_xgboost.py`, `examples/perf_phases.rs`. -- Earlier changes: the wide-data Criterion group in - `benches/forust_benchmarks.rs`; the Python packaging fix - (`py-forust/README.md`, SPDX license, `.gitignore`); `polars` and the Titanic - example removed (by you). -- `perf.data` in the repository root is the raw capture from your flamegraph - run and can be deleted. -- `cargo test --all-targets`: 42 passed with the new example. The Python suite - last passed (111 tests) before these scripts were added; no library code has - changed since. \ No newline at end of file diff --git a/agent/feat/optimize/requirements.md b/agent/feat/optimize/requirements.md deleted file mode 100644 index 86364e2..0000000 --- a/agent/feat/optimize/requirements.md +++ /dev/null @@ -1,9 +0,0 @@ -Review this package and identify speed bottlenecks, and ways that we can optimize, this should not come from purely from code review and speculation, this should come from running benchmarks on the actual package library. - -When this package is used in the wild, evaluation datasets are commonly used as well, this should be considered in the optimizations. - -If you are able to use flamegraph for understanding where the bottlenecks are that's even better. - -Additionally we should identify if the benchmarks should be updated to better identify performance bottlenecks. The python tests are the true validation safetey net. Those should continue to pass with the parity check against XGBoost. - -Please run some initial testing and review the code to determine high performance changes that could be made, create a recomendation markdown document, that we can look at and weigh any pros and cons for speed improvments. \ No newline at end of file diff --git a/agent/feat/optimize/results.jsonl b/agent/feat/optimize/results.jsonl deleted file mode 100644 index 00ff2e6..0000000 --- a/agent/feat/optimize/results.jsonl +++ /dev/null @@ -1,158 +0,0 @@ -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 69.46959152, "bin_s": 0.437525949, "per_iteration_ms": {"eval_predict_metric": 0.5789104999999999, "grad_hess": 0.1910351333333333, "train_predict": 1.6216669500000003, "tree": 69.76122168333333}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [69.46959152, 69.65518834000001, 68.14989166000001], "bin_s_samples": [0.437525949, 0.438140565, 0.437448845], "label": "baseline", "commit": "4a6d828", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 52.72685084, "bin_s": 0.436547018, "per_iteration_ms": {"eval_predict_metric": 0.3928104333333334, "grad_hess": 0.19065013333333344, "train_predict": 0.5185271999999999, "tree": 51.393246583333344}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [50.879969519999996, 52.72685084, 52.92546764], "bin_s_samples": [0.436547018, 0.435753344, 0.438757037], "label": "baseline", "commit": "4a6d828", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 165.37640113999998, "bin_s": 1.939842161, "per_iteration_ms": {"eval_predict_metric": 2.1063131999999993, "grad_hess": 0.7734422666666668, "train_predict": 5.801678933333336, "tree": 169.85902786666668}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [166.7688987, 165.37640113999998, 163.36389540000002], "bin_s_samples": [1.939842161, 1.928059809, 1.9491521440000001], "label": "baseline", "commit": "4a6d828", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 72.54935842, "bin_s": 1.909040288, "per_iteration_ms": {"eval_predict_metric": 1.1507776833333336, "grad_hess": 0.7722273833333333, "train_predict": 1.35429975, "tree": 73.07848834999999}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [72.54935842, 73.70935806, 72.1067068], "bin_s_samples": [1.9228914499999998, 1.907580031, 1.909040288], "label": "baseline", "commit": "4a6d828", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 340.7701525, "bin_s": 5.519331965, "per_iteration_ms": {"eval_predict_metric": 5.2829633, "grad_hess": 1.9559463999999998, "train_predict": 13.820900016666675, "tree": 346.8344234833333}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [340.7701525, 340.36360486, 340.78514792], "bin_s_samples": [5.519331965, 5.508273225, 5.538246969], "label": "baseline", "commit": "4a6d828", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 108.33892336000001, "bin_s": 5.782610459, "per_iteration_ms": {"eval_predict_metric": 2.572827666666666, "grad_hess": 2.150916366666667, "train_predict": 2.774376483333334, "tree": 109.06071794999997}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [107.79606736, 109.83367934, 108.33892336000001], "bin_s_samples": [5.805085562, 5.649859095, 5.782610459], "label": "baseline", "commit": "4a6d828", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 242.68829905, "bin_s": 0.434946006, "per_iteration_ms": {"eval_predict_metric": 1.5758595666666664, "grad_hess": 0.19796540000000004, "train_predict": 5.799344400000001, "tree": 271.35716629999996}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [255.23519255, 239.52746735, 242.68829905], "bin_s_samples": [0.434946006, 0.43527252, 0.434381128], "label": "baseline", "commit": "4a6d828", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 236.6468329, "bin_s": 0.436738132, "per_iteration_ms": {"eval_predict_metric": 0.5208074, "grad_hess": 0.19775866666666664, "train_predict": 1.053641566666667, "tree": 248.01862636666664}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [238.58191425, 236.6468329, 233.6555961], "bin_s_samples": [0.43708605, 0.436738132, 0.435563428], "label": "baseline", "commit": "4a6d828", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 560.5880748999999, "bin_s": 1.9525601689999998, "per_iteration_ms": {"eval_predict_metric": 6.375328333333333, "grad_hess": 0.7730809333333333, "train_predict": 22.83946866666667, "tree": 579.7484771999999}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [560.5880748999999, 560.87988435, 531.71152215], "bin_s_samples": [1.9609985330000002, 1.9525601689999998, 1.95242711], "label": "baseline", "commit": "4a6d828", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 358.13609335, "bin_s": 1.9139849899999999, "per_iteration_ms": {"eval_predict_metric": 1.6465802, "grad_hess": 0.7737273333333334, "train_predict": 3.5979854666666666, "tree": 361.8860509999999}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [353.80761745, 358.2753975, 358.13609335], "bin_s_samples": [1.9139849899999999, 1.9050529859999998, 1.924304735], "label": "baseline", "commit": "4a6d828", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 190.94636265, "bin_s": 1.097706692, "per_iteration_ms": {"eval_predict_metric": 0.6392739666666667, "grad_hess": 0.19672193333333332, "train_predict": 1.6246235, "tree": 189.30125903333328}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [186.8271379, 190.94636265, 192.2119754], "bin_s_samples": [1.097706692, 1.098993946, 1.095035878], "label": "baseline", "commit": "4a6d828", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 128.14974565, "bin_s": 1.096697823, "per_iteration_ms": {"eval_predict_metric": 0.3841668, "grad_hess": 0.19447196666666666, "train_predict": 0.5308942, "tree": 128.5561528}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [127.658573, 132.4411582, 128.14974565], "bin_s_samples": [1.097851831, 1.096697823, 1.089410621], "label": "baseline", "commit": "4a6d828", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 433.8777002, "bin_s": 5.296791118, "per_iteration_ms": {"eval_predict_metric": 2.2977944, "grad_hess": 0.7660136333333333, "train_predict": 6.257479366666667, "tree": 420.2903304000001}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [431.7713069, 433.8777002, 436.12740249999996], "bin_s_samples": [5.282921762, 5.327296441, 5.296791118], "label": "baseline", "commit": "4a6d828", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 174.8739138, "bin_s": 5.138339381, "per_iteration_ms": {"eval_predict_metric": 1.1591382333333333, "grad_hess": 0.7895282000000001, "train_predict": 1.4158118000000002, "tree": 172.42408873333335}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [174.8739138, 177.83162199999998, 172.47661695], "bin_s_samples": [5.147487526, 5.115332534, 5.138339381], "label": "baseline", "commit": "4a6d828", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 20, "tree_ms": 261.0441173, "bin_s": 24.725260587, "per_iteration_ms": {"eval_predict_metric": 8.9735312, "grad_hess": 7.69143665, "train_predict": 8.67225315, "tree": 269.72727359999993}, "eval_logloss": 0.4052419048375606, "tree_ms_samples": [263.7996291, 261.0441173, 260.9039164], "bin_s_samples": [24.29816179, 24.725260587, 24.726936749], "label": "baseline", "commit": "4a6d828", "grid": "G4", "extra": ["--max-depth", "5"]} -{"label": "baseline", "commit": "4a6d828", "kind": "serial_share", "data": "/tmp/forust-perf/w200", "rows": 100000, "max_depth": 5, "window_s": 7.74, "main_samples": 4890, "total_samples": 25188, "serial_fraction": 0.633, "main_thread_top": [["Splitter::evaluate_split", 0.308], ["perf_phases::main", 0.182], ["[unknown]", 0.119], ["HistogramMatrix::new", 0.097], ["_int_free", 0.091], ["HistogramMatrix::from_parent_child", 0.08], ["Splitter::handle_split_info", 0.048], ["__memmove_evex_unaligned_erms", 0.028]]} -{"label": "baseline", "commit": "4a6d828", "kind": "serial_share", "data": "/tmp/forust-perf/w200", "rows": 25000, "max_depth": 8, "window_s": 9.18, "main_samples": 7456, "total_samples": 17340, "serial_fraction": 0.813, "main_thread_top": [["Splitter::evaluate_split", 0.401], ["perf_phases::main", 0.164], ["[unknown]", 0.132], ["_int_free", 0.099], ["HistogramMatrix::new", 0.082], ["HistogramMatrix::from_parent_child", 0.08], [".5", 0.014], ["__memmove_evex_unaligned_erms", 0.014]]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 69.81053074, "bin_s": 0.435306677, "per_iteration_ms": {"eval_predict_metric": 0.5636467666666668, "grad_hess": 0.19023173333333335, "train_predict": 1.6770216166666665, "tree": 70.75476136666666}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [69.81053074, 69.5778413, 70.01196404], "bin_s_samples": [0.435306677, 0.435160973, 0.435464275], "label": "item1", "commit": "c111e36", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 35.85318732, "bin_s": 0.437190146, "per_iteration_ms": {"eval_predict_metric": 0.37964359999999997, "grad_hess": 0.19359513333333336, "train_predict": 0.5206356000000001, "tree": 36.386394349999996}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [35.85318732, 36.764681780000004, 35.46399266], "bin_s_samples": [0.440363532, 0.436570029, 0.437190146], "label": "item1", "commit": "c111e36", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 169.79813908, "bin_s": 1.969967665, "per_iteration_ms": {"eval_predict_metric": 2.3279405999999994, "grad_hess": 0.7821008999999998, "train_predict": 6.066850316666668, "tree": 178.00947385000003}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [174.9803429, 165.97511207999997, 169.79813908], "bin_s_samples": [1.9575992869999999, 1.969967665, 1.981600716], "label": "item1", "commit": "c111e36", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 56.3905667, "bin_s": 1.928826607, "per_iteration_ms": {"eval_predict_metric": 1.1246750166666668, "grad_hess": 0.7749025499999997, "train_predict": 1.3512112166666665, "tree": 56.301933133333335}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [54.90712116, 56.3905667, 56.93049636], "bin_s_samples": [1.928826607, 1.9291983849999998, 1.9217654990000002], "label": "item1", "commit": "c111e36", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 340.05684212, "bin_s": 5.47907255, "per_iteration_ms": {"eval_predict_metric": 5.274108183333334, "grad_hess": 1.9394604, "train_predict": 13.719495166666665, "tree": 345.4624450166667}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [339.19718424, 340.05684212, 343.10781882000003], "bin_s_samples": [5.464031071, 5.511676594, 5.47907255], "label": "item1", "commit": "c111e36", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 89.21600588000001, "bin_s": 5.75627353, "per_iteration_ms": {"eval_predict_metric": 2.4963781166666674, "grad_hess": 1.9135514, "train_predict": 2.6797962833333324, "tree": 88.6433665833333}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [87.9014496, 89.21600588000001, 90.05856505999999], "bin_s_samples": [5.779580888, 5.702051921, 5.75627353], "label": "item1", "commit": "c111e36", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 241.67713865000002, "bin_s": 0.434509673, "per_iteration_ms": {"eval_predict_metric": 1.6278735666666668, "grad_hess": 0.19520189999999996, "train_predict": 5.956649766666667, "tree": 278.9519632}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [264.29187505, 240.25264585, 241.67713865000002], "bin_s_samples": [0.445215949, 0.434481447, 0.434509673], "label": "item1", "commit": "c111e36", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 147.03271765, "bin_s": 0.438076666, "per_iteration_ms": {"eval_predict_metric": 0.5056469, "grad_hess": 0.1905451, "train_predict": 1.0713721666666667, "tree": 153.61278589999992}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [146.10826925, 147.03271765, 151.9490293], "bin_s_samples": [0.435751204, 0.438076666, 0.439756192], "label": "item1", "commit": "c111e36", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 528.6121228000001, "bin_s": 1.944683045, "per_iteration_ms": {"eval_predict_metric": 6.4236425666666666, "grad_hess": 0.7652473333333333, "train_predict": 23.241960766666672, "tree": 547.3844587666666}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [528.6121228000001, 522.43539805, 532.70656355], "bin_s_samples": [1.944683045, 1.938849329, 1.9629755], "label": "item1", "commit": "c111e36", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 245.0629158, "bin_s": 1.910363998, "per_iteration_ms": {"eval_predict_metric": 1.681991733333333, "grad_hess": 0.8040148999999999, "train_predict": 3.569705833333333, "tree": 253.92226230000003}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [245.0629158, 245.4923455, 245.01043755], "bin_s_samples": [1.917898249, 1.903986727, 1.910363998], "label": "item1", "commit": "c111e36", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 190.18595439999999, "bin_s": 1.094923241, "per_iteration_ms": {"eval_predict_metric": 0.5372545, "grad_hess": 0.19877536666666668, "train_predict": 1.4362419333333336, "tree": 187.48143319999997}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [185.95485324999999, 190.18595439999999, 190.7640832], "bin_s_samples": [1.094923241, 1.097001918, 1.092561681], "label": "item1", "commit": "c111e36", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 84.9507787, "bin_s": 1.092896081, "per_iteration_ms": {"eval_predict_metric": 0.4141777000000001, "grad_hess": 0.1931252, "train_predict": 0.5042468666666665, "tree": 85.44767873333333}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [84.9507787, 86.4902627, 82.99517125], "bin_s_samples": [1.093979045, 1.092239698, 1.092896081], "label": "item1", "commit": "c111e36", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 434.23386895, "bin_s": 5.266637135, "per_iteration_ms": {"eval_predict_metric": 2.344956533333333, "grad_hess": 0.7669239333333333, "train_predict": 6.063823466666666, "tree": 432.04873539999994}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [443.4125824, 432.92225075, 434.23386895], "bin_s_samples": [5.3660727, 5.263242021, 5.266637135], "label": "item1", "commit": "c111e36", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 127.72912985, "bin_s": 5.183664367, "per_iteration_ms": {"eval_predict_metric": 1.1557579000000002, "grad_hess": 0.7736606, "train_predict": 1.3519428666666664, "tree": 128.2257553}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [128.97670715, 127.72912985, 126.3522621], "bin_s_samples": [5.19463169, 5.14765144, 5.183664367], "label": "item1", "commit": "c111e36", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 20, "tree_ms": 247.59789609999999, "bin_s": 24.642479796, "per_iteration_ms": {"eval_predict_metric": 9.05855285, "grad_hess": 7.550097549999999, "train_predict": 8.94945505, "tree": 258.19115565000004}, "eval_logloss": 0.4052419048375606, "tree_ms_samples": [252.1289214, 242.2849681, 247.59789609999999], "bin_s_samples": [24.954603472, 24.642479796, 24.462511743], "label": "item1", "commit": "c111e36", "grid": "G4", "extra": ["--max-depth", "5"]} -{"label": "item1", "commit": "c111e36", "kind": "serial_share", "data": "/tmp/forust-perf/w200", "rows": 100000, "max_depth": 5, "window_s": 5.55, "main_samples": 2388, "total_samples": 25794, "serial_fraction": 0.431, "main_thread_top": [["[unknown]", 0.235], ["HistogramMatrix::new", 0.182], ["_int_free", 0.175], ["HistogramMatrix::from_parent_child", 0.148], ["Splitter::handle_split_info", 0.105], ["__memmove_evex_unaligned_erms", 0.035], ["perf_phases::main", 0.029], [".5", 0.025]]} -{"label": "item1", "commit": "c111e36", "kind": "serial_share", "data": "/tmp/forust-perf/w200", "rows": 25000, "max_depth": 8, "window_s": 5.45, "main_samples": 3094, "total_samples": 19578, "serial_fraction": 0.568, "main_thread_top": [["[unknown]", 0.325], ["_int_free", 0.223], ["HistogramMatrix::new", 0.19], ["HistogramMatrix::from_parent_child", 0.16], ["__memmove_evex_unaligned_erms", 0.03], [".5", 0.03], ["Splitter::handle_split_info", 0.023], ["perf_phases::main", 0.004]]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 62.36494508, "bin_s": 0.434893052, "per_iteration_ms": {"eval_predict_metric": 0.6127864166666664, "grad_hess": 0.19119843333333333, "train_predict": 1.5896309333333332, "tree": 66.17605811666667}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [64.55898178, 62.063670959999996, 62.36494508], "bin_s_samples": [0.443136673, 0.433495414, 0.434893052], "label": "item2a", "commit": "763e861", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 34.9079135, "bin_s": 0.436532199, "per_iteration_ms": {"eval_predict_metric": 0.4615823500000001, "grad_hess": 0.18933173333333334, "train_predict": 0.48946645000000016, "tree": 35.213388649999985}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [34.9079135, 34.849794, 37.6829519], "bin_s_samples": [0.439932187, 0.436532199, 0.436081387], "label": "item2a", "commit": "763e861", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 151.58123124, "bin_s": 1.949594749, "per_iteration_ms": {"eval_predict_metric": 2.0717572333333334, "grad_hess": 0.7630904000000002, "train_predict": 5.782246483333334, "tree": 154.60147533333333}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [151.58123124, 151.42749004, 151.92276732], "bin_s_samples": [1.958760668, 1.949594749, 1.9437784009999999], "label": "item2a", "commit": "763e861", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 51.29674252, "bin_s": 1.92003919, "per_iteration_ms": {"eval_predict_metric": 1.1478641333333337, "grad_hess": 0.8529328333333329, "train_predict": 1.3101816500000003, "tree": 51.695109916666645}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [51.29674252, 51.66723024, 51.23324678], "bin_s_samples": [1.92003919, 1.928700896, 1.9102520809999999], "label": "item2a", "commit": "763e861", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 311.41205372, "bin_s": 5.482303308, "per_iteration_ms": {"eval_predict_metric": 5.035733283333334, "grad_hess": 1.9190314166666669, "train_predict": 13.39755913333333, "tree": 315.22710405000004}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [310.13624476, 311.41205372, 312.25750089999997], "bin_s_samples": [5.450111053, 5.482303308, 5.497398603], "label": "item2a", "commit": "763e861", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 81.9523143, "bin_s": 5.754775023, "per_iteration_ms": {"eval_predict_metric": 2.4796098166666662, "grad_hess": 1.9118813000000001, "train_predict": 2.601998116666666, "tree": 82.29246405000002}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [81.9523143, 81.16988581999999, 82.07537106], "bin_s_samples": [5.754775023, 5.660847098, 5.777812417], "label": "item2a", "commit": "763e861", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 240.3807964, "bin_s": 0.434902179, "per_iteration_ms": {"eval_predict_metric": 1.630487166666667, "grad_hess": 0.19502183333333328, "train_predict": 5.9681239, "tree": 253.59377823333338}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [238.8502552, 240.3807964, 241.46047685], "bin_s_samples": [0.434473073, 0.437318938, 0.434902179], "label": "item2a", "commit": "763e861", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 143.85457265, "bin_s": 0.435007596, "per_iteration_ms": {"eval_predict_metric": 0.5071336333333334, "grad_hess": 0.19138186666666662, "train_predict": 1.0667186000000002, "tree": 154.5820894}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [143.85457265, 145.66967005, 139.8081545], "bin_s_samples": [0.435007596, 0.435792317, 0.43497249], "label": "item2a", "commit": "763e861", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 521.4690327, "bin_s": 1.945016328, "per_iteration_ms": {"eval_predict_metric": 6.492004233333334, "grad_hess": 0.7630905333333332, "train_predict": 23.466990333333335, "tree": 533.8431897666667}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [518.0319599000001, 523.44818475, 521.4690327], "bin_s_samples": [1.938104836, 1.945016328, 1.9684916289999999], "label": "item2a", "commit": "763e861", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 236.5429835, "bin_s": 1.9109886440000001, "per_iteration_ms": {"eval_predict_metric": 1.6655144666666666, "grad_hess": 0.7874811, "train_predict": 3.5061236333333334, "tree": 245.64930449999994}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [235.4584154, 236.5429835, 238.7372086], "bin_s_samples": [1.922205211, 1.9109886440000001, 1.906038999], "label": "item2a", "commit": "763e861", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 184.2786318, "bin_s": 1.093891502, "per_iteration_ms": {"eval_predict_metric": 0.6198665666666666, "grad_hess": 0.19394866666666669, "train_predict": 1.5956894666666663, "tree": 185.94806126666666}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [184.2786318, 183.74937605, 184.44155675000002], "bin_s_samples": [1.093891502, 1.095879079, 1.093423396], "label": "item2a", "commit": "763e861", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 83.75228485, "bin_s": 1.098263839, "per_iteration_ms": {"eval_predict_metric": 0.3757068, "grad_hess": 0.20276546666666664, "train_predict": 0.5223672666666664, "tree": 84.04291276666667}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [83.79693615000001, 83.75228485, 83.48150645000001], "bin_s_samples": [1.107784488, 1.098263839, 1.097455404], "label": "item2a", "commit": "763e861", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 404.0627449, "bin_s": 5.259694921, "per_iteration_ms": {"eval_predict_metric": 2.1964489000000005, "grad_hess": 0.7548769000000001, "train_predict": 6.0099577333333345, "tree": 391.9307721000001}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [403.15112115, 404.0627449, 405.1045873], "bin_s_samples": [5.259694921, 5.281528615, 5.241016614], "label": "item2a", "commit": "763e861", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 120.83615425, "bin_s": 5.144885519, "per_iteration_ms": {"eval_predict_metric": 1.146383966666667, "grad_hess": 0.7846477666666666, "train_predict": 1.3450794999999998, "tree": 120.27462073333336}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [120.83615425, 118.83561830000001, 121.8814572], "bin_s_samples": [5.167309297, 5.144885519, 5.106251848], "label": "item2a", "commit": "763e861", "grid": "G3", "extra": ["--max-depth", "5"]} -{"label": "item2a", "commit": "763e861-dirty", "kind": "serial_share", "data": "/tmp/forust-perf/w200", "rows": 100000, "max_depth": 5, "window_s": 5.57, "main_samples": 2587, "total_samples": 24267, "serial_fraction": 0.465, "main_thread_top": [["[unknown]", 0.301], ["_int_free", 0.166], ["HistogramMatrix::new", 0.151], ["HistogramMatrix::from_parent_child", 0.14], ["Splitter::handle_split_info", 0.104], [".5", 0.027], ["__memmove_evex_unaligned_erms", 0.027], ["perf_phases::main", 0.021]]} -{"label": "item2a", "commit": "763e861-dirty", "kind": "serial_share", "data": "/tmp/forust-perf/w200", "rows": 25000, "max_depth": 8, "window_s": 5.15, "main_samples": 2783, "total_samples": 19385, "serial_fraction": 0.54, "main_thread_top": [["[unknown]", 0.266], ["_int_free", 0.241], ["HistogramMatrix::new", 0.202], ["HistogramMatrix::from_parent_child", 0.184], [".5", 0.031], ["__memmove_evex_unaligned_erms", 0.028], ["Splitter::handle_split_info", 0.024], ["perf_phases::main", 0.005]]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 54.62255302, "bin_s": 0.441155219, "per_iteration_ms": {"eval_predict_metric": 0.5302308833333335, "grad_hess": 0.19095013333333333, "train_predict": 1.5396095333333328, "tree": 55.080786933333336}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [54.62255302, 54.04775608, 55.28021114], "bin_s_samples": [0.441478323, 0.441155219, 0.440926813], "label": "item2b", "commit": "18744aa-dirty", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 24.415310780000002, "bin_s": 0.442935569, "per_iteration_ms": {"eval_predict_metric": 0.3958406666666667, "grad_hess": 0.19065843333333332, "train_predict": 0.5198655999999999, "tree": 26.126156716666667}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [26.07663272, 23.88288186, 24.415310780000002], "bin_s_samples": [0.441982943, 0.442935569, 0.443196276], "label": "item2b", "commit": "18744aa-dirty", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 143.20925277999999, "bin_s": 1.9874393860000001, "per_iteration_ms": {"eval_predict_metric": 2.2394551499999995, "grad_hess": 0.7684639833333333, "train_predict": 6.013653316666666, "tree": 147.19299843333332}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [143.83896226000002, 141.84467228, 143.20925277999999], "bin_s_samples": [1.991192125, 1.9874393860000001, 1.982345988], "label": "item2b", "commit": "18744aa-dirty", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 42.4273529, "bin_s": 1.94174806, "per_iteration_ms": {"eval_predict_metric": 1.1297904666666665, "grad_hess": 0.7923446999999999, "train_predict": 1.31905235, "tree": 43.05716893333333}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [42.5827849, 40.73327834, 42.4273529], "bin_s_samples": [1.944380505, 1.94174806, 1.9414715710000001], "label": "item2b", "commit": "18744aa-dirty", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 303.1078622, "bin_s": 5.605754604, "per_iteration_ms": {"eval_predict_metric": 5.333011766666669, "grad_hess": 1.9378437499999999, "train_predict": 13.734163800000001, "tree": 311.40408028333337}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [305.65032118, 303.1078622, 301.87951644], "bin_s_samples": [5.611416029, 5.588344405, 5.605754604], "label": "item2b", "commit": "18744aa-dirty", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 74.44776118, "bin_s": 5.822714712, "per_iteration_ms": {"eval_predict_metric": 2.5028675666666667, "grad_hess": 1.9581078999999995, "train_predict": 2.7191484833333335, "tree": 76.19906685}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [74.44776118, 73.46710128000001, 74.68634382], "bin_s_samples": [5.822714712, 5.810806913, 5.85963559], "label": "item2b", "commit": "18744aa-dirty", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 192.11954425, "bin_s": 0.441290716, "per_iteration_ms": {"eval_predict_metric": 1.6114296666666668, "grad_hess": 0.195342, "train_predict": 5.916364733333335, "tree": 203.51863669999997}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [190.6485558, 192.44107805, 192.11954425], "bin_s_samples": [0.442351825, 0.440890784, 0.441290716], "label": "item2b", "commit": "18744aa-dirty", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 99.2469185, "bin_s": 0.442565296, "per_iteration_ms": {"eval_predict_metric": 0.5073136, "grad_hess": 0.19305176666666665, "train_predict": 1.0346112, "tree": 107.75017286666665}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [98.56537429999999, 99.2469185, 103.845881], "bin_s_samples": [0.44239099, 0.442565296, 0.44482669], "label": "item2b", "commit": "18744aa-dirty", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 482.59553355, "bin_s": 1.976040227, "per_iteration_ms": {"eval_predict_metric": 6.4657067, "grad_hess": 0.7735879, "train_predict": 23.269712566666662, "tree": 498.77318740000004}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [487.5059525, 479.83096045, 482.59553355], "bin_s_samples": [1.985783082, 1.976040227, 1.971430252], "label": "item2b", "commit": "18744aa-dirty", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 183.7891911, "bin_s": 1.938960601, "per_iteration_ms": {"eval_predict_metric": 1.6449345999999998, "grad_hess": 0.8002182333333334, "train_predict": 3.524778499999999, "tree": 189.8888726}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [183.7891911, 186.52808670000002, 181.9774197], "bin_s_samples": [1.938960601, 1.940662243, 1.9388724069999999], "label": "item2b", "commit": "18744aa-dirty", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 164.48178994999998, "bin_s": 1.112666611, "per_iteration_ms": {"eval_predict_metric": 0.5853424666666668, "grad_hess": 0.19319186666666668, "train_predict": 1.559925566666667, "tree": 165.5335692333333}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [165.57202560000002, 164.48178994999998, 163.8747643], "bin_s_samples": [1.112666611, 1.112894082, 1.111188577], "label": "item2b", "commit": "18744aa-dirty", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 56.2669603, "bin_s": 1.110770757, "per_iteration_ms": {"eval_predict_metric": 0.40403760000000005, "grad_hess": 0.22390936666666666, "train_predict": 0.49581666666666674, "tree": 59.02136393333332}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [58.7287944, 55.0582201, 56.2669603], "bin_s_samples": [1.110770757, 1.110336246, 1.119080342], "label": "item2b", "commit": "18744aa-dirty", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 390.37011205, "bin_s": 5.370121168, "per_iteration_ms": {"eval_predict_metric": 2.318095533333334, "grad_hess": 0.7692674333333334, "train_predict": 6.1522020333333325, "tree": 386.03881566666666}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [397.60050775, 390.37011205, 387.05651905], "bin_s_samples": [5.382020591, 5.345707668, 5.370121168], "label": "item2b", "commit": "18744aa-dirty", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 102.75244074999999, "bin_s": 5.244606183, "per_iteration_ms": {"eval_predict_metric": 1.1395340666666667, "grad_hess": 0.7744976000000001, "train_predict": 1.384753766666667, "tree": 100.92454333333336}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [102.75244074999999, 104.64367625, 101.6818097], "bin_s_samples": [5.244606183, 5.261662083, 5.217806732], "label": "item2b", "commit": "18744aa-dirty", "grid": "G3", "extra": ["--max-depth", "5"]} -{"label": "item2b", "commit": "18744aa-dirty", "kind": "serial_share", "data": "/tmp/forust-perf/w200", "rows": 100000, "max_depth": 5, "window_s": 4.46, "main_samples": 1614, "total_samples": 22071, "serial_fraction": 0.362, "main_thread_top": [["[unknown]", 0.466], ["HistogramMatrix::from_parent_child", 0.221], ["Splitter::handle_split_info", 0.147], ["perf_phases::main", 0.037], ["HistogramMatrix::new", 0.032], ["__ieee754_exp_fma", 0.025], ["__ieee754_log_fma", 0.024], ["exp@@GLIBC_2.29", 0.012]]} -{"label": "item2b", "commit": "18744aa-dirty", "kind": "serial_share", "data": "/tmp/forust-perf/w200", "rows": 25000, "max_depth": 8, "window_s": 3.51, "main_samples": 1239, "total_samples": 15932, "serial_fraction": 0.354, "main_thread_top": [["[unknown]", 0.515], ["HistogramMatrix::from_parent_child", 0.377], ["Splitter::handle_split_info", 0.04], ["HistogramMatrix::new", 0.016], ["perf_phases::main", 0.007], ["syscall", 0.007], ["__memmove_evex_unaligned_erms", 0.007], ["_int_free", 0.006]]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 57.85148712, "bin_s": 0.438329437, "per_iteration_ms": {"eval_predict_metric": 0.5654384666666666, "grad_hess": 0.19049838333333335, "train_predict": 1.547496333333333, "tree": 58.93218119999999}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [57.85148712, 58.19980054, 57.78974526], "bin_s_samples": [0.438329437, 0.439248217, 0.438319206], "label": "item2c", "commit": "aefa101", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 20.04327438, "bin_s": 0.44143969, "per_iteration_ms": {"eval_predict_metric": 0.3743266999999999, "grad_hess": 0.1917817666666667, "train_predict": 0.43889173333333326, "tree": 21.017529566666663}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [19.9909778, 20.09700934, 20.04327438], "bin_s_samples": [0.44221861, 0.44143969, 0.441065605], "label": "item2c", "commit": "aefa101", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 153.89786416, "bin_s": 1.976381141, "per_iteration_ms": {"eval_predict_metric": 1.8097984833333338, "grad_hess": 0.7534485333333334, "train_predict": 4.701375966666668, "tree": 161.06310523333335}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [158.16950262, 153.89786416, 152.72367208], "bin_s_samples": [1.98208395, 1.976381141, 1.976331343], "label": "item2c", "commit": "aefa101", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 38.9563416, "bin_s": 1.944444468, "per_iteration_ms": {"eval_predict_metric": 1.1286886333333332, "grad_hess": 0.7927778833333333, "train_predict": 1.3460076166666666, "tree": 39.322201333333325}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [39.10830928, 37.870358440000004, 38.9563416], "bin_s_samples": [1.944444468, 1.92397302, 1.946185837], "label": "item2c", "commit": "aefa101", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 332.27188012, "bin_s": 5.5840569030000005, "per_iteration_ms": {"eval_predict_metric": 4.9808303, "grad_hess": 1.9290567666666663, "train_predict": 13.403468133333337, "tree": 338.15789110000003}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [332.27188012, 333.19588276, 332.1625287], "bin_s_samples": [5.568784481, 5.5840569030000005, 5.60086744], "label": "item2c", "commit": "aefa101", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 73.84313498, "bin_s": 5.802353338, "per_iteration_ms": {"eval_predict_metric": 2.56996515, "grad_hess": 2.0953988666666663, "train_predict": 2.7507051, "tree": 74.92827691666666}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [74.50885382, 73.84313498, 73.7464768], "bin_s_samples": [5.822967643, 5.802353338, 5.732712285], "label": "item2c", "commit": "aefa101", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 195.434199, "bin_s": 0.439163331, "per_iteration_ms": {"eval_predict_metric": 1.5718155333333335, "grad_hess": 0.19270193333333333, "train_predict": 5.760238066666668, "tree": 209.65611399999997}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [198.4554603, 195.434199, 194.98115059999998], "bin_s_samples": [0.439194894, 0.438321173, 0.439163331], "label": "item2c", "commit": "aefa101", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 80.7750602, "bin_s": 0.440469607, "per_iteration_ms": {"eval_predict_metric": 0.5152174333333333, "grad_hess": 0.18693503333333333, "train_predict": 1.0078070666666668, "tree": 83.33125609999999}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [81.8851786, 80.3912184, 80.7750602], "bin_s_samples": [0.442256356, 0.440469607, 0.439289465], "label": "item2c", "commit": "aefa101", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 498.91678675, "bin_s": 1.9718031219999999, "per_iteration_ms": {"eval_predict_metric": 6.333940766666664, "grad_hess": 0.7776693000000002, "train_predict": 22.57424920000001, "tree": 513.1767767333333}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [498.91678675, 495.29916065000003, 500.3623478], "bin_s_samples": [1.969894256, 1.9718031219999999, 1.996895241], "label": "item2c", "commit": "aefa101", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 149.4450353, "bin_s": 1.929036, "per_iteration_ms": {"eval_predict_metric": 1.6439668000000003, "grad_hess": 0.7827572666666668, "train_predict": 3.4786252666666666, "tree": 157.27334506666665}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [153.3930061, 149.2841306, 149.4450353], "bin_s_samples": [1.929036, 1.925551754, 1.929286077], "label": "item2c", "commit": "aefa101", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 176.42237350000002, "bin_s": 1.105661177, "per_iteration_ms": {"eval_predict_metric": 0.6282766333333335, "grad_hess": 0.19203183333333337, "train_predict": 1.6151599666666667, "tree": 173.21282433333334}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [171.31084795, 177.60460989999999, 176.42237350000002], "bin_s_samples": [1.102730509, 1.105661177, 1.105884583], "label": "item2c", "commit": "aefa101", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 40.2903273, "bin_s": 1.104376143, "per_iteration_ms": {"eval_predict_metric": 0.3917207999999999, "grad_hess": 0.23163473333333331, "train_predict": 0.4576986333333333, "tree": 42.48510143333333}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [41.6056254, 40.2903273, 38.9208262], "bin_s_samples": [1.104376143, 1.107289744, 1.104357184], "label": "item2c", "commit": "aefa101", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 410.17604875, "bin_s": 5.334983417, "per_iteration_ms": {"eval_predict_metric": 2.110109866666667, "grad_hess": 0.7645504999999998, "train_predict": 5.7993783666666685, "tree": 408.68899590000007}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [422.2071608, 408.60113105, 410.17604875], "bin_s_samples": [5.338150445, 5.300208883, 5.334983417], "label": "item2c", "commit": "aefa101", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 86.24686700000001, "bin_s": 5.226729188, "per_iteration_ms": {"eval_predict_metric": 1.2542766999999997, "grad_hess": 0.7762199333333332, "train_predict": 1.3214724333333334, "tree": 84.89245013333331}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [86.24686700000001, 85.09254944999999, 87.41631955], "bin_s_samples": [5.226729188, 5.20909275, 5.244104929], "label": "item2c", "commit": "aefa101", "grid": "G3", "extra": ["--max-depth", "5"]} -{"label": "item2c", "commit": "aefa101", "kind": "serial_share", "data": "/tmp/forust-perf/w200", "rows": 100000, "max_depth": 5, "window_s": 3.93, "main_samples": 674, "total_samples": 23945, "serial_fraction": 0.172, "main_thread_top": [["Splitter::handle_split_info", 0.346], ["[unknown]", 0.237], ["perf_phases::main", 0.092], ["HistogramMatrix::new", 0.086], ["__ieee754_exp_fma", 0.064], ["__ieee754_log_fma", 0.049], ["__memmove_evex_unaligned_erms", 0.028], ["exp@@GLIBC_2.29", 0.025]]} -{"label": "item2c", "commit": "aefa101", "kind": "serial_share", "data": "/tmp/forust-perf/w200", "rows": 25000, "max_depth": 8, "window_s": 2.58, "main_samples": 315, "total_samples": 15964, "serial_fraction": 0.122, "main_thread_top": [["[unknown]", 0.467], ["Splitter::handle_split_info", 0.146], ["syscall", 0.086], ["HistogramMatrix::new", 0.07], ["_int_malloc", 0.022], ["perf_phases::main", 0.019], ["malloc", 0.019], ["LockLatch::wait_and_reset", 0.016]]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 80.67388565, "bin_s": 0.435033587, "per_iteration_ms": {"eval_predict_metric": 0.5054301333333332, "grad_hess": 0.20014213333333333, "train_predict": 0.9839798666666667, "tree": 83.88423359999999}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [79.04915179999999, 80.67388565, 80.9793504], "bin_s_samples": [0.438779605, 0.434916201, 0.435033587], "label": "item2d-tune-0", "commit": "aefa101-dirty", "extra": ["--max-depth", "8"], "note": "FORUST_TUNE_HIST_MIN_ELEMENTS small-node threshold experiment (dropped)"} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 19.132505899999998, "bin_s": 0.436407224, "per_iteration_ms": {"eval_predict_metric": 0.3867703666666666, "grad_hess": 0.19462855000000004, "train_predict": 0.45910241666666673, "tree": 20.37825903333334}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [19.132505899999998, 19.58128562, 18.469159259999998], "bin_s_samples": [0.436272921, 0.436407224, 0.436788006], "label": "item2d-tune-0", "commit": "aefa101-dirty", "extra": [], "note": "FORUST_TUNE_HIST_MIN_ELEMENTS small-node threshold experiment (dropped)"} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 35.887019380000005, "bin_s": 1.913833585, "per_iteration_ms": {"eval_predict_metric": 1.1165152166666668, "grad_hess": 0.8286724166666667, "train_predict": 1.1413091000000006, "tree": 36.75836696666667}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [36.24155562, 35.08194536, 35.887019380000005], "bin_s_samples": [1.905596609, 1.922196303, 1.913833585], "label": "item2d-tune-0", "commit": "aefa101-dirty", "extra": [], "note": "FORUST_TUNE_HIST_MIN_ELEMENTS small-node threshold experiment (dropped)"} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 92.7928449, "bin_s": 0.4356769, "per_iteration_ms": {"eval_predict_metric": 0.5053304, "grad_hess": 0.1914217333333333, "train_predict": 1.0390382333333332, "tree": 96.2149793333333}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [93.86221725, 92.7928449, 90.72750735], "bin_s_samples": [0.4356769, 0.435187888, 0.436812666], "label": "item2d-tune-50000", "commit": "aefa101-dirty", "extra": ["--max-depth", "8"], "note": "FORUST_TUNE_HIST_MIN_ELEMENTS small-node threshold experiment (dropped)"} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 18.981821659999998, "bin_s": 0.436349517, "per_iteration_ms": {"eval_predict_metric": 0.36700820000000006, "grad_hess": 0.19109018333333339, "train_predict": 0.44396186666666676, "tree": 19.018378999999996}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [18.981821659999998, 18.43999548, 20.499748920000002], "bin_s_samples": [0.436349517, 0.435006289, 0.439213602], "label": "item2d-tune-50000", "commit": "aefa101-dirty", "extra": [], "note": "FORUST_TUNE_HIST_MIN_ELEMENTS small-node threshold experiment (dropped)"} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 41.03865482, "bin_s": 1.904537978, "per_iteration_ms": {"eval_predict_metric": 1.1240085666666664, "grad_hess": 0.7651606499999997, "train_predict": 1.2748275833333333, "tree": 38.8692485}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [38.29982202, 41.50921346, 41.03865482], "bin_s_samples": [1.904537978, 1.905002981, 1.904036541], "label": "item2d-tune-50000", "commit": "aefa101-dirty", "extra": [], "note": "FORUST_TUNE_HIST_MIN_ELEMENTS small-node threshold experiment (dropped)"} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 99.25009625, "bin_s": 0.436143127, "per_iteration_ms": {"eval_predict_metric": 0.5184241333333334, "grad_hess": 0.21820580000000003, "train_predict": 1.1177737666666665, "tree": 105.83758996666668}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [99.25009625, 97.09164635, 99.53770755000001], "bin_s_samples": [0.436143127, 0.435628562, 0.436908997], "label": "item2d-tune-200000", "commit": "aefa101-dirty", "extra": ["--max-depth", "8"], "note": "FORUST_TUNE_HIST_MIN_ELEMENTS small-node threshold experiment (dropped)"} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 20.31147826, "bin_s": 0.436128914, "per_iteration_ms": {"eval_predict_metric": 0.3789335666666666, "grad_hess": 0.1885034333333334, "train_predict": 0.43427841666666667, "tree": 20.64369425000001}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [20.31147826, 21.32704572, 18.57341912], "bin_s_samples": [0.436844433, 0.43488218, 0.436128914], "label": "item2d-tune-200000", "commit": "aefa101-dirty", "extra": [], "note": "FORUST_TUNE_HIST_MIN_ELEMENTS small-node threshold experiment (dropped)"} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 48.89803422, "bin_s": 1.917245752, "per_iteration_ms": {"eval_predict_metric": 1.1122466666666668, "grad_hess": 0.7658224166666667, "train_predict": 1.35240995, "tree": 50.29458686666666}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [50.245616899999995, 48.59093064, 48.89803422], "bin_s_samples": [1.899600087, 1.917245752, 1.918760572], "label": "item2d-tune-200000", "commit": "aefa101-dirty", "extra": [], "note": "FORUST_TUNE_HIST_MIN_ELEMENTS small-node threshold experiment (dropped)"} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 107.21519885, "bin_s": 0.43722503, "per_iteration_ms": {"eval_predict_metric": 0.5465147666666667, "grad_hess": 0.23965313333333335, "train_predict": 1.0563952333333337, "tree": 117.44229443333333}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [112.0053757, 107.21519885, 103.71211989999999], "bin_s_samples": [0.43722503, 0.436305206, 0.440710777], "label": "item2d-tune-1000000", "commit": "aefa101-dirty", "extra": ["--max-depth", "8"], "note": "FORUST_TUNE_HIST_MIN_ELEMENTS small-node threshold experiment (dropped)"} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 28.6277556, "bin_s": 0.436856074, "per_iteration_ms": {"eval_predict_metric": 0.3926106166666666, "grad_hess": 0.2078056166666667, "train_predict": 0.47570793333333333, "tree": 29.932418}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [30.14656604, 28.6277556, 27.48299082], "bin_s_samples": [0.436856074, 0.436071985, 0.439240671], "label": "item2d-tune-1000000", "commit": "aefa101-dirty", "extra": [], "note": "FORUST_TUNE_HIST_MIN_ELEMENTS small-node threshold experiment (dropped)"} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 72.68402236, "bin_s": 1.933788504, "per_iteration_ms": {"eval_predict_metric": 1.0953946666666667, "grad_hess": 0.7740075833333333, "train_predict": 1.365223583333333, "tree": 73.52970160000001}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [72.91284755999999, 72.68402236, 71.10014991999999], "bin_s_samples": [1.914322446, 1.941587902, 1.933788504], "label": "item2d-tune-1000000", "commit": "aefa101-dirty", "extra": [], "note": "FORUST_TUNE_HIST_MIN_ELEMENTS small-node threshold experiment (dropped)"} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 20, "tree_ms": 230.9395978, "bin_s": 25.167862914, "per_iteration_ms": {"eval_predict_metric": 9.18623765, "grad_hess": 7.614095150000002, "train_predict": 9.080129750000001, "tree": 238.71057434999997}, "eval_logloss": 0.4052419048375606, "tree_ms_samples": [230.9395978, 226.4722435, 232.6398268], "bin_s_samples": [25.229504426, 25.167862914, 25.036657716], "label": "item2c", "commit": "aefa101", "grid": "G4", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 59.443356599999994, "bin_s": 0.285774075, "per_iteration_ms": {"eval_predict_metric": 0.5598416833333333, "grad_hess": 0.1933885166666666, "train_predict": 1.6178184500000004, "tree": 59.976747183333345}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [58.97266592, 60.25453624, 59.443356599999994], "bin_s_samples": [0.285823067, 0.284973945, 0.285774075], "label": "item3", "commit": "86a77b3", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 16.531746260000002, "bin_s": 0.038937446, "per_iteration_ms": {"eval_predict_metric": 0.35443275, "grad_hess": 0.19677364999999997, "train_predict": 0.41120281666666664, "tree": 16.74295491666667}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [16.531746260000002, 17.1692973, 16.24893274], "bin_s_samples": [0.038937446, 0.045959535, 0.038228228], "label": "item3", "commit": "86a77b3", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 158.01826256, "bin_s": 1.17564201, "per_iteration_ms": {"eval_predict_metric": 2.2605473166666665, "grad_hess": 0.7737541, "train_predict": 5.9286441666666665, "tree": 160.38992866666666}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [158.01826256, 156.70740554000002, 160.17827392], "bin_s_samples": [1.174501964, 1.17564201, 1.178716022], "label": "item3", "commit": "86a77b3", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 39.3458792, "bin_s": 0.159414776, "per_iteration_ms": {"eval_predict_metric": 1.1126764333333337, "grad_hess": 0.7710907666666663, "train_predict": 1.2565654499999999, "tree": 40.04417896666667}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [39.3458792, 40.1208941, 39.12868516], "bin_s_samples": [0.159905488, 0.156421596, 0.159414776], "label": "item3", "commit": "86a77b3", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 333.48428479999995, "bin_s": 3.111925037, "per_iteration_ms": {"eval_predict_metric": 5.1051467833333355, "grad_hess": 1.91081955, "train_predict": 13.334768950000004, "tree": 339.20815309999983}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [333.21505614, 334.66754596, 333.48428479999995], "bin_s_samples": [3.102278771, 3.111925037, 3.121979016], "label": "item3", "commit": "86a77b3", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 73.35859784, "bin_s": 0.48569624, "per_iteration_ms": {"eval_predict_metric": 2.5503200999999995, "grad_hess": 2.020965933333333, "train_predict": 2.7528607500000004, "tree": 76.43078198333332}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [74.19795204, 73.35859784, 72.31859266], "bin_s_samples": [0.48569624, 0.480345313, 0.505560439], "label": "item3", "commit": "86a77b3", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 220.45975135, "bin_s": 0.285648686, "per_iteration_ms": {"eval_predict_metric": 1.5835259000000002, "grad_hess": 0.19383183333333334, "train_predict": 5.813873166666667, "tree": 230.63638153333338}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [220.45975135, 210.7597792, 221.63085275], "bin_s_samples": [0.284524519, 0.285648686, 0.28698156], "label": "item3", "commit": "86a77b3", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 76.26261935000001, "bin_s": 0.038109324, "per_iteration_ms": {"eval_predict_metric": 0.4989002666666667, "grad_hess": 0.19103856666666666, "train_predict": 1.0054909333333333, "tree": 80.39037016666666}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [74.94077825000001, 78.530337, 76.26261935000001], "bin_s_samples": [0.038109324, 0.03869257, 0.038020252], "label": "item3", "commit": "86a77b3", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 509.6877283, "bin_s": 1.17354263, "per_iteration_ms": {"eval_predict_metric": 6.394585566666666, "grad_hess": 0.7761474999999998, "train_predict": 22.790000099999997, "tree": 523.1007545999998}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [509.6877283, 512.0003495, 500.0293563], "bin_s_samples": [1.177998252, 1.170988418, 1.17354263], "label": "item3", "commit": "86a77b3", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 152.3313428, "bin_s": 0.160041921, "per_iteration_ms": {"eval_predict_metric": 1.6399509333333329, "grad_hess": 0.7826311666666667, "train_predict": 3.4490430999999995, "tree": 157.66238186666666}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [152.3313428, 154.75143255, 151.3420174], "bin_s_samples": [0.155126484, 0.160041921, 0.169477472], "label": "item3", "commit": "86a77b3", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 170.14766895, "bin_s": 0.71510815, "per_iteration_ms": {"eval_predict_metric": 0.5569084000000001, "grad_hess": 0.19252516666666664, "train_predict": 1.4316521333333336, "tree": 171.46993743333334}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [170.14766895, 170.07940015, 173.9159208], "bin_s_samples": [0.712772227, 0.71510815, 0.715461249], "label": "item3", "commit": "86a77b3", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 38.7723136, "bin_s": 0.094381341, "per_iteration_ms": {"eval_predict_metric": 0.36444659999999995, "grad_hess": 0.19836869999999998, "train_predict": 0.4316082666666667, "tree": 38.403676999999995}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [38.7723136, 37.24363255, 39.69205345], "bin_s_samples": [0.092761697, 0.111557103, 0.094381341], "label": "item3", "commit": "86a77b3", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 411.86101214999997, "bin_s": 3.3946596749999998, "per_iteration_ms": {"eval_predict_metric": 2.2838515000000004, "grad_hess": 0.7649638666666667, "train_predict": 5.964216633333333, "tree": 402.1496569}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [414.3322956, 411.726509, 411.86101214999997], "bin_s_samples": [3.3946596749999998, 3.385653907, 3.403723262], "label": "item3", "commit": "86a77b3", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 84.03381115, "bin_s": 0.405171604, "per_iteration_ms": {"eval_predict_metric": 1.1268169666666663, "grad_hess": 0.9027008999999999, "train_predict": 1.3017250666666667, "tree": 85.00247846666666}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [86.28442294999999, 82.64337995, 84.03381115], "bin_s_samples": [0.405171604, 0.403136153, 0.439706537], "label": "item3", "commit": "86a77b3", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 20, "tree_ms": 229.6138088, "bin_s": 1.7879490169999999, "per_iteration_ms": {"eval_predict_metric": 8.966737049999999, "grad_hess": 7.5092574999999995, "train_predict": 8.205811300000002, "tree": 232.82641120000002}, "eval_logloss": 0.4052419048375606, "tree_ms_samples": [228.47729529999998, 229.6138088, 235.4305871], "bin_s_samples": [1.7879490169999999, 1.7460353149999999, 1.792441417], "label": "item3", "commit": "86a77b3", "grid": "G4", "extra": ["--max-depth", "5"]} -{"label": "baseline", "kind": "end_to_end", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "max_depth": 5, "iterations": 100, "threads": 8, "fit_s": 52.7, "eval_logloss": 0.3363535089725551, "commit": "4a6d828"} -{"label": "item3", "kind": "end_to_end", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "max_depth": 5, "iterations": 100, "threads": 8, "fit_s": 27.2, "eval_logloss": 0.3363535089725551, "commit": "86a77b3"} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 60.151627139999995, "bin_s": 0.286478963, "per_iteration_ms": {"eval_predict_metric": 0.5999760333333335, "grad_hess": 0.19268011666666665, "train_predict": 1.7406499666666668, "tree": 61.403277633333325}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [60.151627139999995, 63.65079474, 59.42340778], "bin_s_samples": [0.284748518, 0.287186783, 0.286478963], "label": "item4", "commit": "c1baff6", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 13.147788420000001, "bin_s": 0.041508912, "per_iteration_ms": {"eval_predict_metric": 0.3158700833333333, "grad_hess": 0.1915651, "train_predict": 0.3667048333333334, "tree": 11.179534666666662}, "eval_logloss": 0.36808370982635397, "tree_ms_samples": [10.90360626, 14.376613180000001, 13.147788420000001], "bin_s_samples": [0.038383628, 0.042107528, 0.041508912], "label": "item4", "commit": "c1baff6", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 156.94657292, "bin_s": 1.181276657, "per_iteration_ms": {"eval_predict_metric": 1.8859439666666664, "grad_hess": 0.7758174333333335, "train_predict": 5.385347866666665, "tree": 159.6154316833333}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [157.03831724, 155.81717536, 156.94657292], "bin_s_samples": [1.181276657, 1.184246637, 1.1736463], "label": "item4", "commit": "c1baff6", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 32.49998448, "bin_s": 0.170587387, "per_iteration_ms": {"eval_predict_metric": 1.0488982166666665, "grad_hess": 0.7626388500000001, "train_predict": 1.2047441166666664, "tree": 33.18850908333333}, "eval_logloss": 0.35641406751706667, "tree_ms_samples": [33.62587218, 31.95325122, 32.49998448], "bin_s_samples": [0.152966613, 0.170979898, 0.170587387], "label": "item4", "commit": "c1baff6", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 60, "tree_ms": 333.01327584, "bin_s": 3.104903805, "per_iteration_ms": {"eval_predict_metric": 5.050117633333333, "grad_hess": 1.9307601666666667, "train_predict": 13.195850250000003, "tree": 339.34423255000013}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [333.01327584, 335.04510748, 330.75675232000003], "bin_s_samples": [3.114733602, 3.104903805, 3.065326691], "label": "item4", "commit": "c1baff6", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 60, "tree_ms": 66.11935008, "bin_s": 0.506013762, "per_iteration_ms": {"eval_predict_metric": 2.461680966666667, "grad_hess": 1.9331185166666665, "train_predict": 2.6640964666666673, "tree": 68.59095655000002}, "eval_logloss": 0.35595460801706497, "tree_ms_samples": [66.8091205, 66.11935008, 65.46252038], "bin_s_samples": [0.507787308, 0.506013762, 0.471341531], "label": "item4", "commit": "c1baff6", "grid": "G1", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 214.5040344, "bin_s": 0.285712957, "per_iteration_ms": {"eval_predict_metric": 1.5635953666666667, "grad_hess": 0.19628536666666666, "train_predict": 5.7386808, "tree": 223.76530009999996}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [214.5040344, 230.57628125000002, 213.31792645], "bin_s_samples": [0.285712957, 0.285261966, 0.28587407], "label": "item4", "commit": "c1baff6", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 45.9466113, "bin_s": 0.042381636, "per_iteration_ms": {"eval_predict_metric": 0.4519189333333334, "grad_hess": 0.19613513333333332, "train_predict": 0.9213645999999999, "tree": 48.121332866666656}, "eval_logloss": 0.35978313313637317, "tree_ms_samples": [45.9466113, 50.6799033, 39.260242299999994], "bin_s_samples": [0.046045634, 0.042381636, 0.041687317], "label": "item4", "commit": "c1baff6", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 495.433256, "bin_s": 1.171448491, "per_iteration_ms": {"eval_predict_metric": 6.3336429999999995, "grad_hess": 0.7693906666666667, "train_predict": 22.5515943, "tree": 507.29823156666674}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [494.44822995, 495.433256, 505.51367324999995], "bin_s_samples": [1.169936858, 1.171448491, 1.172516157], "label": "item4", "commit": "c1baff6", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 98.1889654, "bin_s": 0.170564678, "per_iteration_ms": {"eval_predict_metric": 1.6648044666666664, "grad_hess": 0.7739675, "train_predict": 3.6877157333333326, "tree": 128.71078763333338}, "eval_logloss": 0.33651327399569986, "tree_ms_samples": [123.8890399, 97.49076155, 98.1889654], "bin_s_samples": [0.202299429, 0.15501016, 0.170564678], "label": "item4", "commit": "c1baff6", "grid": "G2", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 173.78728230000002, "bin_s": 0.723152629, "per_iteration_ms": {"eval_predict_metric": 0.6421005666666668, "grad_hess": 0.2027588333333333, "train_predict": 1.666264866666667, "tree": 176.50155726666668}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [173.78728230000002, 177.79341265000002, 172.42936645], "bin_s_samples": [0.726915509, 0.712807126, 0.723152629], "label": "item4", "commit": "c1baff6", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 25000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 37.61205965, "bin_s": 0.101167816, "per_iteration_ms": {"eval_predict_metric": 0.3485692666666667, "grad_hess": 0.18887176666666672, "train_predict": 0.44629526666666663, "tree": 39.47090629999998}, "eval_logloss": 0.41869487952917916, "tree_ms_samples": [38.95110065, 31.098814750000003, 37.61205965], "bin_s_samples": [0.101167816, 0.095653568, 0.111793001], "label": "item4", "commit": "c1baff6", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 1, "parallel": false, "iterations": 30, "tree_ms": 416.6293701, "bin_s": 3.406395757, "per_iteration_ms": {"eval_predict_metric": 2.2015627000000006, "grad_hess": 0.7741275000000002, "train_predict": 6.0286929, "tree": 404.8444218666665}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [415.8391113, 416.6293701, 420.01921604999995], "bin_s_samples": [3.406395757, 3.395171381, 3.44141515], "label": "item4", "commit": "c1baff6", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w500", "rows": 100000, "cols": 500, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 30, "tree_ms": 79.4577177, "bin_s": 0.42050685, "per_iteration_ms": {"eval_predict_metric": 1.0573351666666666, "grad_hess": 0.7659541333333332, "train_predict": 1.1856686666666667, "tree": 74.88955133333333}, "eval_logloss": 0.4129764726227001, "tree_ms_samples": [76.9209361, 81.1009539, 79.4577177], "bin_s_samples": [0.421187605, 0.42050685, 0.386096121], "label": "item4", "commit": "c1baff6", "grid": "G3", "extra": ["--max-depth", "5"]} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 20, "tree_ms": 228.90968859999998, "bin_s": 1.775385495, "per_iteration_ms": {"eval_predict_metric": 9.0054042, "grad_hess": 7.65706775, "train_predict": 8.702826050000002, "tree": 230.14402875000002}, "eval_logloss": 0.4052419048375606, "tree_ms_samples": [224.3267686, 228.90968859999998, 232.45199390000002], "bin_s_samples": [1.775385495, 1.78284543, 1.7363478909999999], "label": "item4", "commit": "c1baff6", "grid": "G4", "extra": ["--max-depth", "5"]} -{"label": "item4", "commit": "c1baff6", "kind": "serial_share", "data": "/tmp/forust-perf/w200", "rows": 100000, "max_depth": 5, "window_s": 1.62, "main_samples": 4, "total_samples": 10171, "serial_fraction": 0.002, "main_thread_top": [["[unknown]", 1.0]]} -{"label": "item4", "commit": "c1baff6", "kind": "serial_share", "data": "/tmp/forust-perf/w200", "rows": 25000, "max_depth": 8, "window_s": 1.26, "main_samples": 1, "total_samples": 8380, "serial_fraction": 0.001, "main_thread_top": [["[unknown]", 1.0]]} -{"label": "baseline", "commit": "pre-4a6d828", "kind": "drift_1000", "rows": 100000, "cols": 200, "threads": 16, "total_s": 80.8, "phases_s": {"bin": 1.97, "tree": 75.43, "train_predict": 1.34, "eval_predict_metric": 1.27, "grad_hess": 0.8}} -{"label": "item4", "commit": "c1baff6", "kind": "drift_1000", "rows": 100000, "cols": 200, "threads": 16, "total_s": 37.167717122, "phases_s": {"bin": 0.136399647, "eval_predict_metric": 1.2341079629999991, "grad_hess": 0.7918761129999998, "train_predict": 1.232725531000001, "tree": 33.770520915}} -{"label": "item4", "commit": "c1baff6", "kind": "end_to_end", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "max_depth": 5, "iterations": 100, "threads": 8, "fit_s": 26.1, "eval_logloss": 0.3363535089725551} -{"label": "xgboost-final", "kind": "xgboost", "data": "w200", "rows": 25000, "max_depth": 5, "threads": 8, "iteration_ms": 10.1} -{"label": "xgboost-final", "kind": "xgboost", "data": "w200", "rows": 100000, "max_depth": 5, "threads": 8, "iteration_ms": 30.1} -{"label": "xgboost-final", "kind": "xgboost", "data": "w200", "rows": 250000, "max_depth": 5, "threads": 8, "iteration_ms": 63.8} -{"label": "xgboost-final", "kind": "xgboost", "data": "w200", "rows": 25000, "max_depth": 8, "threads": 8, "iteration_ms": 39.9} -{"label": "xgboost-final", "kind": "xgboost", "data": "w200", "rows": 100000, "max_depth": 8, "threads": 8, "iteration_ms": 87.6} -{"label": "xgboost-final", "kind": "xgboost", "data": "w500", "rows": 25000, "max_depth": 5, "threads": 8, "iteration_ms": 37.1} -{"label": "xgboost-final", "kind": "xgboost", "data": "w500", "rows": 100000, "max_depth": 5, "threads": 8, "iteration_ms": 107.1} -{"label": "xgboost-final", "kind": "xgboost", "data": "w200-1m", "rows": 1000000, "max_depth": 5, "threads": 8, "iteration_ms": 373.2} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 4, "parallel": true, "iterations": 40, "tree_ms": 16.781396466666667, "bin_s": 0.078603807, "per_iteration_ms": {"eval_predict_metric": 0.32948132500000005, "grad_hess": 0.18841747499999997, "train_predict": 0.529656675, "tree": 17.59580425}, "eval_logloss": 0.3860608457388993, "tree_ms_samples": [17.39065623333333, 16.734341933333333, 16.781396466666667], "bin_s_samples": [0.078010891, 0.078603807, 0.07870761], "label": "item4-threads", "commit": "c1baff6", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 40, "tree_ms": 15.2728093, "bin_s": 0.042085828, "per_iteration_ms": {"eval_predict_metric": 0.33809897500000013, "grad_hess": 0.19145517499999995, "train_predict": 0.41567875, "tree": 15.034622925000003}, "eval_logloss": 0.3860608457388993, "tree_ms_samples": [15.2728093, 15.960969533333333, 12.036424299999998], "bin_s_samples": [0.042085828, 0.04512301, 0.038353023], "label": "item4-threads", "commit": "c1baff6", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 16, "parallel": true, "iterations": 40, "tree_ms": 15.612093066666667, "bin_s": 0.032574769, "per_iteration_ms": {"eval_predict_metric": 0.4277640000000001, "grad_hess": 0.19162010000000007, "train_predict": 0.516843725, "tree": 16.034720224999997}, "eval_logloss": 0.3860608457388993, "tree_ms_samples": [15.612093066666667, 19.140777266666667, 14.745630033333333], "bin_s_samples": [0.032574769, 0.03297338, 0.031672145], "label": "item4-threads", "commit": "c1baff6", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 4, "parallel": true, "iterations": 40, "tree_ms": 46.23637526666666, "bin_s": 0.303462139, "per_iteration_ms": {"eval_predict_metric": 1.1165623750000004, "grad_hess": 0.7528425750000001, "train_predict": 1.5146654, "tree": 47.74266604999999}, "eval_logloss": 0.37647776675685285, "tree_ms_samples": [46.897870966666666, 45.607358266666665, 46.23637526666666], "bin_s_samples": [0.30025181, 0.303462139, 0.31455804], "label": "item4-threads", "commit": "c1baff6", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 40, "tree_ms": 31.388113366666666, "bin_s": 0.170324024, "per_iteration_ms": {"eval_predict_metric": 1.041068225, "grad_hess": 0.756905575, "train_predict": 1.1782420500000002, "tree": 33.26705809999999}, "eval_logloss": 0.37647776675685285, "tree_ms_samples": [32.21435783333333, 30.4575684, 31.388113366666666], "bin_s_samples": [0.184675214, 0.170324024, 0.169923466], "label": "item4-threads", "commit": "c1baff6", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 16, "parallel": true, "iterations": 40, "tree_ms": 36.8773315, "bin_s": 0.12954026, "per_iteration_ms": {"eval_predict_metric": 1.20993, "grad_hess": 0.8288097499999999, "train_predict": 1.4197356500000002, "tree": 38.5916095}, "eval_logloss": 0.37647776675685285, "tree_ms_samples": [38.84676053333333, 35.3693205, 36.8773315], "bin_s_samples": [0.124466744, 0.129788187, 0.12954026], "label": "item4-threads", "commit": "c1baff6", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 4, "parallel": true, "iterations": 40, "tree_ms": 98.35924686666667, "bin_s": 0.9548333, "per_iteration_ms": {"eval_predict_metric": 2.66371115, "grad_hess": 1.9310914500000003, "train_predict": 3.801744025, "tree": 100.66450597500001}, "eval_logloss": 0.3762912848327184, "tree_ms_samples": [98.35924686666667, 98.01377303333334, 98.37588286666667], "bin_s_samples": [0.9548333, 0.93422095, 0.965518299], "label": "item4-threads", "commit": "c1baff6", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 8, "parallel": true, "iterations": 40, "tree_ms": 67.92310549999999, "bin_s": 0.51527495, "per_iteration_ms": {"eval_predict_metric": 2.3413353499999996, "grad_hess": 1.9133289499999993, "train_predict": 2.450840875, "tree": 66.76468864999998}, "eval_logloss": 0.3762912848327184, "tree_ms_samples": [66.36106943333334, 67.92310549999999, 68.6935559], "bin_s_samples": [0.4764414, 0.519423462, 0.51527495], "label": "item4-threads", "commit": "c1baff6", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 250000, "cols": 200, "max_depth": 5, "nbins": 256, "threads": 16, "parallel": true, "iterations": 40, "tree_ms": 69.5873865, "bin_s": 0.376090401, "per_iteration_ms": {"eval_predict_metric": 2.616290325, "grad_hess": 2.00426385, "train_predict": 2.90213045, "tree": 72.0958072}, "eval_logloss": 0.3762912848327184, "tree_ms_samples": [69.5873865, 69.12910403333333, 70.1277002], "bin_s_samples": [0.35060022, 0.380575326, 0.376090401], "label": "item4-threads", "commit": "c1baff6", "extra": []} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 20, "tree_ms": 38.871135200000005, "bin_s": 0.041909727, "per_iteration_ms": {"eval_predict_metric": 0.43251169999999994, "grad_hess": 0.19698035000000003, "train_predict": 0.9128043999999997, "tree": 51.149548800000005}, "eval_logloss": 0.37856465304823506, "tree_ms_samples": [46.7806363, 38.871135200000005, 35.350590700000005], "bin_s_samples": [0.039851471, 0.049086218, 0.041909727], "label": "item4-threads", "commit": "c1baff6", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 25000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 16, "parallel": true, "iterations": 20, "tree_ms": 64.6603486, "bin_s": 0.032710779, "per_iteration_ms": {"eval_predict_metric": 0.55827, "grad_hess": 0.20074550000000002, "train_predict": 0.85627815, "tree": 74.89098869999998}, "eval_logloss": 0.37856465304823506, "tree_ms_samples": [72.9080505, 64.6603486, 48.301259], "bin_s_samples": [0.031144938, 0.032710779, 0.033463499], "label": "item4-threads", "commit": "c1baff6", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 20, "tree_ms": 100.9977406, "bin_s": 0.168885841, "per_iteration_ms": {"eval_predict_metric": 1.5557568499999999, "grad_hess": 0.7727308000000002, "train_predict": 3.3159489500000014, "tree": 100.0247506}, "eval_logloss": 0.360139872242242, "tree_ms_samples": [100.9977406, 104.94419669999999, 95.0364908], "bin_s_samples": [0.168885841, 0.167378992, 0.170106165], "label": "item4-threads", "commit": "c1baff6", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200", "rows": 100000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 16, "parallel": true, "iterations": 20, "tree_ms": 126.30489240000001, "bin_s": 0.129796592, "per_iteration_ms": {"eval_predict_metric": 1.4783498000000004, "grad_hess": 0.7987962999999999, "train_predict": 2.6536663500000004, "tree": 130.41624875}, "eval_logloss": 0.360139872242242, "tree_ms_samples": [123.803101, 126.74488980000001, 126.30489240000001], "bin_s_samples": [0.127404727, 0.133849001, 0.129796592], "label": "item4-threads", "commit": "c1baff6", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 8, "nbins": 256, "threads": 8, "parallel": true, "iterations": 20, "tree_ms": 633.6740773, "bin_s": 1.774890692, "per_iteration_ms": {"eval_predict_metric": 14.562104799999997, "grad_hess": 7.581352849999999, "train_predict": 31.451457149999992, "tree": 628.7355155}, "eval_logloss": 0.361692034994103, "tree_ms_samples": [633.6740773], "bin_s_samples": [1.774890692], "label": "item4-1m-depth8", "commit": "d26345a", "extra": ["--max-depth", "8"]} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 6, "nbins": 256, "threads": 8, "parallel": true, "iterations": 20, "tree_ms": 299.7908149, "bin_s": 1.760453238, "per_iteration_ms": {"eval_predict_metric": 11.030639699999998, "grad_hess": 7.924411600000001, "train_predict": 15.787297600000002, "tree": 312.90538545000004}, "eval_logloss": 0.3930815710991101, "tree_ms_samples": [305.066795, 297.456743, 299.7908149], "bin_s_samples": [1.763585155, 1.7347879019999999, 1.760453238], "label": "lossguide-forust", "commit": "1cc3a4a-dirty", "extra": ["--max-depth", "6", "--grow-policy", "LossGuide"]} -{"library": "xgboost", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "threads": 8, "max_depth": 6, "grow_policy": "lossguide", "max_leaves": 0, "nbins": 256, "dmatrix_s": 10.169106015000011, "iteration_ms": 403.19467550000354} -{"library": "xgboost", "data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "threads": 8, "max_depth": 6, "grow_policy": "lossguide", "max_leaves": 0, "nbins": 256, "dmatrix_s": 9.503570269999955, "fit_s": 8.146005285000001, "iteration_ms": 407.30026425000005} -{"data": "/tmp/forust-perf/w200-1m", "rows": 1000000, "cols": 200, "max_depth": 6, "nbins": 256, "threads": 8, "parallel": true, "iterations": 20, "tree_ms": 310.04808299999996, "bin_s": 1.794623013, "per_iteration_ms": {"eval_predict_metric": 10.736746949999999, "grad_hess": 7.8801359500000006, "train_predict": 15.585925650000002, "tree": 313.0998498500001}, "eval_logloss": 0.3930815710991101, "tree_ms_samples": [305.2696742, 310.04808299999996, 310.1514883], "bin_s_samples": [1.778949452, 1.794623013, 1.8193301549999998], "label": "lossguide-forust-corrected", "commit": "e11e8f1-dirty", "extra": ["--max-depth", "6", "--grow-policy", "LossGuide"]} diff --git a/agent/feat/optimize/results.md b/agent/feat/optimize/results.md deleted file mode 100644 index f6d39ea..0000000 --- a/agent/feat/optimize/results.md +++ /dev/null @@ -1,289 +0,0 @@ -# Optimization Results - -Running log of each change in `plan.md` and its effect on speed. Raw data is in -`results.jsonl`. Tables are generated with `scripts/perf_compare.py`. - -## How to read this - -- **Tree ms:** steady-state milliseconds per tree (the first 10 iterations are - skipped), median of 3 runs, from `scripts/perf_grid.py`. -- **Serial fraction:** share of training-loop wall time spent in serial code - at 8 threads, from `scripts/perf_serial_share.py`. Lower is better. -- **Unchanged models:** after every change, `scripts/perf_golden.py check` - confirms the trained trees are byte-identical to the baseline (5 - configurations, both splitters), and `cargo test` passes, including a test - that serial and parallel training give identical trees. -- **Setup:** synthetic binary-classification data from - `scripts/make_perf_data.py`, learning rate 0.1, 256 bins, evaluation set of - 1/4 the training rows. Azure VM with 8 physical cores (16 vCPU). Repeat runs - vary by about 5%. -- **Serial timings also carry about ±6% from code placement.** Between item 2b - and 2c, the serial histogram loop slowed 3-8% even though its instructions - were byte-identical; only its position relative to 64-byte boundaries - changed (details under item 2c). Read small serial differences as noise. - -| Grid | What it isolates | -| --- | --- | -| G1 | 200 columns, depth 5, 25k-250k rows: fixed vs per-row cost | -| G2 | 200 columns, depth 8: many small nodes | -| G3 | 500 columns, depth 5: wide data | -| G4 | 1M rows x 200 columns, depth 5 | - -## Summary - -All four plan items are done. Trained models are byte-identical to the -baseline throughout, and the Python suite passes (112 tests, including the -XGBoost parity checks and a new `num_threads` test). - -| Workload | Baseline | Final | Change | -| --- | ---: | ---: | ---: | -| 100-iteration `fit`, 1M x 200, 8 threads | 52.7 s | 26.1 s | 2.0x faster | -| 1,000 iterations, 100k x 200, default 16 threads | 80.8 s | 37.2 s | 2.2x faster | -| Tree, 8 threads, across G1-G3 | | | 39-81% less time | -| Binning, 8 threads | | | 91-93% less time | - -Tree ms per change. Each column includes all earlier changes; "item2" is -items 2a-2c (2d was dropped). Per-step numbers are in the step log. - -| Grid | Data | Rows | Depth | Mode | baseline | item1 | item2 | item3 | item4 | final vs baseline | -| --- | --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | -| G1 | w200 | 25k | 5 | serial | 69.5 | 69.8 | 57.9 | 59.4 | 60.2 | -13.4% | -| G1 | w200 | 25k | 5 | 8 threads | 52.7 | 35.9 | 20.0 | 16.5 | 13.1 | -75.1% | -| G1 | w200 | 100k | 5 | serial | 165.4 | 169.8 | 153.9 | 158.0 | 156.9 | -5.1% | -| G1 | w200 | 100k | 5 | 8 threads | 72.5 | 56.4 | 39.0 | 39.3 | 32.5 | -55.2% | -| G1 | w200 | 250k | 5 | serial | 340.8 | 340.1 | 332.3 | 333.5 | 333.0 | -2.3% | -| G1 | w200 | 250k | 5 | 8 threads | 108.3 | 89.2 | 73.8 | 73.4 | 66.1 | -39.0% | -| G2 | w200 | 25k | 8 | serial | 242.7 | 241.7 | 195.4 | 220.5 | 214.5 | -11.6% | -| G2 | w200 | 25k | 8 | 8 threads | 236.6 | 147.0 | 80.8 | 76.3 | 45.9 | -80.6% | -| G2 | w200 | 100k | 8 | serial | 560.6 | 528.6 | 498.9 | 509.7 | 495.4 | -11.6% | -| G2 | w200 | 100k | 8 | 8 threads | 358.1 | 245.1 | 149.4 | 152.3 | 98.2 | -72.6% | -| G3 | w500 | 25k | 5 | serial | 190.9 | 190.2 | 176.4 | 170.1 | 173.8 | -9.0% | -| G3 | w500 | 25k | 5 | 8 threads | 128.1 | 85.0 | 40.3 | 38.8 | 37.6 | -70.6% | -| G3 | w500 | 100k | 5 | serial | 433.9 | 434.2 | 410.2 | 411.9 | 416.6 | -4.0% | -| G3 | w500 | 100k | 5 | 8 threads | 174.9 | 127.7 | 86.2 | 84.0 | 79.5 | -54.6% | -| G4 | w200-1m | 1000k | 5 | 8 threads | 261.0 | 247.6 | 230.9 | 229.6 | 228.9 | -12.3% | - -At 1M rows, per-row histogram work dominates and already scaled well, so the -tree time barely moved; the 1M fit gain comes mostly from binning (item 3). - -8-thread speedup over serial: - -| Case | baseline | item1 | item2 | final | -| --- | ---: | ---: | ---: | ---: | -| 25k x 200, depth 5 | 1.3x | 1.9x | 2.9x | 4.6x | -| 100k x 200, depth 5 | 2.3x | 3.0x | 3.9x | 4.8x | -| 250k x 200, depth 5 | 3.1x | 3.8x | 4.5x | 5.0x | -| 25k x 200, depth 8 | 1.0x | 1.6x | 2.4x | 4.7x | -| 100k x 200, depth 8 | 1.6x | 2.2x | 3.3x | 5.0x | -| 100k x 500, depth 5 | 2.5x | 3.4x | 4.8x | 5.2x | - -Compared with XGBoost 1.7.6 `hist` on the same data and settings, 8 threads, -ms per tree. XGBoost times are full iterations; Forust's are tree-only, which -understates Forust's iteration time by 3-10%. - -| Case | Forust baseline | Forust final | XGBoost | -| --- | ---: | ---: | ---: | -| 25k x 200, depth 5 | 52.7 | 13.1 | 10.1 | -| 100k x 200, depth 5 | 72.5 | 32.5 | 30.1 | -| 250k x 200, depth 5 | 108.3 | 66.1 | 63.8 | -| 25k x 200, depth 8 | 236.6 | 45.9 | 39.9 | -| 100k x 200, depth 8 | 358.1 | 98.2 | 87.6 | -| 25k x 500, depth 5 | 128.1 | 37.6 | 37.1 | -| 100k x 500, depth 5 | 174.9 | 79.5 | 107.1 | -| 1M x 200, depth 5 | 261.0 | 228.9 | 373.2 | -| 1M x 200, depth 8 | not run | 633.7 | 454.6 | - -The 1M x 200 depth 8 row is a single run of 20 iterations. From depth 5 to -depth 8, Forust's time grows 177% at 1M rows versus 22% for XGBoost; serial -row partitioning after each split is the likely cause (not yet profiled). - -### LossGuide comparison - -Matched 1M x 200 binary-classification runs, depth 6, 20 iterations, 256 bins, -one evaluation set, and 8 threads. XGBoost used `grow_policy=lossguide` and -`max_leaves=0` (unlimited); Forust used `GrowPolicy::LossGuide` and its -unlimited `max_leaves` default. - -| Measurement | Forust | XGBoost | -| --- | ---: | ---: | -| Tree/iteration phase | 310.0 ms | 407.3 ms | -| Full fit | 8.07 s median of 3 | 8.15 s | -| Final evaluation log-loss | 0.39308 | not collected | - -This is effectively tied within machine noise, with Forust about 1% faster on -the full fit. The tree-only comparison is not apples-to-apples because the -Forust phase timer excludes prediction and evaluation while XGBoost's fit timer -includes them; the full-fit row is the meaningful comparison. - -Serial fraction at 8 threads (item 4 moved training onto a pool thread, so -the main thread now only waits and this metric no longer applies): - -| Case | baseline | item1 | item2a | item2b | item2c | -| --- | ---: | ---: | ---: | ---: | ---: | -| 100k x 200, depth 5 | 63% | 43% | 47% | 36% | 17% | -| 25k x 200, depth 8 | 81% | 57% | 54% | 35% | 12% | - -Binning time per fit (seconds), after item 3: - -| Data | Rows | Mode | baseline | item3 | Change | -| --- | ---: | --- | ---: | ---: | ---: | -| w200 | 100k | serial | 1.94 | 1.18 | -39% | -| w200 | 100k | 8 threads | 1.91 | 0.16 | -92% | -| w200 | 250k | serial | 5.52 | 3.11 | -44% | -| w200 | 250k | 8 threads | 5.78 | 0.49 | -92% | -| w500 | 100k | serial | 5.30 | 3.39 | -36% | -| w500 | 100k | 8 threads | 5.14 | 0.41 | -92% | -| w200-1m | 1M | 8 threads | 24.73 | 1.79 | -93% | - -End to end, a real 100-iteration `fit` on 1M x 200 with 8 threads went from -52.7 s (baseline) to 27.2 s (item 3) and 26.1 s (final), with bit-identical -evaluation log-loss. - -## Step log - -### baseline (`4a6d828`) - -Analysis tooling, the determinism test, and the golden models. No library -changes. Serial and parallel training already produce identical trees. - -Main thread at 100k x 200, depth 5, 8 threads: `evaluate_split` 31%, loops -inlined into the caller (mostly the split scan) 18%, unresolved (likely kernel) -12%, `HistogramMatrix::new` 10%, `free` 9%, histogram subtraction 8%. - -### item1: parallel split search (`c111e36`) - -`Splitter::best_split` scans features in parallel when `parallel` is set. Ties -go to the earliest feature, as in the serial loop. A new unit test checks -parallel against serial with many exact gain ties. - -- Models unchanged; 44 Rust tests pass. -- Features per task: 1 (Rayon's default) beat 8 and 32. At 25k depth 8: - 149.8 / 156.2 / 170.0 ms; at 100k depth 5: 54.4 / 55.9 / 57.0 ms. -- 8 threads: 17-38% faster across the grid, largest for many small nodes - (depth 8) and wide data. 1M rows: 5% faster, because histogram building - dominates there. -- Serial: unchanged within noise (control). -- Main thread now: unresolved (likely kernel) 24%, `HistogramMatrix::new` 18%, - `free` 18%, histogram subtraction 15%, row partitioning 11%. - `evaluate_split` is gone from the main thread. - -### item2a: borrow gradients instead of copying (`763e861`) - -`HistogramMatrix::new` borrowed the full gradient/Hessian arrays at the root -(where no reordering is needed) instead of copying them every tree, and -gathers them in parallel for nodes with at least 16,384 rows. - -- Models unchanged. -- Serial: 3-10% faster (no per-tree full copy). 8 threads: a further 1-9%. -- Serial fraction about the same (47% / 54%). - -### item2b: write histograms in place (`18744aa`) - -Each node's histogram is allocated once and split into one slice per column, -and the columns are filled in parallel. Previously every column produced its -own vector and Rayon concatenated them on the calling thread. - -- Models unchanged. -- 8 threads: a further 15-33% faster than item2a (25k depth 8: 143.9 to - 99.2 ms). Serial: 3-20% faster (fewer allocations). -- Serial fraction: 36% / 35%. Histogram subtraction became the largest named - item on the main thread. - -### item2c: parallel histogram subtraction (`aefa101`) - -`from_parent_child` and `from_parent_two_children` subtract in parallel -(4,096 bins per task) when `parallel` is set. Also applied `rustfmt` to code -added on this branch. - -- Models unchanged. -- 8 threads: a further 1-28% faster than item2b (25k x 500: 56.3 to - 40.3 ms; 25k depth 8: 99.2 to 80.8 ms). 1M rows: 230.9 ms (-11.5% vs - baseline). -- Serial fraction: 17% / 12%. The main thread now mostly runs row - partitioning (`handle_split_info`) and unresolved (likely kernel) code. -- **Serial regression investigated:** serial runs were 3-8% slower than - item2b, although this change doesn't touch the serial histogram loop. Item 2b - and 2c builds timed back to back confirmed it (100k: ~145 vs ~155 ms), and - `perf` placed the extra time in the histogram fill loop. Its disassembly is - byte-identical in both builds; only the address differs. In item2b the hot - 31-byte inner loop sits within one 64-byte block; in item2c it starts - 48 bytes in and crosses a boundary. This is a code-placement effect that - any change can trigger, not a cost of item 2c. Forcing loop alignment is - a possible later experiment. - -### item2d: small-node threshold (dropped) - -Build histograms serially when a node's rows x columns is below a threshold. -Tested with a temporary environment variable, 8 threads, tree ms: - -| Threshold (rows x columns) | 25k depth 8 | 25k depth 5 | 100k depth 5 | -| ---: | ---: | ---: | ---: | -| 0 (always parallel) | 80.7 | 19.1 | 35.9 | -| 50k | 92.8 | 19.0 | 41.0 | -| 200k | 99.3 | 20.3 | 48.9 | -| 1M | 107.2 | 28.6 | 72.7 | - -Every threshold was slower. With 200+ columns even small nodes have enough -per-column work to keep the workers busy, so the change was not committed. - -### item3: parallel binning, one sort per column (`86a77b3`) - -- **3a:** `percentiles_or_value` sorted a copy of each column to count - distinct values, then `percentiles` sorted an index array again. Now each - column is sorted once. With uniform weights it sorts the values directly; - tied values can't change the percentile walk when every weight is equal. - Otherwise it does one index sort shared by both steps. The percentile walk - moved to `utils::percentiles_of_sorted`; the public `percentiles` (used by - Python) is unchanged. -- **3b:** `bin_matrix` takes a `parallel` argument (API change, approved). - Column cuts and bin assignment run per column in parallel, which also - removes a division per value. -- Models unchanged. New tests: the new `percentiles_or_value` matches the old - algorithm on uniform and weighted, continuous and tied data; serial and - parallel `bin_matrix` give identical cuts and bins with missing values and - weights. 46 Rust tests pass. -- Binning: serial (3a only) 34-44% faster; 8 threads 91-93% faster (11-14x). - 1M x 200: 24.7 s to 1.8 s, beating the 5 s target. -- Tree times unchanged within noise. - -### item3a': pair sort for weighted data (not adopted) - -Sorting (value, weight) pairs instead of an index array would only help -weighted data (uniform weights already use a direct sort), and could change a -cut in its last digit when tied values carry different weights. Timed in a -scratch program on 1M-row columns: continuous 68.4 to 49.0 ms, 50 levels 23.9 -to 15.0 ms per column. For a weighted 1M x 200 fit on 8 threads that's about -0.5 s of a 27 s fit (~2%), so it isn't worth the risk. - -### item4: thread count control (`fcbc1c0`, `c1baff6`) - -- **`num_threads` (`fcbc1c0`):** new optional setting on `GradientBooster` - (Rust setter, Python constructor argument, `get_params`, save/load). - When set, `fit`, `predict`, `predict_contributions`, - `predict_leaf_indices`, and partial dependence run in a dedicated Rayon - pool of that size. Defaults to `None` (today's behavior). Older saved - models still load. New Rust and Python tests: identical trees and - predictions for 1, 2, and 4 threads, and the setting survives save/load. -- **Unexpected finding:** `num_threads=8` was much faster than - `RAYON_NUM_THREADS=8` (25k depth 8: 0.88 s vs 1.57 s per fit). Inside a pool, - `fit` runs on a worker thread, so each parallel step is split among workers - directly. Called from the main thread, every parallel step is a blocking - hand-off to the pool, which adds up over thousands of small nodes. -- **Fix (`c1baff6`):** parallel `fit` and prediction now always run on a pool - thread (`rayon::scope`), so the default path gets the same benefit. Serial - mode is unchanged. The harness was updated to match. -- Models unchanged; 47 Rust and 112 Python tests pass. -- 8 threads, vs item 3: a further 4-40% faster; 25k depth 8: 76.3 to - 45.9 ms, 100k depth 8: 152.3 to 98.2 ms. -- `num_threads=8` now matches `RAYON_NUM_THREADS=8` within noise. -- Building a pool on every `predict` call adds no measurable cost: 1,000 calls - on 1,000 rows with 100 trees took 2.46 ms per call with `num_threads=8` - versus 3.1 ms on the default 16-thread pool, so no pool caching is needed. - -## Final confirmation - -- Full G1-G4 grid (table above), XGBoost rerun, and Python suite (112 - passed). -- 1,000-iteration run at 100k x 200, default 16 threads: 80.8 s to 37.2 s. - Per-tree time is stable (39.6 ms early, 33 ms steady state) and tree sizes - match the baseline run. Binning is now 0.14 s of the 37.2 s; trees are 91%. diff --git a/agent/feat/optimize/thread_tests.md b/agent/feat/optimize/thread_tests.md deleted file mode 100644 index ff704ae..0000000 --- a/agent/feat/optimize/thread_tests.md +++ /dev/null @@ -1,143 +0,0 @@ -# Thread Count Tests - -Goal: decide whether `num_threads` should default to the number of physical -cores instead of Rayon's default (one thread per logical CPU). On the -development VM (8 physical cores, 16 vCPUs), 8 threads beat 16 by 2-18% at -depth 5 and 25-66% at depth 8. These tests check whether that holds on other -machines. - -Run everything from the repository root. Expected total time: about 30-45 -minutes, depending on the machine. - -## 1. Get the branch onto the machine - -The `feat/optimize` branch exists only as local commits on the development VM. -Either push it to a remote you control, or copy it as a bundle: - -```sh -# On the development VM -git bundle create /tmp/forust-optimize.bundle main..feat/optimize - -# On the test machine, inside an existing clone of the repository -git fetch /path/to/forust-optimize.bundle feat/optimize:feat/optimize -git checkout feat/optimize -``` - -## 2. Record the machine - -```sh -lscpu | grep -E 'Model name|^CPU\(s\)|Thread\(s\) per core|Core\(s\) per socket|Socket\(s\)|NUMA node\(s\)' -nproc -# Physical cores (unique core/socket pairs) -lscpu -p=core,socket | grep -v '^#' | sort -u | wc -l -# CPU limit if running in a container ("max" means no limit) -cat /sys/fs/cgroup/cpu.max 2>/dev/null -free -g -``` - -From the output, set these two values for the rest of the steps: - -```sh -export PHYSICAL=8 # physical cores -export LOGICAL=16 # logical CPUs (nproc) -export LABEL=my-machine-name -``` - -If the container CPU limit is lower than `LOGICAL`, note it; that limit is the -real ceiling. - -## 3. Set up - -```sh -# Rust (if not installed): https://rustup.rs -# uv (if not installed): https://docs.astral.sh/uv/ -export UV_PROJECT_ENVIRONMENT=$HOME/uv-environments/forust -(cd py-forust && uv sync --group dev) -PY=$UV_PROJECT_ENVIRONMENT/bin/python - -cargo build --release --example perf_phases - -$PY scripts/make_perf_data.py --out /tmp/forust-perf/w200 --rows 250000 --eval-rows 62500 --cols 200 -$PY scripts/make_perf_data.py --out /tmp/forust-perf/w500 --rows 100000 --eval-rows 25000 --cols 500 -``` - -Check the build reproduces the reference models (only meaningful if you also -copied `/tmp/forust-perf/golden` from the development VM; otherwise skip): - -```sh -$PY scripts/perf_golden.py check -``` - -## 4. Tree-time sweep - -Steady-state milliseconds per tree for each thread count, median of 3 runs. -The thread counts are half the physical cores, the physical cores, and all -logical CPUs. A serial run is included as a reference. - -```sh -OUT=agent/feat/optimize/thread_results.jsonl -THREADS="$((PHYSICAL / 2)) $PHYSICAL $LOGICAL" - -# Depth 5, 200 columns -$PY scripts/perf_sweep.py --data /tmp/forust-perf/w200 --rows 25000 100000 250000 \ - --threads $THREADS --iterations 40 --repeats 3 --label $LABEL --out $OUT - -# Depth 8, 200 columns (many small nodes; most sensitive to thread count) -$PY scripts/perf_sweep.py --data /tmp/forust-perf/w200 --rows 25000 100000 \ - --threads $THREADS --iterations 20 --repeats 3 --label $LABEL --out $OUT -- --max-depth 8 - -# Depth 5, 500 columns -$PY scripts/perf_sweep.py --data /tmp/forust-perf/w500 --rows 25000 100000 \ - --threads $THREADS --iterations 30 --repeats 3 --label $LABEL --out $OUT -``` - -If `LOGICAL` equals `PHYSICAL` (no hyperthreading), also add `$((PHYSICAL * 2))` -to `THREADS` to confirm oversubscribing doesn't help. - -## 5. End-to-end Python fit - -This is what users see: a full `fit` from Python with a single evaluation set, -comparing the default with `num_threads` set to the physical and logical -counts. - -```sh -$PY - <<'PY' -import os, sys, time -from pathlib import Path -sys.path.insert(0, "scripts") -from make_perf_data import load -from forust import GradientBooster - -physical, logical = int(os.environ["PHYSICAL"]), int(os.environ["LOGICAL"]) -d = load(Path("/tmp/forust-perf/w200")) -for depth in (5, 8): - for num_threads in (None, physical, logical): - times = [] - for _ in range(2): - model = GradientBooster(iterations=100, learning_rate=0.1, max_depth=depth, - num_threads=num_threads) - start = time.perf_counter() - model.fit(d["X_train"], d["y_train"], evaluation_data=[(d["X_eval"], d["y_eval"])]) - times.append(time.perf_counter() - start) - print(f"depth={depth} num_threads={num_threads!s:>4}: best of 2 = {min(times):6.1f} s", flush=True) -PY -``` - -`num_threads=None` and `num_threads=$LOGICAL` should take about the same time -(both use every logical CPU), which also confirms the setting is wired up. - -## 6. What to send back - -- The output of step 2. -- `agent/feat/optimize/thread_results.jsonl`, or the tables printed in step 4. -- The lines printed in step 5. - -## How to read the results - -- **Physical cores faster than logical by more than ~5%, consistently:** - supports defaulting `num_threads` to the physical core count. -- **Within ~5% of each other:** keep the current default; document - `num_threads` as an option. -- **Logical faster:** keep the current default. -- Depth 8 and 25k-row cases are the most sensitive; depth 5 at 250k rows the - least. Repeat runs vary by about 5%, so treat smaller differences as noise. diff --git a/py-forust/Cargo.toml b/py-forust/Cargo.toml index db7d03e..109e08d 100644 --- a/py-forust/Cargo.toml +++ b/py-forust/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "py-forust" -version = "0.5.0" +version = "0.7.0" edition = "2021" # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html @@ -10,7 +10,7 @@ crate-type = ["cdylib"] [dependencies] pyo3 = { version = "0.28", features = ["extension-module"] } -forust-ml = { version = "0.5.0", path = "../" } +forust-ml = { version = "0.7.0", path = "../" } numpy = "0.28" ndarray = "0.17" serde_plain = { version = "1.0" } diff --git a/py-forust/docs/release-notes.md b/py-forust/docs/release-notes.md new file mode 100644 index 0000000..5354df1 --- /dev/null +++ b/py-forust/docs/release-notes.md @@ -0,0 +1,178 @@ +# Performance Release Notes + +## Scope + +This page preserves the results and reasoning from the performance work recorded +in October 2026. The original working notes and raw JSONL benchmark output were +kept under `agent/` while the work was in progress. They are now summarized here +so the repository keeps the durable conclusions without carrying a temporary +experiment archive. + +The measurements were taken on an Azure VM with 8 physical cores, 16 vCPUs, +and 62 GB of RAM. Unless noted otherwise, runs used release builds, 8 threads, +synthetic data generated by `scripts/make_perf_data.py` with seed 0, 256 bins, +learning rate 0.1, depth 5, and one evaluation set. Timings vary by roughly +5-10% between runs. + +## Executive Summary + +The main bottleneck was tree construction, not prediction or evaluation. In the +original 1,000-iteration workload with 100,000 rows and 200 columns, tree +building accounted for 93.3% of total time. The optimization work moved the +largest serial sections into parallel execution, reduced repeated allocations +and copies, and made binning parallel. + +The recorded end-to-end results were: + +| Workload | Before | After | Result | +| --- | ---: | ---: | --- | +| 100 iterations, 1M x 200, 8 threads | 52.7 s | 26.1 s | 2.0x faster | +| 1,000 iterations, 100k x 200, default threads | 80.8 s | 37.2 s | 2.2x faster | +| Tree time, 8 threads, 25k-250k rows | baseline | -39% to -81% | substantially lower | +| Binning, 8 threads, 1M x 200 | 24.7 s | 1.8 s | 91-93% less | +| 8-thread speedup over serial | 1.0-3.1x | 4.6-5.2x | much better scaling | + +Every kept change was checked for byte-identical model output. The recorded +Python test suite passed with 112 tests during the main optimization work and +121 tests during the histogram follow-up. + +## Tree-Building Optimizations + +The following changes were measured and kept: + +| Change | Recorded effect | +| --- | --- | +| Parallel split search | 17-38% faster at 8 threads | +| Borrow gradients instead of copying | 3-10% faster | +| Write histograms in place | 15-33% faster at 8 threads | +| Parallel histogram subtraction | Up to 28% faster at 8 threads | +| Parallel binning with one sort per column | Binning 11-14x faster | +| Configurable `num_threads` | Makes thread-count tuning explicit | +| Run parallel work on a pool thread | 4-40% faster at 8 threads | + +The analysis found that histogram accumulation was only part of the problem. +Split search, histogram subtraction, gradient/Hessian gathering, per-column +histogram assembly, and row partitioning remained serial and dominated smaller +or deeper trees. Parallelizing independent feature scans and per-column work +improved scaling while preserving the existing tie-breaking behavior. + +The recommended setting on the benchmark host was 8 threads, one per physical +core. The default 16 threads was 2-18% slower at depth 5 and 25-66% slower at +depth 8. This is workload and machine dependent; applications should measure +on their own hosts before changing thread settings. + +### Histogram follow-up + +Two additional changes were kept: + +- Skip histogram construction for children at `max_depth`, reducing tree time by + 10-35% in the measured cases. +- Remove the redundant `cut_value` field from `Bin`, reducing tree time by up to + 21% on small nodes and deep trees. + +The combined before/after measurements were: + +| Data | Rows | Setting | Before | After | Change | +| --- | ---: | --- | ---: | ---: | ---: | +| w200 | 100k | none | 3.47 s | 2.49 s | -28.1% | +| w200 | 100k | random80 | 3.21 s | 2.30 s | -28.3% | +| w200 | 100k | GOSS | 2.31 s | 1.94 s | -16.0% | +| w200 | 250k | none | 7.91 s | 5.60 s | -29.2% | +| w200 | 250k | random80 | 7.13 s | 5.18 s | -27.4% | +| w200 | 250k | GOSS | 4.42 s | 3.46 s | -21.7% | +| w500 | 100k | none | 7.92 s | 5.76 s | -27.3% | +| w500 | 100k | random80 | 7.20 s | 4.98 s | -30.9% | +| w500 | 100k | GOSS | 4.88 s | 3.57 s | -26.8% | +| w200-1m | 1M | none | 26.74 s | 19.53 s | -27.0% | +| w200-1m | 1M | random80 | 24.10 s | 18.35 s | -23.8% | +| w200-1m | 1M | GOSS | 15.35 s | 12.92 s | -15.8% | + +These are median total fit times for 100 iterations with one evaluation set. +Evaluation logloss was identical before and after for these comparisons. + +## GOSS Sampling + +Forust now supports Gradient-based One-Side Sampling (GOSS), following the +LightGBM approach. GOSS keeps the rows with the largest gradients and samples +from the remaining rows, scaling the retained small-gradient rows to keep split +statistics close to unbiased. With the default `top_rate=0.2` and +`other_rate=0.1`, the first `int(1 / learning_rate)` trees use all rows as a +warm-up. + +Against random 80% row sampling (`random80`), fixed 100-iteration runs showed: + +| Data | random80 | GOSS | Speedup | +| --- | ---: | ---: | ---: | +| w200, 100k | 3.21 s | 2.31 s | 1.39x | +| w200, 250k | 7.13 s | 4.42 s | 1.61x | +| w500, 100k | 7.20 s | 4.88 s | 1.48x | +| w200-1m, 1M | 24.10 s | 15.35 s | 1.57x | + +Across these datasets, fixed-iteration GOSS logloss was within -0.55% to ++0.19% of the random80 baseline. Early stopping was within +0.98% on w200 +100k and -0.11% on w500 100k, but was +1.63% worse on w200 250k. That last +case exceeded the recorded 1% tolerance and is an important caveat rather than +a result to hide. The early-stopping runs usually reached the 1,000-tree cap, +so they were not all fully converged comparisons. + +GOSS-specific implementation measurements on 1M x 200 were: + +| Configuration | Tree | Row selection | Subset copy | Total | +| --- | ---: | ---: | ---: | ---: | +| No sampling | 220.5 ms | - | - | 220.5 ms | +| GOSS, serial selection | 111.1 ms | 17.6 ms | - | 128.7 ms | +| GOSS plus contiguous row subset | 89.2 ms | 15.2 ms | 11.6 ms | 116.0 ms | +| GOSS plus parallel block selection | 85.5 ms | 4.8 ms | 11.1 ms | 101.4 ms | + +The subset tree output was byte-identical to the full-data tree output in the +recorded checks. Parallel and serial GOSS block selection also produced +identical models. + +## Experiments That Were Not Kept + +Several ideas were measured and rejected, and recording those results prevents +repeating the same investigations: + +- Reusing histogram buffers was slower at depth 5: about 6% slower at 250k + rows and 7% slower at 1M rows. +- Reusing the per-column `sums` buffer was 6-30% slower in the tested cases. +- `u8` bins were 7-20% faster in the histogram-fill microbenchmark, but only + apply when all columns fit within 255 bins. The default 256-bin layout needs + `u16`, and the required generic plumbing was not justified by the measured + end-to-end gain. +- `u32` row indices showed no useful microbenchmark gain. +- Building small nodes serially was slower at every tested threshold. +- A faster weighted pair sort gained about 2% but carried a small risk of + changing last-digit cut values. + +The remaining likely bottlenecks are row partitioning after each split, split +evaluation for every bin, histogram accumulation and Rayon scheduling for many +small deep-tree nodes. These are candidates for future profiling, not claims +of completed work. + +## Reproducing the Measurements + +Synthetic datasets can be regenerated with: + +```shell +python scripts/make_perf_data.py --out /tmp/forust-perf/w200 --rows 250000 --eval-rows 62500 --cols 200 +python scripts/make_perf_data.py --out /tmp/forust-perf/w500 --rows 100000 --eval-rows 25000 --cols 500 +python scripts/make_perf_data.py --out /tmp/forust-perf/w200-1m --rows 1000000 --eval-rows 250000 --cols 200 +``` + +The benchmark helpers are kept under `scripts/`. Their generated result files +default to ignored `target/perf/`, while the larger datasets and golden files +use `/tmp/forust-perf/`. The main tools are: + +- `perf_sweep.py` for steady-state tree timing and thread scaling. +- `perf_grid.py` for repeatable before/after grids. +- `perf_compare.py` for markdown comparisons between labels. +- `perf_golden.py` for byte-identical tree checks. +- `perf_goss.py` for GOSS accuracy and speed checks. +- `perf_reference.py` and `perf_xgboost.py` for external reference timings. +- `perf_serial_share.py` for Linux `perf` sampling of serial work. + +These numbers are historical engineering measurements, not a promise of a +fixed speedup for every dataset or machine. Hardware, feature cardinality, +tree depth, sampling, thread count, and evaluation frequency all affect the +result. diff --git a/py-forust/mkdocs.yml b/py-forust/mkdocs.yml index 7b3bbce..55eba7d 100644 --- a/py-forust/mkdocs.yml +++ b/py-forust/mkdocs.yml @@ -1,6 +1,9 @@ site_name: forust repo_name: forust repo_url: https://github.com/jinlow/forust +nav: + - API Reference: index.md + - Performance Release Notes: release-notes.md theme: name: material icon: diff --git a/rs-example.md b/rs-example.md index d8967cc..8edffd5 100644 --- a/rs-example.md +++ b/rs-example.md @@ -3,7 +3,7 @@ To run this example, add the following code to your `Cargo.toml` file. ```toml [dependencies] -forust-ml = "0.5.0" +forust-ml = "0.7.0" polars = "0.28" reqwest = { version = "0.11", features = ["blocking"] } ``` diff --git a/scripts/check_python_parallel.py b/scripts/check_python_parallel.py new file mode 100644 index 0000000..5854faa --- /dev/null +++ b/scripts/check_python_parallel.py @@ -0,0 +1,166 @@ +r"""Measure native forust parallelism from Python. + +Run this with the release-built package, for example from py-forust: + + python ..\scripts\check_python_parallel.py + +CPU percentages are process-wide and are normalized to one logical CPU. For +example, 100% is one busy logical CPU and 800% is eight busy logical CPUs. +""" + +from __future__ import annotations + +import argparse +import os +import threading +import time +from collections.abc import Callable + +import numpy as np +from forust import GradientBooster +from xgboost import XGBClassifier + + +def measure_fit( + fit: Callable[[], object], + interval: float, +) -> tuple[float, float, float, int]: + samples: list[float] = [] + stop = threading.Event() + + def monitor() -> None: + previous_wall = time.perf_counter() + previous_cpu = time.process_time() + while not stop.wait(interval): + current_wall = time.perf_counter() + current_cpu = time.process_time() + wall_delta = current_wall - previous_wall + if wall_delta > 0: + samples.append(100.0 * (current_cpu - previous_cpu) / wall_delta) + previous_wall = current_wall + previous_cpu = current_cpu + + watcher = threading.Thread(target=monitor, daemon=True) + start = time.perf_counter() + watcher.start() + fit() + elapsed = time.perf_counter() - start + stop.set() + watcher.join() + + average_cpu = sum(samples) / len(samples) if samples else 0.0 + peak_cpu = max(samples, default=0.0) + return elapsed, average_cpu, peak_cpu, len(samples) + + +def measure_forust( + X: np.ndarray, + y: np.ndarray, + *, + parallel: bool, + num_threads: int | None, + iterations: int, + interval: float, +) -> tuple[float, float, float, int]: + model = GradientBooster( + objective_type="LogLoss", + iterations=iterations, + max_depth=6, + nbins=64, + parallel=parallel, + num_threads=num_threads, + ) + return measure_fit(lambda: model.fit(X, y), interval) + + +def measure_xgboost( + X: np.ndarray, + y: np.ndarray, + *, + num_threads: int | None, + iterations: int, + interval: float, +) -> tuple[float, float, float, int]: + model = XGBClassifier( + n_estimators=iterations, + max_depth=6, + max_bin=64, + tree_method="hist", + n_jobs=num_threads, + objective="binary:logistic", + eval_metric="logloss", + random_state=0, + verbosity=0, + ) + return measure_fit(lambda: model.fit(X, y), interval) + + +def warm_up_xgboost(X: np.ndarray, y: np.ndarray) -> None: + model = XGBClassifier( + n_estimators=1, + max_depth=6, + max_bin=64, + tree_method="hist", + n_jobs=1, + objective="binary:logistic", + eval_metric="logloss", + random_state=0, + verbosity=0, + ) + model.fit(X[: min(len(X), 1_000)], y[: min(len(y), 1_000)]) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--rows", type=int, default=100_000) + parser.add_argument("--cols", type=int, default=32) + parser.add_argument("--iterations", type=int, default=50) + parser.add_argument("--interval", type=float, default=0.05) + args = parser.parse_args() + + logical_cpus = os.cpu_count() or 1 + rng = np.random.default_rng(0) + X = rng.random((args.rows, args.cols)) + y = (X[:, 0] + X[:, 1] > 1.0).astype(np.float64) + warm_up_xgboost(X, y) + + cases = [ + ("serial", False, None, None), + ("default", True, None, None), + ("rayon-1", True, 1, None), + ("rayon-8", True, 8, None), + (f"rayon-{logical_cpus}", True, logical_cpus, None), + ("xgboost-default", None, None, None), + ("xgboost-1", None, 1, 1), + ("xgboost-8", None, 8, 8), + (f"xgboost-{logical_cpus}", None, logical_cpus, logical_cpus), + ] + print(f"data={args.rows}x{args.cols} iterations={args.iterations}") + print(f"logical_cpus={logical_cpus}") + print("case elapsed_s avg_cpu_% peak_cpu_% samples") + for name, parallel, num_threads, xgb_threads in cases: + if parallel is None: + elapsed, average_cpu, peak_cpu, sample_count = measure_xgboost( + X, + y, + num_threads=xgb_threads, + iterations=args.iterations, + interval=args.interval, + ) + else: + elapsed, average_cpu, peak_cpu, sample_count = measure_forust( + X, + y, + parallel=parallel, + num_threads=num_threads, + iterations=args.iterations, + interval=args.interval, + ) + print( + f"{name:<16} {elapsed:>9.3f} {average_cpu:>10.1f} " + f"{peak_cpu:>10.1f} {sample_count:>8}" + ) + + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/scripts/perf_compare.py b/scripts/perf_compare.py index 14bda50..af28fd3 100644 --- a/scripts/perf_compare.py +++ b/scripts/perf_compare.py @@ -1,7 +1,7 @@ """Print a markdown before/after table of tree time for labels in a results file. Example: - python scripts/perf_compare.py baseline item1 + python scripts/perf_compare.py baseline item1 --results target/perf/results.jsonl """ from __future__ import annotations @@ -10,7 +10,7 @@ import json from pathlib import Path -RESULTS = Path(__file__).resolve().parents[1] / "agent/feat/optimize/results.jsonl" +RESULTS = Path(__file__).resolve().parents[1] / "target/perf/results.jsonl" def key(record: dict) -> tuple: diff --git a/scripts/perf_goss.py b/scripts/perf_goss.py index f07d867..447662e 100644 --- a/scripts/perf_goss.py +++ b/scripts/perf_goss.py @@ -13,10 +13,10 @@ Example: python scripts/perf_goss.py run --data /tmp/forust-perf/w200 --rows 100000 --threads 8 \ - --iterations 100 --out agent/feat/goss/results.jsonl --references + --iterations 100 --out target/perf/goss-results.jsonl --references python scripts/perf_goss.py run --data /tmp/forust-perf/w200 --rows 100000 --threads 8 \ - --iterations 1000 --early-stopping-rounds 20 --out agent/feat/goss/results.jsonl - python scripts/perf_goss.py report --results agent/feat/goss/results.jsonl + --iterations 1000 --early-stopping-rounds 20 --out target/perf/goss-results.jsonl + python scripts/perf_goss.py report --results target/perf/goss-results.jsonl """ from __future__ import annotations diff --git a/scripts/perf_grid.py b/scripts/perf_grid.py index 738b9ee..e7b53d2 100644 --- a/scripts/perf_grid.py +++ b/scripts/perf_grid.py @@ -1,4 +1,4 @@ -"""Run the fixed performance grid from `agent/feat/optimize/plan.md` with a label. +"""Run the fixed performance grid with a label. Example: python scripts/perf_grid.py --label baseline --grids G1 G2 G3 @@ -15,7 +15,7 @@ from perf_sweep import git_commit, run DATA = Path("/tmp/forust-perf") -RESULTS = Path(__file__).resolve().parents[1] / "agent/feat/optimize/results.jsonl" +RESULTS = Path(__file__).resolve().parents[1] / "target/perf/results.jsonl" # name: (dataset, rows, max_depth, iterations, include serial) GRIDS = { @@ -36,6 +36,7 @@ def main() -> None: args = parser.parse_args() commit = git_commit() + args.out.parent.mkdir(parents=True, exist_ok=True) print(f"label={args.label} commit={commit}") print(f"{'grid':>4} {'data':>8} {'rows':>8} {'depth':>5} {'mode':>10} {'tree ms':>9} {'bin s':>7}") with open(args.out, "a") as out: diff --git a/scripts/perf_serial_share.py b/scripts/perf_serial_share.py index 7acb256..efc96c8 100644 --- a/scripts/perf_serial_share.py +++ b/scripts/perf_serial_share.py @@ -23,7 +23,7 @@ sys.path.insert(0, str(Path(__file__).resolve().parent)) from perf_sweep import BINARY, git_commit -RESULTS = Path(__file__).resolve().parents[1] / "agent/feat/optimize/results.jsonl" +RESULTS = Path(__file__).resolve().parents[1] / "target/perf/results.jsonl" PERF_DATA = Path("/tmp/forust-perf/serial_share.data") SAMPLE = re.compile(r"^\s*(\d+)/(\d+)\s+([\d.]+):\s*(?:[0-9a-f]+\s*)?(\S*)") @@ -53,6 +53,7 @@ def main() -> None: parser.add_argument("--delay-ms", type=int, default=3500, help="Skip loading and binning.") parser.add_argument("--out", type=Path, default=RESULTS) args = parser.parse_args() + args.out.parent.mkdir(parents=True, exist_ok=True) command = [ "perf", "record", "-q", "-F", "999", "--delay", str(args.delay_ms), "-o", str(PERF_DATA), "--", diff --git a/scripts/perf_sweep.py b/scripts/perf_sweep.py index 712e9e9..ba7b0dd 100644 --- a/scripts/perf_sweep.py +++ b/scripts/perf_sweep.py @@ -2,7 +2,7 @@ Example: python scripts/perf_sweep.py --data /tmp/forust-perf/w200 --rows 50000 100000 \ - --threads 8 --repeats 3 --label baseline --out agent/feat/optimize/results.jsonl + --threads 8 --repeats 3 --label baseline --out target/perf/results.jsonl """ from __future__ import annotations