Gradient boosting, written twice then halved

Austin Wagner

I implemented gradient boosting from scratch in numpy, ported the compute to a Rust core, and then deleted the numpy version, 6,910 lines of source and 7,677 of tests, because keeping two implementations means maintaining two things that can disagree.

What it is
The algorithm behind most machine learning on spreadsheet-shaped data. Nearly everyone uses an existing implementation; this is one written from the histogram up.
Result
A Rust core doing histogram building, tree construction and prediction, bridged with PyO3 0.27.2, behind a strictly-typed Python façade.
The deletion
Suite went 555 tests to 182 at 100% coverage when the reference implementation was retired.
Stack
Rust, PyO3, numpy, mypy under strict settings.

The sequence is the point, not the Rust

Histogram binning, split-gain computation and the leaf-value update are where the algorithm actually lives, and they are exactly the parts a library hides. Writing them once is the part that teaches you the algorithm.

Writing them twice is what made the port safe. The numpy version was an executable specification the Rust core had to agree with: an oracle to port against, which most from-scratch implementations never have, because they have nothing to be wrong relative to.

Then the oracle was deleted

Once the Rust core matched and was fully covered, the numpy implementation became a liability rather than an asset: two implementations that can drift, and a tempting fallback path that would hide a broken build.

So it went. The native library is now a hard dependency: if it is absent the package raises at import time rather than quietly running a slower, different answer. What remains in Python is a typed façade that pins every native callable to a protocol, so the type checker sees real signatures instead of an opaque object. The façade's own statement of the rule:

Strict typing only: no ``Any``, no ``cast``, no ``type: ignore``. If
``cleargbm_rs`` is not installed, this module raises ``ImportError`` at import
time -- there is no Python fallback.

libs/cleargbm/src/cleargbm/_rust.py, module docstring.

The port is the achievement. The reference implementation was scaffolding, and scaffolding comes down.

It competes in someone else's harness

It is not a standalone repository with a benchmark notebook. It is two of the 12 model backends in a risk-prediction service, a classifier and a regressor, sitting behind the same interface as XGBoost and LightGBM and selectable per request, so it is evaluated on the same datasets, through the same training and explainability machinery, as the production libraries I wrote it to understand.

The companion evidence is the opposite shape: nine merged patches into LightGBM itself. One shows the algorithm is understood; the other shows a large unfamiliar codebase can be worked in under review.

Record

Every figure above is bound to a named section of the wiki page cleargbm and checked mechanically before this page can deploy. That page pins the retirement to the commit that performed it and separates it from a later packaging change four weeks afterwards, because the two are routinely collapsed into one story and only the first one removed compute.

Known limits. "Pure Rust" is true of the compute path and of the crate's contents, not of the shipped package: a typed Python API surface remains, and it has grown by about half since mid-August. No benchmark is claimed here: being selectable alongside XGBoost and LightGBM in one harness is a statement about structure, not about winning. Until 2026-09-14 this page said "roughly 10k lines" were deleted, a figure that matched neither the source nor the source-plus-tests count.

Sources