Jiayang Ren
Research

Optimal and Deep Decision Trees

Contents
  1. Overview
  2. From the Papers
  3. Code
  4. Publications

Training decision trees close to optimality on up to ten million samples.

Overview

I work on algorithms that train decision trees close to optimality: a scalable deterministic global optimization algorithm for optimal decision trees, GPU-accelerated moving-horizon methods that train deep classification trees on up to ten million samples, and a differentiable “ReLU+Argmin” reformulation of decision trees for gradient-based training.

Moving-horizon training of deep decision trees
Moving-horizon training: each step optimizes a small subtree, then moves down the tree.

From the Papers

Moving-horizon refinement of a depth-4 decision tree
Moving-horizon refinement of a depth-4 tree: subtrees of depth 4, 3 and 2 are re-optimized in turn (TMLR 2026, Figure 2).
An oblique decision tree with linear branching conditions
An oblique decision tree: each branch node tests a linear condition aᵀx ≤ b, and a sample follows one path to a leaf (NeurIPS 2025).

Code

The moving-horizon method is available as the open-source Julia package MHDETrees.jl (MIT License).

Publications

TMLR 2026

A Moving-Horizon Approximate Branch-and-Reduce Method for Deep Classification Trees

Zou, C., Ren, J., Mao, Q., Liu, J., Lai, M., and Cao, Y.

Transactions on Machine Learning Research (2026).

J2C CertificationPDFOpenReview
ESWA 2026

A Moving-Horizon Differential Evolution Algorithm for Training Deep Classification Trees on Large Datasets

Ren, J., Osuna-Enciso, V., and Cao, Y.

Expert Systems with Applications, 333(B): 133846 (2026).

DOICode
NeurIPS 2025

Differentiable Decision Tree via “ReLU+Argmin” Reformulation

Mao, Q., Ren, J., Wang, Y., Zou, C., Zheng, J., and Cao, Y.

Advances in Neural Information Processing Systems, 38: 166874–166909 (2025).

Spotlight (top 3%)PDFOpenReview
NeurIPS 2022

A Scalable Deterministic Global Optimization Algorithm for Training Optimal Decision Tree

Hua, K., Ren, J., and Cao, Y.

Advances in Neural Information Processing Systems, 35: 8347–8359 (2022).

Best Theoretical Paper, INFORMS Data Mining WorkshopPDFProceedings
Software

MHDETrees.jl: Moving-Horizon Differential Evolution for Optimal Classification and Regression Trees

Ren, J., Osuna-Enciso, V., and Cao, Y.

Open-source Julia package, MIT License (2026).

GitHub