Oblivious piecewise-linear decision trees

Abstract

Gradient boosting ensembles of decision trees are a very popular type of machine learning model, with several popular implementations. Some of those implementations utilize symmetric decision trees - decision trees of a specific structure that improve regularization and speed predictions. Over the last years, several research works have been published related to piecewise-linear decision trees and their utilization in gradient boosting ensembles. In this paper, symmetric piecewise-linear decision trees were introduced, and it was shown that it ispossible to efficiently train such trees in gradient boosting ensembles. It was shown that such symmetric piecewise-lineardecision trees have significantly faster prediction time compared to regular decision trees and piecewise-linear decision trees ofsimilar depths and that ensembles of symmetric piecewise-linear decision trees achieve competitive quality on open datasets.