Tree Neural Tangent Kernel for Arbitrary Binary Trees

Abstract

We present a formulation for the Neural Tangent Kernel (NTK) induced by soft tree ensembles, applicable to arbitrary tree architectures. Our analysis reveals that, when the number of leaves at each depth is fixed, the training behavior of infinite tree ensembles in function space remains identical across different tree architectures, even those that are not isomorphic.