Learning Prototype-guided Semantic Decision Trees

抄録

Clear reasoning and semantic understanding are essential for making human decision-making interpretable and trustworthy. Our model uses tree structures and semantic prototypes to represent hidden features, enabling comprehensible explanations of its predictions. We propose a method that combines Vision-Language Models (VLMs) with manually crafted text prompts to guide the learning of class-specific semantic prototypes. These prototypes are then integrated into a decision tree, where adjusting the thresholds at each node optimizes decision-making by aligning it with the prototypes. The semantic prototypes provide a level of explainability that standard feature splits lack, as they highlight salient regions in images. Additionally, the model demonstrates adaptive behavior across data classes, allowing it to adjust to variations in data distribution during the testing phase. We demonstrate the effectiveness of our approach in classification tasks, delivering accurate predictions alongside reasoned explanations.