2023 Volume 11 Issue 2 Pages 30-37
Hepatocellular carcinoma (HCC), a prevalent liver cancer, poses a substantial mortality risk. Surgical resection is the primary treatment choice, but post-resection recurrence challenges patient outcomes, especially in early recurrenced cases. Developing preoperative early recurrence prediction methods is crucial for personalized treatment plans, improving survival time for HCC patiens. Existing clinical data-based predictions overlook imaging modalities, while radiomics-based methods suffer from limitations imposed by predefined features. In this light, we propose ModalityFormer, a deep learning model leveraging multimodality MRI/CT for HCC early recurrence prediction task. ModalityFormer utilize tranformer architecture to capture inter-modality context and utilize adaptive fusion module to effectively combine prediction logits of all MRI/CT modalities. Consequently, promising results are achieved in our prediction task. Furthermore, we introduce a fusion model called ModalityFormer++, which integrates multimodality MRI/CT with clinical data. Through detailed experiments, we demonstrate that our ModalityFormer model outperforms other state-of-the-art methods. Additionally, ModalityFormer++ exhibits superior performance compared to models relying solely on multimodality MRI/CT or clinical data.
