A Geometric Multimodal Foundation Model Integrating Bp-MRI and Clinical Reports in Prostate Cancer Classification
Research paper by Juan A. Olmos and collaborators. arXiv record dated 2026-01-30. Research area: Computer vision.
Resource information
- Authors
- Juan A. Olmos, Antoine Manzanera, Fabio Martínez
- Research area
- Computer vision
- Application domain
- Healthcare
- AI methods
- Deep learning
- Research topics
- cs.CV, cs.AI
- arXiv record date
- 2026-01-30
- Publication
- 2026 IEEE 23rd International Symposium on Biomedical Imaging (ISBI)
- DOI
- 10.1109/ISBI61048.2026.11515916
- Upstream last modified
- Jul 29, 2026
- License
- CC BY 4.0
- Last record update
- Nov 4, 2026
Abstract
Prostate cancer (PCa) is one of the most common cancers in men worldwide. Bi-parametric MRI (bp-MRI) and clinical variables are crucial for PCa identification and improving treatment decisions. However, this process is subjective to expert interpretations. Furthermore, most existing computer-aided diagnosis methods focus on imaging-based models, overlooking the clinical context and suffering from data scarcity, limiting their ability to learn robust representations. We propose a geometric multimodal Foundation Model (FM), named MFM-Geom, that learns representations from bp-MRI and clinical reports, encoding visual findings and information from the context of clinical variables. In the representations classification head, the approach leverages symmetric positive definite (SPD) matrices and Riemannian deep learning to integrate imaging-text representations from a biomedical multimodal FM. Using 10% of the training data, MFM-Geom outperformed baseline class token embedding-based classification (+8.3%, AUC-PR of 90.67). Generalization on external dataset confirmed the robustness of fine-tuning biomedical FM, achieving an AUC-PR of 90.6.