
Multimodal, Generative, and Agentic AI for Pathology
Clark S361
Abstract: Advances in digital pathology and artificial intelligence have presented the potential to build models for objective diagnosis, prognosis and therapeutic-response and resistance prediction. In this talk we will discuss our work on: (1) Data-efficient methods for weakly-supervised whole slide classification with examples in cancer diagnosis and subtyping (Nature BME, 2021), identifying origins for cancers of unknown primary (Nature, 2021) and allograft rejection (Nature Medicine, 2022) (2) Discovering integrative histology-genomic prognostic markers via interpretable multimodal deep learning (Cancer Cell, 2022; IEEE TMI, 2020; ICCV, 2021; CVPR, 2024; ICML, 2024). (3) Building unimodal and multimodal foundation models for pathology, contrasting with language and genomics (Nature Medicine, 2024a, Nature Medicine 2024b, CVPR 2024). (4) Developing a universal multimodal generative co-pilot and chatbot for pathology (Nature, 2024). (5) 3D Computational Pathology (Cell, 2024) (6) Bias and fairness in computational pathology algorithms (Nature Medicine, 2024; Nature BME 2023) (7) Agentic AI workflows for diagnostic pathology and biomedical research.
Reading list:
- Lu, M.Y., Chen, B., Wi liamson, D.F., Chen, R.J., Zhao, M., Chow, A.K., Ikemura, K., Kim, A., Pouli, D., Patel, A. and Soliman, A., Mahmood, F. 2024. A multimodal generative AI copilot for human pathology. Nature, 634(8033), pp.466-473. https://www.nature.com/articles/s41586-024-07618-3
- Chen, R.J., Ding, T., Lu, M.Y., Wiliamson, D.F., Jaume, G., Song, A.H., Chen, B., Zhang, A., Shao, D., Shaban, M. and Wiliams, M., Mahmood, F. 2024. Towards a general-purpose foundation model for computational pathology. Nature Medicine, 30(3), pp.850-862. https://www.nature.com/articles/s41591-024-02857-3
- Song, A.H., Wi liams, M., Wi liamson, D.F., Chow, S.S., Jaume, G., Gao, G., Zhang, A., Chen, B., Baras, A.S., Serafin, R. and Coling, R., Mahmood, F. 2024. Analysis of 3D pathology samples using weakly supervised AI. Cell, 187(10), pp.2502-2520. https://www.cel.com/cel/abstract/S0092-8674(24)00351-9