Collective and Evolving AI in Medicine
AG Grabe
Our team understands AI in medicine not as a one-time development software project but as a snapshot in a continuous development life-cycle happening between many interacting partners with interdisciplinary competences. It is important to understand that medicine is different as medical know-how is rather a fluid opinion of how diagnosis and therapy is done best, stored in an ever evolving collective brain of medical experts around the world. Only if a collective of experts agrees that, based on sufficiently large medical studies, a specific diagnostic or therapeutic decision is better, any novel way of decision-making can become reality. On the other hand, AI is under continuous evolutionary pressure by fast technical advances and the availability of more and more data. Therefore, any medical AI should not be regarded as a fixed product, but should be understood rather as an ongoing, iteratively improving development process, performed collectively by medical and technical experts. This way of thinking holds the key for future AI in medicine and we are therefore researching ways to enhance this collective process by developing and validating own medical AI, and develop software supporting this collective evolutionary development process, especially in the areas of cancer screening, -diagnostics, -therapy and digital pathology in general.
Our group at the Institute of Pathology at the UMG is also a member of the Lower-Saxony Center for Causal Artificial Intelligence in Medicine CAIMED and also running the Hamamatsu Tissue Imaging and Analysis (TIGA) Center at the University of Heidelberg in the BIOQUANT Institute Heidelberg. Our group is also part of the Medical Oncology Department at the National Center for Tumor Diseases Heidelberg (NCT). We are also a member of the Goettingen Campus Institute Data Science.
We are offering HIWI contracts for students, experienced in software development. Please contact Prof. Dr. Niels Grabe by e-mail
Research Group Members
Prof. Dr. Niels Grabe is Professor of AI in Medicine and Co-Director of CAIMED. He studied computer science and obtained his PhD in Bioinformatics. After co-founding the aging-focused biotech start-up ageLab at the University Hospital Hamburg-Eppendorf, he joined the Department of Medical Informatics at the University Hospital Heidelberg in 2006. In 2007, he established a BMBF-funded junior research group on tissue systems biology at the BIOQUANT Center and founded the Hamamatsu TIGA Center at the University of Heidelberg. In 2013, he created the Steinbeis Center for Medical Systems Biology (STCMED), where—together with Dr. Bernd Lahrmann and Dr. Nicolas Wentzensen (NCI)—he co-developed Cytoreader for AI-based automatic evaluation of dual-stain cytology.As a pioneer in digital pathology, Prof. Grabe, in collaboration with Hamamatsu Photonics Germany, organized the first series of European workshops in this field. He subsequently moved to the NCT Heidelberg, where he and his colleagues pioneered the use of CD3 and CD8 on whole-slide images for cancer therapy response prediction. Since 2021, Prof. Dr. Grabe has served as Professor of AI in Medicine at the University Medical Center Göttingen (UMG), where he continues to advance the development of AI in digital pathology with his group.
Dr. Bernd Lahrmann is an experienced software developer in digital pathology. He studied Bioinformatics at the University of Applied Sciences Emden, Germany and completed his diploma thesis in 2008 at the Hamamatsu TIGA Center, University of Heidelberg. Since 2009 he worked at the TIGA Center as a research assistant and completed his PhD in 2013 at the National Center for Tumor Diseases Heidelberg under supervision of Prof. Dr. Niels Grabe. Together with Prof. Grabe and Dr. Wentzensen he co-developed Cytoreader for dual-stain cytology and has been working on several projects in the field of digital pathology. He is interested in the practical application of machine learning based algorithms within the field of medical research and diagnostics.
Liam Bartels studied International Relations at the University of Oklahoma where he completed his Master’s degree in 2013. Pursuing his interests in medical research and informatics, he has been working at the Hamamatsu TIGA Center, University of Heidelberg since 2017. His current focus is imaging and quality control.
Felipe Miranda (PhD student) studied Industrial Engineering at the University of Chile in Santiago, Chile, where he also undertook a MSc in Medical Informatics. Since 2019 he has been working at the Hamamatsu TIGA Center, University of Heidelberg, and is currently working on robustness in the application of deep learning for computational histopathology and cytopathology at UMG.
Andreas Keil (PhD student) studied Medical Informatics at the University of Heidelberg, where he also undertook his MSc studies. Since 2021 he is working at the Hamamatsu TIGA Center, University of Heidelberg, and is currently focusing on our decentralized Digital Pathology blockchain CENTAURON as well as decentralized Deep Learning at UMG.
Alexandra Krauthoff completed her technical assistant training at the Institute of Pharmacy and Molecular Biotechnology, University of Heidelberg. Since 2014 she has been working at the Hamamatsu TIGA Center, University of Heidelberg where she is responsible for all laboratory processes, including tissue preparation and staining, Whole Slide Image generation, quality control and image management.
Contact: niels.grabe(at)med.uni-goettingend.de
Tel: +49 171 558 1923
Selected Research Projects
Improving Global Cervical Cancer Screening with AI
Cervical cancer screening is one of the most globally impactful cancer preventive measures and stands out due to its high degree of automation when primary HPV testing is used, which enables efficient and large-scale implementation but is lacking specificity. Triage with p16/Ki-67 dual-stain (DS) testing has shown high sensitivity and specificity for detection of cervical precancers. DS is by far superior to the until now used PAP cytology, dual-stain requires fewer subsequent colposcopies and detects more, and earlier, cervical intraepithelial neoplasia grade 3 or greater. We are studying the application and evolution of our AI development in this field with CYTOREADER (www.cytoreader.com). Specifically we study how dual-stain AI can become a substantial improvement in global cancer screening. To this end, we are performing AI validation studies to advance global efforts to reduce cervical cancer mortality, especially also with applicability in low-resource settings.
CENTAURON Blockchain for Collaborative and Evolving AI
Centauron (www.centauron.net) is a decentralized open source software and blockchain developed in our lab for multi-center, decentralized collaboration, training and validation of AI with a focus on digital pathology. It allows assembling virtual cohorts across institutions for training and validating AI models in a decentralized manner. Centauron exists as a decentralized peer-to-peer network of nodes at different institutions with each node having equal rights to ensure data safety and privacy protection. Each node can contribute data (cases, files, labels) to a cohort. Its intellectual property (IP) is protected by a fine-grained permission concept and the decentralized architecture of the network running the private CENTAURON blockchain which is mirrored on the public Ethereum network. This way contributions to decentralized shared projects on CENTAURON are protected. We are researching how such blockchain networks can be utilized to become the only solution for collective development of iteratively improving AI in medicine.
T-CELL-PREDICT
In 2011 we discovered, together with colleagues, the use of CD3 and CD8 for the prediction of the response of a patient to chemo or immunotherapy in oncology (PCT/EP2011/004710). Since then, it was broadly confirmed by many research groups and products (marketed e.g. for some time under the name “Immunoscore” e.g.) that the exact quantification of several biomarkers that are related to the innate and adaptive immune response (like CD3 and CD8, Granzyme B) holds an important key to predict the treatment response of a cancer patient to chemotherapy or immunotherapy. AI and digital pathology are thereby the essential components as only with whole slide imaging and AI-based image analysis, tumor sections can be profiled systematically for said immune cells, so for each patient a quantitative decision for or against a specific therapy can be taken. Up till now, no digital pathology AI for quantifying CD3 and CD8 exists for therapy prediction. Therefore, based on a license for the according patent, we are currently developing and evaluating our AI-based tool T-CELL-PREDICT for tumor immune cell profiling in Colorectal and Lung Cancer (CRC) based on the CANCERSCOUT cohort and others.
Selected publications
Interobserver reproducibility of cervical histology interpretation with and without p16 immunohistochemistry. Tao AS, Zuna R, Darragh TM, Grabe N, Lahrmann B, Clarke MA, Wentzensen N. Am J Clin Pathol. 2024 Aug 1;162(2):202-209. doi: 10.1093/ajcp/aqae029. PMID: 38527169
CNN stability training improves robustness to scanner and IHC-based image variability for epithelium segmentation in cervical histology. Miranda Ruiz F, Lahrmann B, Bartels L, Krauthoff A, Keil A, Härtel S, Tao AS, Ströbel P, Clarke MA, Wentzensen N, Grabe N. Front Med (Lausanne). 2023 Jul 5;10:1173616. doi: 10.3389/fmed.2023.1173616. eCollection 2023. PMID: 37476610 Free PMC article.
Automated Evaluation of p16/Ki-67 Dual-Stain Cytology as a Biomarker for Detection of Anal Precancer in Men Who Have Sex With Men and Are Living With Human Immunodeficiency Virus. Cohen CM, Wentzensen N, Lahrmann B, Tokugawa D, Poitras N, Bartels L, Krauthoff A, Keil A, Miranda F, Castle PE, Lorey T, Hare B, Darragh TM, Grabe N, Clarke MA. Clin Infect Dis. 2022 Oct 29;75(9):1565-1572. doi: 10.1093/cid/ciac211. PMID: 35325073 Free PMC article.
Ruiz, F. M., Lahrmann, B., Bartels, L., Krauthoff, A., Keil, A., Härtel, S., ... & Grabe, N. (2023). CNN stability training improves robustness to scanner and IHC-based image variability for epithelium segmentation in cervical histology. Frontiers in Medicine, 10.
Wentzensen, N., Lahrmann, B., Clarke, M. A., Kinney, W., Tokugawa, D., Poitras, N., ... & Grabe, N. (2021). Accuracy and efficiency of deep-learning–based automation of dual stain cytology in cervical Cancer screening. JNCI: Journal of the National Cancer Institute, 113(1), 72-79.
Large-scale in-silico identification of a tumor-specific antigen pool for targeted immunotherapy in triple-negative breast cancer. Kaufmann J, Wentzensen N, Brinker TJ, Grabe N. Oncotarget. 2019 Apr 2;10(26):2515-2529. doi: 10.18632/oncotarget.26808. eCollection 2019 Apr 2.
Robust and accurate quantification of biomarkers of immune cells in lung cancer micro-environment using deep convolutional neural networks. Aprupe L, Litjens G, Brinker TJ, van der Laak J, Grabe N. PeerJ. 2019 Apr 10;7:e6335. doi: 10.7717/peerj.6335. eCollection 2019. PMID: 30993030 Free PMC article.
A 3D self-organizing multicellular epidermis model of barrier formation and hydration with realistic cell morphology based on EPISIM. Sütterlin T, Tsingos E, Bensaci J, Stamatas GN, Grabe N. Sci Rep. 2017 Mar 6;7:43472. doi: 10.1038/srep43472.
Exemplary current thesis offer
We do have bachelor and master thesis topics available. Please contact felipeignacio.mirandaruiz(at)med.uni-goettingen.de or andreas.keil(at)med.uni-goettingen.de
Experimental design for the development of a multi-source vision-language Computational Pathology foundation model
Background
The field of Deep Learning (DL) has seen the emergence and rapid progress of vision-language foundation models (VLM). These models capture and align large quantities of image and text representations, and can be used for a wide range of downstream tasks such as multimodal classification, object detection, image segmentation, interactive question answering, etc.
Within Computational Pathology, Whole Slide Images (WSIs) of cells or tissue, together with their associated clinical reports, patient diagnoses, region annotations and/or omics analyses could be used to train these VLMs. With the university’s institute of pathology being fully digitized, there is an enormous opportunity to process and integrate large amounts of Whole Slide Image and patient data for cancer research.
Some potential applications include:
- As a baseline model that can be further fine-tuned for specific clinical use cases, reducing the need for large amounts of annotated/curated data,
- For downstream Whole Slide Image processing,
- For medical report generation and interactive question answering.
At the moment, VLMs in pathology are still being trained through image-data pairs, with small image patches and their respective captions. However, it could be possible to train these models directly with WSIs, and allow the VLM to infer regional representations of the data within the WSIs. Potentially, either a Multiple Instance Learning approach or WSI-level feature extraction approach could be implemented. By directly training using on WSIs, advantage
An interesting question that emerges is: How can we design a training setup to train a VLM directly on WSIs and/or image patches, with multiple different modalities (e.g. region annotations, omics data and WSIs) and on incomplete data (e.g. training samples without omics data/annotations/diagnosis)?
- The institute für pathology is completely digitized, meaning we have access to tens of thousands of WSIs + patient data.
- Computational Pathology is migrating from the direct development of use-case based applications, to the development of foundation models that fuse large amounts of image and text data, which can be used to develop downstream applications.
- Uses of foundation models:
- Fine-tune prediction layer on smaller training datasets (meaning, less annotated data required for training),
- Assist in the process of generating annotations,
- Advantages:
- Scalability
- Robustness (can be trained on large amounts of heterogeneous data)
- Reduce need for highly annotated data
- SOTA:
- Solutions: CONCH, BiomedCLIP, PLIP
- Limitations: trained on image patches instead of WSIs, only image + text, single image captions per image (corroborate)
Requirements
- Essential
- Python programming
- Experience with tensorflow or pytorch
- Desirable:
- Experience with Docker