Background Immune checkpoint inhibitors (ICIs) have revolutionized cancer therapy but can cause serious immune-related adverse events (irAEs), with pneumonitis (ICI-P) being among the most severe. Early identification of high-risk patients before ICI initiation is critical to close monitoring, enable timely intervention, and optimize outcomes.

Purpose To develop and validate a deep learning foundation model to predict ICI-P from baseline CT scans in patients with lung cancer.

Methods We designed the Checkpoint-Inhibitor Pneumonitis Hazard EstimatoR (CIPHER), a deep learning-powered foundation model combining contrastive learning with a transformer-based masked autoencoder to predict ICI-P from baseline CT scans in lung cancer patients. Using self-supervised learning, CIPHER was pre-trained on 590,284 CT slices from 2,500 non-small cell lung cancer (NSCLC) patients, to understand heterogeneous lung parenchyma. Following pre-training, the model was fine-tuned on an internal NSCLC cohort for ICI-P risk prediction, with images from 254 patients used for model development and from 93 patients for internal validation. We compared CIPHER with classical radiomic models. We also validated CIPHER on an external NSCLC cohort of 116 patients.

Results In our internal immunotherapy cohort, CIPHER consistently distinguished patients at elevated risk of ICI-P from those without the event, with AUCs ranging from 0.77 to 0.85. In head-to-head benchmarking, CIPHER achieved an AUC of 0.83, outperforming radiomic model. In the external validation cohort, CIPHER maintained high performance (AUC=0.83; balanced accuracy=81.7%), exceeding the radiomic models (Delong p=0.0318) and demonstrating superior specificity without sacrificing sensitivity. By contrast, radiomic model, despite high sensitivity (85.0%), showed markedly lower specificity (45.8%). Confusion matrix analyses confirmed CIPHER’s robust classification, correctly identifying 80 of 96 non–ICI-P cases and 16 of 20 ICI-P cases.

Conclusions We developed and externally validated CIPHER for predicting future risk of developing ICI-P from pre-treatment CT scans. With prospective validation, CIPHER can be incorporated into routine patient management to improve outcomes.

Highlights

The first chest CT AI foundation model for immune toxicity – We introduce CIPHER (Checkpoint-Inhibitor Pneumonitis Hazard EstimatoR), a transformer-based masked autoencoder trained through self-supervised contrastive learning on 590,284 CT slices from 4,242 NSCLC patients’ scans. This large-scale pretraining enables CIPHER to learn intrinsic lung parenchymal representations linked to immune toxicity risk.

Early risk prediction prior to therapy initiation – CIPHER predicts the likelihood of ICI-induced pneumonitis directly from baseline CT scans, offering the first non-invasive foundation model for early risk assessment before ICI.

Robust validation and benchmarking – We fine-tuned and evaluated CIPHER across independent internal and external NSCLC immunotherapy cohorts, achieving AUCs of 0.77–

0.85 internal cross validation and 0.83 external testing, surpassing conventional radiomic models in both performance and generalizability.

Interpretability and clinical readiness – We demonstrate how model-derived attention maps align with clinically relevant pulmonary patterns, enhancing interpretability and enabling seamless integration into radiology workflows.

Translational potential – CIPHER’s performance and scalability underscore its potential as decision-support tool to guide treatment planning, pre-emptive monitoring, and toxicity mitigation in immunotherapy practice.

T.C. reports speaker fees and honoraria, including travel and meeting expenses, from ASCOPost, AstraZeneca, Bio Ascend, Bristol Myers Squibb, Clinical Care Options, IDEOlogy Health, Medical Educator Consortium, Medscape, OncLive, PeerView, Physicians' Education Resource, and Targeted Oncology; advisory and consulting fees, including travel and meeting expenses, from AstraZeneca, Bristol Myers Squibb, Daiichi Sankyo, Genentech, Johnson & Johnson, Merck, Nuvalent, oNKo-innate, Pfizer, and RAPT Therapeutics; and institutional research funding from AstraZeneca, Bristol Myers Squibb, and Merck. N.I.V. receives consulting fees from Sanofi, Regeneron, Oncocyte, and Eli Lilly, and research funding from Mirati, outside the submitted work. D.L.G. has served on scientific advisory committees for Menarini Ricerche, 4D Pharma, Onconova, and Eli Lilly, and has received research support from Takeda, Astellas, NGM Biopharmaceuticals, Boehringer Ingelheim, and AstraZeneca. J.V.H. reports scientific advisory roles for AstraZeneca, Boehringer Ingelheim, Genentech, GlaxoSmithKline, Eli Lilly, Novartis, Spectrum, EMD Serono, Sanofi, Takeda, Mirati Therapeutics, BMS, and Janssen; research support from AstraZeneca, Takeda, Boehringer Ingelheim, and Spectrum; and licensing fees from Spectrum. J.Z. reports research funding from Helius, Johnson & Johnson, Merck, Novartis, and Summit, and personal fees from AstraZeneca, Catalyst, GenePlus, Helius, Hengrui, Innovent, Johnson & Johnson, Novartis, Oncohost, Takeda, and Varian, outside the submitted work. J.W. reports research funding from Siemens Healthcare. All other authors declare no competing interests.

This work was supported by generous philanthropic contributions to The University of Texas MD Anderson Lung Moon Shot Program, the MD Anderson Cancer Center Support Grant P30CA016672, and the Tumor Measurement Initiative through the MD Anderson Strategic Initiative Development Program (STRIDE). This research was partially funded by National Institutes of Health grants R01CA262425 (T.C. and J.W.), R01CA276178 (N.I.V. and J.W.), and CPRIT RP240117 (J.W.). This work was also supported by generous philanthropic contributions from Mrs. Andrea Mugnaini and Dr. Edward L. C. Smith, as well as the Rexanna's Foundation for Fighting Lung Cancer, QIAC Partnership in Research (QPR) funding, and Permanent Health Funds. The funding sources had no role in study design, data collection, analysis, interpretation, or manuscript preparation.

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The Institutional Review Board of The University of Texas MD Anderson Cancer Center gave ethical approval for this work for the MD Anderson cohort (protocol 2021-0123) and waived the requirement for informed consent because of the retrospective nature of the study. The Institutional Review Board of Johns Hopkins gave ethical approval for this work for the JHU cohort (protocol 00186276) and waived the requirement for informed consent because of the retrospective nature of the study.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes