Eur Respir J. 2023 Apr 20:2201720. doi: 10.1183/13993003.01720-2022. Online ahead of print.
ABSTRACT
RATIONALE: Few studies have investigated the collaborative potential between artificial intelligence (AI) and pulmonologists for diagnosing pulmonary disease. We hypothesized that the collaboration between pulmonologist and AI with explanations (explainable AI, XAI) is superior in diagnostic interpretation of pulmonary function tests (PFTs) than a pulmonologist without support.
MATERIALS AND METHODS: The study was conducted in two phases, a mono-centre (P1) and a multi-centre intervention study (P2). Each phase utilized two different sets of 24 PFT reports of patients with a clinically validated gold-standard diagnosis. Each PFT was interpreted without (control) and with XAI’s suggestions (intervention). Pulmonologists provided a differential diagnosis consisting of a preferential diagnosis and optionally up to three additional diagnoses. Primary endpoint compared accuracy of preferential and additional diagnoses between control and intervention. Secondary endpoints were number of diagnoses in differential diagnosis, diagnostic confidence and inter-rater agreement. We also analysed how XAI influenced pulmonologists’ decisions.
RESULTS: In P1 (N=16 pulmonologists), mean preferential and differential diagnostic accuracy significantly increased by 10.4% and 9.4%, respectively between control and intervention (p<0.001). Improvements were somewhat lower but highly significant (p<0.0001) in P2 (5.4% and 8.7% respectively, N=62 pulmonologists). In both phases, the number of diagnoses in differential diagnosis did not reduce, but diagnostic confidence and inter-rater agreement significantly increased during intervention. Pulmonologists updated their decisions with XAI’s feedback and consistently improved their baseline performance if AI provided correct predictions.
CONCLUSION: A collaboration between pulmonologist and XAI is better at interpreting PFTs than individual pulmonologists reading without XAI support or XAI alone.
PMID:37080566 | DOI:10.1183/13993003.01720-2022
