Can AI-assisted microscope facilitate breast HER2 interpretation? A multi-institutional ring study

The level of human epidermal growth factor receptor-2 (HER2) protein and gene expression in breast cancer is an essential factor in judging the prognosis of breast cancer patients. Several investigations have shown high intraobserver and interobserver variability in the evaluation of HER2 staining by visual examination. In this study, we aim to propose an artificial intelligence (AI)–assisted microscope to improve the HER2 assessment accuracy and reliability. Our AI-assisted microscope was equipped with a conventional microscope with a cell-level classification-based HER2 scoring algorithm and an augmented reality module to enable pathologists to obtain AI results in real time. We organized a three-round ring study of 50 infiltrating duct carcinoma not otherwise specified (NOS) cases without neoadjuvant treatment, and recruited 33 pathologists from 6 hospitals. In the first ring study (RS1), the pathologists read 50 HER2 whole-slide images (WSIs) through an online system. After a 2-week washout period, they read the HER2 slides using a conventional microscope in RS2. After another 2-week washout period, the pathologists used our AI microscope for assisted interpretation in RS3. The consistency and accuracy of HER2 assessment by the AI-assisted microscope were significantly improved (p < 0.001) over those obtained using a conventional microscope and online WSI. Specifically, our AI-assisted microscope improved the precision of immunohistochemistry (IHC) 3 + and 2 + scoring while ensuring the recall of fluorescent in situ hybridization (FISH)–positive results in IHC 2 +. Also, the average acceptance rate of AI for all pathologists was 0.90, demonstrating that the pathologists agreed with most AI scoring results