USCAP 111th Annual Meeting
Los Angeles, California
March 19 – 24, 2022
Poster VI – Wednesday PM – Informatics
Background:
Quantitative immunohistochemistry (IHC) is an integral part of histopathology reporting, providing vital information such as tumour characterisation and prognosis. Ki-67 proliferation index has traditionally been manually estimated on light microscopy over several fields of view, a laborious task given to interobserver variation. AI can potentially overcome this. We chose to evaluate the performance of an AI assistance tool for grading of sarcomas.
Design:
The system design makes use of traditional computer vision algorithms which, unlike machine learning methods, do not require training. The algorithm identifies nuclei in a given field of view and categorizes them as positive or negative based on the intensity of the IHC staining. In this study, the algorithm was evaluated by 3 pathologists independently on 440 regions of interest (ROIs) from 88 Ki-67 stained sarcoma slides. To compare the output of the algorithm, each pathologist was first asked to score every ROI as per their traditional workflow. Next, the pathologist was shown the outputs of the AI (both cell segmentation and final Ki-67 score) and asked to score each ROI again. The discordance between the pathologist’s initial impression and the final score after viewing the AI outputs was calculated.
Results:
The metrics calculated show that, on average, the pathologists agree with 93.5% of the ROIs scored by the AI (number of ROIs accepted / total number of ROIs in the study). In the cases where the pathologist disagreed with the outputs of the AI, a majority of the differences lie between 0 and 10 percentage points (Figure 1). To quantify inter-pathologist differences, we compared their Ki67 scores before and after they were assisted by the AI outputs. We observed that the root mean squared error (RMSE) between the pathologists themselves was 14.08% on average before they viewed the AI outputs. In contrast, the average RMSE between the pathologists after AI assistance was 2.52 (Table 1). These results show that after AI assistance, the pathologists generally provide more accurate estimates of Ki-67 and discordance was reduced by 82.10%.
Conclusion:
We show that AI, correctly used in quantitative immunohistochemistry, can increase the accuracy of the output and reduced the discordance between the pathologists by 82.10%. This adds objective specificity to the scoring.
