Accuracy of online symptom checkers for diagnosis of orofacial pain and oral medicine disease

Abstract

Purpose: The aim of this study was to compare and contrast the diagnostic accuracy of multiple online symptom checkers when used for the diagnosis of orofacial pain and oral medicine related disease vignettes. The comparison condition used in this study was the diagnostic accuracy achieved by advanced specialty residents on these same vignettes using a virtual patient system.
Methods: 27 individual disease vignettes were utilized. These vignettes had a variety of orofacial pain and oral medicine diseases. Post graduate orofacial pain and oral medicine residents at our University of Southern California interacted with their randomly assigned virtual patients were analyzed [n=574]. Virtual patient accuracy was based on whether the user selected the primary diagnosis as one of their top four choices after interviewing. Eleven English-language symptom checkers accuracy was based on whether the vignettes produced the primary diagnosis as one of their top four choices. Using these data, symptom checker and virtual patient accuracy rates were calculated.
Results: The primary diagnosis on virtual patient encounters was found within the top four choices a mean of 67.2% of the time. The primary diagnosis for the same vignettes entered into the 11 symptom checkers was found within the top four choices a mean of 5.9% of the time.
Conclusions: The accuracy of currently available symptom checkers that patient might use for self-diagnosis of common orofacial pain and oral medicine diseases was low, this result suggest that the improved diagnostic algorithms are needed.