Communication features of ChatGPT responses and care-seeking intention for syphilis in Japan: a controlled dialogue experiment

Abstract

Objective: Syphilis is a major re-emerging infectious disease, and pre-consultation support may be particularly important in rural and remote areas where access to confidential care is limited and stigma can delay help-seeking. We examined how communication features in ChatGPT-generated responses influence perceived consultation satisfaction and intention to seek professional care in a simulated syphilis-related anxiety scenario in Japan.

Patient/Materials and Methods: We conducted a controlled dialogue experiment in Japanese using standardized interactions between a simulated patient concerned about syphilis and ChatGPT (GPT-4o mini). Across 48 three-turn sessions, artificial intelligence (AI) responses varied by requested communication stance and personal information disclosure. Supportive communication features, stance alignment, personalization consistency, readability (jReadability), and information volume (character count) were coded. Three health professional evaluators rated consultation satisfaction using the Consultation and Relational Empathy (CARE) Measure and rated care-seeking intention using a Readiness Ruler.

Results: CARE scores showed moderate inter-rater reliability, whereas intention ratings showed low agreement. Two AI responses contained medical misinformation. In hierarchical regression analyses, longer responses and higher readability were associated with higher satisfaction, and among supportive communication behaviors, praise expressions independently associated with satisfaction. Satisfaction was strongly associated with care-seeking intention (β=0.80, P<0.001).

Conclusion: In Japanese-language simulated syphilis consultations, information volume, readability, and praise expressions were associated with higher perceived consultation satisfaction, which in turn was associated with greater intention to seek professional care. These findings suggest that conversational AI may serve as a low-threshold, anonymous pre-consultation support tool to encourage timely help-seeking, particularly in rural and remote settings where confidential access to sexual health services is limited. Low inter-rater agreement for intention ratings and the presence of misinformation underscore the need for robust outcome measurement and safety safeguards.