2026 年 8 巻 4 号 p. 127-134
BACKGROUND
This study retrospectively evaluated the user experience and safety outcomes of medical chatbot service providing medicine use information to pregnant and lactating women.
METHODS
The chatbot utilized in this study did not employ generative artificial intelligence (such as large language models). Rather, it was a rule-based system in which all response patterns were predefined. We collected user survey data between December 13, 2021, and August 10, 2022. User status, satisfaction, addressed consultation categories, and post-use health problems were analyzed.
RESULTS
The study included 1,000 records, comprising 419 (41.9%) cases of pregnant women and 581 (58.1%) cases of lactating women across Japan. The overall satisfaction was 93.9%, with no significant difference between pregnant and lactating women (P = .866). Headache was the most frequently consulted category followed by cold symptoms and hay fever symptoms. High satisfaction levels were observed across the consultation categories. None of the participants indicated post-use health problems.
CONCLUSION
A medical chatbot service providing medicine use information to women during pregnancy and lactation can achieve high satisfaction and safety.
Medication use is a major concern for pregnant and lactating women. Most women take one or more types of medications before and during pregnancy1). Studies have reported concerns about medication use among pregnant and lactating women owing to the knowledge that not all medicines are safe for their babies2–4). Although these women face such concerns daily, their opportunities to communicate with healthcare professionals are limited because of the limited availability of services and the difficulty in making appointments. In Japan, for example, contact with obstetricians usually ceases after the one-month postpartum checkup. In addition, the development of the Internet and social media has increased the exposure to not only extensive medical information but also substantial misinformation5),6). Hence, reliable sources of evidence-based information on the use of medications during pregnancy and lactation should be made available and readily accessible at any time.
A possible solution could be the use of chatbots as a tool to support pregnant and lactating women at any time and connect them to medical services when needed. A chatbot, or a conversational agent, is a software program that simulates human conversations and allows online communication with users7). Equipped with an interactive and intuitive interface, a chatbot is an easy-to-use and readily available tool that allows users to access prepared sets of information anytime7). Chatbots are increasingly being deployed in healthcare fields to reduce the burden of healthcare practitioners, improve accessibility to evidence-based medical knowledge, and support medical decision-making processes for healthcare professionals and patients8),9). By improving access to information, chatbots could be an effective approach to supporting patients’ decisions regarding treatment and medication10–12). Although challenges in installation (e.g., users’ mixed responses, privacy concerns, usability, and acceptability) remain13–18), chatbots are expected to play a key role in future healthcare structures19–21).
However, despite increasing attention toward chatbots and the potential of these programs in healthcare practices, chatbots are not widely deployed in obstetrics and gynecology practices. Only a few studies have introduced the use of chatbots in obstetrics and gynecology, such as those used to provide psychological support to parents22),23), nutrition education24), breastfeeding education23), and disease-specific information support25). However, based on our review, no studies so far have reported the use of health chatbots among women during pregnancy and lactation for concerns regarding medication use.
“Kusuribo” is a simple database-based chatbot service that provides evidence-based information on medication use for pregnant and lactating women in Japan. Kusuribo was accessible and fully functional on both smartphones and personal computers. The chatbot uses a rule-based system and does not employ generative artificial intelligence (such as large language models). The service was developed and has been provided by Kids Public Inc., a Japanese healthcare company since December 202126). As of August 2022, Kids Public Inc. has contracted with 10 organizations, including local governments, private companies, and medical institutions, and it also allows unlimited usage of the chatbot service among individual users without charging any fee. This chatbot only provides evidence-based recommendations and does not offer medical services, such as screening, referral, diagnosis, and prescription. Users are instructed to answer questions about their health conditions and select the consultation category. Pregnant women are asked about the weeks of pregnancy, as well as pre-existing conditions and complications, while lactating women are asked about the number of postpartum months, breastfeeding methods, and pre-existing conditions and complications. Users can then immediately view response statements, which include safe medications, self-care tips for their symptoms, and symptoms that require immediate medical attention. The chatbot stores approximately 2,000 response patterns that are grouped into 15 categories of concern: fever, headache, abdominal pain, nausea/vomiting, diarrhea/soft stools, constipation, stomach discomfort, hay fever symptoms, itchy skin, joint pain, uterine cramps, common cold symptoms, anesthetic use in dental care, antibiotic use in dental treatment, and sleeping difficulty. All responses provided by the chatbot are prewritten and reviewed by multiple board-certified obstetricians based on drug inserts, academic papers, and the Drugs and Lactation Database27).
In the present study, we aimed to evaluate the user experience of Kusuribo, a medical chatbot service that provides medication use information for pregnant and lactating women. To this end, we analyzed Kusuribo’s user data and investigated the user demographics, frequently addressed consultation categories, post-use satisfaction levels, and post-use health problems. As available reports on the evaluation of a medical chatbot designed to provide medication use consultation for pregnant and lactating women are few, our findings will have important implications for further development and improvement of chatbot services in the healthcare field.
We conducted a retrospective cross-sectional evaluation of Kusuribo’s user data. Kusuribo collects and stores user information and inquiries, including pregnancy status (number of weeks of pregnancy, number of postpartum months, and breastfeeding method), health conditions, and consultation categories. Each record corresponds to a unique inquiry, indicating that repeated use by the same user could not be identified. We measured self-reported user satisfaction using a voluntary online questionnaire. The questionnaire was placed below the chatbot’s recommendation statements and inquired about users’ satisfaction level using a four-level Likert scale: “satisfied,” “somewhat satisfied,” “somewhat unsatisfied,” and “not satisfied” (Fig. 1). To collect the data related to post-use health issues, Kids Public Inc. also collaborated with a local government and conducted an additional survey of local users using Google Forms. An e-mail containing the survey link was distributed to the users living in the municipality one to three weeks after their use of Kusuribo. The following question was asked: “Did you have any health problems after using the service?”
Our evaluation comprised three steps. First, the users’ status was presented as descriptive statistics. Second, the average level of satisfaction (the percentage of users who answered “satisfied” or “somewhat satisfied” in the questionnaire) was calculated for overall service use and for each consultation category. Third, the proportion of post-use problems was evaluated using the responses to the question, “Did you have any health problems after using this service?” To assess user status and satisfaction, we set the categorical variables as numbers and percentages, which were compared by conducting χ2 or Fisher’s exact test, as appropriate. Stata Version 16.0 software (StataCorp LP, College Station, TX, USA) was used for the statistical analyses. A two-tailed test was conducted, and P < .05 was used to determine statistical significance.
ETHICAL STATEMENTThis study used the data through the service. This study was approved by the Institutional Review Board of the University of Tokyo for joint research between Kids Public Inc and the University of Tokyo (number 2020043NI). No personal information was included in the data, and consent for research use of the data was obtained from all users. Additionally, Author 4 and Author 5 were involved in the data analysis and evaluation to ensure objective evaluation.
Table 1 presents the summary statistics for service use. A total of 1,000 records were identified between December 13, 2021, and August 10, 2022. Pregnant and lactating users accounted for 419 (41.9%) and 581 (58.1%) cases, respectively. Among the pregnant women, 36.3% were under 12 weeks of gestation, while 8.4% were at 37 weeks of gestation or later. Among the lactating women, 35.5% were lactating at 1–3 months postpartum, 7.9% were lactating at less than 1 month postpartum, and 26.5% were lactating at 12 months or more postpartum. In addition, 53.5% of the respondents were exclusively breastfeeding, while 5.9% were on exclusive formula feeding. The most frequently consulted categories were headache, cold symptoms, and hay fever symptoms. Nausea/vomiting and stomach discomfort were reported more than twice as frequently during pregnancy than during lactation, whereas abdominal pain, joint pain, and antibiotic use for dental treatment occurred more than twice as frequently during lactation than during pregnancy. Of all users, 11.8% reported having pre-existing medical or chronic complications, with bronchial asthma being the most prevalent (35.6%).
| All use (n = 1000) |
Use during pregnancy (n = 419) |
Use during lactation (n = 581) |
||
|---|---|---|---|---|
| Gestational age (weeks), n (%) | ||||
| <12 | — | 152 (36.3) | — | |
| 12–21 | — | 111 (26.5) | — | |
| 22–27 | — | 48 (11.5) | — | |
| 28–36 | — | 73 (17.4) | — | |
| 37–40 | — | 30 (7.2) | — | |
| >40 | — | 5 (1.2) | — | |
| Postpartum months (months), n (%) | ||||
| <1 | — | — | 46 (7.9) | |
| 1–3 | — | — | 190 (32.7) | |
| 4–6 | — | — | 109 (18.8) | |
| 7–11 | — | — | 128 (22.0) | |
| >11 | — | — | 108 (18.6) | |
| Method of breastfeeding, n (%) | ||||
| Exclusive breastfeeding | — | — | 311 (53.5) | |
| Mixed feeding | — | — | 236 (40.6) | |
| Total artificial feeding | — | — | 34 (5.9) | |
| Categories of concern, n (%) | ||||
| Fever | 86 (8.6) | 28 (6.7) | 58 (10.0) | |
| Headache | 218 (21.8) | 82 (19.6) | 136 (23.4) | |
| Abdominal pain | 39 (3.9) | 10 (2.4) | 29 (5.0) | |
| Nausea/vomiting | 29 (2.9) | 24 (5.7) | 5 (0.9) | |
| Diarrhea/soft stools | 28 (2.8) | 12 (2.9) | 16 (2.8) | |
| Constipation | 75 (7.5) | 36 (8.6) | 39 (6.7) | |
| Stomach discomfort | 58 (5.8) | 35 (8.4) | 23 (4.0) | |
| Hay fever symptoms | 101 (10.1) | 43 (10.3) | 58 (10.0) | |
| Itchy skin | 75 (7.5) | 35 (8.4) | 40 (6.9) | |
| Joint pain | 55 (5.5) | 17 (4.1) | 38 (6.5) | |
| Uterine cramps | 20 (1.0) | 10 (2.4) | 0 (0.0) | |
| Common cold symptoms | 126 (12.7) | 50 (11.9) | 76 (13.8) | |
| Anesthetic use in dental care | 20 (2.0) | 7 (1.7) | 13 (2.2) | |
| Antibiotic use in dental treatment | 25 (2.5) | 6 (1.4) | 19 (3.3) | |
| Sleeping difficulty | 55 (5.5) | 24 (5.7) | 31 (5.3) | |
| Pre-existing conditions and complications, n (%) | ||||
| Hypertension or hypertensive disorder of pregnancy | 15 (1.5) | 7 (1.7) | 8 (1.4) | |
| Diabetes mellitus or gestational diabetes mellitus | 18 (1.8) | 9 (2.2) | 9 (1.6) | |
| Impaired liver function | 9 (0.9) | 6 (1.4) | 3 (0.5) | |
| Impaired kidney function | 5 (0.5) | 3 (0.7) | 2 (0.3) | |
| Bronchial asthma | 43 (4.3) | 24 (5.7) | 19 (3.3) | |
| Thyroid disorders | 35 (3.5) | 14 (3.3) | 21 (3.6) | |
Table 2 presents the satisfaction survey results. The survey was sent to 1,000 users, and the response rate was 32.7%. The overall satisfaction rate was 93.9%; specifically, 71.9% of users reported being “satisfied,” while 22.0% reported being “somewhat satisfied.” No significant difference in satisfaction was found between pregnant and lactating users (P = .866). Table 3 presents the level of satisfaction for each of the 15 consultation categories. All categories showed at least 84% satisfaction, and seven categories (46.7%) showed 100% satisfaction.
| Total (n = 327) | Use during pregnancy (n = 125) | Use during lactation (n = 202) | P value | ||
|---|---|---|---|---|---|
| Binary scale, n (%) | .866 | ||||
| Satisfied | 307 (93.9) | 117 (93.6) | 190 (94.1) | ||
| Unsatisfied | 20 (6.1) | 8 (6.4) | 12 (5.9) | ||
| Four-point scale, n (%) | .645 | ||||
| Satisfied | 235 (71.9) | 85 (68.0) | 150 (74.3) | ||
| Somewhat satisfied | 72 (22.0) | 32 (25.6) | 40 (19.8) | ||
| Somewhat unsatisfied | 7 (2.1) | 3 (2.4) | 4 (2.0) | ||
| Unsatisfied | 13 (4.0) | 5 (4.0) | 8 (4.0) | ||
| Satisfied | Unsatisfied | P value | ||
|---|---|---|---|---|
| Categories of concerns, n (%) | .173 | |||
| Fever | 18 (100.0) | 0 (0.0) | ||
| Headache | 81 (96.4) | 3 (3.6) | ||
| Abdominal pain | 9 (90.0) | 1 (10.0) | ||
| Nausea/vomiting | 11 (100.0) | 0 (0.0) | ||
| Diarrhea/soft stools | 12 (100.0) | 0 (0.0) | ||
| Constipation | 19 (86.4) | 3 (13.6) | ||
| Stomach discomfort | 18 (90.0) | 2 (10.0) | ||
| Hay fever symptoms | 32 (100.0) | 0 (0.0) | ||
| Itchy skin | 32 (97.0) | 1 (3.0) | ||
| Joint pain | 11 (84.6) | 2 (15.4) | ||
| Uterine cramps | 1 (100.0) | 0 (0.0) | ||
| Common cold symptoms | 26 (86.7) | 4 (13.3) | ||
| Anesthetic use in dental care | 7 (100.0) | 0 (0.0) | ||
| Antibiotic use in dental treatment | 8 (100.0) | 0 (0.0) | ||
| Sleeping difficulty | 22 (84.6) | 4 (15.4) | ||
An additional questionnaire regarding the presence or absence of post-use health problems was sent to 70 users, 28 responses were received (40% response rate). None of the users indicated experiencing any health problems after using the service.
This study evaluated the user experience and safety of Kusuribo, one of the first medical chatbot services to provide medication use information for pregnant and lactating women in Japan. It retrospectively analyzed 1,000 cases of chatbot use records and evaluated the user status, consultation categories, satisfaction levels, and post-use health problems. The largest user group among the pregnant women comprised those in the early stage of pregnancy (<12 weeks), and the largest user group among the lactating women comprised those who were 1–3 months postpartum. The most frequently addressed consultation category was headache, followed by cold and hay fever symptoms. The overall satisfaction rate was 93.9%, and no significant difference in satisfaction was found between the pregnant and lactating users. The satisfaction rate was 84% or higher across all consultation categories, with 7 of the 15 categories having a 100% satisfaction rate. An additional question on post-use health problems indicated no such problems among the 28 users who responded to it.
Considering the importance of identifying the topics of interest among users and understanding their service use experience, our findings identified service use patterns among pregnant and lactating women24). Pregnant women were concerned about medication use during early pregnancy, corresponding to the period of fetal organogenesis. Approximately 40% of the lactating women used the service up to 3 months postpartum. However, more than one in six lactating women continued using the service beyond 12 months postpartum, suggesting the demand over a long period after childbirth. The trends of frequently addressed consulting categories also differed between pregnant and lactating women, but all 15 categories were queried. Although we could not find any previous studies on similar medical chatbots, we assumed that health problems commonly queried in daily medical care are also frequently queried in chatbots. Our finding suggests the significance of developing a service that meets a wide range of needs among women.
The findings also demonstrated high satisfaction levels among users. Regarding post-use satisfaction, 93.9% of the users reported high satisfaction levels with the chatbot’s recommendations, and no significant difference was found between the pregnant and lactating women. Previous studies on medical chatbots for cancer patients reported satisfaction rates of 85%–97%28–30), which appears to be comparable to that of Kusuribo. In this regard, future research should quantitatively and qualitatively explore categories with lower satisfaction levels to improve the user experience. Some users might report lower satisfaction levels owing to the unavailability of any safe medication for their specific conditions at a particular stage during pregnancy or lactation. Additionally, past studies used a variety of measurements, including usefulness, ease of use, and usability scores22). Further satisfaction evaluations with such measurements should be conducted to continue improving user experience.
The evaluation of safety outcomes is important for chatbot services8),9). Given the small data size and limited time period, our findings on the safety of the chatbot service for maternal medication use are preliminary. With increasing medical applications of chatbot services, the risks of providing misinformation and overtreatment must be carefully addressed.9
STRENGTHS AND LIMITATIONSThis study focused on a new medical chatbot service that provides information on medication use and analyzed its retrospective data to show the differences in the service use characteristics between pregnant women and lactating mothers. It had some limitations, which could be attributable to the data. First, data were not available for items not included in the surveys (e.g., consultation categories, complications, and pre-existing conditions). Expanding the categories in the future would improve user experience and allow for a more detailed analysis. Second, besides the user inquiries and consultation categories, participants were asked to answer only one question per item (satisfaction and post-health problems). These questions were not prepared for this study, so their readability and validity could not be confirmed. The selection of outcome measures also had limitations. The outcomes of interest were selected based on previous chatbot service evaluation studies, no conceptual or theoretical framework was referred for this study. In addition, although satisfaction is a useful outcome, it does not fully reflect the multidimensional nature of implementation. In future evaluations, we recommend considering additional implementation outcomes such as acceptability, appropriateness, and feasibility31). Although recent reviews reported that only a few studies of chatbots for maternal health settings have employed such validated multi-dimensional user experience measures22),32–34). These constructs would allow for a more comprehensive understanding of how users perceive the relevance, clarity, and practical value of the service. Third, the assessment of post-use health problems relies on a small sample (n = 28), which limits the robustness of safety conclusions. Future research should consider larger-scale or prospective follow-up studies.
This study retrospectively analyzed 1,000 cases of medical chatbot service use and evaluated the user status, satisfaction, and post-use health problems among pregnant and lactating women. These findings suggest that a medical chatbot providing medicine use information during pregnancy and lactation may help achieve high satisfaction and support safe use, although evidence on safety remains limited due to low follow-up rates.
D.S., R.T., and A.O. were employed by Kids Public Inc., the developer and provider of Kusuribo. The other authors report no conflicts of interest.
None.
We would like to thank all obstetricians and midwives affiliated with Kids Public, Inc., as well as all study participants for their contributions to the study.
K.S. interpreted the data and was responsible for drafting and revising the manuscript. D.S. contributed to the study conceptualization and design, acquired and analyzed the data, interpreted the findings, and reviewed the manuscript. R.T. supported data acquisition. N.M. contributed to the study conceptualization, design, and interpretation of the data. H.Y. also contributed to the study conceptualization, design, and interpretation of the data.
Hideo Yasunaga is one of the Editorial Board members of Annals of Clinical Epidemiology (ACE). This author was not involved in the peer-review or decision-making process for this paper.
