2025 年 7 巻 4 号 p. 120-127
BACKGROUND
Routinely collected medical data, such as electronic medical records (EMRs) and medical claims, are necessary for developing disease registries. This study aimed to develop an otologic surgery registry by integrating data from EMRs, medical claims, and otorhinolaryngology department information systems (ORL-DIS) and to assess the agreement of hearing tests between registry-determined evaluations (RDE) and surgeon-determined evaluations (SDE).
METHODS
A stapes surgery registry was developed by linking data from ORL-DIS, EMRs, and medical claims from two hospitals in Japan. SDE were recorded by the surgeons, whereas RDE were automatically assigned by the registry system. This study focused on pre- and postoperative hearing evaluations. Pure-tone averages (PTA) for air conduction (AC) and bone conduction (BC) were calculated. Agreement between SDE and RDE was assessed using Bland–Altman plots, and mean differences and 95% limits of agreement (95% LoA) were calculated. In SDE, cases with incomplete data were excluded.
RESULTS
A total of 164 patients (187 cases) were included. The Bland–Altman analysis revealed a high agreement between preoperative AC-PTA (mean difference: −1.61 dB; 95% LoA: −12.5 to 9.29 dB) and BC-PTA (mean difference: −1.05 dB; 95% LoA: −13.9 to 11.8 dB) measurements by SDE and RDE. Additionally, postoperative improvements showed a moderate agreement. The integration of audiometric data into the registry significantly reduced manual errors.
CONCLUSION
This study successfully established the first otologic surgery registry in Japan that integrates audiometric data from EMRs. This registry provides a valuable resource for analyzing surgical outcomes and a framework for future otologic research.
Routinely collected medical data, including data from electronic medical records (EMRs) and medical claims, play an important role in disease registries1). A disease registry is a systematic and organized collection of information about individuals with specific diseases or conditions. In the past, disease registries were manually populated by healthcare professionals. This process is labor-intensive and limits the number of available data fields. Combining disease registries with routinely collected data can streamline data input processes, thereby facilitating efficiency and enabling the creation of large-scale disease registries2).
Pure-tone audiometry is the most commonly laboratory test for hearing assessment. Audiometry is a standardized procedure worldwide3),4) and is conducted approximately 5.4 million times annually in Japan5). The audiometry results are stored in otorhinolaryngology department information systems (ORL-DIS) as numerical data, such as hearing thresholds. Although ORL-DIS are available for research, the audiometry results in the ORL-DIS are not used for medical database research or disease registries.
In this study, we developed a new otologic surgery registry using EMRs, medical claims, and ORL-DIS. Additionally, this study aimed to confirm the agreement of hearing tests between registry-determined evaluations (RDE) conducted through programming and traditional surgeon-determined evaluations (SDE) performed manually.
An otologic surgery registry was developed by linking data from hospital-based registry, ORL-DIS, EMRs, and medical/Diagnosis Procedure Combination claims. Claio (FINDEX Inc.) and MediRepo (RION Co., Ltd.) are the major ORL-DIS in Japan. In this study, two hospitals were included across different ORL-DIS. Kurashiki Central Hospital uses Claio, and Shizuoka General Hospital uses MediRepo.
This study focused on stapes surgery as the topic of the registry. Stapes surgery is considered the standard treatment for otosclerosis and is one of the major otologic procedures6). Stapes surgery aims to improve hearing, and hearing improvement can be evaluated using ORL-DIS. In Sweden, a national registry has been established for quality control because experience can improve surgical outcomes7). Since Japan does not have such a registry, we chose this topic for this study.
STUDY DESIGN AND PARTICIPANTSThis was a retrospective agreement study of pre- and postoperative hearing examinations for stapes surgery based on SDE and RDE. This study developed a stapes surgery registry that combined the following data: chart review, EMRs, claims, and ORL-DIS. A chart review was conducted for patients who underwent stapes surgery for congenital otosclerosis. Patients who underwent stapes surgery for traumatic diseases were excluded. Kurashiki Central Hospital and Shizuoka General Hospital are otologic surgery training facilities approved by the Japan Otological Society. Otologic surgery training facilities report stapes surgery cases annually to the Japan Otological Society. EMRs and claims data, including diagnosis, drug, and procedure information, were collected and standardized according to a previous report8). These data were anonymized, combined, and sent to JMDC Co., a vendor of the Health, Clinic, and Education Information Evaluation Institute, whose database was used for this registry.
The dates for pre- and postoperative hearing evaluations, which served as the basis for SDE, were predetermined by the surgeons at each facility. For RDE, preoperative hearing evaluations were selected within 3 months (90 days) before surgery, whereas postoperative evaluations were selected between 6 weeks and 2 years (730 days) after surgery. Postoperative hearing can temporarily deteriorate within 6 weeks due to the effects of surgery, and if the interval between pre- and postoperative evaluations is too long, it is difficult to assess improvements accurately. Therefore, this period was set accordingly9). If multiple evaluations were available within the defined periods, those that simultaneously measured AC and BC and those with the smallest AC-PTA threshold were considered. These were then compared with RDE, which were programmatically extracted from the registry by merging stapes surgery cases with the hearing examination database.
HEARING ASSESSMENTPre- and postoperative pure-tone averages (PTA) were calculated for the hearing examinations according to the Committee on Hearing and Equilibrium of the American Academy of Otolaryngology-Head and Neck Surgery guidelines9). The PTA were determined as the mean of the thresholds at 0.5, 1, 2, and 3 kHz. Based on the guidelines, if 3 kHz is not tested, the threshold at 3 kHz should be estimated by averaging the thresholds at 2 and 4 kHz. This study followed this guideline, and 2 and 4 kHz data were used. The hearing examinations assessed air conduction (AC) and bone conduction (BC), and AC-PTA and BC-PTA were calculated accordingly. Postoperative hearing improvement was evaluated by subtracting the preoperative PTA from the postoperative PTA for both AC and BC.
STATISTICAL ANALYSISContinuous variables were presented as mean (standard deviation), whereas categorical variables were presented as number (percentage). Patient characteristics included gender, age at the time of the first surgery, history of previous stapes surgery before the study period, and total number of surgeries. Preoperative hearing evaluations by SDE and RDE were categorized into the following timeframes: “within 30 days before surgery”, “31–90 days before surgery”, “more than 91 days before surgery”, “no hearing examination or no preoperative decision date by the surgeon”, and “unclassifiable”. The postoperative hearing evaluation dates were similarly categorized as “within 6 weeks”, “6 weeks to 1 year”, “1 to 2 years”, “more than 2 years”, “no hearing examination including missing data of SDE”, and “unclassifiable”. Preoperative and postoperative evaluations by SDE were categorized as “both AC and BC”, “AC only”, “BC only”, or “neither” for both preoperative and postoperative evaluations.
The agreement between SDE and RDE was assessed by generating Bland-Altman plots to evaluate differences between the two methods in measuring preoperative AC-PTA and BC-PTA as the primary endpoint10). This method involved plotting the mean PTA from the two methods (calculated as (SDE + RDE)/2) on the x-axis and the difference between the two methods (calculated as SDE–RDE) on the y-axis. The mean difference and 95% limits of agreement (95% LoA: mean difference ±2 SD) were reported in the same units as the measured variable, with a mean difference of 0 indicating perfect alignment between the methods. We defined differences within ±5 dB as indicating high agreement and calculated the proportion of cases11). Additionally, the postoperative hearing improvement was analyzed using the same approach as the secondary outcome. Furthermore, the linear relationship and inter-rater reliability between these measurements were assessed by calculating their correlation coefficients and the intraclass correlation coefficient (ICC) using a two-way random-effects model for absolute agreement with single measurements (ICC[2,1])12).
For the final analysis, cases with missing SDE data for stapes surgery, incorrect entries where no hearing test was performed on SDE, unclassifiable cases, and cases where postoperative SDE was recorded within 6 weeks or more than 2 years after surgery were excluded. Additionally, for revision surgeries on the same side, if <1 year had passed since the previous surgery, data from the previous surgery were excluded. The agreement between SDE and RDE was assessed using data that included both evaluations, and the distribution of missing SDE data was simultaneously evaluated to account for its potential impact. Data cleaning and analysis were performed using RStudio (version 2024.4.2.764).
ETHICAL APPROVALThis study followed the Strengthening the reporting of observational studies in epidemiology statement13), and was approved by the Institutional Ethics Committee of Kurashiki Central Hospital (approval number: KCH 4,297) and conducted in accordance with the tenets of the Declaration of Helsinki. A waiver of informed consent was obtained under an opt-out approach for the target patients at the time of study participation at Kurashiki Central Hospital and Shizuoka General Hospital. Patients were informed that they could opt out of disclosing their information on the hospital’s website.
In total, 187 cases (164 patients) who underwent stapes surgery between March 2014 and December 2022 at Kurashiki Central Hospital and Shizuoka General Hospital were extracted from the claims database. Table 1 shows the characteristics of patients who underwent stapes surgery. Of the 164 patients, 104 (63%) were females. The average age at the first surgery was 39 (24) years. Of the 164 patients, 4 (2.4%) had a history of previous stapes surgery before registration, 142 (87%) had one surgery, 21 (13%) had two surgeries, and 1 (0.6%) had three surgeries.
| n = 164 | |
|---|---|
| Age, years, mean (SD) | 39 (24) |
| Sex, n (%) | |
| Female | 104 (63%) |
| Male | 60 (37%) |
| History of previous stapes surgery before registry, n (%) | |
| Presence | 4 (2.4%) |
| None | 160 (98%) |
| Number of stapes surgeries in the right ear, n (%) | |
| 0 | 62 (38%) |
| 1 | 100 (61%) |
| 2 | 2 (1.2%) |
| Number of stapes surgeries in the left ear, n (%) | |
| 0 | 84 (51%) |
| 1 | 77 (47%) |
| 2 | 3 (1.8%) |
| Total number of stapes surgeries, n (%) | |
| 1 | 142 (87%) |
| 2 | 21 (13%) |
| 3 | 1 (0.6%) |
SD, standard deviation
Table 2 shows details of the pre- and postoperative SDE and RDE. Preoperatively, SDE was recorded in 135 cases (72.2%), whereas RDE was recorded in 186 cases (99.5%). Postoperatively, SDE was recorded in 111 cases (59.4%), whereas RDE was recorded in 184 cases (98.4%). Additionally, 48 SDE cases did not have evaluation dates for both periods. Unclassifiable cases where SDE dates were inconsistent with surgery included four pre- and four postoperative cases. RDE had one preoperative and three postoperative cases without hearing evaluation records.
| SDE | RDE | |
|---|---|---|
| Perioperative evaluation time, n (%) | ||
| Within 30 days | 107 (57%) | 120 (64%) |
| 31–90 days | 13 (7.0%) | 66 (35%) |
| Over 90 days | 15 (8.0%) | 0 (0%) |
| No hearing examinationa | 48 (26%) | 1 (0.5%) |
| Unclassifiable | 4 (2.1%) | 0 (0%) |
| Time of postoperative evaluation, n (%) | ||
| Within 6 weeks | 22 (12%) | 0 (0%) |
| 6 weeks to 1 year | 53 (28%) | 127 (68%) |
| 1–2 years | 58 (31%) | 57 (30%) |
| Over 2 years | 2 (1.1%) | 0 (0%) |
| No hearing examinationa | 48 (26%) | 3 (1.6%) |
| Unclassifiable | 4 (2.1%) | 0 (0%) |
| Perioperative PTA category, n (%) | ||
| Both AC and BC | 84 (45%) | 95 (51%) |
| AC only | 53 (28%) | 91 (49%) |
| BC only | 0 (0%) | 0 (0%) |
| Neither | 50 (27%) | 1 (0.5%) |
| Postoperative PTA category, n (%) | ||
| Both AC and BC | 90 (48%) | 52 (28%) |
| AC only | 46 (25%) | 132 (71%) |
| BC only | 1 (0.5%) | 0 (0%) |
| Neither | 50 (27%) | 3 (1.6%) |
SDE, surgeon-determined evaluation; RDE, registry-determined evaluation; PTA, pure-tone averages; AC, air conduction; BC, bone conduction
a No hearing examination including missing data of SDE.
The preoperative SDE hearing tests included both AC and BC in 84 cases (44.9%) and AC only in 53 cases (28.3%). Regarding RDE hearing tests, both AC and BC were performed in 95 cases (50.8%) and AC only was performed in 91 cases (48.7%). Postoperatively, SDE included both AC and BC in 90 cases (48.1%) and AC only in 46 (24.6%). Regarding RDE, both AC and BC were performed in 52 cases (27.8%), and AC only was performed in 132 cases (70.6%). Seventy cases were excluded from the preoperative analysis, and 19 were excluded from the improvement analysis (Fig. 1).
Fig. 2 shows the preoperative AC-PTA and BC-PTA agreement between SDE and RDE using a Bland-Altman plot. For preoperative AC-PTA, the analysis included 117 cases and showed a correlation of 0.91 and an ICC of 0.95, with a mean difference of −1.68 dB (95% LoA: −13.4 to 10.0 dB; within ±5 dB: 91.5%). For BC-PTA, the analysis included 70 cases and showed a correlation of 0.77 and an ICC of 0.88, with a mean difference of −1.22 dB (95% LoA: −13.4 to 10.0 dB; within ±5 dB: 91.5%).
(a) AC-PTA and (b) BC-PTA. SDE, surgeon-determined evaluation; RDE, registry-determined evaluation; AC, air conduction; BC, bone conduction; PTA, pure-tone averages; LoA, limit of agreement (blue plot, right ear; red plot, left ear).
Fig. 3 shows the Bland–Altman plot for the agreement of postoperative improvements. For AC-PTA, the analysis included 98 cases and showed a correlation of 0.77 and an ICC of 0.86, with a mean difference of 2.46 dB (95% LoA: −11.1 to 16.1 dB; within ±5 dB: 89.8%). For BC-PTA, the analysis included 39 cases and showed a correlation of 0.45 and an ICC of 0.64, with a mean difference of 1.88 dB (95% LoA: −8.72 to 12.5 dB; within ±5 dB: 74.4%).
(a) AC-PTA and (b) BC-PTA. SDE, surgeon-determined evaluation; RDE, registry-determined evaluation; AC, air conduction; BC, bone conduction; PTA, pure-tone averages; LoA, limit of agreement (blue plot, right ear; red plot, left ear).
This study represents a significant milestone as the first registry for stapes surgery in Japan that integrates audiometric data from EMRs. This comprehensive database provides a valuable resource for analyzing pre- and postoperative outcomes using structured and quantifiable data. This study examined the integration of a hearing database using a registry primarily focused on otosclerosis, a condition commonly treated to improve hearing. Regarding the agreement between SDE and RDE, preoperative AC-PTA demonstrated strong agreement between the evaluations.
Traditionally, audiometric results have been used visually as audiograms, limiting their usability for detailed analysis. Our database enables automated postoperative outcome assessment by converting these results into numerical data like EMRs. This automation reduces the manual effort required for data entry and minimizes the potential for human error, thus streamlining the overall process. In this study, many human errors were observed, such as missing diagnosis dates in SDE, incorrect entries, and registrations made within 6 weeks postoperatively, highlighting the usefulness of RDE. As a future development, the use of ORL-DIS, EMRs, claims data, and the hospital database at JMDC will enable the automatic creation of registries by integrating various otological diseases and hearing databases into a comprehensive system, facilitating multicenter studies. Because this study served as a pilot study validated with key ORL-DIS data, expanding this approach to different medical institutions may allow its broader application across larger databases, such as the JMDC database, as used in this study14).
The integration of audiometric data into the registry represents a substantial advancement in otologic research. Age-related hearing loss has been reported to affect approximately one-third of patients aged >65 years10). Thus, this integration is expected to become valuable in a super-aging society like Japan. This approach not only allows for a more precise evaluation of surgical outcomes but also facilitates the identification of trends and patterns that may have been overlooked by traditional methods. In the UK, the National Institute for Health and Care Research Health Informatics Collaborative has been integrating routinely collected hearing health data since 2022 to address clinical needs and improve hearing health. Given that the National Health Service provides 80% of healthcare in the UK, it aims to better understand hearing data and repurpose it for research15). However, database research in the field of otolaryngology in Japan is limited, particularly in integrating audiometric and claims data. This study set a precedent that can be expanded to other diseases, enhancing the scope and depth of otologic research. Future studies should incorporate surgeon and facility performance metrics to improve healthcare quality by identifying best practices and areas for improvement.
There are several limitations to this study. First, this study was conducted in a limited number of facilities, and the timing of evaluations varied between facilities. Thus, generalizability of the findings may be limited, and larger-scale studies are warranted. Second, to investigate the agreement between the two methods, cases with missing data were excluded. These missing data were likely due to human error and were treated as missing completely at random. However, some cases in the SDE dataset had postoperative evaluations scheduled immediately after surgery, which could have introduced bias and negatively affected the agreement if they were included. Third, there are various ways to assess hearing outcomes following stapes surgery, such as measuring the air–bone gap or evaluating overclosure. However, this study focused solely on the agreement of AC-PTA and BC-PTA, which is the most commonly used measures. The reliability of disease-specific hearing evaluation metrics should be assessed in future studies to provide a more comprehensive understanding.
In conclusion, the creation of a registry with integrated audiometric data represents a significant leap forward in the study of stapes surgery outcomes. By overcoming the limitations of traditional data collection methods, this registry can facilitate more accurate and comprehensive evaluations, ultimately leading to better patient care and medical advancements in otolaryngology.
The authors declare that they have no conflict of interest with respect to this research study and paper.
This research received no specific funding.
The authors would like to thank Enago (www.enago.jp) for the English language review.
TF: conceptualization, methodology, investigation, formal analysis, data curation, visualization, writing –original draft, and supervision. KU: conceptualization, methodology, data curation, formal analysis, visualization, investigation, writing –original draft, and supervision. All authors reviewed and approved the final version of the manuscript.
