An international comparison of the involvement of epidemiology in the most frequently cited publications in the field of clinical medicine.

The objectivity, validity and credibility of research in clinical medicine can be enhanced by the appropriate involvement of epidemiology. However, the overall contribution of epidemiology to clinical research, either as a methodology or as a resource for research, has been poorly quantified. We therefore assessed the involvement of epidemiology in influential publications in the field of clinical medicine, and made an international comparison on a quantitative basis. The 500 most frequently cited papers published during 1981-96 in the field of clinical medicine in the US, the UK, and Japan were compared in terms of epidemiological involvement using predetermined criteria. The three criteria were based on the indexing of relevant MeSH keywords, publication types, or the departmental affiliations of the authors. For all three criteria, the proportion of clinical papers with epidemiological involvement was the highest in the US, followed by the UK, whereas it was the lowest in Japan. The difference was almost four-fold between the US and Japan. There was also an increasing trend of epidemiological involvement in publications of clinical medicine over the years, which was more apparent in the US than in either the UK or Japan. These findings may reflect inter-country differences in resources as well as in the stance towards evidence-based health sciences.

The objectivity, validity and credibility of research in clinical medicine can be enhanced by the appropriate involvement of epidemilogy. However, the overall contribution of epidemiology to clinical research, either as a methodology or as a resource for research , has been poorly quantified. We therefore assessed the involvement of epidemilogy in influential publications in the field of clinical medicine, and made an international comparison on a quantitative basis . The 500 most frequently cited papers published during 1981-96 in the field of clinical medicine in the US, the UK, and Japan were compared in terms of epidemilogical involvement using predetermined criteria. The three criteria were based on the indexing of relevant MeSH keywords , publication types, or the departmental affiliations of the authors. For all three criteria, the proportion of clinical papers with epidemiological involvement was the highest in the US, followed by the UK, whereas it was the lowest in Japan. The difference was almost four-fold between the US and Japan. There was also an increasing trend of epidemiological involvement in

INTRODUCTION
Epidemiology has its roots in the investigation of epidemics of infectious diseases in communities. Subsequently, epidemiology has been applied to the study of non-infectious chronic diseases. An example of a simple definition of epidemiology reads as follows: Epidemiology is the branch of medicine concerned with the determination of cause and distribution of diseases in specific populations and its application in the prevention of diseases. Over the years, the definition of epidemiology has been broadened to encompass all phenomena related to health, including, for example, clinical trials.
The objectivity, validity and credibility of research in clinical medicine can be enhanced by consideration and inclusion of epidemiologic principles and procedures. Different "types" of epidemiological study designs have been used for medical research in many subspecialties of medicine. For example. controlled clinical trials are used in the evaluation of drug efficacy, whereas longitudinal cohort studies are used to assess the carcinogen of industrial chemicals. However, the overall contribution of epidemiology to clinical research, either as a methodology or as a resource for scientific research has been poorly quantified. The objective of the present investigation is therefore to assess the role of epidemiology in influential papers (judged by the frequency of citation) in the field of clinical medicine, and to make an international comparison (between the US, UK and Japan) on a quantitative basis. 3. Affiliation: involved if any of the following terms was included in the affiliation of any author: "epidemiology," "statistics ," ..biostatistics (e.g., biostatistical center)," "biometrics," "biometry," "public health (e.g., department of public health or school of public health)," "prevention (e.g., bureau of prevention or cancer prevention div: sion)," "biomathematics," or "social medicine." Only the 500 most frequently cited papers for each county were analyzed for comparison. In ISI'S original database, th total number of citations received from 1981 through 1997 was counted along with the number of expected citations. Th expected citation is a baseline number that is unique to the typ of publication (original article, review, editorial, etc.) and alsi unique to the year in which the publication appeared in a par ticular journal 4. For example, two original articles publishes in the same journal published in the same year would have identical expected citation. However, an editorial published in the same journal and in the same year would have a differen expected citation figure.
For the purpose of a time-trend analysis, the entire publics tion period (1981-96) was divided into five consecutive inter vals (1981-83, 1984-86, 1987-89, 1990-92, and 1993-96). I should be noted that the number of publications in earlier inter vals tended to be higher than that in recent intervals because of the longer period available to receive citations.
The proportion of the most frequently cited papers with epidemiological involvement according to each of the three criteria was compared in pairs among the three countries using the chi-square test (US vs. UK, US vs. Japan, UK vs. Japan). The time-trend analysis of proportion of publications with epidemiological involvement by the content criterion for each country was based on the Cochran-Armitage test . A p-value less than, 0.05 was considered statistically significant , 0.05<p<O.1 marginally significant, and p>O.1 non-significant. The original database was converted into an Excel file and all analyses were performed using SAS (version 6.12) on a personal computer.

RESULTS
The average number of citations received per paper by country is presented in the Table. The number of actual citations per paper was the highest for the US (672.7), the lowest for Japan (156.3), and in-between for the UK (304.1). Some of the most frequently cited US papers received more than 4 ,000 citations during 1981-97. A similar international pattern was also observed in terms of the number of expected citations per paper.
Informed judgment was used to assess the sensitivity and specificity of screening papers for the involvement of epidemiology using Medline information. The criterion based on "publication type" was judged to be 100% sensitive and specific . For the criterion based on "content," selection by the aforementioned MeSH keywords produced no false-positives (i .e., specificity =100%), but missed several papers with epidemiological involvement (i.e., sensitivity < 100%) based on our judgment of their content by reviewing the papers themselves. In addition to the 198 papers (96 US , 76 UK, 26 Japan) origi-nally identified as having epidemiological involvement using MeSH keywords, a further 19 papers (10 US, 6 UK, 3 Japan) were included based on our review of the contents of these papers. Thus, the sensitivity of the criterion based on MeSH keywords was 198/(198+19) or 91%.
In Figure 1, the proportion of publications with epidemiological involvement in the most frequently cited clinical papers was compared among the three countries according to each of the three criteria. When the criterion based on content was applied, the proportion of papers with epidemiological involvement was 21.2% (n =106) for the US, 16.4% (n =82) for the UK, and 5.8% (n =29) for Japan. The differences between Japan and the other two countries were statistically significant, while that between the US and the UK was only marginally significant. Based on the criterion of publication type, the respective proportions were 14.0% (n =70) for the US, 13.2% (n =66) for the UK, and 2.8% (n =14) for Japan. Only the differences between Japan and the other two countries reached statistical significance. For the criterion based on affiliation, the respective proportions were 19.0% (n =95) for the US, 14.0% (n =70) for the UK, and 4.6% (n =23) for Japan: all the differences being statistically significant.  Hence, for all three criteria, the proportion of clinical papers with epidemiological involvement was the highest in the US, followed by the UK, and the lowest in Japan. The difference in proportion between the US and the UK was relatively small, whereas those between Japan and the other two countries were much larger in magnitude and consistently significant (p<0.001).

DISCUSSION
Whether or how the content of a paper is related to a specific discipline such as epidemiology, depends on the definition of the discipline, and in practice, the criteria applied. As the present analyses depended on the use of predefined indices available in the Medline database, the validity and reliability of the indexing procedure need to be discussed. The fact that the indexed information has been designed for the purpose of retrieving papers from the database justified, in part, the use of this method 3) Furthermore, different criteria for content were applied objectively to further explore this issue. The narrower criterion for content, which was equivalent to the criterion applied in the current analysis except for exercising human judgment, produced 19.2% (n =96) of US papers, 15.2% (n =76) of UK papers, and 5.2% (n =26) of Japanese papers categorized as having epidemiological involvement, with marginal statistical significant difference between the US and the UK (p =0.094) and statistically significant differences between Japan and the other two countries (p<0.001, respectively). Subsequently, the following descriptors were added to the narrower criterion: "mortality," "follow-up," "prevention and control," "prospective," "longitudinal," "retrospective," "risk factor ," and "statistical." The application of this broader criterion produced 34.4% (n =172) of US papers, 32.2% (n =161) of UK papers, and 15.0% (n =75) of Japanese papers to be categorized as having epidemiological involvement: with no statistically significant difference between the US and the UK, but with statistically significant differences between Japan and the other two countries. Although these new terms are perhaps more general, their inclusion produced no difference in the conclusions reached. These results are similar to the results obtained in the current analyses and thus inferences were unchanged.
It is unlikely that potential bias due to indexing procedures had operated differentially in the three countries. Hence, the role of epidemiology in influential papers dealing with clinical medicine, assessed for its involvement as a research method and/or as resource for scientific research, assumed importance to a variable degree in the three countries. This may reflect a citations. If the frequency of citation is indicative of the quality of the paper (i.e., objectivity, validity or credibility), then one interpretation is that the involvement of epidemiology enhances the quality of the paper. In the US, the involvement of epidemiology and/or biostatistics has become a standard requirement in many major proposals for funding in clinical research, although the extent of reference to epidemiology depends on the nature of the study, and varies from investigation to investigation. Based on the international comparison presented in the present paper, the requirement for epidemiologic considerations appears to have been rightfully justified.
In conclusion, the most frequently cited papers in clinical medicine from the US, the UK, and Japan were compared in terms of epidemiological involvement. When pre-determined criteria were applied based on the indexing of relevant MeSH keywords and publication type, in addition to the identification of relevant terms describing affiliation of authors, the involvement of epidemiology in clinical medicine was most apparent in the US, followed by the UK, but very weak in Japan. There was also an increasing trend in the proportion of clinical papers with epidemiological involvement in recent years in all three countries, but the trend was much stronger in the US than in either the UK or Japan.