The total cases of randomized controlled trials (RCT) from the 1950s in the area of medicine and healthcare have exceeded 600,000. Interest in RCT has been increasing since the mid-1990s with the global movement of evidence-based medicine (EBM). Several problems were identified in the EBM movement, leading to the development of new solutions. It was found that these problems were discovered at “use” and “cascading evidence” stages rather than the “generate” stage of the evidence usage whose categories were introduced in the previous reports. This report introduces CONSORT statements for the improvement in quality of RCT reports and their utilization in many aspects at the “stream down” stage. Feedback from narrative based information and the informative best case as one of the current movements of post-RCT at the “use” stage are also discussed. Furthermore, the latest trends toward the need for continuous evaluation of public investment, and the patient's and the citizen's roles in EBM development are discussed.

View full abstract

The world of evidence-based policies and practices has not always been widely understood in the field of education. However, this has appeared to change in recent times with a considerable surge in the number of studies that discuss the educational policies of various countries across the world. In this paper, firstly, issues related to production, mediation and the use of evidence will be clarified, mainly based on the rigorous definitions provided by the Cochrane and Campbell collaborations, which are known to gather data through randomized control trials (RCTs), undertake systematic reviews, and conduct exhaustive meta-analyses. Secondly, the fact that OECD has had to frequently make policy recommendations using the world of evidence-based policies,-that is, making political decisions with the help of scientific data,-will be discussed in juxtaposition with the human capital theory. Finally directions for evidence-based policies will be suggested.

View full abstract

Innovative approaches for research and practices of Evidence-Based Practices (EBP) that have gained the worldwide attention in the field of mental health have been greatly developed, and the approaches seem to throw a new light on methodology of program evaluation. In this article, we reviewed current research trends and results of EBP service research, focusing on critical components of effective EBP program models associated with favorable program outcomes, development of fidelity scales for the effective models, and EBP toolkit approaches that are new measures for technology transfers in the field of human services. We finally discussed possible contribution of EBP research experiences to the methodology of program evaluation and issues to develop the perspectives.

View full abstract

Even when a program is shown to have evidence that it is effective, using the evidence, or implementing the program while retaining or confirming/renewing its effectiveness is not easy. This article proposes two views of the process of using evidence. One view is of the technological transfer, the process of retaining the effectiveness of the original program. The view of the technological transfer suggests that the implementation of the original program while retaining its effectiveness cannot be achieved by knowledge provision, but must include a change in intra- or inter-organizational environment to be more collaborative. The other view is of the replication, the process of confirming/renewing the effectiveness of the original program. The view of replication suggests that core components of a program can be defined as predictors of the outcome, or components which can enhance external validity of the program in different contexts, such as local additions and factors promoting applicability of the program. The next future step for the promotion of using evidence is to establish a purveyor (an entity responsible for the development of organizations, staff, and leaders, technical assistance and the third-party evaluation as well as the provision of information) in various fields, equipped with the view of the process of using evidence for both technological transfer and replication.

View full abstract

Rigorous impact evaluation in the field of international development aid has been increasingly implemented since around 2000. In line with the growing importance of rigorous impact evaluation, a number of RCT is conducted. Leading development economists are discussing the pros and cons of implementing RCT, which has been a controversial issue in evaluation studies for a long time. In addition, criticism on “measurement without theory” in the field of economics in the past seems to have been revived. However, compared with other areas such as epidemiology, RCT in development economics needs to be standardized and institutionalized. From Japan's standpoint rigorous impact evaluation including the implementation of RCT is considered.

View full abstract

“Evidence-based Development Aid Evaluation” is a hot topic in the aid evaluation field and this movement has been led by Poverty Action Lab (J-PAL) since it's establishment. By reviewing the history, three origins can be identified about this movement. The first is the evaluation study which root is deeply embedded the Campbell and Stanley's proposal; the second one is the aid evaluation which has been uniquely developed due to the unique characteristics of the field; and the third one is relatively newcomer which is development economics field. After discussing each origin, the thoughts of Scriven, program director of the Evaluation Center, Western Michigan University (-2008), and Banerjee, director of J-PAL, are examined about advantages and constrains of randomized experimental design. One conclusion is: there are some rooms for employing RCT even though it cannot be dominant. It should be considered to put some more resources on the aid activities which effectiveness is verified by this approach.

View full abstract

Participatory evaluation, which is conducted by project stakeholders, holds advantages for effective assessment to improve programs. On the other hand, ensuring objectivity and the appropriate representation of stakeholders are challenges. In the participatory evaluation process, the evaluator plays the role of facilitator, not a position of assessor, as is the case of conventional evaluation.
The author facilitated the participatory evaluation of a process oriented educational project, which is implemented by a Japanese NGO, “Salt Payatas Foundation Philippines”. During the evaluation process, stakeholder empowerment and raising awareness for improvement were observed. However, insufficient evaluation skills and the tendency of subjective judgments were also encountered. Therefore, the facilitator made necessary measures corresponding to the situation.
This case study verifies; the effectiveness of participatory evaluation for process oriented projects, and also identifies three important roles of the facilitator in order to take advantage of participatory evaluation and make up for its weakness.

View full abstract

Balanced Scorecard (BSC) is well utilized by local governments in the U.S.A. However, there are a few good examples in Japan as well. The financial perspective, which takes an approach similar to a company, is in relation to BSC. From the financial perspective, indicators express achievements and results for stakeholder satisfaction in financial terms. Measuring achievements monetarily is difficult, especially for local governments. Furthermore, the aim of local governments differs from the aim of for-profit companies, which the financial perspective advocates. Customer satisfaction, which is not a financial indicator, must also be taken into consideration by local governments. Therefore, it is necessary to integrate the financial perspective with the customer perspective.

View full abstract