Although there is a social demand for scientific evidence of public investment, experience in areas other than healthcare are quite rare in Japan. However, in the United Kingdom and the United States of America, several activities of evidence-based practice in decision making for public policy are in existence, and these evidences are steadily increasing in number. These examples are: the Campbell Collaboration, What Works Clearinghouse of US Department of Education, and the Poverty Action Laboratory in the area of official development assistance. Taking into account such a background, an overview of the genealogy of the Evidence-based Medicine (EBM) and the Cochrane Collaboration as the precedent experience is reviewed in this paper. In addition, the role and the future direction of EBM with regard to the importance of accumulating quality information as a social infrastructure and social epistemology are discussed.

View full abstract

Scientific evidence produced by RCTs (randomized controlled trials) and quasi-experiments and SRs (systematic reviews) has been accumulated in criminal justice, especially as regards delinquency prevention and treatment. We now face the challenge of how to utilize the evidence so that it can be put into practice. In this article, the author describes two approaches to evidence utilization so far developed in the U. S. and discusses the relative applicability of these approaches to a Japanese context. The first approach is a model program approach which encourages practitioners to discard ineffective and harmful programs and to adopt effective model programs based on the results of RCTs and/or SRs while the second approach is a model principle approach which encourages practitioners to assess and improve their programs in terms of effective program principles derived from SRs. The second approach is preferred as a method of utilizing evidence in Japan firstly because it does not require the robustness of programs across contexts and secondly because it allows us to improve Japanese criminal justice practices without replacing them.

View full abstract

The No Child Left Behind Act, which was enacted in the United States in 2002, includes challenging provisions that require states and local schools to adopt programs and practices based on scientifically-based researches. The act focuses especially on studies using experimental designs. In order to provide the education community reliable information on “what works, ” Department of Education opened a web site called What Works Clearinghouse (WWC). WWC review teams conduct systematic reviews of studies and the results are disseminated as WWC evidence reports. WWC is now under development, and is likely to influence U.S. education programs by its contents.
The idea of evidence-based education is suggestive for the Japanese education community. It is desirable that we accumulate reliable evaluation studies and conduct systematic reviews.

View full abstract

A challenge of evidence-based evaluation has already started at official development assistance (ODA) sector, just like the trial of Cochran Collaboration in Medicine and that of Campbell Collaboration in social policies. The name of the project is MIT's Poverty Action Lab. It has actively conducted and produced evaluation results using rigorous randomized experimental design since its establishment in 2003. In this paper, firstly the progress of the project is reviewed, and then, based on its review, several issues relating to randomized design are discussed. They include ethical issue, cost issue, bias issue, the reason why true experimental design is more preferable than quasiexperimental design, and the reason why this design has not been used in this sector. Then, the current situation and issues in Japanese ODA relating to introduction of experimental design is discussed. Finally it is recommended that Japan should actively commit this project because Japan is the largest financer of the World Bank that has financially supported this project.

View full abstract

Since early 1990s, Capacity Development and Capacity Development in Environment (CDE) are discussed in the field of aid assistance. In the current literature, although there is consensus on necessity of capacity development, there is none on its definition as well as its measurement. Regarding this, this study tries to develop a new evaluation method for capacity development. As a case study, we examine the elements of governmental capacity for urban air quality management in major cities of Japan. In this respect, city-level panel data in 1970-2000 period are used in the analysis. By using Factor analysis, we estimate four elements of capacity that contribute to urban air quality improvement. These four elements are further arranged into three factors. Using these factors, we propose an evaluation form called as Actors-Factors matrix. Evaluation method based on Actors-Factors matrix is superior to UNDP's method in a systematic classifies of evaluation criteria, efficient and effectives evaluation, and appropriate policy design for capacity development.

View full abstract

While there is a trend towards decentralization in educational policies, it is not a panacea for the problems faced by developing countries. The disparity between regions or schools in educational achievements has been pointed out as one of the defects attributable to decentralization.
This paper offers a justification for such proposition by studying the case in Indonesia. The method of covariance structure analysis was used in order to find out the influences and effects of the factors related to educational environment upon educational outcome, and to make comparison between before and after the introduction of the decentralized system in Indonesia.
The main findings and conclusion are following; generally speaking, the increase in the school budget improved the quality of educational environment, and led to higher test scores and lower dropping out rates. Such positive results, however, turned out to be less significant, and the relationship between factors became weaker during the period after the decentralization system had started. This was particularly the case for the group of schools whose budget level was relatively small.
In addition to the efforts to improve the quality of teachers and headmasters, it is important to consider the measures aimed at correcting the disparity between schools as well as regions.

View full abstract

The Japanese Ministry of Foreign Affairs recently conducted the first comprehensive review of Structural Adjustment Loans (SAL) provided by the Japanese government. For the roughly two decades from 1986 to the present, approximately 9, 676 billion yen has been disbursed as SAL. This paper attempts to uncover the political background for the decision to start SAL in the 1980s. The author focuses on the fact that the Japanese government faced a serious problem from the US-Japan trade imbalance at the time. Under Prime Minister Yasuhiro Nakasone's initiative, it announced the Financial Recycling Plan with the aim of circulating about US$65 billion in Japanese money to international institutions such as the IMF and World Bank, and to the developing countries, which were facing serious debt problems. SAL was selected as one of the components of the Recycling Plan, as it allowed the quicker disbursement of loans than project loans. The question here is the rationale for the selection of SAL as the modality for ODA. However, it is impossible to evaluate the relevance of the SAL program, which had a political mission beyond the purpose of simply providing ODA, under the current guideline for evaluating ODA. This paper raises the question about how to evaluate the program that is directly connected to the grave political purposes.

View full abstract

In this paper, the author proposes an evaluation method to measure the standardized effect by measuring the change of key indicators of the intervened group (in comparison with a non-intervened group). The standardized effect can be dealt as unit. She gives it a unit name ‘effect’. Impact can be calculated by multiplying the standardized effect and the target population. Efficiency can be calculated by dividing this impact by the total input expressed in monetary unit. She shows two cases to which she applied this method: the evaluation of Japan's Foreign Student Policy and that of a regional Reproductive Health Program in the Philippines.
One of the merits of this method is to quantify the effect of intervention and to express it by a standardized unit ‘effect’. By this quantification, it became possible to measure impact and efficiency in a standardized way.
By utilizing the same key indicators and conducting evaluation in this method, it will be possible to compare the standardized effect, impact and efficiency of different interventions. The difference of effect and efficiency will be caused by the conditions of the intervening side and the intervened side. So, this comparison will give clues to explore not only the effective and efficient way of intervention, but also the promoting and hindering factors in the target population.

View full abstract

As design and the execution of a policy by a few policy actors became difficult, and the policy problems that they have deal with became complicated, various policy actors participated in a policy process, and the collection and disclosure of policy-related information became do. The evaluation of information approach policies and citizen's participation policy cannot enough analysis by existing evaluation approaches. Therefore I devised that I evaluated an information approach policies and citizen's participation policy in the basis by the policy network analysis that integrated quantitative society network analysis in mathematics sociology with a qualitative policy network theory in political science. In this thesis, Eco-labeling for use in informational approach is analyzed by policy network analysis method. Consequently, in the process of these analyses, it became clear to be different in a role of the policy actors to achieve for the information flowing by each eco-labeling goods, and between each structure of a network of the information flowing.

View full abstract

This paper analyses the diffusion and development process of evaluation and performance measurement in Japan's local governments. We indicate that unprecedented is the analysis of such process focusing on contributing factors to the diffusion and developmental process, and employs event history analysis of 47 prefectures and complementarily a case study approach for 10 representative local authorities. The results of these two approaches suggest three points: first, an eagerness of governors or mayors, a tradition of administrative reform, and change of administration contribute to the initiation and developmental process; second, socialeconomic factors do not contribute so much as is generally expected, and third, reference group has not impacted on the diffusion process, but a countrywide catching up/imitating strategy among local governments has proved an important contribution to this diffusion. To sum up, political factors are significant, in addition to this strategy.

View full abstract

This paper reconsiders evaluation methods for donor-funded SCPs at the primary and secondary education levels in developing countries and improvement of future prospects for education assistance. Through literature reviews and interviews with those concerned, this paper examines recent characteristics of SCPs in Japan's ODA, and presents a list of indicators related to SC and conceptual models of assistance framework in education cooperation based on stakeholders analysis. An examination of the actual state of SC in Indonesia as a case study reveals recent trends in developing countries. Finally, evaluation guidelines for donor-funded SCPs in developing countries are established, and key issues are observed, such as: 1) more emphasis on relevance of the means of assistance for SC; 2) the importance of applying the DAC's five criteria from holistic viewpoints, considering stakeholders' roles; and 3) the necessity of considering recently established local standards regarding SC in developing countries.

View full abstract

Evaluator training systems in Japan have just started and “The Japan Evaluation Society (JES)” gives great impacts on their promotion, especially student intern program managed by training committee seems to be successful as its dawn.
Therefore, it can safely be said that student intern program mainly aims at evaluator training, but it doesn't mean there's no need to pay attention to the quality of evaluation. If student intern neglects “utilization of evaluation findings” at all, time and cost participants (clients) paid will be wasted because neither reports nor comments are useless for most of them.
This paper first outlines the case study of student intern's evaluation on Japanese nonprofit organization and then examines the utility of it in the context of utilization-focused evaluation. Consequently, looking at the case with six viewpoints of utilization-focused evaluation, some positive elements and challenges are found out through it. In order to develop our evaluator training systems hereafter, student intern should pay more attention to utilization of evaluation findings based on utilization-focused evaluation.

View full abstract