Evaluators can and should help policymakers and managers to overcome the challenges in use ofperformance information to improve policy decisionmaking and program performance. Results-orientedmanagement is likely to increase the demand for and supply of evaluation studies.

View full abstract

A systematic review is a literature review following a scientific protocol beginning with the identification of a research question, literature search, screening and coding of literature and statistical analysis, and ending with structured reporting. In medicine, the Cochrane Collaboration, an international network of individuals, which aims to produce, disseminate and update systematic reviews of evidence, has evolved. Leaning from its success, the Campbell Collaboration was established to inform social policy-making. Evidence generated by the Campbell Collaboration will be distributed electronically and utilized by policy makers as well as service providers. The Campbell Collaboration which will provide the best available evidence on “what works” will form a social infrastructure of the experimenting society.

View full abstract

The purpose of this paper is to discuss Japan' s international cooperation evaluation and policy evaluation focusing on program theory from the viewpoint of program theory evaluation. Methodology of Japan's international cooperation evaluation and policy evaluation have been developed heavily depending on actual administrative works so that they are weak at providing clear and accurate program theory. Consequently the utilities of evaluations are reduced. Considering this situation, to advance studies on how to construct more accurate and suitable program theory is indispensable and an acute issue for improving the effects in Japan's international cooperation evaluation and policy evaluation.

View full abstract

Theoretical concern in program evaluation has for long focused on the choice of evaluation method, with the quantitative-qualitative debate occupying its central place. The principal question had to do with scientific nature, objectivity and generalizability of evaluation findings. Michael Q. Patton joined the debate with a theory of utilization-focused evaluation, proposing a pragmatic synthesis between the two propositions. This theory suggests that the utility of evaluation should be measured in terms of utilization of evaluation findings, and that the key role of an evaluator should be to identify those stakeholders who possess strong interest in the use of the results and to strengthen their ownership of the evaluation process through their participation. This paper first outlines the debate which formed the background to the birth of the utilization-focused evaluation, and then presents its main ideas, illustrates its uses and discusses some questions and issues surrounding it. The paper concludes by commenting on issues that need to be tackled if the high potential it promises for the Japanese evaluation context is to be realized.

View full abstract

Empowerment Evaluation by Dr.Fetterman is the use of evaluation concepts and techniques to foster transformation and self-determination. It provides the process for a group of participants concerned to evaluate by themselves as well as to provide internalization of evaluation system, so that they can continue self-evaluation. Major roles of evaluators include facilitator, trainer of evaluation know-how and even direct advocate to help the group be empowered. Empowerment Evaluation appears most appropriate when the program itself has agenda of empowerment and social change. The mixed use of Empowerment Evaluation and traditional types of evaluations would make it possible to implement valuepluralistic evaluations, and that would certainly provide us with opportunity to reconsider development of community by voluntary people. The accumulation of experiences of Empowerment Evaluation in Japan is expected for further testing and adjustment of its theory and methodology.

View full abstract

Peter F. Drucker is one of the world famous authorities of Management. Although he was considered as a management expert of the for profit business sector, he has shown his interest in the private non-profit sector from more than 50 years ago. He predicted the growth of the nonprofit sector at the global level since 1980s. He also pointed out the fundamental problem of the nonprofit organization. It is the lack of the mechanism and consensus with respect to a bottom line of performance.
“Self-Assessment Tool for the Nonprofit Organization” was developed by Drucker Foundation as a tool of management improvement, in which Drucker guides to find out their problems and to think about innovating their business with simple five questions. This paper explains the outline of the selfassessment tool with Drucker's theory of nonprofit organization. And, since evaluation experts show their interest in how the result of performance measurement is integrated in management, we will try to identify the difference between the result based management and self-assessment tool simply. Drucker's assessment, which focuses on the innovation, always looks not at verifying attainment of the objectives but at comprehending the gap between the surrounding environment's changes and the organization's current achievement. This is the major difference of approaches between the RBM and self-assessment tool, and may tell us the limitations and possibilities of such evaluations.

View full abstract

In the process of introducing evaluation activities into Japanese public sector, ‘strategic planning’, which is generally recognized as counterpart of performance measurement in the United States, has not been seriously examined and considered. This fact has made it very smoothly to introduce evaluation activities in Japan, but also this has led the result that evaluation activities have not well contributed to realization of the idea and the goals sought in the trials of the public sector reform in Japan. Based on this situation, the followings should be seriously examined in revision of the current policy evaluation law, which is expected to be conducted soon. Firstly, strategic planning and performance measurement should be introduced as ‘one set’ into all governmental ministries and agencies as obligation. Secondly, Japanese term of ‘project evaluation’, which is almost similar to ex-ante cost-benefit analysis in the United States, and Japanese term of ‘integrated evaluation’, which is almost the same as program evaluation in the United States, should be employed in order to assess and evaluate programs and projects scrutinized and selected as a result of strategic planning.

View full abstract

This article proposes a new method of secondary evaluation as a means for the public to participate in governmental activities, using a JICA project as a case study. In this method, various procedures are followed, such as: 1) collecting, using the internet and libraries, primary data mainly composed of evaluation reports; 2) collecting secondary data based on literature review and the law concerning access to information held by administrative organs; 3) formulating the Program Theory Matrix; and 4) conducting secondary evaluation using the DAC five criteria.
This method makes it possible for the public to carry out evaluation on governmental activities from their own standpoint and viewpoint, using existing materials at low costs without implementing research at the project site. Thus, secondary evaluation enhances the contents of primary evaluation.
The concept of secondary evaluation is relatively new in the history of evaluation, and is meaningful in the sense that the potential of evaluation is widened. Through non-profit organizations actively conducting secondary evaluations on activities of public utilities, public's voice will be raised in formulating policies of the government and transparency and efficiency of governmental activities will be increased.

View full abstract

This paper quantitatively analyzes the incidence of public education spending on the poor in developing countries by using the results of standard benefit-incidence studies and additional national data. Although there is considerable variation across studies, it is found that public education spending generally does not favor the poor, but it does favor them at a lower level of education. The poorest quintile (20% of the population), on average, receives 16.3% of total public education spending while the richest quintile receives 25.9%. In primary education, the poorest quintile receives 22.4% of public spending but only 5.5% in higher education. The cross-country analysis of variations in the incidence of public education spending implies that increased spending on education will be associated with increased share for the poor given that it is not devoted to spending on higher education.

View full abstract

In his treatise on strategy entitled, A Book of Five Rings, famed samurai, Miyamoto Musashi, puts forward a nine-step method for learning Kendo or “the Way of the sword.” Musashi's words also speak to those of us who practice the evaluation profession. The same nine-step method can be applied for learning “the Way of evaluation.”

View full abstract

When developing social indicators, a method is needed to transform each variable into units ona common scale so that they can be compared and combined. Recently, this issue has become commonto performance measurement and program evaluation, because they use so many outcome measures manyof which are a kind of social indicators. In order to analyze the outcome of programs, it is often necessary to compare the indicators in a common framework.
We review the method for converting the indicators used in the new social indicators developedby the Cabinet Office of the Government of Japan to evaluate the outcome of structural reform in Japan, and propose another method of conversion which are applicable to performance measurement and program evaluation.

View full abstract

The purpose of this paper is to present a model for a new role of evaluators for local government reform based on the current practice of its evaluation. Evaluation was introduced as a highlight for local government reform. Local government reform has the three stages: system reform, management reform, and norm reform. At the stages of management and norm reforms, evaluation must be utilized, however, not a remarkable outcome has been seen and obstacles exist for the evaluation to be fully utilized. In order to overcome the obstacles, the author argues that local government foster an Empowerment-oriented Evaluator for local government reform, who would lead local government reform and improve organizational capacity. The evaluator utilizes the roles of mentor, facilitator and producer as well as evaluator, and realizes management and norm reforms. The evaluator may revitalize government reform. Creating Empowerment-oriented Evaluator Model and training of Empowerment-oriented Evaluator are one of the most pressing challenges.

View full abstract

With the development of decentralization and administrative reform, local self-governing bodies are being asked to enter into a partnership with residents. The administration must fully consider the environment, not just economics.
The purpose of this research is to determine measures for the water cycle by assembling a solution through a questionnaire analysis of prefecture Gifu on consciousness of the water cycle among residents.

View full abstract

Gender Statistics are basis for gender-responsive policy formulation and planning.
In Indonesia lack of appropriate gender-responsive statistics system hindered the promotion ofthe formulation of gender-responsive policies. In this situation the Japan International Co-operation Agency (JICA) started assisting the Government of Indonesia in developing the functional gender-responsive statistics system (Gender Statistics Project) in February 2000 and continued until March 2002.
The overall goal of the Gender Statistics Project was to utilize gender statistics to formulate policies and plan, and the direct objectives were 1) to promote cooperation between gender statistics producers and users 2) to improve capacities of producers 3) to improve capacities of users 4) to increase users' access to gender statistics. Under the Project the following activities took place; Needs assessment, Trainings, Production and publication of gender statistics booklets, Dissemination of gender statistics, Review of the published gender statistics booklets.
In this paper first the outputs of the project were reviewed, and then the possibility of the utilization of the presented statistics was examined from the gender-responsive policy formulation point of view. Finally it was evaluated whether this Project contributed to promote the gender-responsive policy formulation. It was recognized that the Project made the concrete system of gender statistics necessary for formulating gender-responsive policies based on the positive external factors.

View full abstract