Japanese Journal of Evaluation Studies
Online ISSN : 1884-7161
Print ISSN : 1346-6151
ISSN-L : 1346-6151
Volume 20, Issue 2
Displaying 1-10 of 10 articles from this issue
Special Issue: The Current Status and Issues on Evidence-Based Policy Making (EBPM)
  • Ryo Sasaki, Tomoya Masaki
    2020Volume 20Issue 2 Pages 1-2
    Published: July 31, 2020
    Released on J-STAGE: June 01, 2023
    JOURNAL FREE ACCESS
    Download PDF (283K)
  • Tomoya MASAKI, Kiichiro TSUTANI
    2020Volume 20Issue 2 Pages 3-18
    Published: July 31, 2020
    Released on J-STAGE: June 01, 2023
    JOURNAL FREE ACCESS

      In Japan, since the late 2010s, “evidence-based policy-making(EBPM)” has been introduced into the public sector. In the medical field, the CONSORT statement was developed in 1996 to improve the quality of Randomized Control Trial(RCT)articles. It was the pioneer of reporting guidelines. The third edition of CONSORT 2010, consisting of all 25 items, has been translated into 16 languages and is widely used throughout the world. In addition, over 400 reporting guidelines have been developed for basic study designs such as observational studies(cross-sectional, retrospective, and prospective studies)and systematic reviews, as well as expanded versions such as cluster randomization and nonpharmacologic treatment(NPT). We also introduced the EQUATOR Network, which covers many reporting guidelines that will be useful for people starting to create research plans. Together, these findings suggest the use of reporting guidelines is a feasible option for EBPM.

    Download PDF (2193K)
  • Implications from experiences in the healthcare sector
    Satoko Tanabe
    2020Volume 20Issue 2 Pages 19-31
    Published: July 31, 2020
    Released on J-STAGE: June 01, 2023
    JOURNAL FREE ACCESS

      Evidence-based policymaking (EBPM) has its origins in evidence-based medicine (EBM). Policymaking is believed to be separate from healthcare, where evidence is created and used in an ideal setting. However, even in healthcare, medical providers are not able to make decisions based only on evidence. Additionally, they face the issue of the evidence-practice gap.

      This paper distilled suggestions for EBPM based on EBM and experiences in the healthcare sector, namely, creating a definition of evidence that focuses on effectiveness, accumulating and sharing evidence, understanding the factors to be considered in decision-making, combining the application of rigorous evidence and simple performance measurement, resolving difficulties in diffusing evidence, and realizing the government’s role in developing infrastructure for EBPM.

    Download PDF (1657K)
  • Yohei Kobayashi
    2020Volume 20Issue 2 Pages 33-48
    Published: July 31, 2020
    Released on J-STAGE: June 01, 2023
    JOURNAL FREE ACCESS

      Evidence-based policy making (EBPM) has been gradually introduced in Japan. This paper summarizes and evaluates its current status it focuses on ex-ante analysis and logic model building, and deviates from the its primary role. We compare EBPM with the other field such as evidence-based medicine in order to clarify the differences between them. Based on the comparison, we discuss how to promote EBPM. Specifically, we emphasize the importance of setting questions beforehand, reconsidering the definition of evidence, generating evidence, and creating demand for evidence.

    Download PDF (2149K)
  • Yutaka Sunayama
    2020Volume 20Issue 2 Pages 49-64
    Published: July 31, 2020
    Released on J-STAGE: June 01, 2023
    JOURNAL FREE ACCESS

      The Japanese government has engaged in creating, disseminating, sharing and accumulating concrete examples of EBPM, which is one of the most immediate issues to address concerning administrative management in Japan. In the process of the on-going joint study on EBPM among the Ministry of Internal Affairs and Communications (MIC), related ministries and academics, some elements that are effective and useful for EBPM of the government are being identified. In addition, the MIC has conducted several surveys of government programs and their operations with empirical analyses using statistical data to get evidence, which contributes to boosting the government’s efforts in promoting EBPM. It is indispensable hereafter to arrange a stable institutional infrastructure to establish EBPM within the government. In this respect, the policy evaluation system introduced in 2001 could be one of the most prospective options.

    Download PDF (2626K)
  • Koji Nishihata
    2020Volume 20Issue 2 Pages 65-76
    Published: July 31, 2020
    Released on J-STAGE: June 01, 2023
    JOURNAL FREE ACCESS

      In recent years, some local governments in Japan have begun to introduce Evidence-Based Policy Making (EBPM). It is predicted that more and more local governments in Japan will introduce EBPM in the near future, but the author thinks it is important that rather being diffused than each local government sporadically collecting their own evidence, the system should be rooted into an organization in each local government body. This article focuses on the current status and issues for institutionalizing the EBPM. The author will shed light on some points wherein what behavioral patterns the EBPM would require, and what management of conventional local governments will call for will differ, which is an issue applicable to most Japanese local governments. Then, the author will make clear possible solutions by referring to the case of Hiroshima Prefecture.

    Download PDF (1407K)
  • Toshiro Mori, Yoshihiro Okazaki
    2020Volume 20Issue 2 Pages 77-88
    Published: July 31, 2020
    Released on J-STAGE: June 01, 2023
    JOURNAL FREE ACCESS

      The Evidence-informed decision making has spread to the education field, and the evidence institution that mediate the evidence in the education field have been established in foreign countries since about 2000. This paper investigated the features of evidence mediators in foreign countries. As a result of investigating each feature of three evidence intermediation organizations(WWC, EEF, CUREE), WWC and the EEF rigorously made the standard which evaluates the research, and while the WWC mainly collects randomization comparative research, the EEF mainly collects systematic review and meta-analysis. Then, the frame which showed robustness of evidence and intervention effect was also different. In addition, the WWC and the EEF focused on the preparation of the report and the guidebook which summarized the research and the CUREE have supported on the utilization of evidence through teacher training. Finally, it was discussed that evidence institution from the viewpoint of EBPM in the education field.

    Download PDF (1365K)
  • Ryo Sasaki
    2020Volume 20Issue 2 Pages 89-103
    Published: July 31, 2020
    Released on J-STAGE: June 01, 2023
    JOURNAL FREE ACCESS

      As a practice of EBPM: Evidence-Based Policy Making (EBPM), impact evaluation has been widely practiced in international development field. The design widely used in the field is Randomized Controlled Trial (RCT). This article compares and discusses the three emerging not-for-profit research institutes that disseminate the database of evaluation reports, J-PAL (The Abdul Latif Jameel Poverty Action Lab), IPA (Innovations for Poverty Action) and 3ie (International Initiative for Impact Evaluation). In addition, the traditional aid agencies, such as the World Bank, USAID of the United States, DFID of the United Kingdom and Japanese JICA will be compared and analyzed.

      Based on those analysis, the emergent agenda will be proposed. They are: (1) the idea of “experimenting society” proposed by Campbell is now in reality and this current practice of impact evaluation should be disseminated more widely; (2) The reason why application of impact evaluation results is limited would be the old discussion is still continuing, that is “Island Effect”. All aid agencies claimed their success of individual project but the vast ocean (i.e., a whole country or a whole sector) remains unchanged; (3) The confusion between the practice of strategic planning/performance measurement and the practice of program evaluation which root is social science is widely observed, and they are actually complement each other, thus the camp for pushing impact evaluation using RCT design should understand there is another thought of strategic planning/performance measurement and there is possibility of collaboration. (4) Possibility of collaboration between research institutes and international NGOs should be pursued in the process of intervention and data collection at field; (5) Major private foundations which roots are IT industries provide vast financial support to the emerging not-for-profit research institutes and this might be the same trend observed at the whole international development field, that is “from government to private sector”

    Download PDF (2294K)
Article
  • Yuichi Murakami
    2020Volume 20Issue 2 Pages 105-119
    Published: July 31, 2020
    Released on J-STAGE: June 01, 2023
    JOURNAL FREE ACCESS

      This paper analyses survey results on the Local Revitalisation Policy(LRP)enacted in Japan in 2014, by focusing on differences in answers provided by the merger and non-merger municipalities of the governmental campaign that took place around 2000(Heisei-no-Dai-Gappei). To evaluate municipality mergers as a policy tool, a case study on Japan was taken into consideration and compared with(non-)merger cases in France, aiming to construct the general theory of its conditions and effects. It was found that ‘non-mergers’ were not strongly influenced by LRP and did not feel obliged to cooperate with neighbours as the LRP desired. On the other hand, they were likely to cooperate with industries, according to their actual and potential partnerships. It cannot be determined whether ‘mergers’ have developed their own competencies as was desired, although they seem to have decreased the extent of democratic participation because their decision-making takes place inside their office. However, they can take relatively more aggressive actions towards other local and central governments without onerous coordination with neighbours.

    Download PDF (2149K)
Report
  • - Consider three cases in Japan focusing on learning and improvement objective evaluation -
    Hiroshi Tanaka
    2020Volume 20Issue 2 Pages 121-136
    Published: July 31, 2020
    Released on J-STAGE: June 01, 2023
    JOURNAL FREE ACCESS

      MSC (Most Significant Change) is a participatory and qualitative evaluation method used by Western NGOs and is suitable for evaluation of learning and improvement purposes, but there are few examples of its use in Japan. In this paper, Author compared and reviewed the three cases of MSC evaluation conducted in the disaster recovery, education, welfare in domestic social sectors, and present reference information for effective use of MSC in Japan. As a result of case studies, following points are revealed. The existence of evaluation experts as trainers and coaches, the setting of evaluation purposes according to the characteristics of the method, the collection of data from the viewpoint of the field, the need to intervene to improve the validity of data analysis, etc.. In addition, the difference from other participatory evaluation methods that internalization is relatively easy was recognized. It is expected that MSC will be utilized in domestic social sector projects in the future, but there are issues such as a lack of the experts to guide and appropriate feedback style.

    Download PDF (1945K)
feedback
Top