日本における若手薬剤師の臨床能力評価ツールの評価

Abstract

本邦では標準化された卒後臨床教育プログラムはなく評価方法も確立されていない．そこで，我々は薬剤師レジデントを対象として，英国で開発された臨床能力の評価方法の1つであるCase-based discussion（CbD）を実施し，成長過程を可視化し得るかを検証した．CbDは「薬物治療の必要性評価」等の5項目で評価した．薬剤師レジデント78名と，専門・認定薬剤師（Board certified pharmacists; BCP）5名を対象とした．CbDを実施した結果，5項目すべてのCbDスコアが病棟の経験と共に増加し，4項目が統計学的に有意であった．BCPのCbDスコアは，全項目で最高点の6であった．以上より，CbDは，本邦においても臨床能力の成長プロセスを可視化し，若手薬剤師の教育ツールとして臨床能力の評価に応用できる可能性が示された．今後，多施設を含めた更なる検討が必要である．

Purpose

With the remarkable progress in medical care, health care providers are required to have a high degree of expertise and up-to-date knowledge and skills; moreover, the roles and responsibilities of pharmacists have shifted from dispensing to medical therapy management^1,2). Pharmacy residency programs were developed in the U.S. in the 1960s with the aim of improving the clinical skills of pharmacists. The American Society of Health Systems Pharmacists (ASHP) has described competency areas for pharmacy residents³⁾, whereas the Competency Development and Evaluation Group (CoDEG) in the U.K. consisting of university faculty and hospital pharmacists has established the standard expected of pharmacists as a General Level Framework (GLF)⁴⁾. ASHP has developed competency requirements for graduation from the first and second years of the residency program, whereas CoDEG evaluates the competence of young to mid-career pharmacists.

In order to evaluate the utility of the GLF, the CoDEG has developed various online tools⁵⁾. For example, the Mini Peer Assessment Tool (mini-PAT) is used to evaluate daily clinical practice from self and other professions, whereas the Mini Clinical Evaluation Exercise (mini-CEX) evaluates patient interview skills. Case-based discussion (CbD) assesses whether the pharmacist has sufficient problem-solving ability regarding drug selection, ethical/legal questions, and pharmacokinetic or pharmacodynamic appropriateness. CbD is conducted through oral examination with an assessor and covers the following five items: 1) pharmaceutical needs assessment; 2) treatment recommendations; 3) follow-up/monitoring; 4) professionalism; and 5) overall clinical judgment (Table 1). Each item is scored on a 6-point Likert scale, with a score of 4 defined as the minimum level that should be attained by pharmacists-in-training.

Table 1 Case-based discussion (CbD) form

Item		Below expectations for GLF completion		Borderline for GLF completion	Meets expectations for GLF completion	Above expectations for GLF completion		U/C
		1	2	3	4	5	6
1	Pharmaceutical needs assessment
2	Treatment recommendations
3	Follow-up/monitoring
4	Professionalism
5	Overall clinical judgment

Rating scale
1	Significantly below	Performs poorly; very rarely meets the required standard
2	Below	Performs poorly; occasionally meets the required standard
3	Borderline	Performs satisfactorily; should meet expectations with appropriate support and direction
4	Meets expectations	Performs well and to the standard expected of a pharmacist at that stage in their training
5	Above expectations	Performs to a standard higher than what is expected from a pharmacist at that stage in their training
6	Significantly above expectations	Performs to an excellent standard; trainee is ahead of his/her peer group
U/C	Unable to comment	Unable to comment as performance was not observed whilst s/he was there

CbD item	Description of activity
Pharmaceutical needs assessment	Has correctly identified the pharmaceutical care issues for the patient and is able to appropriately prioritize the pharmaceutical care issues.
Treatment recommendations	Can discuss the treatment of the main medical problem—evidence-based treatment guidelines, drug therapy (mechanism of action of drugs, dosage range, key pharmacokinetic data, cautions, contra-indications, common side effects, major drug interactions, patient counseling points).
Follow-up/monitoring	Can discuss the rationale for monitoring patient’s pharmaceutical care.
	Demonstrates practical ongoing and appropriate monitoring of therapy (including Kardexes as well as biochemistry and hematology)
	Can discuss and demonstrate how care of the patient was managed in conjunction with the wider healthcare team
Professionalism	Demonstrates an ability to prioritize and works efficiently. Has an ethical approach to work and is aware of any relevant legal frameworks. Has insight into own limitations. Considers interface issues.
Overall clinical judgment	Can discuss his/her own judgment and synthesize information, and cares about the patient.

Kitasato University introduced the pharmacy residency program in 2002; as of 2018, similar programs have been adopted by over 40 hospitals in Japan⁶⁾. However, there is no standardized core curriculum or method for assessing the clinical skills of pharmacists-in-training in Japan, with each hospital developing its own curriculum^7,8). The present exploratory study was carried out in order to establish a standard assessment tool for evaluating the clinical skills of trainee pharmacists in Japan so that educational programs can be tailored to the degree of competence of each pharmacist and education gaps between hospitals can be reduced. We also evaluated the applicability of CbD by examining changes in CbD score over the residency period.

Method

1. Participants

A total of 78 pharmacy residents who entered the program in April 2016 and April 2017 were included in this study. As a control, five board-certified pharmacists (BCPs) were enrolled along with two Japanese Society of Pharmaceutical Health Care and Sciences (JSPHCS)-certified oncology pharmacists, two JSPHCS-certified pharmacotherapy specialists, and one Japanese Society for Emergency Medicine-certified pharmacist for emergency medicine.

2. Training curriculum

The training period of the pharmacy residency program is 1 year, during which residents must acquire the level of competency required by Showa University (Table 2). Pharmacy residents trained at four wards including internal medicine, surgery, and oncology wards over a 1-year period (2–3 months per ward). The residents engaged in CbD with their preceptor at the end of each ward training period.

Table 2 Competency for pharmacy residency at Showa University

1	Properly carry out brought-in medicine confirmation and prepare for the first meeting
2	Collect information for assessing the patient’s medical condition from an interview
3	Assess patient information obtained from the interview, and share important information with medical staff
4	Understand treatment policy agreed upon by medical staff
5	Plan and propose optimal drug therapy based on patient’s condition
6	Audit the medication prescribed to the patient
7	Confirm clinical examination data and evaluate patient’s condition
8	Prepare for patient counseling
9	Implement patient education and counseling
10	Document assessment of patient’s pharmacotherapy and education
11	Communicate effectively with medical staff and patients
12	Implement patient education and counseling at discharge
13	Discuss with medical staff at ward conference to decide each patient’s treatment
14	Create a case summary of patient
15	Rapidly and accurately answer questions from medical staff
16	Understand the role of pharmacists in medical teams and cooperate with other medical staff to provide optimal treatment
17	Professionalism

Assessor evaluates the trainee on a 6-point Likert scale for each item four times a year.

3. Development of the Japanese version of CbD

The CbD was translated into Japanese by several independent pharmacists (including those with a PharmD degree) with a minimum of 5 years of clinical experience.

4. Assessment of participants

A preceptor with at least 2 years of ward experience assessed each pharmacy resident.

5. Implementation of CbD

Each item in the CbD (“Pharmaceutical needs assessment”, “Treatment recommendations”, “Follow-up/monitoring”, “Professionalism”, and “Overall clinical judgment”⁵⁾) was scored on a 6-point Likert scale with minimum and maximum values of 1 and 6, respectively.

According to the implementation method of CoDEG original version⁹⁾, each pharmacy resident presented one case in which he/she intervened to the preceptor, who interviewed the resident to verify their degree of comprehension and ability according to the CbD assessment form (Table 1). The resident was required to obtain at least 4 points for each item to achieve the general level set by the CoDEG. BCPs were assessed in the same manner.

6. Statistical analysis

Average CbD scores for each training period were evaluated by one-way analysis of variance (ANOVA). A two-sided P value of 0.05 was considered statistically significant. When a significant difference was observed by ANOVA, the group showing a significant difference in multiple comparisons was identified. Data were analyzed using JMP Pro v.14.0.0 software (SAS, Cary, NC, USA).

7. Ethical considerations

This study was approved by the institutional review board of the School of Pharmacy of Showa University.

Results

1. Participant characteristics

Table 3 shows the characteristics of pharmacy residents and BCPs. All 78 residents were new graduates without work experience as a pharmacist; their average age at the start of training was 24.8 years (range: 24–31 years). Two residents abandoned the training, and 76 completed the 1-year curriculum. BCPs had an average of 12.4 years of pharmacist experience (range: 8–19 years).

Table 3 Participant characteristics

Resident pharmacists	n = 78
Sex (male/female)	29/49
Age at the start of training, mean (range)	24.8 (248–31)
Board-certified pharmacists	n = 5
Sex (male/female)	4/1
Type of certification
JSPHCS-certified oncology pharmacist	2
JSEM-certified pharmacist for emergency medicine	1
JSPHCS-certified pharmacotherapy specialist	2
Years of experience as a pharmacist, mean (range)	12.4 (8–19)

JSEM, Japanese Society for Emergency Medicine; JSPHCS, Japanese Society of Pharmaceutical Health Care and Sciences.

2. CbD score

Figure 1 shows the changes in CbD score for each item during the residency period. The change in score for “Pharmaceutical needs assessment” was statistically significant (P = 0.01). The results of multiple comparisons showed that the CbD score for this item increased significantly in wards 2–4 as compared to ward 1. The score for “Treatment recommendations” did not change significantly during the residency (P = 0.07). However, the results of multiple comparisons showed a significant increase in ward 2 as compared to ward 1 (P = 0.02), whereas no significant differences were observed between wards 3 and 4 and ward 1. CbD scores for “Follow-up/monitoring”, “Professionalism”, and “Overall clinical judgment” changed significantly over the course of the residency (all P < 0.01), showing an increase in wards 2–4 relative to ward 1 in multiple comparisons. The BCPs had CbD scores of 6 for all items.

Fig. 1.

Changes in CbD score over the pharmacy residency. Upper and lower points of each diamond represent confidence intervals; the line crossing the center of each diamond represents the group average; lines drawn up and down from the group average are overlap marks calculated as group mean ± ($\sqrt{2}$)/2 × CI/2; dotted and solid lines indicate the results for BCPs and the general level defined by the CoDEG, respectively. A–E: Changes in CbD score for the five items were evaluated by ANOVA. P < 0.05 for all items except for “Treatment recommendation” (P = 0.08).

Discussion

In this study, we established a standard measure of the clinical skills of pharmacy residents based on CbD developed by the CoDEG. The CbD score increased with the number of training wards, indicating that CbD can be used to assess the degree of improvement in clinical competence of trainee pharmacists.

The CbD score in ward 1 was initially less than 4 for all items, but was over 4 by the end of training. This suggests that the CbD score of 4 recommended by the CoDEG⁵⁾ is appropriate as an initial training target value in Japan.

CbD may also be useful for assessing the curriculum itself. In this study, the CbD score for “Treatment recommendations” did not increase after ward 2, suggesting that the current curriculum should further develop or emphasize this aspect of training so that pharmacy residents are better able to identify drug-related problems based on patients’ conditions. On the other hand, “Treatment recommendations” requires that patients have a relationship of trust with their doctor; a 1-year program may not allow sufficient time to attain competence in this regard. Based on these results, the duration of the pharmacy residency program at Showa University has been extended to 2 years starting in 2019.

The controls in this study were BCPs, whose knowledge and skill can improve drug therapy efficacy and safety^10,11). All of the BCPs had CbD scores of 6 on all five items, reflecting their advanced clinical skills including critical thinking (“Pharmaceutical needs assessment”), planning (“Treatment recommendations”), and decision making based on disease state and the patient’s condition (“Overall clinical judgment”) as well as their strong sense of responsibility (“Professionalism”).

The CoDEG has a consistent educational program for young through senior pharmacists, but mainly applies the GLF to the former and the Advanced and Consultant Level Framework (ACLF) to the latter¹²⁾. The ACLF consists of the following six categories: 1) Expert professional practice; 2) Building working relationships; 3) Leadership; 4) Management; 5) Education; and 6) Training/development and research/evaluation. A similar clinical skills assessment method for advanced (e.g., board-certified) pharmacists and their competencies is needed in Japan.

Although a pharmacy residency program has been established at more than 40 hospitals in Japan, a unified core curriculum is lacking. The CbD used in this study can be helpful for measuring differences in the standard of training among hospitals and programs. Implementing CbD at multiple hospitals in Japan will allow comparisons with other countries such as Singapore¹³⁾ and Australia¹⁴⁾ that have introduced the curriculum established by the CoDEG; through these comparisons, clinical competency in specific areas can be improved in the development of a standardized clinical education curriculum for pharmacists in Japan.

This study had five major limitations. Firstly, one of the four training hospitals was unable to conduct the fourth evaluation due to scheduling conflicts. As such, the fourth evaluation of CbD score showed greater variation than the first through third evaluations. However, this had little impact on the results since the CbD score increased from the start of the program up to the fourth ward.

A second limitation was that although the assessor was defined as a pharmacist with at least 2 years of clinical experience, this included pharmacists with anywhere from 2 to over 20 years of experience. Thus, inter-individual differences in clinical experience may have contributed to variations in the evaluation of CbD, although statistically significant increases in CbD score over time were nonetheless observed. In this study, CbD was performed by one assessor for one trainee according to the CoDEG original version⁹⁾. However, evaluation by multiple assessors is generally necessary to ensure consistency. Although the variation from the results of this study was expected to be small, evaluation of by multiple assessors and verification of the degree of agreement may be necessary.

A third limitation was that this study did not compare with or without CbD. As this is an exploratory study to verify the applicability in Japanese, a group not performing CbD was not set. In the trial of CoDEG original version¹⁵⁾, the change from the baseline also verified the usefulness. This exploratory study found changes similar to the original version in Japanese, but it is necessary to carry out randomized controlled trial to verify the usefulness of CbD.

A fourth limitation was that CbD is one of tools for evaluating competency as defined by the GLF of the CoDEG. Because we focused on the point of problem-solving ability regarding pharmacotherapy, we used CbD as an evaluation item in this study. In the future, it is necessary to evaluate the comprehensive pharmacist’s ability other than problem solving ability by combining and evaluating mini-PAT which is evaluation from other professions and mini-CEX which evaluates patient interview skills. Furthermore, in this study, CbD served as a program aimed at achieving the competencies required by Showa University. The GLF of CoDEG includes competency in the following four areas: 1) Delivery of patient care; (2) Personal; (3) Problem-solving; and (4) Management and organization. CbD mainly evaluates 1) and 3). Since Showa University’s competencies include these items (Table 2), the impact of the competency difference on the results might be limited.

Finally, there may be a difference between the clinical ability of young UK pharmacists and young Japanese pharmacists. At present, international comparisons are difficult because there is no standard tool to compare the clinical ability of pharmacists worldwide. CbD may be one of the tools to compare clinical competency internationally if the reliability and the validity of CbD are established in Japanese pharmacists.

In conclusion, the results of this study demonstrate that CbD can be used to evaluate the acquisition and improvement of clinical skills by pharmacists-in-training. The CbD is thus useful not only as an educational tool but also for standardizing the educational curriculum of pharmacy residency programs in Japan.

Conflicts of interest

There are no conflicts of interest to be disclosed in relation to the contents of this paper.

References

• 1)  Worley MM, Schommer JC, Brown LM, et al. Pharmacists’ and patients’ roles in the pharmacist-patient relationship: Are pharmacists and patients reading from the same relationship script? Res Social Adm Pharm. 2007; 3(1): 47–69.
• 2)   Sweeney  MA. Physician-pharmacist collaboration: A millennial paradigm to reduce medication errors. J Am Osteopath Assoc. 2002; 102(12): 678–681.
• 3)  Postgraduate Year One Pharmacy Residencies—Competency Areas, Goals, and Objectives (CAGO) and Guidance [Internet]. ASHP; [cited June 1, 2018]. Available at: https://www.ashp.org/-/media/assets/professional-development/residencies/docs/required-competency-areas-goals-objectives.
• 4)  A Framework for Pharmacist Development in General Pharmacy Practice [Internet]. CoDEG; [cited June 1, 2018]. Available at: http://www.codeg.org/fileadmin/codeg/pdf/glf/GLF_October_2007_Edition.pdf.
• 5)  Assessment_Tools [Internet]. CoDEG; [cited June 1, 2018]. Available at: http://www.codeg.org/fileadmin/codeg/pdf/Assessment_Tools.pdf.
• 6)  Tohru H. 新しい薬剤師卒後研修制度の構築を目指した薬剤師レジデント制．Farumashia. 2016; 52(8): 749–751.
• 7)   Washiyama  A,  Otsuka  M,  Futagami  K. Evaluation of the resident pharmacist program of Fukuoka University Hospital. Yakugaku Zasshi. 2014; 134(1): 127–132.
• 8)   Yoshikazu  H,  Rena  N,  Satoshi  T. Developing a residency curriculum dedicated to training highly skilled clinical pharmacists. RSMP. 2016; 6(2): 151–162.
• 9)  The Pharmacy Vocational Training Scheme [Internet]. Scotland, UK: National Health Service; 2015 Oct [cited 2019 Apr 25]. Available from: https://webcache.googleusercontent.com/search?q=cache:-n3yB_RLAuUJ:https://www.nes.scot.nhs.uk/media/2988464/vt2_faqs_vt2__hints___tips__october_2014-2.doc+&cd=6&hl=ja&ct=clnk&gl=jp[]
• 10)   Imamura  M,  Ogawa  D,  Takatori  T, et al. A retrospective study of the effects of oncology pharmacist participation in treatment on therapeutic outcomes and medical costs. Biol Pharm Bull. 2017; 40(11): 1956–1962.
• 11)   Takashi  N. Establishment of antimicrobial stewardship intervention to promote the appropriate use of antimicrobial injection in all inpatients and outcomes evaluation. Jpn J Pharm Health Care Sci. 2013; 39(3): 125–133.
• 12)  Advanced and Consultant Level Framework [Internet]. CoDEG; [cited June 1, 2018]. Available at: http://www.jpbsoutheast.org/fileadmin/codeg/pdf/ACLF.pdf.
• 13)   Victoria  R,  Camilla  W,  Ian  C, et al. Use of a general level framework to facilitate performance improvement in hospital pharmacists in Singapore. Am J Pharm Educ. 2012; 76(6): 107.
• 14)   Jackson  S,  Martin  G,  Bergin  J, et al. An advanced pharmacy practice framework for Australia. Pharmacy (Basel). 2015; 3(2): 13–26.
• 15)   Ian  C,  Minyon  A,  Lynda  C, et al. Improvement in pharmacist’s performance facilitated by an adapted competency-based general level framework. J Pharm Pract Res. 2010; 40: 111–118.