Evaluation of undergraduate students’ nursing assessment and communication skills through an objective structured clinical examination within a high-fidelity simulation using a student-simulated patient

: Objective: To evaluate nursing students’ assessment and communication skills through an objective structured clinical examination (OSCE) within a high-fidelity simulation using a student-simulated patient (SSP) and analyze the validity, reliability, degree of difficulty, and discriminability of the OSCE. Methods: A descriptive study was conducted at one nursing school in Macao. All scenarios were designed using a high-fidelity simulator combined with SSPs. A convenience sample of 54 baccalaureate nursing students completed the OSCE. The nursing assessment OSCE checklist (NAOC) and the communication evaluation rubric (CER) were used as observational measurements with three-point Likert scales (2 = Achieved, 1 = Partly achieved, 0 = Not completed/Incorrect). Results: Difficulty coefficients of the exam were 0.63 for nursing assessment skills and 0.56 for communication skills. The discrimination index of the majority items of the NAOC (86.4%) was >0.20, showing a better to good discriminability. The items of the CER had satisfactory indexes of item discrimination (from 0.38 to 0.84). Students received high scores in conducting blood oxygen saturation and cardiac and lung auscultation but low scores in vomiting and diarrhea assessment. Students presented good communication skills in eye contacting and listening, but culture assessment needs to be improved. The students with experiences in simulation or simulated patient (SP) interactions had better assessment and communication skills than students without those experiences. There was a positive relationship between nursing assessment and communication skills ( r = 0.67, P = 0.000). Conclusions: SSPs were involved in enhancing the realism of interactions in simulated scenarios. Nursing students can conduct nursing assessments specific to patient conditions, explain the conditions to the patient, and ensure that the patient remains informed at all times of the precautions to be taken. However, students’ cultural awareness and some communication skills need further training. With moderate difficulty and high discrimination index, OSCE showed satisfactory reliability and validity. competence, clinical skills, clinical problem-solving, communication ability.


Introduction
Holistic human-centered care requires the competency to establish good therapeutic relationships with patients and provide safe nursing services. To face changeable and complex clinical situations, nursing graduates need the requisite knowledge, skills, and competency to deliver qualified nursing care. Nursing education is the basis for cultivating qualified nursing graduates who can provide safe patient care. 1 Simulation is a learning method to imitate or replace real experiences within a particular context through an interactive manner and help nursing students to acquire insight into the interconnected structures of professional knowledge under teachers' guidance. 2 It is used for developing social-cognitive-emotional competency and team-work cooperation within a controlled simulated environment. 3 The skills of nursing assessment and therapeutic communication are important in clinical teaching. The educators should find innovative ways to educate nursing students more effectively in health assessment and the therapeutic use of communication. 4 Simulation provides an appropriate method for observing and evaluating clinical skills. The simulation experience focuses on training in nursing assessment and empathy communication through the interaction with simulated patients (SPs). Such experience is highly beneficial for nursing students because it ensures that they can improve their level of confidence in clinical nursing care. 3 A metaanalysis reported that simulation using SPs in nursing education had a significant effect on cognitive, affective, and psychomotor learnings. 5 The use of SPs vs. manikins to teach cardiac and pulmonary examinations revealed that SPs were significantly preferred by nursing students as a learning method. 6 Student-simulated patients (SSPs) are the trained nursing students who can portray patient concerns in a realistic style. 7 Thus, SSPs are among the effective means to coach psychomotor skills, improve therapeutic communication, enhance clinical knowledge, and promote clinical judgment in undergraduate nursing education. 8 Regarding student evaluation, SSPs usually focused on students' shortcomings in competence and clinical performance. 9 However, the actual knowledge that students have gained was not taken into consideration. 10 To be clear and systematic, assessment needs to focus on objective criteria to ensure that SSPs can encourage students' continuous learning process. The educators should consider the means by which evaluation of students' competencies can be made feasible through adoption of the objective standardized approaches. The criterion-referenced assessments could enhance the reliability of measurements for achievement assessment of learning outcome. 7 As a summative assessment, objective structured clinical examination (OSCE) played a key role in the assessment of students' abilities to complete specific skills and demonstrate holistic nursing for SPs. The OSCE was considered to be a reliable and valid assessment tool in assessing knowledge application and specific clinical skills. SSPs were also used as a learning assessment tool in nursing education. 8 However, the available literature on the use of SSPs in OSCE for nursing assessment and communication skills was limited. It was therefore considered necessary to create contextualized scenarios with a high-fidelity simulation and SSP for developing the objective assessment tools and explore their evaluation quality.
The objectives of this study were to evaluate students' nursing assessment and communication skills through an OSCE within a high-fidelity simulation using an SSP and to analyze the validity, reliability, degree of difficulty, and discriminability of the OSCE.
Experience learning was defined as the learning process whereby knowledge was derived from it and continuously modified by experience. Knowledge was constructed through grasping it from learning-experiences which the work-environment offers; further, transforming that knowledge into skills that are of practical use is a challenge which can be addressed only through more experience. Simulation-based learning provides students a unique chance to learn through experience and to promote their in-depth reflection about learning. 11 The simulation-based learning centered on the opportunity for students to practice and learn in the environment that is as close to reality as possible. The closer a learning experience resembled a real event, the easier it was for a student to transfer gained knowledge and skills to real situations. Reflection was the process to reexamine the experience. Students may think about what comes to mind first, what knowledge they learned, and what needs to be improved. Students were expected to demonstrate successfully that they can transfer what they have learned and experienced from one situation to the next. 5 The clinical education focuses on facilitating knowledge application, accurate clinical judgment, and skill improvement. 12 The OSCE involves performance-based testing. The OSCE assessment method consisted of a given number of "show how to do in simulated scenarios" stations at a given time. 13 It was used to assess students' clinical competence and to test some knowledge, skills, judgment, and clinical reasoning through direct observations. The OSCE was used to evaluate the abilities to obtain and interpret data and handle unpredictable patient conditions in clinical examinations. 14 The OSCE was designed to evaluate the performances of the knowledge, skill, and cognitive/affective domains. It was a standard method of assessment in clinical competence, clinical skills, clinical thinking, problemsolving, and communication ability. 13 The SP was a standardized patient, who was trained to act as a real patient to simulate a set of symptoms or problems. SPs were extensively used in health care education for students to practice clinical and communication skills. 15 The OSCEs in high-fidelity simulationscombined with SPs-were conducted for achieving the objective clinical performance assessment. Working with simulators and SPs helped the students to promote conceptual understanding, reasoning, and critical thinking skills. The students were observed and evaluated as they went through a series of stations in which they assessed the patient conditions, conducted the interventions, and treated the patients. 13 The OSCE within a simulated context not only provided students with a realistic clinical experience but also offered an opportunity for structured reflection on their performance. Students received comments openly from teachers and SSPs. The integration of feedback and reflection increased the students' abilities to construct knowledge from multiple sources and make a safe and sound decision. 16 The OSCEs within high-fidelity simulations that deployed SSPs enhanced students' understanding and respect of diverse cultures and religions and additionally minimized student and evaluator variation in the assessment of clinical skills. 13

Design
A descriptive study was conducted at one nursing school in Macao, China.

Participants
This study involved a convenience sample of baccalaureate nursing students who had completed courses of fundamentals of nursing, health assessment, and medical-surgical nursing. A total of 54 students completed nursing assessment OSCE (Table 1).

The simulated scenario in OSCE
The OSCE was designed to evaluate students' assessment ability, communication skills, and professional attitude. The OSCE included both generic nursing care skills as well as skills specific to each of the scenarios and involved comprehensive abilities to problem-solving and ethical consideration. Scenarios were designed using the high-fidelity simulator, which was a computercontrolled human patient simulator (HPS, SimMan). The SimMan was a full-body manikin with a realistic upper airway, chest movement, variable cardiac and breath sounds, and a palpable pulse, with an ability to detect some verbal symptoms and respond to interventions. The simulated environments had wireless capabilities to run the simulations using a laptop computer. The scenarios progressed chronologically and were run sequentially. Students who passed the SP training were required to participate in enhancing the realism of interactions. SSPs were proficient in simulating a set of symptoms and emotional states while providing instant feedback and psychosocial interactions. They were expected to demonstrate an emotional connection, ask some appropriate questions, or answer questions from the "nurses." SSPs used a checklist to record the details of the encounter for assessing student performance. Students provided care for the "patient" in two simulated environments. Station I (nursing assessment, 10 min) was for admitting the patient to the emergency department and conducting an assessment specific to patient conditions. Station II (patient instruction, 10 min) was for explaining the conditions and informing the precautions. Students were expected to assess patient conditions and recognize the abnormal signs and symptoms; meanwhile they should have dealt with the patient's conditions and complaints, such as pain, breathlessness, vomiting, nausea, cough, and thirst. After reviewing the data from assessment and laboratory examination, students defined the problems which required nursing care and thereafter explained the conditions and informed the precautions to the "patient". Finally, the debriefing was conducted following each scenario in terms of safe practice, priority setting, continuous assessment, and communication in the simulation laboratory. An example of simulated scenarios for OSCE is shown in Figure 1. The situation took place in a simulated emergency room. Mrs. Hong, a 25-year old female, was diagnosed with myocarditis and admitted to the emergency department for chest pain.

Nursing assessment OSCE checklist (NAOC)
The evaluation indicators of nursing assessment in OSCE were grounded in a preliminary Delphi study which was 2 rounds of consultation with 20 nursing experts. The expert authority coefficient was 0.87. The coordination coefficient was 0.613. Based on those indicators, the NAOC was developed by the research team to assess students' nursing assessment skills in terms of patient identification, history inquiry, symptom assessment, physical examination, and humanistic care. It was a 22-item observational measurement with a three-point Likert scale (2 = Achieved, 1 = Partly achieved, 0 = Not completed/Incorrect). A higher score indicated a better performance in nursing assessment.
Content validity (CVI) of the NAOC was established using 3 experienced nurse educators who were the experts in simulation, SP, and OSCE for baccalaureate nursing students. When the level of inter-rater agreement was reported as 0.99, the CVI value was reported as 0.98. The parameter of internal consistency reliability, Cronbach's alpha, was 0.839. The inter-rater reliability should be established for minimizing the possibility that the scores would vary from rater to rater. The inter-rate reliability by two raters was 0.834.  OSCE Station I Health assessment OSCE introduction SSP • "After I got wet last week, I had a sore throat and cough." • "The sputum was white at first and became yellow after three days. " • "I had a fever about 37.8-38.5°C. After I took antipyretic medicine, it dropped to 37.5-37.8°C., and then increased again." • " I was too busy at work to see a doctor at that time." • " I felt weak and dizzy." • "I felt uncomfortable on the left arm and in the middle of the stomach." • "I felt stuffy in my chest, it was like a stone being pressed." • "I felt better after taking some rest." • "I had vomiting for two days, 5-6 times in total, a little bit at a time." • "I defecated 3-4 times a day with t hin stools."

Communication evaluation rubric (CER)
Based on communication theory and principles of therapeutic communication, the CER was developed by the research team to assess therapeutic communication when students conducted nursing assessment and patient education exercises (Table 2). It was a 14-item observational measurement with a three-point Likert scale (2 = Achieved, 1 = Partly achieved, 0 = Not completed/Incorrect). The higher score indicated better communication between students and SSPs.
The CVI of the checklist was established using 3 experienced nurse educators who were considered experts in conducting simulation teaching, SP, and OSCE evaluation for baccalaureate-level nursing students. When the level of inter-rater agreement was reported as 1.00, the CVI of the CER was reported as 1.00. The parameter of internal consistency reliability, Cronbach's alpha, was 0.896. The inter-rate reliability by 2 raters was 0.942.

Data collection and analysis
The audiovisual system was used to record the interactions among participants, the SimMan and SSPs. Each station was videotaped to enable the examiners to make the recording objectively during the OSCEs and to provide detailed feedback to the participants after OSCE. The NAOC and the CER were used by 2 raters to assess participants' performance on nursing Respond appropriately to the patient's doubts and express concern towards the patient's understanding Ask the patient's doubts, but do not concern the patient's understanding Neither ask patient's doubts nor concern patient's understanding The provided information is clear and understandable Most of the information provided is clear, but some information is difficult to be understood The explanation was confusing, and consequently the patient is unable to understand it  Descriptive analyses were performed to identify the characteristics of participants, to generate descriptive statistics on the Likert-type data, and to describe the central tendency and dispersion for the scores on the NAOC and the CER. Independent samples' t-test was carried out to compare the mean scores of the NAOC and the CER between male and female students, year 2 and year 3 students, and with or without the experience of simulations or SP interactions. The degree of difficulty was calculated by dividing the mean score by the highest score. The difficulty factor was the exact opposite of the actual difficulty of the item. The greater the difficulty coefficient, the more the number of students who completed the item correctly, thereby implying that the easier the item was for students to answer. It was generally recommended that moderate difficulty could more objectively reflect the learning effects of students and should be distributed between 0.30 and 0.70. Furthermore, the degree of discriminability reflected the ability of items to distinguish students' actual level. It was analyzed by the Pearson correlation coefficient (r-value) between the score of each item and the total score. It was defined as good discriminability (0.40 ≤ r ≤ 1.00), better discriminability (0.20 < r < 0.40), and poor discriminability (0.00 ≤ r ≤ 0.20). 17

Results
The overall item mean score of nursing assessment was from 0.55 to 1.95 with the mean of 1.26 (SD 0.36). Students received high scores in conducting blood oxygen saturation and cardiac and lung auscultation but low scores in vomiting and diarrhea assessment. Difficulty coefficients of the exam were 0.63 while the discrimination index of the majority items of the NAOC (86.4%) was >0.20, showing a better to good discriminability ( Table 3).
As shown in Table 4, the overall item mean score of communication was from 0.43 to 1.93 with the mean of 1.12 (SD 0.43). Students presented good communication skills in maintaining eye contact and listening, but culture assessment and explanation for visiting need to be improved. Difficulty coefficients of the exam were 0.56 for communication skills. The items of the CER had satisfactory indexes of item discrimination (from 0.38 to 0.84).
There were no differences based on gender, but year 3 students with a 30 week clinical practice (CP) had better assessment and communication skills than the year 2 student with 16 week-CP. The students who had experiences in simulations or interactions with SPs presented better nursing assessment and communication skills than the students without those experiences (Table 5). Nursing assessments were positively related to communication (r = 0.67, P = 0.000).

Discussion
Nursing teachers should continuously look for teaching methods to ensure that students are prepared to provide safe and competent nursing care. 18 SSPs accurately portrayed the patients as encountered in the real clinical situation. Using SSPs was definitely helpful and realistic for promoting clinical reasoning, critical thinking, and communication, all of which are essential skills for nursing assessment. This study showed that the students with experiences in simulation or SP interactions had better assessment and communication skills than the students without those experiences. A previous study reported that practical patient simulations offered the opportunity for students to practice the implementation of these skills with no risk to real patients. 19 Students engaged in role-playing to analyze real-life scenarios and practice communicating with SPs. This might provide them with the contextualization of information, which enhanced their learning experiences. Students viewed SP scenarios as being more useful than roleplaying when learning communication skills. The specified interactions with SPs emphasized the importance of communication skills for ensuring safe patient care. 20 The SP simulated experience increased the perceived learning in the nursing therapeutic communication process. The practical experience of communicating with an SP allowed students to ask and discuss communication skills and identify areas for improvement during the debriefing. 21 The application of SPs in nursing education enhanced communication skills, improved student self-confidence, promoted critical thinking, and helped students to overcome nervousness and anxiety in skill examinations. 22 This study showed a positive relationship between nursing assessment and communication skills. As reported in previous studies, students preferred using SSPs for nursing assessment examinations because the experience was more realistic and also because it prepared them for doing physical assessments in real clinical settings. 21 Students repeatedly played a role in SP interaction, thereby ensuring that they mastered the technology learned from their own experience. Seeing the peers' demonstrations on communication skills also provided these students with the necessary opportunities to reflect on their own communication skills. Comprehensive and meaningful reflections from SSPs can facilitate the development of psychomotor and communication skills. Feedback from peers and facilitators during the debriefing reduced their anxiety and meanwhile  enhanced their self-confidence towards the task of communicating with patients. Students who interacted with SPs performed better on their performance evaluations as compared with students who practiced on peers. 21 In this study, students performed well in patient identification, blood oxygen saturation checking, heart and lung auscultation, and nonverbal communication behaviors (especially initiating and maintaining eye contact and listening). This may be attributed to the strengthening of knowledge and skill training in classroom teaching and CP. Students paid more attention to the assessment of chest discomfort but ignored the assessment of gastrointestinal reactions related to it, and neglected cultural assessment, the explanation for visiting reasons, and information reconfirmation. It was necessary to foster the nursing students' cultural awareness and to train students to assess the patients' feelings and ideas through a systematic clinical thinking mode in which the focus is on both the main clinical manifestations and related clinical manifestations. 23 Besides, nursing students had better communication skills through developing eye contac, listening, and empathy. Empathy denoted an understanding of the patient's perception and feelings. Nursing students presented a sense of duty to the "patient" and empathy and advocacy for patients. Students integrated   the intellectual understanding of empathy with their concrete behaviors to ensure that empathetic communication with other persons would become habitually ingrained behaviour. Such empathy and advocacy were noted to be essential for establishing rapport and safe patient care. 24 Students should demonstrate genuineness and honesty for facilitating therapeutic communication. They should not be proficient merely in using open-ended and closed-ended questions, but also imperatively develop the nonverbal communication skills necessary in clarifying, interpreting, validating, and paraphrasing the information provided by the "patient." 25 SPs provided the opportunity to engage the learner in active learning through the provision of a realistic scenario with a human interaction experience. SPs were useful in developing empathic skills, which would in turn facilitate nursing students to successfully implement therapeutic communication. 26 Students were expected to interact with SSPs and high-fidelity simulations as they would with the patients encountered in real clinical situations. The simulation experience provided an avenue for students to implement therapeutic communication within a controlled environment. 27 As consistent and measurable evaluation methods, SSP-based simulations were used to assess students not only on the basis of objectively tested assessment and communication skills, but also on the basis of the intentionally inculcated critical-thinking and problem-solving skills. Working and being tested in SSPs improved the clinical skills, communication, and teamwork-abilities of students. As a formative assessment method, the collaborative testing method can help students reflect on their peers' progressaccording to specific criteria-when preparing for a summary assessment. Using standardized clinical simulated scenarios in OSCE was essential to conduct the direct comparison of the students' assessment skills and communication skills. 28 The OSCE was designed as a structured checklist that facilitated the objective assessment. The trained SSPs presented the patient's conditions and feelings reliably and consistently. OSCE standardized checklists were highly content-specific and provided objective ratings for measuring a range of important skills which were regarded to be necessary for nursing practice, including assessment, empathy, questioning, and initiating and maintaining eye contact and wording expressions in interactions with SPs. 29 Video recording was reported to be a reliable testing alternative that offered fairness in grading, evaluation, and debriefings.

Conclusions
SSPs were involved for enhancing the realism of interactions in scenarios. Nursing students can conduct nursing assessments specific to patient conditions, explain the relevant conditions to the patient, and inform the patient the precautions required to be followed. However, students' cultural awareness and some communication skills need further improvement through training. It was worth noting that two assessment tools were tested for psychometric properties. With moderate difficulty and high discrimination index, OSCE showed satisfactory reliability and validity.

Limitations
A small convenience sample of students from one nursing school was used; for this reason, the findings were limited in their usefulness to be extrapolated to larger populations and the similar demographic profiles resulted in the difficulty in generalization of findings in other nursing programs. The assessment tools need to be further tested to verify their validities and reliabilities, especially when they are used in different contexts or among students with different cultural backgrounds in SSP-based simulations. A larger sample size and additional undergraduate nursing programs may contribute to the genesis of a more robust data set and are expected verify the findings to a greater extent. Moreover, student stress and anxiety in OSCEs also need to be explored in further studies.

Ethics approval
Ethics approval was granted from the research committee and administration council of the institute. Participations were voluntary. Confidentiality of data was maintained. Only aggregate data was reported.

Conflicts of interest
The author declares that there exist no conflicts of interest.