Running head: EFFECT OF SIMULATION Effect of Simulation on Interprofessional Communication and Collaboration for Nurses in China Mary T. McFadden Arizona State University 1 2 EFFECT OF SIMULATION Abstract Nurses working in China do not have access to hospital data, access to professional organizations, or to the internet for reviewing evidence-based practice (EBP). Chinese healthcare organizational leaders are seeking international support to provide nurse leaders with necessary skills to lead China based organizations in safe, quality, healthcare delivery. With the opening of a new hospital in Yinchuan, China, it is imperative to ensure that a climate of collaboration, teamwork, and clear communication methods exist between nurses, doctors, and other interprofessional staff members. Evidence indicates that use of simulation with standardized communication tools and processes (use of Situation-Background- AssessmentRecommendation [SBAR], TeamSTEPPS, and checklists) can facilitate interprofessional collaboration and teamwork and improve communication among interprofessional staff. Designing effective simulation scenarios with sensitivity to Chinese culture, with an interprofessional staff will enhance quality and patient safety in Chinese hospitals. Keywords: Chinese nurses, interprofessional, simulation, collaboration, communication 3 EFFECT OF SIMULATION Problem Statement There is a significant amount of literature related to patient safety, communication, and the role that effective teams play in improving patient safety. Ineffective communication with health care professional teams is noted to be one of the leading causes of medical errors and patient harm (Dept. of Defense & Agency for Healthcare Research & Quality [AHRQ], (2018); Donaldson (2008); Institute of Medicine Committee on Quality of Health Care in America (2001); Joint Commission International [JCI], (2017); Leonard, Graham, & Bonocum, 2004). Timely evidence suggests that there are many barriers to effective communication. Healthcare complexity, distraction due to managing multiple priorities, and lack of a coordinated structure and standardization are but a few (Foronda, MacWilliams, & McArthur, 2016). Organizational structure and hierarchy shift or schedule changes, and multiple modalities used to communicate create additional complexity (Kostoff, Burkhardt, Winter, & Shrader, 2016). Research has also demonstrated that Chinese nurses face more barriers then those in the Western world (Cheng & Yu, 2017; Wang, Jiang, Wang, Wang, & Bai, 2013). Lack of authority, time, and language were noted as the top three barriers. Blurred boundaries related to managerial requirements, peer influence, administrative power and influence from physicians, could be facilitator or a barrier (Cheng & Yu, 2017). Purpose and Rationale With the opening of the new hospital in Yinchuan, China, it will be imperative to ensure a climate of collaboration, teamwork, and clear communication methods exist between nurses, doctors, and other interprofessional staff members. Research completed by Foronda et al. (2016) indicated that standardized communication tools and processes (use of Situation Background Assessment Recommendation [SBAR], checklists, and/or 4 EFFECT OF SIMULATION simulations) can facilitate interprofessional collaboration and teamwork and improve communication among interprofessional staff. Background and Significance Nursing and Evidenced-Based Practice (EBP) in China The World Health Organization (WHO) connects the importance of global collaborative practice in engaging an interprofessional team by reinforcing the positive quality outcomes that stakeholders (patients, families, caregivers, and communities) can receive when collaboration occurs (Poore, Dawson, Dunbar & Parrish, 2019). Literature suggests that all healthcare organizations need to strive to improve consistency in quality and patient safety through the rigor of EBP (Cheng, Feng, Yu & Broom, 2018; Melnyk & Fineout-Overholt, 2014). Despite efforts to engage professional nurses in EBP, continuous efforts are necessary (Greenhalgh et al., 2014; International Council of Nurses [ICN], 2012; World Health Organization [WHO], 2011). EBP nursing was introduced into China during 2001, yet there have only been three published papers; two carried out in a Chinese general hospital, and one in a traditional Chinese hospital (Zhou et al., 2015). Barriers to initiating EBP in China includes lack of evidence-based knowledge, lack of information and data, and administrative support (Cheng & Yu, 2017; Normille, 2017). The number of Chinese registered nurses has reached 3.24 million, with 62.5 % of these nurses holding higher than an associate degree (Cheng & Yu, 2017). With this number of nurses, China is well primed for use of EBP in nursing and healthcare. Nurses in Yinchuan, China work in a hierarchical system, where harmony is sought over managing conflict, and where the culture supports obedience to leaders of authority (Holroyd, Wai-wan, Yue-kuen, Sauwai, & Fung-shan, 2003). A published literature review reported a number of articles (n=95) related to EBP in China. The review found that in Chinese nursing practice, strategies and EFFECT OF SIMULATION 5 barriers related to EBP resulted from lack of leadership at the organizational level (Cheng, Feng, Hu, & Broome, 2018). The study indicated that when top leaders embraced EBP, frontline nurses would be more likely to follow new protocol. Simulation, Communication, Collaboration and Patient Safety Improving communication, collaboration, and teamwork among healthcare professionals is a key focus area for the Inter-Professional Education Collaborative [IPE], (Interprofessional Education Collaborative, 2016; Klipfel et al., 2014). It is noted that this education fosters learning with each other and about each other (Kostoff et al., 2016). Mariani & Doolan (2016) suggest simulation in healthcare is an exceptional method for teaching interprofessional skill building. A prospective, pre-post comparative cross-sectional pilot study was completed in a Chinese medical center, to evaluate improvement in interdisciplinary team attitudes (Yang et. al, 2017). The study included 34 physicians, 30 nurses, and 24 pharmacists who participated in simulation courses. The study concluded that IPE using simulation enhanced participants’ IPE attitudes, self-reflection, work-place transfer, and practice of learned skills. An integrative review on interprofessional communication by Foronda et al. (2016) was comprised of 26 research studies, six papers, and one theoretical framework. Their research concluded that interprofessional communication, collaboration, and teamwork, can be significantly improved with the use of evidence-based standardized tools such as simulation or the creation of a consistent use of a standardized communication tool’s. Key themes from Foronda’s integrative review include: awareness of communication styles due to professional training; impact of hierarchical, historical subservient role of nurses; simulation, and/or use of an SBAR for structured communication, may warrant consideration as an IPE gold standard; and culture and diversity in collaboration and 6 EFFECT OF SIMULATION teambuilding are an important component of developing successful models (Foronda, et al., 2016). Another study conducted by 96 pharmacy students and 96 BSN students provided support for the use of SBAR (Kostoff et. al., 2016). Pharmacy and nursing students collaborated on multiple patient cases using the SBAR communication tool. Using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), they rated satisfaction with the tool at a mean of 4.2. Additional studies have been done in situ utilizing TeamSteppsR (an evidencebased teambuilding model) as a method to promote teamwork, communication, and patient safety (Klipfel et al., 2014). A quality improvement project was initiated with the use of two instruments using the Mayo High Performance teamwork scale (Klipfel et al., 2014). A team of nurses and physician residents used the Plan Do Check Act (PDCA) process improvement method, to test three simulated medical emergencies (Klipfel et al., 2014). They used a regular clinical setting, with simulation, briefing, scenarios, and debriefing structures. Each of these structures have distinct communication processes within them. Overall results included improved confidence in the team, more trust, superb interactions with the team. Recommendations included the use of the same process for all nursing, residents, and allied staff. Internal Evidence Yinchuan Goulong Hospital, is located in Yinchuan, Northwest China. It is home to 1,290,170 people spread between three urban districts (Yinchuan, 2020). Two private, specialized Yinchuan Hospitals were opened in 1995; these hospitals consolidated into one, new Yinchuan Goulong hospital with 600-beds during the Fall, 2019. The hospital is recognized for specialty in Orthopedics and Women and Children’s services. Approximately 700 new nurses were hired at the new Yinchuan Hospital, and more are expected to be hired over the next year. The patients, nurses and associated interprofessional staff will be affected by many 7 EFFECT OF SIMULATION system changes and will require astute communication, teamwork and collaborative practice to ensure patient safety. In the previously existing hospitals, physicians, nurses, and ancillary teams worked in silos. There was a lack of interprofessional teamwork as noted by a request from the Chinese nurses. Physician and leadership hierarchy was evident. This led to a fragmented communication infrastructure, which can lead to medical errors. The Vice-President/Chief Nursing Officer (VP/CNO) of the PreferUS consulting group developed a simple internal survey, asking 344 nurses at Goulong Hospital what type of training or education would be of most interest to them. Two-hundred thirty-three (68%) responses were received. The survey was narrowed to the top three learning interests which were, Interprofessional Teamwork/Collaboration, Infection Control, and Cardio-Pulmonary Emergency Response (Code Blue). With the new hospital opening, it is imperative that nursing staff are able to collaborate and communicate with a large interprofessional staff. The literature strongly supports EBP tools to improve collaboration, teamwork, and communication to promote patient safety. Common themes supported by the evidence include utilizing EBP structures for consistency (Foronda et al., 2016). Examples of EBP structures include use of SBAR, TeamSTEPPS, debriefing, and simulation. This review and inquiry has led to the clinically relevant PICOT question: for nurses working in an acute care hospitals in Yinchuan China (P), how does actively participating in an interprofessional hospital orientation (role-play) simulation (I) compared to traditional orientation (lecture) (C) affect perceived teamwork, communication, and collaboration (O)? Search Strategy Five databases were extensively searched to develop a systematic review for the EFFECT OF SIMULATION 8 PICOT question related to nurses and interprofessional teams, use of simulation, and effect on communication, collaboration, and teamwork. Databases searched included the Cumulative Index to Nursing and Allied Health Literature (CINAHL), PubMed, Cochrane Library, the Grey Literature-Health Sciences Online (HSO), and Agency for Healthcare Research and Quality (AHRQ). Initial keywords searched included combinations that addressed the PICO: nurses, Chinese nurses, interprofessional, interdisciplinary, simulation, communication, and improvement. As the PICOT was modified, an expanded search included the terms collaboration, teamwork, employee and orientation. The term Chinese nurses was later removed due to limited numbers of articles. Search modes using MESH and Boolean/Phrases included use of AND/OR to expand options. Applied filters included use of the English language, peer-reviewed and evidence-based journals, from 2013-2018. Reference lists were abundant and scanned for a review of additional relevant articles. Overall, 30 articles were pulled from this extensive list for further review. CINAHL Advanced Search Yield The initial review yielded a total of 100 results; using keywords Chinese nurses, nurses, interprofessional, simulation, communication, and improvement. A further search excluded the term Chinese nurses, added the terms role play, collaboration, teamwork, and education. By adding these keywords with the same filters, there were 2,438 results (role playing: 2,274; collaboration: 26; and education: 138). PubMed Advanced Search Yield The initial review yielded 106 results with use of the terms nurses, simulation, interprofessional, and communication. The term Chinese nurses was excluded. Two additional search terms were added, employee and orientation, for one result. EFFECT OF SIMULATION 9 Cochrane Library Database Yield One review was completed utilizing the terms interprofessional, simulation, communication, and improvement. Results yielded 4 reviews and 192 trials. The four articles scanned were not relevant to the PICOT. Grey Literature: Health Services Online (HSO) Key terms included interprofessional, simulation, collaboration, teamwork, communication, and improvement. There were 199 results; 20 were available focused on high-fidelity simulation; 11 for In-situ simulation, and 9 for collaborative practice. Agency for Healthcare Quality and Research (AHRQ) This search yielded a wide range of results from a low of 805 to a high of 36,079. Key words for the lowest yield utilized lesser terms of teamwork, collaboration, and simulation. The highest yield searched used the key words nurses, interprofessional, simulation, communication, and collaboration. A librarian e-mail referral was sought out due to the high volume of results. The librarian reinforced that the AHRQ filters are not as robust. Search terms were reduced to simulation and orientation with a yield of 162. In conclusion, after a review of all citations using rapid critical appraisal, 30 studies were carefully scanned; 10 studies were retained for inclusion in an evidence table, based on a combination of qualitative, quantitative, and one systematic review. Critical Appraisal and Synthesis of Evidence Ten studies were selected for this mixed method review (Appendix A, Table 1 and 2). This included five qualitative descriptive studies (QDS), four quantitative studies (integrative review, psychometric exploratory study, experimental design, comparative study), and one systematic review. Six of the studies demonstrate level VI evidence, three demonstrate level II evidence, and one study represents level I evidence (Melnyk & Fineout-Overholt, EFFECT OF SIMULATION 10 2015). Grounded theory was utilized throughout the QDS to explore the complexity involved, guiding data collection by constant comparative methods, ensuring data was accurately coded. Because qualitative data is sensitive to evaluator bias, the researchers in these studies created structures to minimize this. Three of the four quantitative studies utilized survey instruments which demonstrated reliability through Cronbach’s alpha index of internal consistency (alpha >.75); validity was tested using confirmatory and exploratory factor analysis. Sample populations were fairly homogeneous: smaller sample sizes, utilization of RN’s and/or interprofessional staff, with the studies located in academic medical centers, universities, and classrooms with simulation sites. There was one outlier sample size related to a survey sent out to 2200 student nurses developed to research the reliability and validity of the Simulation Design Scale [SDS] (Appendix A, Table 2, Franklin). Studies located in China vs. the United States are outliers yet relevant to the PICOT (Appendix A, Table 1 through Table 3). Nurse participants were educated as BSN students, BSN graduates, MSN, and /or PhD level. Most of the studies included interprofessional staff as a component of the population. Length of time for nurses who have worked in their role range from 0-29 years. Major variables of interest relate to interventional outcomes. Interventions included interprofessional simulation-based education (IPSE), other structured communication interventions (Situation, Background, Assessment, Recommendation [SBAR]), training to TeamSTEPPS, and factors that influence adoption of Evidence-Based Practice (EBP) for nurses working in Chinese hospitals. The Kouzes & Posner Transformational Leadership Model was utilized well as a framework for nurse managers implementing new EBP scenarios in China (Kouzes & Posner, 2007). Mariani & Doolan (2016) [Appendix A, Table 1] provided an excellent overview on the current state of simulation research noting gaps for 11 EFFECT OF SIMULATION participants in psychological safety and research rigor in relation to evaluation methods, longitudinal studies, multisite simulation studies, and academic-service partnerships. Heterogenous measurement instruments were utilized across QDS utilizing internetbased surveys, tape recorded and transcribed interviews and focus groups. Qualitative studies were tested for rigor and integrity using the criteria for credibility, dependability, confirmability, and transferability (Reavy, 2016). Rigor in data collection was solid, and included audit trails, confirmability, triangulation, and multiple-rators (Reavy, 2016). Utilizing a variety of data collection methods ensures a rigorous and productive analytic process (Maher, Hadfield, & Hutchins, & Eyto (2018). Several survey instruments were utilized for the quantitative studies: the simulation design scale (SDS), the Interprofessional Education Collaborative (IPEC) competency instrument, the Interprofessional Attitudes Survey (IPAS), and the Medical Education Research Quality Instrument (MERSQ). Of major interest is the SDS due to its widescale use for measuring beliefs and attitudes about simulation learnings (Franklin, Burns, & Lee, 2014). The scale assesses 20 items in six categories: perceptions of objectives, information, support, problem solving, feedback, and fidelity in simulation (Franklin et al., 2014). Responses are categorized using a Likert type scale. Franklin et al., 2014 reported strong reliability and validity based on Cronbach’s alpha and confirmatory and exploratory factor analysis (Appendix A, Table 2, Franklin). Conclusions from the Evidence Strong evidence indicated that the use of structured communication processes and use of simulation, lead to better understanding of professional roles, increased participant confidence, and knowledge. There is a strong theme around the significance of hierarchy, power distance, the role of physician’s and managers, in managing culture. The Kouzes & Posner Transformational 12 EFFECT OF SIMULATION Leadership Model provides an exceptional framework for implementation for leading these EBP modules in China (Kouzes & Posner, 2007). Supporting data also suggested a need for more simulation research in the areas of psychological safety, research rigor in relation to evaluation methods, longitudinal studies, multisite simulation studies, and academic-service partnerships. Theory Application The effectiveness of simulation has been well documented (Foronda, 2016; Mariani, 2016). However, questions that need to be addressed with simulation practice include: What simulation learning practices lead to a positive outcome; what is the role of the simulation educator; and how does the design of the simulation experience contribute to learning (Jeffries, 2005)? The NLN Jeffries Simulation theory can serve to help guide, design, implement, and evaluate teaching methods (Jeffries, 2005; Jeffries, Rodgers, & Adamson, 2015; LaFond & Van Hulle Vincent, 2012). The Jeffries Simulation Framework was developed during 2005, and since that time has evolved into the NLN Jeffries Simulation Theory (Jeffries, Rodgers, & Adamson, 2015). The NLN Jeffries Simulation Theory (Appendix B, Figure 1) consists of context, background, simulation design, experience, facilitator strategies and participant attributes. It concludes with outcomes based on system, patient, or participant need (Jeffries, Rodgers, & Adamson, 2015). Briefly, context portrays the overarching purpose; background provides the goals and necessary resources needed to inform the design; the design includes elements of fidelity, participant and observer roles-progression of activities and briefing/debriefing methods (Jeffries, Rodgers, & Adamson, 2015). Simulation experience requires an environment facilitating trust between the facilitator and participant, to promote psychological safety within the experience. Facilitator responsiveness is required to help adjust for learner needs that affect the participant experience. 13 EFFECT OF SIMULATION Finally, the outcomes are demonstrated in a triangular form based on hierarchy for the system, patient, and participant (Jeffries, Rodgers, & Adamson, 2015). The majority of studies used simulation as an intervention, combined with use of SBAR, Simulated Interprofessional Bedside Rounds (SBIR), and TeamSTEPPS. Evidence Based Practice Model The Rosswurm & Larrabee Model for Evidence-Based Practice Change (Appendix B, Figure 2) was selected for this project (Rosswurm & Larrabee, 1999). This model was chosen for its simplicity, and is based on grounded literature related to EBP and change theory, with a focus on culture change (Reavy, 2016; Rosswurm & Larrabee, 1999). It is designed for guiding multiple practice changes and will assist nurse leaders in diffusing updated EBP changes in a Chinese culture (Melnyk & Fineout-Overholt, 2014). The model also integrates well with the Kouzes & Posner Transformational Leadership Model, which will be utilized to assist nurse Managers in China in adapting EBP changes (Kouzes & Posner, 2007). The Rosswurm & Larabee Model for EBP Change model has six components: 1). Assess the need for change in practice; 2). Locate the best evidence; 3). Critically analyze the evidence; 4). Design practice change; 5). Implement and evaluate change in practice; and 6). Integrate and maintain change in practice. Implications for Practice Change The Rosswurm & Larrabee Model for EBP Change can be easily applied to the simulation design project. In sharing this model with the PreferUS CNO and collaborating on each essential step, five of the six components were adapted, and a plan for the sixth component is in place. An example follows: We assessed the need for change through surveying the frontline nurses on their perceived priority needs. Based on this internal data, we then focused on one simulation with a targeted group of Chinese nurse managers related to 14 EFFECT OF SIMULATION interprofessional communication. I then located the best evidence and critically analyzed the evidence- completed through an extensive literature review and critical appraisal related to interprofessional communication. This review provided clear evidence on the use and benefits of simulation, informing key elements of the process. We designed the simulation scenarios with interprofessional teams. The designed simulation was based on a new workflow for opening a new hospital and included use of SBAR. A post-survey was distributed within 12 months of the simulation experience, to assess nurse perception related to improvement in interprofessional communication and collaboration utilizing the NLN Jeffries Simulation Design Participant Evaluation (SDS) survey tool and the Socrative electronic platform (Jeffries, 2005; Socrative.com, n.d.). The evidence, once summarized and completed, will be sent to the PreferUS CNO to inform the future Goulong Hospital nursing strategic plan related to a sustained practice change. Methods Ethical Considerations and Human Subject Protection A protocol entitled “Using simulation to facilitate interprofessional collaboration in Yinchuan China” was submitted and approved through the Arizona State Institutional Review Board (IRB). This process was initiated to ensure all participants are treated in an equal and ethical manner, and that their rights are protected (“Research integrity and assurance,” n.d.). The IRB study number, STUDY00010468, is considered to be exempt pursuant to Federal Regulation 45CFR46 (shown in Appendix C). Population and Setting Participants included Chinese nurses and physicians who participated in the simulationrole play component of the new hospital orientation between Aug. 28, 2018-Sept. 30, 2019. Simulation/role play was a required component of the physician and nurse orientation. EFFECT OF SIMULATION 15 More than 700 nurses and an unknown number of practicing physicians attended. Minors, adults unable to consent, prisoners, native Americans, undocumented and administrative staff were excluded. Project Description and Timeline The purpose of this study was to determine if there was improvement in interprofessional collaboration and teamwork after implementing a scenario-based simulation. The simulation included nurses, physicians, and interprofessional staff, based on new workflow through a new hospital, utilizing tools reinforced through EBP. Nurses and physicians were asked to complete the NLN Jeffries Post Simulation SDS, to determine if they perceive improved interprofessional communication, collaboration and teamwork. New hospital orientation occurred from Aug. 31, 2018-Sept 30, 2019. The survey was distributed from October 27-Nov. 19, 2019. Instrumentation, Data Collection, and Data Analysis Plan The NLN Jeffries Post Simulation Participant Evaluation Survey/Simulation Design Scale (SDS) [Appendix D] was utilized to assess improvement in interprofessional communication and teamwork. The Chinese translated survey utilizes a 31 question Likert scale ranging from 1 (strongly agree) to 6 (strongly disagree). Questions one through 20 are original questions from the NLN Jeffries SDS; five of the 31 questions (2125), were developed to measure improvement in interprofessional collaboration and teamwork through a teamwork composite. These questions were sent to the NLN prior to distribution and permission was granted to add these five questions to this specific survey. Questions 26-31 were added by the DNP student to assess and compute population demographics. EFFECT OF SIMULATION 16 The SDS original 20 questions have been tested for validity and reliability. Jeffreys & Rizzolo (2006) reported that “content validity was established through expert review and reliability was tested through using Cronbach’s alpha with reported values of .90 and .92 for the presence of simulation design characteristics and .95 and .96 for importance” (LaFond & Van Hulle Vincent, 2012, p.469). Questions 21-25 represent a Team Composite, and were tested in the DNP survey evaluation, discussed in Statistical Summary. LaFond & Van Hulle Vincent (2012) reported that the SDS has been utilized to measure learner perception of five variables in the concept of simulation design characteristics: objectives, fidelity, problem-solving, student support, and debriefing. A project introductory/recruitment letter was translated and back translated by certified Chinese nurse interpreters and was sent to all participants meeting inclusion criteria. Participation was voluntary and took no longer then 25 minutes to complete. The survey was presented and answered via computer entry. Responses have been statistically analyzed utilizing the Socrative platform (Socrative, 2020), an Excel spreadsheet, Intellectus software (Intellectus Statistics, 2020), descriptive and nonparametric statistics. Budget and Funding The funding for this project was received from the CEO at Goulong Hospital, Yinchuan, China. He provided the DNP student with travel, housing, and meal expenses during the twoweek simulation development process in 2018. He also supported the orientation and participation of all nurses, physicians, and ancillary staff, providing funding, time out, and replacement teams, to attend training sessions. Each participant attended an 8-hour simulation training session which was built into the new hospital budget performa. An estimate of project cost is provided based on the average China salaries in various roles, including RN’s (Salary Explorer, 2020). 17 EFFECT OF SIMULATION Overall cost for DNP project two-week simulation design and orientation of employees to their new hospital environment over one year was approximately $111, 023.00 (US dollars). This project cost encompasses human capital, project development, orientation of employees to the new hospital-simulation role-play, supplies, DNP student travel, lodging, and meals. An Estimated DNP Project Budget is shown in Appendix A ,Table 4. Project Results Descriptive statistics were utilized to describe the sample and outcome data. In addition, Two-tailed Independent Sample t-Tests were performed to examine mean differences in demographics and ANOVA tests were performed to determine significant differences in the Teamwork Composite variables. Data Analysis and Outcomes Demographic variable frequency distributions. Frequency Distributions were utilized to describe and display the sample and summarize the demographic variables from the survey; representing the frequency and percentage of each variable (Cronk, 2014). Seven hundred nurses and an unknown number of physicians and interdisciplinary healthcare personnel received the survey; 327 (46.7%) of the sample (n=327) responded. Nominal descriptive data included gender, marital status, age, years of work experience, current professional discipline, and highest degree. Frequencies and percentages were calculated for each nominal variable and are shown in Appendix A, Table 5: Frequency Table for Nominal Variables/Demographics. Females represented 96% of the sample; single individuals versus married, represented 65%; 79% of respondents were between 20-29 and 72% have worked less than five years. Seventy-two percent of the respondents were RN’s; and 61% of these RN’s held an EFFECT OF SIMULATION 18 associate degree. Physicians (MD) and Physician Assistant’s (PA) comprised 5.2%. of the sample. Simulation survey outcome variables. A 6-point Likert scale was utilized to measure a level of agreement or disagreement with the use of simulation to improve interprofessional collaboration and teamwork. The 6-point Likert scale consisted of the following responses: 1). Strongly agree (SA); 2). Agree (A); 3). Not Applicable (NA); 4). Undecided (U); 5). Disagree (D); or 6). Strongly disagree (SD). Five of these questions (2125) reflect a teamwork composite and are shown in Appendix A, Table 6: Teamwork Composite. Teamwork composite results: Frequencies and percentages. Frequencies and percentages were calculated for Questions 21-25 (noted as Q21, Q22, Q23, Q24, and Q25). The most frequently observed category of Q21 was Agree (n = 258, 79%). The most frequently observed category of Q22 was Agree (n = 252, 77%). The most frequently observed category of Q23 was Agree (n = 260, 80%). The most frequently observed category of Q24 was Agree (n = 260, 80%). The most frequently observed category of Q25 was Agree (n = 236, 72%). Frequencies and percentages are shown in Appendix A, Table 7: Frequency Table for Variables Q21-25. Teamwork composite results: Cronbach’s alpha coefficient. Cronbach’s alpha coefficient is a measure to determine reliability and determines internal consistency; to assess how closely related a set of items are (Cronk, 2014). In this case, to assess the teamwork composite scoring for consistency. A Cronbach alpha coefficient was calculated for the demographic 1-5 scale, consisting of Q21, Q22, Q23, and Q24. Question 25 was excluded due to the manner it was interpreted. The Cronbach's alpha coefficient was evaluated using the guidelines suggested by George and Mallery (2016) where > .9 excellent, > .8 good, > .7 EFFECT OF SIMULATION 19 acceptable, > .6 questionable, > .5 poor, and ≤ .5 unacceptable. The items for Q21-Q24 had a Cronbach's alpha coefficient of 0.71, indicating acceptable reliability. Appendix A, Table 8: Reliability Table for Q21-24 presents the results of the reliability analysis. Teamwork composite summary: Statistical results. Summary statistics were calculated for Teamwork composite. The observations for Teamwork composite had an average of 5.12 (SD = 0.39, Min = 3.25, Max = 6.00, Mdn = 5.00). The summary statistics can be found in Appendix A, Table 9: Summary Statistics Table for Interval and Ratio Variables. Teamwork composite: Two-tailed independent sample t-Test (gender). A twotailed independent samples t-test was conducted to examine whether the mean of Teamwork composite was significantly different between the Female and Male categories of “What is your gender?” The result of the two-tailed independent samples t-test was not significant based on an alpha value of 0.05, t(325) = -1.09, p = .277, indicating the null hypothesis cannot be rejected. This finding suggests the mean of Teamwork composite was not significantly different between the Female and Male categories of “What is your gender?” The summary statistic is shown in Appendix A, Table 10: Two-Tailed Independent Samples t-Test for Teamwork composite by “What is your gender?” Teamwork composite: Two-tailed independent sample t-Test (marital status). A twotailed independent samples t-test was conducted to examine whether the mean of Teamwork composite was significantly different between the Married and Single categories What is your marital status? The result of the two-tailed independent samples t-test was not significant based on an alpha value of 0.05, t(325) = 1.55, p = .122, indicating the null hypothesis cannot be rejected. This finding suggests the mean of Teamwork composite was not significantly different between the Married and Single categories of “What is your marital status? “The results EFFECT OF SIMULATION 20 are presented in Appendix A, Table 11: Two-Tailed Independent Samples t-Test for Teamwork composite by “What is your marital status?” Teamwork composite: ANOVA (education level). An analysis of variance (ANOVA) was conducted to determine whether there were significant differences in Teamwork Composite by Education Level. The ANOVA was examined based on an alpha value of 0.05. The results of the ANOVA were not significant, F(2, 316) = 1.83, p = .163, indicating the differences in Teamwork composite among the levels of Education Level were all similar, as shown in Appendix A, Table 12: Analysis of Variance Table for Teamwork composite by Education Level. The main effect, Education Level was not significant, F(2, 316) = 1.83, p = .163, indicating there were no significant differences of Teamwork composite by Education Level. The means and standard deviations are presented in Appendix A, Table 13: Mean, Standard Deviation, and Sample Size for Teamwork composite by Education Level. Teamwork composite: ANOVA (years of experience). An analysis of variance (ANOVA) was conducted to determine whether there were significant differences in Teamwork composite by Years of Experience. The ANOVA was examined based on an alpha value of 0.05. The results of the ANOVA were not significant, F(4, 322) = 0.46, p = .768, indicating the differences in Teamwork composite among the levels of Years of Experience were all similar as shown in Appendix A, Table 14: Analysis of Variance Table for Teamwork Composite by Years of Experience). The main effect, Years of Experience was not significant, F(4, 322) = 0.46, p = .768, indicating there were no significant differences of Teamwork composite by Years of Experience. The means and standard deviations are presented in Appendix, A, Table 15: Mean, Standard Deviation, and Sample Size for Teamwork Composite by Years of Experience EFFECT OF SIMULATION 21 Teamwork composite: ANOVA (profession). An analysis of variance (ANOVA) was conducted to determine whether there were significant differences in Teamwork Composite by Profession. The ANOVA was examined based on an alpha value of 0.05. The results of the ANOVA were significant, F(4, 317) = 2.61, p = .036, indicating there were significant differences in Teamwork composite among the levels of Profession (Appendix A, Table 16: Analysis of Variance Table for Teamwork Composite by Profession). The eta squared was 0.03 indicating Profession explains approximately 3% of the variance in Teamwork composite. The means and standard deviations are presented in Appendix A, Table 17 (Mean, Standard Deviation, and Sample Size for Teamwork Composite by Profession). ANOVA (profession), post-hoc. Paired t-tests were calculated between each pair of measurements to further examine the differences among the variables. Tukey pairwise comparisons were conducted for all significant effects based on an alpha of 0.05. For the main effect of Profession, the mean of Teamwork composite for MD (M = 5.54, SD = 0.43) was significantly larger than for Physician Assistant (M = 5.00, SD = 0.46), p = .041. No other significant effects were found. Summation discussion of results. The results from the Jeffries NLN SDS supports the concepts driven by the initial PICO, strongly suggests three interesting findings from the Chinese nurse and physician sampling, n=327: 1. Previous literature notes 62% of Chinese nurse population hold an associate degree (Cheng & Yu, 2017). The Yinchuan Hospital sample of n=327, represents 61.3% of the nurses who held an associate degree. 2. From Q21 and Q22: 95% of the respondents indicate that they Strongly Agree (17%) or Agree (78%) that the simulation scenario strengthened interdisciplinary EFFECT OF SIMULATION 22 teamwork and collaboration overall and interdisciplinary teamwork between nurses and physicians. 3. An analysis of variance (ANOVA) was conducted to determine whether there were significant differences in Teamwork Composite by Profession. The results of the ANOVA, F(4, 317) = 2.61, p = .036, indicated there were significant differences in Teamwork composite among the levels of Profession (Appendix A, Table 17: Analysis of Variance Table for Teamwork Composite by Profession). The MD and the PA were noted to perceive a strengthening of interdisciplinary teamwork due to the simulation experience. Due to the culture, logistics, and timing of the new hospital opening, it was not practical to do a pre-test to make a comparative analysis for traditional orientation versus interprofessional role/play simulation. However, through the literature review and evidencebased findings, the survey results support and are consistent with previous integrative review studies such as those in Foronda et al. (2016). Their research concluded that interprofessional communication, collaboration, and teamwork, can be significantly improved with the use of evidence-based standardized tools such as simulation. An updated statistically correct version of the PICO may be: What are the trends in the perceptions of teamwork and collaboration for nurses and physician’s working in acute care hospitals after the participation in a role-play/simulation? Simulation project impact. The overall impact of the role-play/simulation on a new hospital opening provided for process improvement and validation of interprofessional competencies. By utilizing role-play/simulation case studies, the participants (nurse, physicians, and some ancillary team members) identified gaps in their current processes that provide for patient safety. The direct observation of the ‘aha’ moment during the simulation/role play EFFECT OF SIMULATION 23 development process was telling. Frontline staff began to routinely use SBAR for communication and exchange of information between disciplines. The patient ultimately receives the benefit from these improvements which can be demonstrated through additional quality improvement indicators. According to an interview with the VP/CNO PreferUs, the new simulation process has provided a process to identify, create, and adapt new policies related to EBP delivery (D. Cato, personal communication, February 17, 2020). The VP/CNO PreferUS indicated that the staff began to adapt a seamless approach to patient care versus a working in silos, considering patient flow from admission to discharge. Sustainment. Factors known to enhance sustainability in organizations include the capacity to embed the innovation or change into day-to-day operations (Davidson, Weberg, Porter-O’Grady, & Malloch, 2017). Video vignettes were produced by the Chinese nursing leadership during the final stages of the roleplay/simulation development. A Chinese nurse with a master’s degree in Nursing (MSN) has been selected to serve as a full-time Simulation Coordinator. In this role she will ensure an integrated and consistent approach to the orientation and training process utilizing simulation, SBAR, and debriefing, for all new employees. The CEO authorized the development of a modern simulation center at the medical center to serve their local three-hospital system. This strategy, supported by primary stakeholders, will ensure ongoing training and education. Due to the infancy stage of the simulation development, additional quality improvement processes should be implemented that provide for ongoing feedback of the learner. This process illustrates the sixth component of the Rosswurm & Larrabee Model for Evidence-Based Change (1999). 24 EFFECT OF SIMULATION Discussion Summary, conclusion, and recommendation The evidence from the literature review strongly suggested that the use of simulation combined with structured communication processes, can lead to improvement in interprofessional communication, team enhancement, better understanding of professional roles, increased participant confidence, knowledge, and patient safety. The results of the 2019 NLN Jeffries Simulation Post-Simulation Evaluation survey completed by nurses and physicians in Yinchuan, China, indicated improvement in interprofessional teamwork and collaboration did occur with a simulated new employee orientation. The recommendation from this EBP scenario, is to utilize simulation/role play as a key intervention for improving interprofessional collaboration and teamwork. Strengths The strengths of this project included a transformative Chinese CEO, who advocated for nursing; he hired a United States RN, as the VP/CNO from Prefer US, who has a solid track record for implementing EBP; and has a passion to work in global settings such as China. He also provided 100% funding for ASU DNP student participation as a preceptee under their VP/CNO for a two-week period during 2018 to prepare for the opening of their new 600bed hospital in Yinchuan, China. To be noted was the high acceptance of US Nurse leadership and enthusiasm of the Chinese nurses and some practicing physician leaders to embrace change and implement EBP protocols, such as SBAR and simulation. Limitations Chinese interpreters translated the English survey. Both translators were certified in English as a second language and fluent in Chinese writing. There could be some misinterpretation of the questions due to cultural barriers. In addition, the survey was scheduled EFFECT OF SIMULATION 25 for Sept. 1-30, 2019; due to the hospital grand opening; the survey process was delayed and occurred for a 3-week, versus 4-week period. A concern, not a limitation related to The Cronbach’s alpha coefficient for Team Composite. Four of the five questions (Q21-Q24) were shown to demonstrate reliability with an α of 71. Question 25, “The simulation scenario helped me recognize that collaboration is not required for all decisions however it can happen spontaneously if the right factors are in place” was excluded from testing, as it was not an effective question. Project Challenges The initial new hospital opening was scheduled for Fall, 2018. This timeframe was just prior to the DNP student arrival August 2018. However, when I arrived, the 600-bed 11-story building was brick and mortar. The process of developing the role-play simulation became a paper exercise, utilizing the two, older hospitals. The new hospital actually opened September 2019, one-year later. The interprofessional scenarios were updated by the Chinese project team and adapted for geography. Evidence-Based Practice challenges traditional norms in a system supporting hierarchy (Holroyd, Wai-wan, Yue-kuen, Sau-wai, & Fung-shan, 2003). Roadblocks occurred when some leaders in the traditional hierarchy did not agree with practice changes. It was advantageous that the CEO of Goulong Hospital, selected key physician leaders to help move through some of the obstacles. Chinese nurses are taught with a “task-based/rules.” An interview with the PreferUs VP/CNO (D. Cato, personal communication, February 17, 2020) indicated that a success factor in overcoming roadblocks require that new evidence-based content needs to be “linked to existing hospital rules and strategies to develop critical thinking and rationalization of practices.” EFFECT OF SIMULATION 26 Recommendations for Further Research This project focus related to assessing the perceptions of improvement in interprofessional teamwork and collaboration post-simulation in a Chinese hospital. Previous studies indicated that additional positive outcomes expected from interprofessional collaboration, include patient safety outcomes. Global studies where simulation has been a key intervention, should include patient safety outcomes and a component for psychological safety. Of key interest, would be those participants from Goulong Hospital, Yinchuan, China. 27 EFFECT OF SIMULATION References American Psychological Association (2009). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author. Baik, D., & Zierler, B. (2019). Clinical nurses’ experiences and perceptions after the implementation of an interprofessional team intervention: A qualitative study. Journal of Clinical Nursing, 28(3–4), 430–443. doi.org/10.111/jocn.14605 Cato, D. L., Walker, K., Aders, D., Liu, F., & McFadden, M. T. (2019). The CNO US healthcare immersion program, Part 1. Nursing Administration Quarterly, 43(1), 40– 49. doi.org/10.1097/naq.0000000000000337 Cheng, L., Feng, S., Hu,Y., & Broome, M.E. (2018). Leadership practices of nurse managers for implementing evidence-based nursing in China. Journal of Nursing Management, 26(6), 671-678. doi: 10.1111/jonm.12594 Cheng, L., & Hu, Y. (2017). Evidence-based nursing implementation in Mainland China: A scoping review. Nursing Outlook, 6(5), 27-35. Cronk, B.C. (2014). How to use SPSS: A step-by-step guide to analysis and interpretation (7th ed). London, ENG: Routledge. Davidson, S., Weberg, D., Porter-O’Grady, T., & Malloch, K. (2017). Leadership for evidencedbased innovation in nursing and health professions. Burlington, MA: Jones & Bartlett. Department of Defense & the Agency for Healthcare Research and Quality (2018). Team strategies and tools to enhance performance and patient safety (TeamSTEPPS). Retrieved from http://www.ahrq.gov/qual/teamstepps/ Donaldson, M. S. (2008). An overview of to err is human: Re-emphasizing the message of patient safety. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. doi.org/NBK2673 [bookaccession] EFFECT OF SIMULATION 28 Foronda, C.L, Alhusen, J., Budhathoki, C., Lamb, M., Tinsley, K., MacWilliams, B., Daniels, J., Baptiste, D.L., Reese, K.K., & Bauman, E. (2015). A mixed-methods, international, multisite study to develop and validate a measure of nurse-physician communication in simulation. Nursing Education Perspectives, 36(6): 383-388. Foronda, C., MacWilliams, B., & McArthur, E. (2016). Interprofessional communication in healthcare: An integrative review. Nurse Education in Practice. doi.org/10.1016/jepr.2016.04.005 Franklin, A.E., Burns, P., & Lee, C.S. (2014). Psychometric testing on the NLN student satisfaction and self-confidence in learning, simulation design scale, and educational practices questionnaire using a sample of pre-licensure novice nurses. Nurse Education Today, 34(10), 1298–1304. doi.org/10.1016/j.nedt.2014.06.011 George, D., & Mallery, P. (2016). IBM SPSS statistics 23 step by step: A simple guide and reference (14th ed.). New York, NY: Routledge. Greenhalgh, T., Howick, J., Maskrey, N., Brassey, J., Burch, D., Burton, M., … Spence, D. (2014, June 13). Evidence based medicine: A movement in crisis? BMJ (Online). BMJ Publishing Group. doi.org/10.1136/bmj.g3725 Holroyd, E., Wai-wan, W., Yue-kuen, C., Sau-wai, C., & Fung-shan, L. (2003). A Chinese cultural perspective of nursing care behaviours in an acute setting. Journal of Advanced Nursing, 28(6), 1289–1294. doi.org/10.1046/j.1365-2648.1998.00849.x International Council of Nurses (ICN). (2012). Closing the gap from evidence to action. Retrieved from https://www.nursingworld.org/~4aff6a/globalassets/practiceandpolicy/innovation evidence/ind-kit-2012-for-nnas.pdf EFFECT OF SIMULATION 29 Intellectus Statistics [Online computer software]. (2020). Intellectus statistics: Statistics software for the non-statistician. Retrieved from https://analyze.intellectusstatistics.com/ Institute of Medicine Committee on Quality of Health Care in America. Crossing the Quality Chasm. A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.http://www.nap.edu/openbook.php?isbn=0309072808. Accessed July 20, 2015. Interprofessional Education Collaborative (2016). Core competencies for interprofessional collaborative practice: 2016 Update. Washington, DC: Interprofessional Education Collaborative. Retrieved from https://nebula.wsimg.com/2f68a39520b03336b41038c370497473?AccessKeyId=D C06780E69ED19E2B3A5&disposition=0&alloworigin=1. Jeffries, P. R. (2005). A framework for designing, implementing, and evaluating simulations used as teaching strategies in nursing. Nursing Education Perspectives, 26(2), 96–103. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/15921126 Jeffries, P.R., Rodgers, B., & Adamson, K. (2015). NLN Jeffries simulation theory: Brief narrative description. Nursing Education Perspectives, 36(5), 292-293. Joint Commission International (2017). Joint Commission International Accreditation Standards for Hospitals. 6th ed. Oakbrook, Ill: Joint Commission International. http://jointcommissioninternational.org. Accessed Feb. 2, 2019. Klipfel, J.M., Carolan, B.J., Brytowski, N., Mitchell, C.A., Gettman, M.T., & Jacobson, T.M. (2014). Patient safety improvement through in situ simulation interdisciplinary training. Urologic Nursing, 3(1), 39-46. EFFECT OF SIMULATION 30 Kostoff, M., Burkhardt, C., Winter, A., & Shrader, S. (2016). An interprofessional simulation using the SBAR communication tool. American Journal of Pharmaceutical Education, 80(9), 157. Kouzes, J.M., & Posner, B.Z. (2017). The leadership challenge (6th ed.). Hoboken, NJ: John Wiley & Sons. LaFond, C.M., & Van Hulle Vincent, C. (2012). A critique of the national league for nursing/Jeffries simulation framework. Journal of Advanced Nursing, 28, 465-480. Leonard, M., Graham, S., & Bonacum, D. (2004). The human factor: The critical importance of effective teamwork and communication in providing safe care. Quality Safe Health Care, 13, 85-90. Mayer, C., Hadfield, M., Hutchings, M., & Eyto, A.E. (2018). Ensuring rigor in qualitative data analysis: A design research approach to coding combining NVIVO with traditional material methods. International Journal of Qualitative Methods 17(1), 1-13. https://doi.org/10.1177/1609406918786362 Mariani, B., & Doolen, J. (2016). Nursing simulation research: What are the perceived gaps? Clinical Simulation in Nursing, 12(1), 30– 36. doi.org/10.1016/j.ecns.2015.11.004 Melnyk, B.M., Fineout-Overholt, E. (2015). Evidence-based practice in nursing and healthcare: A guide to best practice (3rd ed.). Philadelphia, PA: Lippincott, Williams & Wilkins. Normille, D. (2017). Science suffers as China plugs holes in Great Firewall. Science. 357(6354). Retrieved 2-02-2019 http://science.sciencemag.org.ezproxy1.lib.asu.edu/content/sci/357/6354/856.full.pdf EFFECT OF SIMULATION 31 Oxelmark, L., Amoroe, T. N., Carlzon, L., & Rystedt, H. (2019). Correction to: Students’ understanding of teamwork and professional roles after interprofessional simulation—a qualitative analysis. Advances in Simulation, 4(1). doi.org/10.1186/s41077-019-0089-6 Poore, J. A., Dawson, J. C., Dunbar, D. & Parrish, K. (2019). Debriefing interprofessionally. Nurse Educator, 44(1), 25–28. doi: 10.1097/NNE.0000000000000518. Reavy, K. (2016). Inquiry and leadership: A resource for the DNP project. Philadelphia, PA: F.A. Davis. Research integrity and assurance (n.d.). Retrieved from https://researchintegrity.asu.edu/humansubjects. Rosswurm, M.A. & Larrabee, J.H. (1999). A model for change to evidencebased practice. Journal of Nursing Scholarship, 31(4), 317-322. Rutherford-Hemming, T. (2012). Simulation methodology in nursing education and adult learning theory. Adult Learning, 23(3), 129– 137. doi.org/10.1177/1045159512452848 Salary Explorer (2020). Nurse average salary in China. Retrieved from http://www.salaryexplorer.com/salarysurvey.php?loc=44&loctype=1&job=865&jobtype=3 Socrative [Online computer software]. (2020). About us. Retrieved from https://socrative.com/. Wang, L., Jiang, X., Wang, L., Wang, G., & Bai, Y. (2013). Barriers to and facilitators of research utilization: A survey of registered nurses in China. PloS One, 8, e81908. Will, K. K., Stepanek, J., Brewer, K. K., Colquist, J. A., Cruz, J. E. S., Donald, C. B., Wilson, R. (2016). Interprofessional orientation for health professionals utilizing simulated learning: EFFECT OF SIMULATION 32 Findings from a pilot study. Journal of Interprofessional Care, 30(2), 254– 256. doi.org/10.3109/13561820.2015.1092116 World Health Organization (2011). WHO patient safety curriculum guide. Retrieved Feb 2, 2019 from https://www.who.int/patientsafety/education/mp_curriculum_guide/en/ Yang, Y.Y., Ying-Ying, Y., Chia-Chang, H., Jen-Feng, L., Hao, M.C., Chin, C.H., & Shou, Y.K. (2017). Simulation-based inter-professional education to improve attitudes towards collaborative practice: a prospective comparative pilot study in a Chinese medical centre. BMJ Open 7(11). doi: 10.1136/bmjopen-2016-015105 Yinchuan (2020). Britannica Academic. Retrieved from https://academic-ebcom.ezproxy1.lib.asu.edu/levels/collegiate/article/Yinchuan/77970 Zhou, F., Maier, M., Hao, Y., Tang, L., Guo, H., Liu, H., & Liu, Y. (2015). Barriers to research utilization among registered nurses in traditional Chinese medicine hospitals: A cross-sectional survey in China. Evidence-Based Complementary and Alternative Medicine, 2015, 1-8. doi.org/10.1155/2015/475340 33 EFFECT OF SIMULATION Appendix A Table 1 Evaluation Table of Qualitative Studies Citation Conceptual Framework Mariani, B. & Doolen, J. (2016). Nursing simulation research: What are the perceived gaps? ALT: Cognitive Social Constructivist (RutherfordHemming, 2012). Country: US Funding: None Bias: None How learners gain knowledge with simulation. Design/ Method/ Sampling (Grounded Theory, phenomenology, Narrative…) Sample/Setting (describe) Design: QDS n=90 RN’s: CS Ethnography A naturalist inquiry approach Purpose: Gain perspective of the perceived gaps in SR and identify areas of saturation and those that need more research. INACSL members. MA: 52.47 Gender: F- 93.33% M-: 4.44% NR-2.22% Ed Lev: PhD-23.3% DO 15.56% MSN/MS56.65% BS-3.33% NR-1.11% Major Variables Studied and Their Definitions SR: What areas in SR do INACSL members perceive to be well studied? What areas in SR as perceived by the members need more research? Definition: Measurement/ Instrumentation (focus group, 1:1, researcher(s) Data Analysis Demographic questionnaire SCA Internet -Based E-mail Survey PI, Co-PI 3rd Party Reviewer Findings/ Themes Gap Themes: OutcomesTransfer of KS to positive patient outcomes SD & SettingLOFElements of importance in SD Participants & Reviewers- Level/Quality of Evidence; Decision for practice/ application to practice/ Generalization LOE: VI Strengths: Strong QDS; good n Strong evidence to support PICOT Satisfaction, perception, and SE well-studied. Weaknesses: Only represents the INACSL membership, and not an entire nursing community. Key: ALT-Adult Learning Theory; AMC-Academic Medical Center; CAF-Conceptual Assessment Framework; CCA-Conventional Content Analysis; CEF-Confirmatory Exploratory Factor; CS-Convenience Sampling; DO-Doctorate other; EVR-Evaluation of reliability & validity; F-Female; IBS-Internet based survey; ICCAS-Competency Instrument Interprofessional Attitudes Survey; INACSL- International Nursing Association for Clinical Simulation and Learning; IP-interprofessional team; IPAS-Interprofessional Attitude Survey; IPECInterprofessional Education Collaborative; IPSE-interprofessional simulation-based education; ITA-Inductive Thematic Analysis; KS-knowledge & skills; LOF-Level of Fidelity; LOOLevel of Outcome; LS-last semester M-male; MA-Mean Age;; MD-Medical Student; n- number of participants NLNST-NLN Simulation theory; NR-no response; PCE-Psychometric Confirmatory and Exploratory ; PI-Principal Investigator; QDS-Qualitative Descriptive Study; RE-Research Experience ; RN’s-Registered Nurses; ROL-review of literature; SD-simulation design; SDS-Simulation Design Scale; SCA-Summative Content Analysis; SE-Self-Efficacy; SIBR-Structured IP Bedside Rounds; SIS-Simulation intervention studies; SR-Simulated Research; TS-teamSTEPPS; TW-teamwork; WNH-White, not Hispanic. 34 EFFECT OF SIMULATION Citation Conceptual Framework Design/ Method/ Sampling (Grounded Theory, phenomenology, Narrative…) Sample/Setting (describe) Ethnicity: WNH-88.9% All others: 11.1% RE: Expert5.5%; MidCareer-28.78% Novice-63.33% NR-3.33% Setting: IBS at Constant Contact Exclusions: Not clearly stated Attrition: None Major Variables Studied and Their Definitions SR-to enhance team skills and test new clinical processes. Measurement/ Instrumentation (focus group, 1:1, researcher(s) Data Analysis . Findings/ Themes Studies with large sample sizes Preparation or support of faculty and influence on simulation Research RigorEVRMultisite SIS Longitudinal Studies Academic partnerships and IP studies Level/Quality of Evidence; Decision for practice/ application to practice/ Generalization Lengthy survey (time to take). Application: Advances the science of SR baseline info. Multiple opportunities for DNP scholars to develop and/or strengthen simulation projects Key: ALT-Adult Learning Theory; AMC-Academic Medical Center; CAF-Conceptual Assessment Framework; CCA-Conventional Content Analysis; CEF-Confirmatory Exploratory Factor; CS-Convenience Sampling; DO-Doctorate other; EVR-Evaluation of reliability & validity; F-Female; IBS-Internet based survey; ICCAS-Competency Instrument Interprofessional Attitudes Survey; INACSL- International Nursing Association for Clinical Simulation and Learning; IP-interprofessional team; IPAS-Interprofessional Attitude Survey; IPECInterprofessional Education Collaborative; IPSE-interprofessional simulation-based education; ITA-Inductive Thematic Analysis; KS-knowledge & skills; LOF-Level of Fidelity; LOOLevel of Outcome; LS-last semester M-male; MA-Mean Age;; MD-Medical Student; n- number of participants NLNST-NLN Simulation theory; NR-no response; PCE-Psychometric Confirmatory and Exploratory ; PI-Principal Investigator; QDS-Qualitative Descriptive Study; RE-Research Experience ; RN’s-Registered Nurses; ROL-review of literature; SD-simulation design; SDS-Simulation Design Scale; SCA-Summative Content Analysis; SE-Self-Efficacy; SIBR-Structured IP Bedside Rounds; SIS-Simulation intervention studies; SR-Simulated Research; TS-teamSTEPPS; TW-teamwork; WNH-White, not Hispanic. 35 EFFECT OF SIMULATION Baik, D. & Zierler, B. (2018). Clinical nurses’ experience and perception after implementation of an interprofessional team intervention: a qualitative study Country: US Funding: McLaws’ Research Award, (UW). Sigma Theta Tau International STTI Grant (UW). Bias: None ALT: Cognitive Social Constructivist (RutherfordHemming, 2012). How learners gain knowledge with simulation. QDS Ethnography Grounded Theory Purpose: Explore clinical nurses’ experiences and perceptions following a purposeful IP intervention in practice. n=10 RN’s; CS Setting: AMC Telemetry Unit Magnet site TST site Gender: F Mean Age: 34.7 Educ: BSN Tele Unit Mean Tenure: 6 years Employment: FT or PT Exclusion: Traveler Nurses Understand nurses’ perceptions after a 4-hr TS training intervention Focus Group Interviews IRR Interview Guide; Questionnaire Tape Recorded Understand nurses’ perceptions after an SIBR simulation intervention. CCA Transcription Code Book of Quality Data Analysis IP team building IP relationships and IP communication Psychological safety & cultural change Efficiency in care delivery Quality of patient care Improved job outcomes Team challenges -Lack of consistency -MD engagement in SBIR -Hierarchical Structure LOE: VI Strengths: Thorough design & rigour Weaknesses: Small n Conducted on a single inpatient unit, is it generalizable? Focus groups require followup to evaluate sustainability Application: Effective TW and IP collaboration improve overall care delivery for inpatient settings. Excellent contribution to a toolkit for effective clinical IP collaborative practice. Key: ALT-Adult Learning Theory; AMC-Academic Medical Center; CAF-Conceptual Assessment Framework; CCA-Conventional Content Analysis; CEF-Confirmatory Exploratory Factor; CS-Convenience Sampling; DO-Doctorate other; EVR-Evaluation of reliability & validity; F-Female; IBS-Internet based survey; ICCAS-Competency Instrument Interprofessional Attitudes Survey; INACSL- International Nursing Association for Clinical Simulation and Learning; IP-interprofessional team; IPAS-Interprofessional Attitude Survey; IPECInterprofessional Education Collaborative; IPSE-interprofessional simulation-based education; ITA-Inductive Thematic Analysis; KS-knowledge & skills; LOF-Level of Fidelity; LOOLevel of Outcome; LS-last semester M-male; MA-Mean Age;; MD-Medical Student; n- number of participants NLNST-NLN Simulation theory; NR-no response; PCE-Psychometric Confirmatory and Exploratory ; PI-Principal Investigator; QDS-Qualitative Descriptive Study; RE-Research Experience ; RN’s-Registered Nurses; ROL-review of literature; SD-simulation design; SDS-Simulation Design Scale; SCA-Summative Content Analysis; SE-Self-Efficacy; SIBR-Structured IP Bedside Rounds; SIS-Simulation intervention studies; SR-Simulated Research; TS-teamSTEPPS; TW-teamwork; WNH-White, not Hispanic. 36 EFFECT OF SIMULATION Oxelmark et al. (2017). Students’ understanding of teamwork and professional roles after interprofessional simulation-a qualitative analysis Country: Sweden Funding: Sahlgrenska Academy, Golthenburg University Bias: None ALT: Cognitive Social Constructivist (RutherfordHemming, 2012). How learners gain knowledge with simulation. QDS Ethnography Grounded Theory n=12 4 MD students (LS) 8 RN students (LS) Setting: Classroom IPSE What changes in students’ understanding of teamwork and professional roles can be identified through IPSE? How can IPSE support the transformation of students’? Understanding of teamwork and professional roles? Definition IPSE: students are allowed to practice simulation skills in a controlled environment Focus Groups Trust LOE: VI Strengths: Research fits the study. IP Scenario’s Participation in IPSE more rewarding for participants then traditional textbook learning Audio and Transcription Recordings Realizing and embracing TW fundamentals Weaknesses: Small n Software: NVivo Reconsidering IP roles; shared view of work process Focus Group Guide Lecture ITA Thorough background & ROL Unclear limitations PI Table clarity Application: Co PI Achieving increased confidence IPSE design Power Distance & Hierarchy Reinforcement Key: ALT-Adult Learning Theory; AMC-Academic Medical Center; CAF-Conceptual Assessment Framework; CCA-Conventional Content Analysis; CEF-Confirmatory Exploratory Factor; CS-Convenience Sampling; DO-Doctorate other; EVR-Evaluation of reliability & validity; F-Female; IBS-Internet based survey; ICCAS-Competency Instrument Interprofessional Attitudes Survey; INACSL- International Nursing Association for Clinical Simulation and Learning; IP-interprofessional team; IPAS-Interprofessional Attitude Survey; IPECInterprofessional Education Collaborative; IPSE-interprofessional simulation-based education; ITA-Inductive Thematic Analysis; KS-knowledge & skills; LOF-Level of Fidelity; LOOLevel of Outcome; LS-last semester M-male; MA-Mean Age;; MD-Medical Student; n- number of participants NLNST-NLN Simulation theory; NR-no response; PCE-Psychometric Confirmatory and Exploratory ; PI-Principal Investigator; QDS-Qualitative Descriptive Study; RE-Research Experience ; RN’s-Registered Nurses; ROL-review of literature; SD-simulation design; SDS-Simulation Design Scale; SCA-Summative Content Analysis; SE-Self-Efficacy; SIBR-Structured IP Bedside Rounds; SIS-Simulation intervention studies; SR-Simulated Research; TS-teamSTEPPS; TW-teamwork; WNH-White, not Hispanic. 37 EFFECT OF SIMULATION Appendix A Table 2 Evaluation Table of Quantitative Studies Citation Theory/ Conceptual Framework Design/ Method Sample/ Setting Major Variables & Definitions Measurement/ Instrumentatio n Data Analysis (stats used) Findings/ Results Welsch, et al., (2018) ALT Design: Systematic Review Purpose: Explicitly examine the use of TeamStepps and simulation in an IPE program and subsequent outcomes N=11 DS: EBSCO PubMed Hand Search (HS) IV 1-TS training IV-2 IHS DV-Kirpatrick’s LOO MERSQ NOS-E MERSQ & NOS-E Kirkpatricks LOO: 2 IR ERV ICC .82 (.76-.98) Reactions from learners; satisfied Interprofessional education involving didactic TeamSTEPPSR and interactive healthcare simulation: A systematic review Country: US Funding: Graduate research assistantship from Old Dominion University Modeling and Simulation Jeffries NLN ST Setting: In-situ simulations (3) Simulation training sites (4) Did not indicate (4) Population: IP RNs Rx MD RT MSW Definitions: Kirpatricks LOO (see Findings) Participant attitude: improved even after 1 yr Acquisition of new knowledge: improvement in critical situations Behavioral Changes; benefits of teamwork Level/Quality of Evidence; Decision for practice/ application to practice LOE: I Strengths First known review to fully examine TeamSTEPPS and simulation Similar results among studies Weaknesses Challenging to synthesize multiple variable outcomes Conclusions Great diversity among design and evaluation methods Previous research should guide the Key: ALT-Adult Learning Theory; AMC-Academic Medical Center; CAF-Conceptual Assessment Framework; CCA-Conventional Content Analysis; CEF-Confirmatory Exploratory Factor; CS-Convenience Sampling; DO-Doctorate other; EVR-Evaluation of reliability & validity; F-Female; IBS-Internet based survey; ICCAS-Competency Instrument Interprofessional Attitudes Survey; INACSL- International Nursing Association for Clinical Simulation and Learning; IP-interprofessional team; IPAS-Interprofessional Attitude Survey; IPECInterprofessional Education Collaborative; IPSE-interprofessional simulation-based education; ITA-Inductive Thematic Analysis; KS-knowledge & skills; LOF-Level of Fidelity; LOOLevel of Outcome; LS-last semester M-male; MA-Mean Age;; MD-Medical Student; n- number of participants NLNST-NLN Simulation theory; NR-no response; PCE-Psychometric Confirmatory and Exploratory ; PI-Principal Investigator; QDS-Qualitative Descriptive Study; RE-Research Experience ; RN’s-Registered Nurses; ROL-review of literature; SD-simulation design; SDS-Simulation Design Scale; SCA-Summative Content Analysis; SE-Self-Efficacy; SIBR-Structured IP Bedside Rounds; SIS-Simulation intervention studies; SR-Simulated Research; TS-teamSTEPPS; TW-teamwork; WNH-White, not Hispanic. 38 EFFECT OF SIMULATION Bias: Consider direction and magnitude of possible bias on the effect estimate with each review Length of Study: 1 (Semester) 8 (3 hours to 9 hrs) 2 (Did not indicate) strongly noted after intervention Systemic organizational practices didactic content and evaluation Feasability Use of common instruments can assist with comparision between studies Inclusions: -Didactic learning sessions -Plus an interactive simulation -IP team -Pre-post learning outcomes -English -Peer Reviewed Exclusions: -No pre-post outcome -Sim done electronically -Role playing in lieu of simulation Key: ALT-Adult Learning Theory; AMC-Academic Medical Center; CAF-Conceptual Assessment Framework; CCA-Conventional Content Analysis; CEF-Confirmatory Exploratory Factor; CS-Convenience Sampling; DO-Doctorate other; EVR-Evaluation of reliability & validity; F-Female; IBS-Internet based survey; ICCAS-Competency Instrument Interprofessional Attitudes Survey; INACSL- International Nursing Association for Clinical Simulation and Learning; IP-interprofessional team; IPAS-Interprofessional Attitude Survey; IPECInterprofessional Education Collaborative; IPSE-interprofessional simulation-based education; ITA-Inductive Thematic Analysis; KS-knowledge & skills; LOF-Level of Fidelity; LOOLevel of Outcome; LS-last semester M-male; MA-Mean Age;; MD-Medical Student; n- number of participants NLNST-NLN Simulation theory; NR-no response; PCE-Psychometric Confirmatory and Exploratory ; PI-Principal Investigator; QDS-Qualitative Descriptive Study; RE-Research Experience ; RN’s-Registered Nurses; ROL-review of literature; SD-simulation design; SDS-Simulation Design Scale; SCA-Summative Content Analysis; SE-Self-Efficacy; SIBR-Structured IP Bedside Rounds; SIS-Simulation intervention studies; SR-Simulated Research; TS-teamSTEPPS; TW-teamwork; WNH-White, not Hispanic. 39 EFFECT OF SIMULATION Kostoff, et al., (2016) Jeffries NLN ST ESD Mixed methods research An Interprofessional Simulation Using the SBAR Communication Tool Randomized Sample N=190 n=94 (RNs) n=96 (Rx ) Setting: Univ. of Kansas-School of Rx Country: US IV: Use of SBAR tool on self- perception of interprofessional competence ICCAS SPSS Dedoose software DV: Reactions towards interprofessional collaboration Reflection Paper Telephones SBAR communicatio n tool Funding: Not reported Bias: Some risk with randomized sampling Retrospective pretest posttest design Wilcoxin Rank Sum Interrator agreement: Cohen’s kappa value Theory/ Design/ Method Sample/ Setting Major Variables & Definitions Improvement in self-perception of interprofession al competence in all factors (p<.001) Strongest themes: Interprofessio nal simulation Active listening To IP ideas Videoconferen ce Negotiation of responsibilities within scope of practice Group Debriefing session Citation 58/96 (60%) Rx Students: MeasurementInstrumentation LOE: II Scarcity of information for Pharmacists in use of SBAR tool. Strengths Builds on existing literature, supports use of SBAR as a proactive structured communication tool Weaknesses Could use more information on sampling related to BSN students Conclusions Use of SBAR enhances ability to organize and communicate clearly Feasability Simple tool to teach to for broad base utilization. Data Analysis (stats used) Findings/ Results Level/Quality of Evidence; Decision Key: ALT-Adult Learning Theory; AMC-Academic Medical Center; CAF-Conceptual Assessment Framework; CCA-Conventional Content Analysis; CEF-Confirmatory Exploratory Factor; CS-Convenience Sampling; DO-Doctorate other; EVR-Evaluation of reliability & validity; F-Female; IBS-Internet based survey; ICCAS-Competency Instrument Interprofessional Attitudes Survey; INACSL- International Nursing Association for Clinical Simulation and Learning; IP-interprofessional team; IPAS-Interprofessional Attitude Survey; IPECInterprofessional Education Collaborative; IPSE-interprofessional simulation-based education; ITA-Inductive Thematic Analysis; KS-knowledge & skills; LOF-Level of Fidelity; LOOLevel of Outcome; LS-last semester M-male; MA-Mean Age;; MD-Medical Student; n- number of participants NLNST-NLN Simulation theory; NR-no response; PCE-Psychometric Confirmatory and Exploratory ; PI-Principal Investigator; QDS-Qualitative Descriptive Study; RE-Research Experience ; RN’s-Registered Nurses; ROL-review of literature; SD-simulation design; SDS-Simulation Design Scale; SCA-Summative Content Analysis; SE-Self-Efficacy; SIBR-Structured IP Bedside Rounds; SIS-Simulation intervention studies; SR-Simulated Research; TS-teamSTEPPS; TW-teamwork; WNH-White, not Hispanic. 40 EFFECT OF SIMULATION Conceptual Framework Franklin et al., (2014) Psychometric testing on the NLN student satisfaction and self-confidence in learning, simulation design scale (SDS), and educational practices questionnaire using a sample of pre-license novice nurses CAF Item analysis: PCE Purpose: Establish the psychometric properties of the SDS (& others) using reliability and validity testing CS N=2200 Novice nurses: Pre-licensure BSN Program IV: SDS psychometric properties DV: Reliability and Validity MA: 22.8 Definition: Inclusion Participation in sim activities as part of coursework 18 or older SDS-20 items assess perceptions of objectives, information, support, problem solving, feedback, and fidelity in simulation SDS (6 point Likert Scale) Item analysis Confirmatory Exploratory Factor Analysis Concordant and Discordant validity Cronbachs alpha (.92; 96) for presence & importance of design features Measures are both reliable and valid; construct for SDS needs to change using a revised 5-factor design scale for practice/ application to practice LOE: Not clear, testing for IRR Strengths Large sample size Weaknesses Exclusions are not clearly stated Conclusion Robust evidence that the SDS is valid and reliable Feasability Strong 2007-2010 Country: US Funding: Not reported Exclusion: Not clear Bias: None Key: ALT-Adult Learning Theory; AMC-Academic Medical Center; CAF-Conceptual Assessment Framework; CCA-Conventional Content Analysis; CEF-Confirmatory Exploratory Factor; CS-Convenience Sampling; DO-Doctorate other; EVR-Evaluation of reliability & validity; F-Female; IBS-Internet based survey; ICCAS-Competency Instrument Interprofessional Attitudes Survey; INACSL- International Nursing Association for Clinical Simulation and Learning; IP-interprofessional team; IPAS-Interprofessional Attitude Survey; IPECInterprofessional Education Collaborative; IPSE-interprofessional simulation-based education; ITA-Inductive Thematic Analysis; KS-knowledge & skills; LOF-Level of Fidelity; LOOLevel of Outcome; LS-last semester M-male; MA-Mean Age;; MD-Medical Student; n- number of participants NLNST-NLN Simulation theory; NR-no response; PCE-Psychometric Confirmatory and Exploratory ; PI-Principal Investigator; QDS-Qualitative Descriptive Study; RE-Research Experience ; RN’s-Registered Nurses; ROL-review of literature; SD-simulation design; SDS-Simulation Design Scale; SCA-Summative Content Analysis; SE-Self-Efficacy; SIBR-Structured IP Bedside Rounds; SIS-Simulation intervention studies; SR-Simulated Research; TS-teamSTEPPS; TW-teamwork; WNH-White, not Hispanic. 41 EFFECT OF SIMULATION Citation Theory/ Conceptual Framework Design/ Method Sample/ Setting Major Variables & Definitions Measurement/ Instrumentatio n Data Analysis (stats used) Findings/ Results Costello et al., (2017) Jeffries NLN ST Pre-Post Comparative Study N=122 IV: 2.5 hour simulation lab experience DV: Assessment of student perspectives Pre-Post comparative Cronbach’s alpha coefficient: (.96-.98) Statistically significant findings: Simulation as an effective strategy for interprofessional education Country: US Funding: Not noted Bias: None Purpose: To examine student perspectives before and after completing an interprofessi onal, community health, simulation lab experience 33 RN students 38 PT students 29 Nutrition 22 Social work Setting: Liberal Arts College, Boston, MA Inclusion Participation in simulation training Exclusion No data collection required by students IPEC Competency Instrument IPAS IPEC Competency Instrument analysis: roles and responsibilities, team, and teamwork (p<.001) IPAS: Pt interaction and cultural competence (p< .001) Level/Quality of Evidence; Decision for practice/ application to practice LOE: II Strengths: Use of well studied tools (IPEC/IPAS) Weaknesses Unmatched samples Lack of comparison on sub-scale between groups Conclusion Simulation is an effective teaching strategy for interprofessional teamwork and collaboration skillbuilding. Feasability Future research needed to evaluate the effect in this specific study Key: ALT-Adult Learning Theory; AMC-Academic Medical Center; CAF-Conceptual Assessment Framework; CCA-Conventional Content Analysis; CEF-Confirmatory Exploratory Factor; CS-Convenience Sampling; DO-Doctorate other; EVR-Evaluation of reliability & validity; F-Female; IBS-Internet based survey; ICCAS-Competency Instrument Interprofessional Attitudes Survey; INACSL- International Nursing Association for Clinical Simulation and Learning; IP-interprofessional team; IPAS-Interprofessional Attitude Survey; IPECInterprofessional Education Collaborative; IPSE-interprofessional simulation-based education; ITA-Inductive Thematic Analysis; KS-knowledge & skills; LOF-Level of Fidelity; LOOLevel of Outcome; LS-last semester M-male; MA-Mean Age;; MD-Medical Student; n- number of participants NLNST-NLN Simulation theory; NR-no response; PCE-Psychometric Confirmatory and Exploratory ; PI-Principal Investigator; QDS-Qualitative Descriptive Study; RE-Research Experience ; RN’s-Registered Nurses; ROL-review of literature; SD-simulation design; SDS-Simulation Design Scale; SCA-Summative Content Analysis; SE-Self-Efficacy; SIBR-Structured IP Bedside Rounds; SIS-Simulation intervention studies; SR-Simulated Research; TS-teamSTEPPS; TW-teamwork; WNH-White, not Hispanic. 42 Running head: EFFECT OF SIMULATION Appendix A Table 3 Synthesis Table Author Baik et al. Cheng et al. Cheng et al. Foronda et al. 2016 Kostoff et al. 2018 Franklin et al. 2014 Year Study Characteristics LOA Study Design C.Framework Setting: Country AMC Univ/CR STS Demographics RN NM Student RX 2018 2017 VI QDS; E;GT ALT; CSC USA X Costello et al. 2017 Oxelmark et al. 2016 Mariani et al. 2016 2017 Welsch et al. 2018 VI QDS ALT VI QDS TLT II IA; PCE CAF IV IR; MM JST II ESD JST VI QDS ALT;CSC II Pre-Post CS JST VI QDS ALT I SRV JST China X China X X USA X X USA USA USA USA Sweden USA X X X X X X X X X X X X X X X X X X X X X X X X Key: SYMBOLS: SR Gap★-Simulation Research Gap; WS★★-Well studied; ★Strong influencer;⬆ ️-Improvement; ➡️ Consistent Theme AMC-Academic Medical Center; ALT-Adult Learning Theory; BC-Bejing, China; CAF- Conceptual Assessment Framework; CAA-Constant Analysis Approach; CCA-Constant Comparative Analysis; CN-Clinical Nurse; CS-Comparative study; ICCAS-Competency Instrument Interprofessional Attitudes Survey; CSC-Cognitive Socialist Constructivist; CR-Classroom; DU-USA-Duke Univ.-USA; DV-Dependent Variable; ESD-Experimental Study Design; E-Ethnography; EBI-Evidence based implementation; FUC-Fudan Univ., China; GT-Grounded Theory; IA-Item Analysis; IBS-Internet Based Survey; IHS-Interactive healthcare simulation ; INACSL-International Nursing Assn for Clinical Simulation & Learning; IP-Interprofessional; IPCInterprofessional communication; IPI-Interprofessional Intervention; IPSE-Interprofessional simulation-based education; IR-Integrative Review; IV-Independent variable; JC-Jiangsu China; JST-Jeffries Simulation Theory; LAC-Liberal Arts College; LOE-Level of Evidence; LOO: Level of Outcome; LP-Leadership Practices; LR-Lit Review; MC-Mainland, China; MDphysician; MERSQ-Medical education research quality instrument; MM-Mixed Method; ND-Nurse Dir.; NM-Nurse Mgr; Nursing Simulation Theory; NOS-E-Newcastle-Ottawa scaleEducation; NR-Not recorded; OHSU-OR-Oregon Health Sciences Univ-Oregon-USA; PCE-Psychometric, Confirmatory, Exploratory; PL-Pre-licensure; PR-Professional roles; QDS-Qualitative Descriptive Study; RN-Registered Nurses; R&V-Reliability and Validity; RX-Pharmacy; SC-Sichuan, China; SDS-Simulation Design Scale; SHC-Shanghai, China; SIBR-Simulated Interprofessional Bedside Rounds; SLE-Simulation Lab Experience-Staff nurses; SR-Simulation Research; STE-Standardized Tool Effectiveness; STS-Simulation Training Sites; SU=Students understanding; SRV-Systematic Review; TA-Thematic Analysis; TLT-Transformational Leadership Theory; TS-TeamSTEPPS; TW-Teamwork; UK-Univ. of Kansas; 43 EFFECT OF SIMULATION IP Sample M. Age BSN MS PhD/DO Experience Length of Training 10 34.7 100% 6y 4 hr x5 Variables and Outcomes Measurement CCA Tools Intervention TS SIBR X 56 NR 44.6% 28.57% 21.4% 0-9y 30% 10-19y 39% 20-39 y 30% NA-Interviews 15 41 60% 40% 2200 18 yrs + 100% X 27 Studies NR NR RX; n=94;RN=9 NR Students 22y 10y Mgmt + Students NR Students NAInterviews NA-Survey NA 4 hr X 3 90 52.47 3.33% 56.65% 37.86% Expert 5% MC: 29% Nov-63% NA-Survey CAA CAA SDS LR ICCAS IBS EBI influence factors EBI experiences STE X 122 NR Students X 12 NR NR X 11 Studies NR NR Students NR Varied 2.5 hr 8 hours x1 3-8 hrs IPEC,CIIS TA MERSQ; NOS-E SR studied SR needed Changes in SU of TW/PR through IPSE? IPSE support SU of TW/PR? Variables IV: SDS psychometri c properties DV: R &V IV: Use of SBAR DV: Reactions towards IP collaboration IV: SLE DV: Student perspective IV 1-TS training IV-2 IHS Key: SYMBOLS: SR Gap★-Simulation Research Gap; WS★★-Well studied; ★Strong influencer;⬆ ️-Improvement; ➡️ Consistent Theme AMC-Academic Medical Center; ALT-Adult Learning Theory; BC-Bejing, China; CAF- Conceptual Assessment Framework; CAA-Constant Analysis Approach; CCA-Constant Comparative Analysis; CN-Clinical Nurse; CS-Comparative study; ICCAS-Competency Instrument Interprofessional Attitudes Survey; CSC-Cognitive Socialist Constructivist; CR-Classroom; DU-USA-Duke Univ.-USA; DV-Dependent Variable; ESD-Experimental Study Design; E-Ethnography; EBI-Evidence based implementation; FUC-Fudan Univ., China; GT-Grounded Theory; IA-Item Analysis; IBS-Internet Based Survey; IHS-Interactive healthcare simulation ; INACSL-International Nursing Assn for Clinical Simulation & Learning; IP-Interprofessional; IPCInterprofessional communication; IPI-Interprofessional Intervention; IPSE-Interprofessional simulation-based education; IR-Integrative Review; IV-Independent variable; JC-Jiangsu China; JST-Jeffries Simulation Theory; LAC-Liberal Arts College; LOE-Level of Evidence; LOO: Level of Outcome; LP-Leadership Practices; LR-Lit Review; MC-Mainland, China; MDphysician; MERSQ-Medical education research quality instrument; MM-Mixed Method; ND-Nurse Dir.; NM-Nurse Mgr; Nursing Simulation Theory; NOS-E-Newcastle-Ottawa scaleEducation; NR-Not recorded; OHSU-OR-Oregon Health Sciences Univ-Oregon-USA; PCE-Psychometric, Confirmatory, Exploratory; PL-Pre-licensure; PR-Professional roles; QDS-Qualitative Descriptive Study; RN-Registered Nurses; R&V-Reliability and Validity; RX-Pharmacy; SC-Sichuan, China; SDS-Simulation Design Scale; SHC-Shanghai, China; SIBR-Simulated Interprofessional Bedside Rounds; SLE-Simulation Lab Experience-Staff nurses; SR-Simulation Research; STE-Standardized Tool Effectiveness; STS-Simulation Training Sites; SU=Students understanding; SRV-Systematic Review; TA-Thematic Analysis; TLT-Transformational Leadership Theory; TS-TeamSTEPPS; TW-Teamwork; UK-Univ. of Kansas; 44 EFFECT OF SIMULATION DVKirkpatrickL OO Outcomes Team Enhance Confidence IP Communication Knowledge Satisfaction Prof Role Understanding Psych Safety Cultural influence Quality Hierarchy Consistent Themes Team Enhance Understanding of Prof Roles Cultural Impact Use of Standard tools ️ ️ ⬆️ ️ ️ ️ ★ ⬆️ ⬆️ ️ ️ ️ ️ ★ ★ ️ ️ ★ ★ ★ ️ ️ ️ ★ ★ ️ ️ ★ ★ ★ ️ ️ ⬆️ ️ ️ ️ ️ ️ ️ ️ ️ ️ ️ WS ️ SR Gap★ ️ ⬆️ ★ ️ ★ ️ ★ ️ ⬆️ ★ ⬆️ ⬆️ ⬆️ ⬆️ WS★★ ⬆️ Key: SYMBOLS: SR Gap★-Simulation Research Gap; WS★★-Well studied; ★Strong influencer;⬆ ️-Improvement; ➡️ Consistent Theme AMC-Academic Medical Center; ALT-Adult Learning Theory; BC-Bejing, China; CAF- Conceptual Assessment Framework; CAA-Constant Analysis Approach; CCA-Constant Comparative Analysis; CN-Clinical Nurse; CS-Comparative study; ICCAS-Competency Instrument Interprofessional Attitudes Survey; CSC-Cognitive Socialist Constructivist; CR-Classroom; DU-USA-Duke Univ.-USA; DV-Dependent Variable; ESD-Experimental Study Design; E-Ethnography; EBI-Evidence based implementation; FUC-Fudan Univ., China; GT-Grounded Theory; IA-Item Analysis; IBS-Internet Based Survey; IHS-Interactive healthcare simulation ; INACSL-International Nursing Assn for Clinical Simulation & Learning; IP-Interprofessional; IPCInterprofessional communication; IPI-Interprofessional Intervention; IPSE-Interprofessional simulation-based education; IR-Integrative Review; IV-Independent variable; JC-Jiangsu China; JST-Jeffries Simulation Theory; LAC-Liberal Arts College; LOE-Level of Evidence; LOO: Level of Outcome; LP-Leadership Practices; LR-Lit Review; MC-Mainland, China; MDphysician; MERSQ-Medical education research quality instrument; MM-Mixed Method; ND-Nurse Dir.; NM-Nurse Mgr; Nursing Simulation Theory; NOS-E-Newcastle-Ottawa scaleEducation; NR-Not recorded; OHSU-OR-Oregon Health Sciences Univ-Oregon-USA; PCE-Psychometric, Confirmatory, Exploratory; PL-Pre-licensure; PR-Professional roles; QDS-Qualitative Descriptive Study; RN-Registered Nurses; R&V-Reliability and Validity; RX-Pharmacy; SC-Sichuan, China; SDS-Simulation Design Scale; SHC-Shanghai, China; SIBR-Simulated Interprofessional Bedside Rounds; SLE-Simulation Lab Experience-Staff nurses; SR-Simulation Research; STE-Standardized Tool Effectiveness; STS-Simulation Training Sites; SU=Students understanding; SRV-Systematic Review; TA-Thematic Analysis; TLT-Transformational Leadership Theory; TS-TeamSTEPPS; TW-Teamwork; UK-Univ. of Kansas; 45 EFFECT OF SIMULATION Academic partnerships Evaluate with V &R ⬆️ SR Gap★ ⬆️ ⬆️ SR Gap★ ⬆️ Key: SYMBOLS: SR Gap★-Simulation Research Gap; WS★★-Well studied; ★Strong influencer;⬆ ️-Improvement; ➡️ Consistent Theme AMC-Academic Medical Center; ALT-Adult Learning Theory; BC-Bejing, China; CAF- Conceptual Assessment Framework; CAA-Constant Analysis Approach; CCA-Constant Comparative Analysis; CN-Clinical Nurse; CS-Comparative study; ICCAS-Competency Instrument Interprofessional Attitudes Survey; CSC-Cognitive Socialist Constructivist; CR-Classroom; DU-USA-Duke Univ.-USA; DV-Dependent Variable; ESD-Experimental Study Design; E-Ethnography; EBI-Evidence based implementation; FUC-Fudan Univ., China; GT-Grounded Theory; IA-Item Analysis; IBS-Internet Based Survey; IHS-Interactive healthcare simulation ; INACSL-International Nursing Assn for Clinical Simulation & Learning; IP-Interprofessional; IPCInterprofessional communication; IPI-Interprofessional Intervention; IPSE-Interprofessional simulation-based education; IR-Integrative Review; IV-Independent variable; JC-Jiangsu China; JST-Jeffries Simulation Theory; LAC-Liberal Arts College; LOE-Level of Evidence; LOO: Level of Outcome; LP-Leadership Practices; LR-Lit Review; MC-Mainland, China; MDphysician; MERSQ-Medical education research quality instrument; MM-Mixed Method; ND-Nurse Dir.; NM-Nurse Mgr; Nursing Simulation Theory; NOS-E-Newcastle-Ottawa scaleEducation; NR-Not recorded; OHSU-OR-Oregon Health Sciences Univ-Oregon-USA; PCE-Psychometric, Confirmatory, Exploratory; PL-Pre-licensure; PR-Professional roles; QDS-Qualitative Descriptive Study; RN-Registered Nurses; R&V-Reliability and Validity; RX-Pharmacy; SC-Sichuan, China; SDS-Simulation Design Scale; SHC-Shanghai, China; SIBR-Simulated Interprofessional Bedside Rounds; SLE-Simulation Lab Experience-Staff nurses; SR-Simulation Research; STE-Standardized Tool Effectiveness; STS-Simulation Training Sites; SU=Students understanding; SRV-Systematic Review; TA-Thematic Analysis; TLT-Transformational Leadership Theory; TS-TeamSTEPPS; TW-Teamwork; UK-Univ. of Kansas; 46 Running head: EFFECT OF SIMULATION Appendix A Table 4: Estimated Project Budget Simulation Development Human Capital CEO (16 hrs) PreferUS CNO (80 Hours)* DNP Student (no Charge @80 hrs) MD leader (8 hrs)* Sr. RN Leaders x5 (80 hours) 2 RN Interpreters (80 hrs)* Ancillary team x5 (80 hours)* Frontline RN’s x5 (80 hours)* Orientation and Training RN (8 hours x700) MD and Ancillary, unknown) Leadership Team*(including Train the Trainer’s) Simulation Co-ordinator Simulation Manager (Masters prepared) Supplies Flipcharts, Pens, Videotaping (no data) DNP student Travel, Lodging, Meals Total Cost Estimate Note: Dollars are rounded Total $CYN Total in $US $9285.00 $40,336.00 $1328.00 $5769.00 0 0 $4800.00 $16,000.00 $687.00 $2,288.00 $32,000 $4577.00 Not known Not known $8975.00 $1284.00 $628,266.00 $89,856.00 $8975.00 $1284.00 $11,539.00 $1650.00 $2000.00 $300.00 $13,984.00 $2,000.00 $776,160 $111,023.00 47 EFFECT OF SIMULATION Appendix A Table 5 Frequency Table for Nominal Variables/Demographics Variable What is your gender? Female Male What is your marital status Married Single What is your current age Under 20 20-29 30-39 40-49 Under 20 Years of work experience 11-15 16-20 21-25 26-30 5-10 Less then 5 What is your professional discipline Administration Ancillary MD No Answer Nurse Manager Pharmacist Physician Assistant What is your professional discipline RN RN-Operating Room What is your highest degree level Associate Bachelors Masters None of the above Vocational Note. Due to rounding errors, percentages may not equal 100%. n % 313 14 95.72 4.28 116 211 35.47 64.53 16 259 43 9 16 4.89 79.20 13.15 2.75 4.89 18 9 11 1 52 236 5.50 2.75 3.36 0.31 15.90 72.17 7 21 6 5 11 6 11 2.14 6.42 1.83 1.53 3.36 1.83 3.36 234 26 71.56 7.95 200 74 1 61.16 22.63 0.31 8 2.45 44 13.46 48 EFFECT OF SIMULATION Appendix A Table 6 Teamwork Composite NLN Jeffrey’s Post-Simulation Survey: Teamwork Composite Q21. The simulation scenario strengthened interdisciplinary teamwork and collaboration. Q22. The simulation scenario strengthened teamwork and collaboration between nurses and physicians. Q23. The simulation scenario helped me recognize differences in perspective are essential for effective collaboration. Q24. The simulation scenario helped me recognize that effective teamwork and collaboration is a continuous journey. Q25. The simulation scenario helped me recognize that collaboration is not required for all decisions however it can happen spontaneously if the right factors are in place. Table 7 Frequency Table for Variables Q 21-Q25 Variable Q21 Strongly Agree Agree Not Applicable UndecidedDisagree Q22 Strongly agree Agree Not Applicable UndecidedDisagree Q23 Strongly Agree Agree Undecided Q24 Strongly Agree Agree Not Applicable; UndecidedDisagree Strongly Disagree Q25 Strongly Agree Undecided Disagree Agree Not Applicable Strongly Disagree Note. Due to rounding errors, percentages may not equal 100%. n % 57 258 1 10 1 17.43 78.90 0.31 3.06 0.31 54 252 3 14 4 16.51 77.06 0.92 4.28 1.22 61 260 7 18.65 79.51 1.81 59 260 1 5 1 1 18.04 79.51 0.31 1.53 0.31 0.31 27 29 28 236 3 4 8.26 8.87 8.56 72.17 0.92 1.22 49 EFFECT OF SIMULATION Appendix A Table 8 Reliability Table for Q21-Q24 Scale α No. of Items Lower Bound Upper Bound Demographic 1-5 4 0.71 0.66 0.76 Note. The lower and upper bounds of Cronbach's α were calculated using a 95.00% confidence Table 9 Summary Statistics Table for Interval and Ratio Variables Variable M SD n Min Max Mdn Teamwork composite 5.12 0.39 327 Note. '-' denotes the sample size is too small to calculate statistic. 3.25 6.00 5.00 Table 10 Two-Tailed Independent Samples t-Test for Teamwork composite by What is your gender? Female Male Variable M SD M SD t p Teamwork Composite 5.12 0.39 5.23 0.35 -1.09 .277 Note. N = 327. Degrees of Freedom for the t-statistic = 325. d represents Cohen's d. d 0.31 Table 11 Two-Tailed Independent Samples t-Test for Teamwork composite by What is your marital status? Married Single Variable M SD M SD t p Teamwork composite 5.17 0.41 5.10 0.38 1.55 .122 Note. N = 327. Degrees of Freedom for the t-statistic = 325. d represents Cohen's d. d 0.18 Table 12 Analysis of Variance Table for Teamwork composite by Education Level Term Education Level_ Residuals SS df F p ηp2 0.56 48.05 2 316 1.83 .163 0.01 50 EFFECT OF SIMULATION Appendix A Table 13 Mean, Standard Deviation, and Sample Size for Teamwork composite by Education Level Combination M Associate 5.10 Bachelors/Masters 5.20 Vocational 5.10 Note. A '-' indicates the sample size was too small for the statistic to be calculated. SD 0.41 0.38 0.30 n 200 75 44 Table 14 Analysis of Variance Table for Teamwork Composite by Years of Experience Term Years of Experience_ Residuals SS 0.28 49.57 df 4 322 F 0.46 p .768 ηp2 0.01 Table 15 Mean, Standard Deviation, and Sample Size for Teamwork Composite by Years of Experience Combination M 11-15 5.14 16-20 5.28 21-30 5.17 5-10 5.12 Less than 5 5.11 Note. A '-' indicates the sample size was too small for the statistic to be calculated. SD 0.40 0.49 0.39 0.33 0.40 n 18 9 12 52 236 Table 16 Analysis of Variance Table for Teamwork Composite by Profession Term Profession Residuals SS 1.50 45.53 df 4 317 F 2.61 p .036 ηp2 0.03 SD 0.35 0.45 0.43 0.37 0.46 n 7 27 6 271 11 Table 17 Mean, Standard Deviation, and Sample Size for Teamwork Composite by Profession_ Combination M Administration 5.32 Pharmacist/Ancillary 5.14 MD 5.54 RN 5.12 Physician Assistant 5.00 Note. A '-' indicates the sample size was too small for the statistic to be calculated. 51 EFFECT OF SIMULATION Appendix B Figure 1: NLN Jeffries Simulation Theory. Adapted from NLN Jeffries simulation theory: brief narrative description,” by P.R. Jeffries, B. Rodgers, and K. Adamson, 2017, Nursing Education Perspectives, 36(5), 292-293. Copyright © 2015 by the National League for Nursing. 52 Running head: EFFECT OF SIMULATION Appendix B Figure 2: Rosswurn & Larrabee Model for Evidence-Based Practice Change. Adapted from “A model for change to evidence-based practice,” by M.A. Rosswurm and J.A. Larrabee, 1999, Journal of Nursing Scholarship, 31(4), 317-22. 53 Running head: EFFECT OF SIMULATION Appendix C Arizona State University IRB Study Exemption 54 Running head: EFFECT OF SIMULATION Appendix D The NLN Jeffries Post Simulation Participant Evaluation Survey/Simulation Design Scale (SDS) Question Q1. There was enough information provided at the beginning of the simulation to provide direction and encouragement. Q2. I clearly understood the purpose and objectives of the simulation. Q3. The simulation provided enough information in a clear matter for me to problemsolve the situation. Q4. There was enough information provided during the simulation. Q5. The cues were appropriate and geared to promote my understanding. Q6. Support was offered in a timely manner. Q7. My need for help was recognized. Q8. I felt supported by the facilitator’s assistance during the simulation Q9. I was supported in the learning process. Q10. Independent problem-solving was facilitated. Q11. I was encouraged to explore all possibilities of the simulation. Q12. The simulation was designed for my specific level of knowledge and skill. Q13. The simulation allowed me the opportunity to prioritize care delivery. Q14. The simulation provided me the opportunity to set goals for care delivery Q15. Feedback provided was constructive. Strongly Agree (SA) Agree (A) Not Applicable (NA) n % Undecided (U) Disagree (D) n % n % Strongly Disagree (SA) n % n % n % 49 14.98 263 80.43 2 0.61 11 3.36 1 0.31 1 0.31 51 15.6 266 81.35 0 0 8 2.45 1 0.31 1 0.31 48 14.68 249 76.15 1 0.31 21 6.42 8 2.45 0 0 47 14.37 258 78.90 1 0.31 16 4.89 1 1.22 0 0 47 14.37 249 76.15 2 0.61 21 6.42 5 1.53 3 0.92 46 41 37 14.07 12.54 11.3 264 255 266 80.73 77.98 81.35 2 4 2 0.61 1.22 0.61 14 22 17 4.28 6.73 5.20 0 4 4 0 1.22 1.22 1 1 1 0.31 0.31 0.31 45 52 13.76 15.90 263 254 80.43 77.68 16 1 4.89 0.31 2 18 0.61 5.50 1 2 0.31 0.61 0 0 0 0 50 15.29 250 76.45 1 0.31 24 7.34 2 0.61 0 0 37 11.31 229 70.03 4 1.22 42 12.84 15 4.59 0 0 44 13.46 246 75.23 44 13.46 257 78.59 7 2.14 19 5.81 0 0 37 11.31 272 83.16 1 0.31 12 3.67 5 1.53 8 2.45 25 7.65 3 0’92 1 0 0.31 0 55 EFFECT OF SIMULATION Question Q16. Feedback was provided in a timely manner. Q17. The simulation allowed me to analyze my own behavior and actions. Q18. There was an opportunity after the simulation to obtain guidance/feedback from the facilitator in order to build knowledge to another level. 19. The scenario resembled a real-life situation. 20.Real life factors, situations, and variable were built into the simulation experience scenario. 21. The simulation scenario strengthened interdisciplinary teamwork and collaboration. 22. The simulation scenario strengthened teamwork and collaboration between nurses and physicians. 23. The simulation scenario helped me recognize differences in perspective are essential for effective collaboration. 24. The simulation scenario helped me recognize that effective teamwork and collaboration is a continuous journey. 25. The simulation scenario helped me recognize that collaboration is not required for all decisions however it can happen spontaneously if the right factors are in place. Strongly Agree (SA) Agree (A) Not Applicable (NA) n % Undecided (U) Disagree (D) n % n % Strongly Disagree (SA) n % n % n % 51 15.60 271 82.87 0 0 4 1.22 0 0 1 0.31 51 15.6 255 77.98 0 0 20 6.12 1 0.31 0 0 37 11.31 274 83.79 0 0 13 3.98 2 0.61 1 0.31 50 49 15.29 14.98 247 230 75.54 70.34 3 4 0.92 1.22 16 34 4.89 10.40 11 9 3.36 2.75 0 1 0 0.31 57 17.43 258 78.90 1 0.31 10 3.06 1 0.31 0 0 54 16.51 252 77.06 3 0.92 14 4.28 4 1.22 0 0 61 18.65 270 79.51 1 0.31 5 1.53 0 0 0 0 59 18.04 260 79.51 1 0.31 5 1.53 1 0.31 1 0.31 27 8.26 236 72.17 3 0.92 29 8.87 28 8.56 4 1.22