NURSE PRACTITIONER RESIDENCY PROGRAMS 1 Nurse Practitioner Residency Programs and Provider Satisfaction Erin M. Luedtke Edson College of Nursing and Health Innovation, Arizona State University Author Note Erin M. Luedtke is a registered nurse at HonorHealth Shea Medical Center in the Progressive Coronary Care Unit and works for Dependable Staffing as a triage nurse in the outpatient setting. She is also a student in the Edson College of Nursing and Health Innovation, Arizona State University. She has no known conflict of interest to disclose. Correspondence should be addressed to Erin M. Luedtke, Edson College of Nursing and Health Innovation, Arizona State University, PO BOX 873020, Phoenix, AZ 85004-3020. Email:eroehm@asu.edu NURSE PRACTITIONER RESIDENCY PROGRAMS 2 Acknowledgements I wish to acknowledge and thank my faculty mentors, Dr. Judith Ochieng and Dr. Carol Moffett, who made significant contributions to my project and guided me along the way. I would also like to thank the Advanced Practice Registered Nurse (APRN) Residency Program Director and Coordinator at the project site for the opportunity to work with their organization and for facilitating the project. Finally, thank you to those who supported me throughout the project journey, especially my colleague, Olivia Sitton, and my family. NURSE PRACTITIONER RESIDENCY PROGRAMS 3 Abstract Purpose & Background: Family Nurse Practitioner (FNP) residency programs are meant to ease providers' transition into practice, but there is limited evidence about their overall effectiveness and impact on provider satisfaction. When a FNP residency program in the Southwestern United States found they had high resident provider attrition rates, it prompted an investigation into current and past residents’ satisfaction levels. Methods: Arizona State University’s (ASU’s) Institutional Review Board (IRB) and the project site’s review committee approved the project design for human subject protection. After approval, all current and past residents employed at the practice were e-mailed a link to SurveySparrow with the Misener Nurse Practitioner Job Satisfaction Scale (MNPJSS) and a demographic questionnaire in December 2021 and February 2022. Results: Mean satisfaction scores indicated “minimally satisfied” overall. When satisfaction was compared over time using a two-tailed independent t-test for an alpha value of 0.05, p = 0.731, indicating no significant change in satisfaction over two months. Total satisfaction and subscales of satisfaction were divided by cohort, averaged, and compared on a Likert scale from “1” (Very Dissatisfied) to “6” (Very Satisfied). Current residents’ average satisfaction score was M = 3.77. They were most satisfied with challenge and autonomy, M = 4.28, and least with collegiality, M = 3.26. Providers' one-year post-residency average satisfaction score was M = 3.98. They were most satisfied with benefits, M = 4.53, and least with time, M = 3.04. Providers' two-year postresidency average satisfaction score was M = 3.49. They were most satisfied with benefits, M = 4.56, and least with time, M = 2.90. Using Pearson Correlation tests there was no correlation between average satisfaction and average performance on Uniform Data Systems (UDS), r = 0.01, p = 0.968. NURSE PRACTITIONER RESIDENCY PROGRAMS Conclusions: Overall providers were “minimally satisfied.” Opportunities to make program improvements were identified and could help improve retention and reduce costs and provider shortages. Keywords: Nurse practitioner residencies, family nurse practitioner, new advanced practice registered nurse, provider satisfaction 4 NURSE PRACTITIONER RESIDENCY PROGRAMS 5 Nurse Practitioner Residency Programs and Provider Satisfaction Advanced Practice Registered Nurse (APRN) residency programs provide clinical and procedural opportunities, mentorship, and supplemental education to create more competent and confident providers. Residency programs are widely available for various healthcare professionals to ease the transition into practice. However, postgraduate Family Nurse Practitioner (FNP) residency programs are limited. Background and Significance Problem In 2010, The Institute of Medicine (IOM) recommended expanding the scope of practice for Advanced Practice Registered Nurses (APRNs) and increasing the availability of residency programs to create more confident and competent providers (Bryant & Parker, 2020). However, there are very few programs available to date to meet the needs of those interested in a residency to transition into their new role. One reason for the limited availability is that most programs receive funding through the federal government, and funding for these programs is not guaranteed to be long-term (Bryant & Parker, 2020; Wiltse Nicely & Fairman, 2015). As a result, funding is a common concern among program coordinators. Important aspects of continued federal funding include performance on quality indicators, known as Uniform Data System (UDS) metrics, and resident provider retention rates (D. Potter, personal communication, November 10, 2020). As of 2017, only 45 residency programs were available and the majority were in government-funded programs, such as the Department of Veterans Affairs (VA) and Federally Qualified Health Centers (FQHCs) (Hicks et al., 2017). FQHC settings are particularly challenging for new practitioners due to the complexity of healthcare problems, cultural and NURSE PRACTITIONER RESIDENCY PROGRAMS 6 language barriers, and limited resources for these patients. Therefore, many advocate for postgraduate programs with mentorship, supplemental knowledge, and skills to ease providers into demanding community health settings (Brown et al., 2015). To better address, the needs of patients and new graduate APRNs in the community, a FQHC in the Southwestern United States established the first primary care nurse practitioner residency program in its state (National Nurse Practitioner Residency & Fellowship Training Consortium [NNPRFTC], 2021). Stakeholders at the project site expressed interest in partnering with Arizona State University graduate students on several Quality Improvement (QI) projects related to their NP residency program. First, they addressed concerns about high resident provider attrition rates during the program’s initiation as a reason to investigate job satisfaction. Bush & Lowery (2016) estimated that the loss of a single resident provider could cost an organization up to $100,000 in federal funding and threaten the residency program. Additionally, as a shared project, the APRN program director invited an investigation into residency performance on UDS measures, an important quality indicator for continual funding. Purpose and Rationale The purpose of addressing a gap in knowledge about residents’ job satisfaction is to identify areas for program improvement and retain providers at the project site. Additionally, it is currently unknown whether APRN residency programs have improved provider satisfaction and performance among residents and graduates of the residency program. Therefore, the information gathered through this project will supplement current literature. Epidemiological Data Population NURSE PRACTITIONER RESIDENCY PROGRAMS 7 The APRN role was created to fill gaps in healthcare and reduce provider shortages. FNP graduates are generally trained and educated on common patient conditions and the management of complex problems across the lifespan. However, many graduates of the FNP program choose to work in specialty areas instead. As of 2015, NPs cared for nearly 20% of patients seen in the primary care setting (Wiltse Nicely & Fairman, 2015). Still, there remains a shortage of primary care providers in the United States (Dumphy et al., 2019). According to Dumphy et al. (2019), the transition from experienced registered nurse to novice APRN has proven difficult, leading to abandonment of their APRN role. As a result, programs to ease the transition have emerged (Dumphy, 2019). Intervention In 2000 the first APRN postgraduate programs were established to help ease the transition from nurse to APRN (Hicks et al., 2017). However, it was not until 2010 that IOM called to expand the scope of practice for APRNs and increase residency programs (Bryant & Parker, 2020). Although many were enthusiastic about the call to action, others were fearful that the IOMs recommendations would lead to residency requirements for APRNs (Speight et al., 2019; Wiltse Nicely & Fairman, 2015). Those with concerns argue that nurses were equipped with years of healthcare experience before entering an APRN program, making residencies an unnecessary requirement. Meanwhile, supporters of mandatory postgraduate education point to the changing demographic of APRN students as a challenge for transitioning into practice (Wiltse Nicely & Fairman, 2015). For instance, in recent years, APRN students tend to be recent graduates of a baccalaureate program and have just a few years of experience in the acute care setting (Wiltse Nicely & Fairman, 2015). Despite the changing demographic of APRN students, NURSE PRACTITIONER RESIDENCY PROGRAMS 8 Dillon et al. (2016) found that years of experience in nursing did not make a difference in the transition into advanced practice. Residency versus Non-residency Further, researchers aimed to determine the difference between APRNs that chose a residency versus those that did not. They noticed that a sudden transition to practice led to higher turnover, causing a significant financial burden to the employing organizations (Bryant & Parker, 2020; Hicks et al., 2017). Bryant & Parker (2020) and Hicks et al. (2017) suggested that individuals who completed residencies were better prepared in their role transition, had less turnover, and had overall greater job satisfaction. Bush & Lowery (2016) found that graduates of at least a one-year NP residency program were more satisfied in their current roles than those that did not complete postgraduate education (Bush & Lowery, 2016). Still, limited literature is available, and more research is needed to determine if APRN programs improve job satisfaction. Outcomes of Residency Programs The literature review revealed little about how postgraduate APRN residency programs impact the quality of care delivered or performance on UDS measures. For instance, Rugen et al. (2018) looked at the VA Centers of Excellence in Primary Care Education (VA CoEPCE) NP Residency Program. They found that residents’ competency significantly improved by the end of the residency program. Conversely, Scaglione & Loyd (2021) examined confidence among adult geriatric acute nurse practitioners throughout their VA fellowship. They reviewed skills such as reading a chest x-ray and interpreting an electrocardiogram toward the program's start and end. Their results unexpectedly showed a decrease in the confidence in skills toward the end of the program. The implications of this study suggested the need to reinforce clinical skills throughout NURSE PRACTITIONER RESIDENCY PROGRAMS 9 that program (Scaglione & Lloyd, 2021). However, all of the fore-mentioned studies cited small sample sizes as a limitation of their work. Timing of Programs Lastly, residency programs vary in length. The program's length ranges from several months to two years (Martsolf et al., 2017). Yet most primary care residency programs in the United States are highly structured 12-month programs focusing on mentorship, didactic education, clinical transition, and quality improvement (Hicks et al., 2017). Summary of the Literature The review of the literature revealed many common themes. First, most NP graduates are interested in a residency program, but there are not enough programs and funding to meet the demand (Bryant & Parker, 2020). Secondly, among individuals who do not enroll in APRN residency program, turnover rates are higher, and many abandon their new roles (Dumphy, 2019). Lastly, preliminary findings suggest that individuals that completed an APRN residency program have higher job satisfaction (Bush & Lowery, 2016). Still, there is a lack of evidence regarding the difference in the delivery of care among graduates of a residency program. Internal Evidence Internal evidence to address provider satisfaction among NP residents comes from the APRN director at a Southwestern FQHC. Specifically, the program director reported several APRN residents resigned in their first cohort, prompting concern for provider satisfaction and retention in their new APRN residency program (D. Potter, personal communication, November 10, 2020). The APRN director was interested in learning the reasons that may have led to resignations and identifying ways of improving the program and retaining providers. Since FQHCs use UDS measures to assess healthcare services provided in their organization, the APRN residency director was also interested in understanding how residents performed on UDS measures and if there is any correlation between performance on UDS measures and provider satisfaction (D. Potter, personal communication, November 10, 2020). FQHCs use UDS NURSE PRACTITIONER RESIDENCY PROGRAMS 10 measures to determine the quality of care provided to their patients in comparison to other FQHCs nationally. Information on UDS measures can be found on the HRSA website. The nation's topperforming FQHCs receive financial rewards and website recognition for quality improvement, leadership, and performance measures (HRSA, 2020). The FQHC in the Southwest received a bronze level as a health quality leader, which meant their performance on quality indicators was not among the top achievers (HRSA, 2020). Stakeholders at this FQHC report that providers struggle to meet the screening goals, which is substantiated by the HRSA data (HRSA, 2019). PICOT Question This clinical problem led to the PICOT question: Do Advanced Practice Registered Nurse (APRN) graduates of a twelve-month Nurse Practitioner Residency program experience greater job satisfaction than APRN providers without residency experience? Evidence Synthesis Search Strategy To answer the PICOT question, an exhaustive literature search was performed to find the most current evidence-based literature. Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO, and PubMed databases were selected due to the high probability of generating peer-reviewed journals with a nursing focus. Each database required an advanced search, key terms, and a unique search strategy. All initial searches were published between 2016 and 2021, peer-reviewed, and included the following keywords: nurse practitioner, residency, satisfaction, performance, transition, or skills. Some results produced articles with little value to the topic. Therefore, additional searches were used with the following quoted phrases: “new APRN,” “New NP,” “postgraduate training,” “postgraduate education,” “primary care,” or “family practice.” This search strategy produced 91 articles, of which 10 were selected for rapid and critical appraisal based on relevance to the PICOT question. Of the NURSE PRACTITIONER RESIDENCY PROGRAMS 11 articles, eight were level four evidence from well-designed cohort studies, and two were level six evidence from single qualitative studies. Critical Appraisal The contents of the articles were separated into evaluation tables and simplified into a synthesis table (see Appendix B). Although many articles differed in their purpose and findings, there were some commonalities. For instance, all studies (see Appendix B) were conducted in the United States and published within the last five years. Additionally, many articles used quantitative and cross-sectional designs, sampled master prepared APRNs in the primary care setting, and measured outcomes by a survey. Several articles used postgraduate residency or fellowship programs as the independent variable but evaluated different clinical outcomes. Although the studies measured different outcomes, they generally found increased confidence and competency after completing a residency. This finding is consistent with the literature reviewed earlier in the text. Another article found that NPs that completed a residency felt more valued and had better compensation than those who did not (see Appendix B). Although the studies demonstrated the value of residency programs, a gap in the literature exists between the difference in competency among those who complete a postgraduate program and those that do not. Interestingly, two studies examined a short practicum course for students nearing graduation. The programs lacked the structure of a typical residency, and the results were inconclusive on whether these interventions were effective in improving clinical competency (see Appendix B). Thus, a more structured and more prolonged course, such as a 12-month residency program, demonstrates more value for the refinement of clinical skills. Only one study NURSE PRACTITIONER RESIDENCY PROGRAMS 12 identified the necessary components of a residency program and determined that mentorship, leadership, collaboration, quality improvement, and clinical competency were essential. A few articles examined different aspects of role transition and job satisfaction. One study identified that autonomy, open communication, work-life balance, and meaning in work facilitated role transition (see Appendix B). Conversely, lack of understanding of the NP role, disrespect for NPs, low compensation, and limited time to see patients were deterrents for role transition and satisfaction. Another article indicated correlations between physicians and administrator relationships and better clinical outcomes (see Appendix B). These study findings and those previously stated in the text are important to ease role transition and job satisfaction, critical to retaining nurse practitioners in the field. Synthesis Statement Many new graduate APRNs lack confidence and refinement of clinical skills upon graduation, leading to job dissatisfaction, impaired role transition, and even abandonment of the NP role. Meanwhile, a supportive environment with mentorship, respect, and knowledge of the NP role is necessary for a successful transition and job satisfaction. As a result, most NP students would benefit from a highly structured postgraduate residency program to strengthen confidence and facilitate role transition. However, there remains little evidence on NP job satisfaction from residency and performance on quality measures and delivery of care among graduates of an FNP residency compared to non-graduates. Foundation of the Project Although there is evidence to support NP residency programs, more evidence is needed to show how they directly impact patient care and outcomes. The literature demonstrates the importance of provider satisfaction for retention, improved patient outcomes, and reduced costs. NURSE PRACTITIONER RESIDENCY PROGRAMS 13 Given the high attrition rates of NP residents at the project site, the project aims to evaluate FNP residents’ and graduates’ satisfaction through the Misener Nurse Practitioner Job Satisfaction Scale (MNPJSS) (Misener & Cox, 2001). The potential outcomes of this QI project include program evaluation, modifications, increased satisfaction, recruitment to residency programs, retention of practitioners in the field, and overall improved patient outcomes. Conceptual Framework and Evidence-Based Practice Model Theoretical Framework The theoretical framework behind the QI project is Frederick Herzberg’s Two-Factor Theory. The theory states that factors that increase job satisfaction are called motivators, and those that influence job dissatisfaction are called hygiene factors (Syptak et al., 1999). According to Herzberg, hygiene factors include working conditions, relationships with peers, quality of supervisors, wage or salary, and the organization's policies (Syptak et al., 1999). If the hygiene factors are not met to the worker's satisfaction, dissatisfaction occurs. On the other hand, motivating factors lead to job satisfaction, including the enjoyment of the work itself, achievement, recognition, responsibility, and advancement (Syptak et al., 1999). The theoretical framework guided the use of the MNPJSS, a 44-question Likert scale that measures hygiene and motivating factors that influence job satisfaction. Implementation Framework The implementation framework serves as a roadmap to guide the QI project. The Centers for Disease Control and Prevention’s framework for program evaluation in public health (Center for Disease Control and Prevention [CDC], 2015) is a practical tool for evaluating postgraduate APRN programs in an FQHC setting. Steps within the program evaluation framework include engaging stakeholders, describing the program, focusing the evaluation design, gathering NURSE PRACTITIONER RESIDENCY PROGRAMS 14 credible evidence, justifying conclusions, and disseminating and using lessons learned (CDC, 2015). Additionally, four essential standards are at the center of the framework: utility, feasibility, propriety, and accuracy (CDC, 2015). These core standards guide an ethical, precise, helpful, and accurate project delivered to stakeholders (CDC, 2015). Core standards were met by obtaining approval through the Institutional Review Board (IRB) and the project site’s review committee and close communication with stakeholders throughout the various stages of the project. Methods Ethical Considerations This QI project adheres to ethical principles: autonomy, beneficence, and nonmaleficence. Autonomy refers to the participants' right to self-determination and the ability to make informed decisions regarding participation in the project, which was met by participants signing an informed consent that was pre-approved by Arizona State University’s Institutional Review Board (IRB) and the organization’s review committee. To protect participants’ privacy, all information was de-identified and an option of “prefer not to answer” was provided on demographic questions to reduce the concern of being identified by stakeholders. The concept of beneficence implies that projects are intended to benefit others, which was done in this project by gathering information regarding postgraduate nurse practitioner residents' satisfaction in their program to improve the overall experience. Lastly, non-maleficence is the concept of protecting participants from harm, which will be accomplished by de-identifying information. Population and Setting A FQHC in the Southwest is the only primary care NP residency in the state (National Nurse Practitioner Residency & Fellowship Training Consortium [NNPRFTC], 2021). Clinicians NURSE PRACTITIONER RESIDENCY PROGRAMS 15 in the FQHC provide primary care services to underserved populations and low-income individuals in the Southwestern State. The FQHC-based NP residency programs was developed to increase the number of providers and clinically competent NPs who could serve this more complex patient population. The first residency cohort began their twelve-month program in September 2019 and graduated in August 2020. The program director cited high attrition rates among the first cohort as a reason to evaluate residents’ satisfaction. The average residency cohort size is about eight, with previous classes the majority of resident’s being masters prepared APRNs. Impact This program evaluation can impact a broad group of stakeholders at the microsystem, mesosystem, and macrosystem levels. The individuals affected at the microsystem level include the FNP residents, the individual patients, provider mentors, the partnered University faculty, support staff, and the APRN residency program director. Individuals at the microsystem level are at the organization's center and are all impacted by the slightest changes to the residency program (Porter-O’Grady & Malloch, 2018). Meanwhile, stakeholders at the mesosystem level include the organization’s leaders. For example, the executives of operations, human resources, medical informatics, clinical operations, grant funding, and finance would all be impacted by the findings of provider satisfaction. Turnover rates directly impact these stakeholders due to the cost, recruitment efforts, and impact on reputation. Lastly, stakeholders of the macrosystem level are the organizations. These organizations include the FQHCs, FNP residency programs, University partners, overall community health, and the federal government. The results of this QI project can add evidence to the literature regarding structured NP residency programs and NURSE PRACTITIONER RESIDENCY PROGRAMS 16 influence federal grant funding for residencies, increase interest in FNP residency programs, and increase medical services to underserved communities. Project Design Residents and recent residency graduates from three cohorts were surveyed with the MNPJSS two times. The first survey was sent three months from the most recent cohort’s start date and the second survey five months from the program start date. To be included for consideration, participants must be current residents or recent graduates from the NP residency program and still work for the FQHC or its affiliates. Eligible participants were identified by the Residency Coordinator at the project site and e-mailed a preapproved recruitment letter with a link to SurveySparrow. Participants used the link to access the password-protected survey, typed their names into an electronic consent, and completed the MNPJSS survey and a demographic questionnaire. The anticipated time to complete the survey was about 5 minutes. A reminder email was sent one week after the original e-mail and on day 11. Participants were given 72 hours to complete the survey after sending the final reminder e-mail. Data Collection and Analysis Participants were de-identified and given a unique identification code. Their survey responses, demographic data, and unique identifiers were transcribed into an excel spreadsheet, password-protected, and reviewed twice for accuracy by two project team members. Data from the MNPJSS was scored out of 258 and averaged by cohort. Responses were also separated and averaged by subcategories of satisfaction. For example, the MNPJSS contains several subscales demonstrating validity and reliability: “Interpractice Partnership/Collegiality; Challenge/Autonomy; Professional, Social, and Community Interaction; Professional Growth; Time; and Benefits” (Bush & Lowery, 2016, p. 229). Respectively, the subscales demonstrate the NURSE PRACTITIONER RESIDENCY PROGRAMS 17 following reliability scores: “0.94, 0.89, 0.84, 0.86, 0.89, and 0.79” (Bush & Lowery, 2016, p. 229). Demographic data collected include gender, age in ranges, race, the highest degree of education, number of years of nursing experience, status in the program, employment status, and an option of prefer not to answer. Demographic data and survey responses were analyzed using Intellectus Software and an Intellectus statistician. Analyses included a two-tailed independent sample t-test and descriptive statistics. Outcomes measured included provider satisfaction at different stages of the residency program and after program completion in the FQHC. Lastly, as a shared project with another student, average UDS scores pulled from chart reviews were linked with satisfaction scores using a Pearson correlation test to determine any correlation. Budget Project costs were estimated to be $400 and were personally funded by the student. Direct project funds included a paid subscription to SurveySparrow and the use of an Intellectus Software statistician for data analysis. Indirect costs included travel expenses (See Appendix D). Results Demographic Analysis and Descriptive Statistics Procedures A convenience sample of participants completed the surveys. Initial respondents (n = 11), of those, current residents (n = 3), one-year post-residency (n = 6), and two-years post-residency (n = 2). Total respondents for the second survey (n = 8). Of those, current residents (n = 3), oneyear post-residency (n = 4), and two-years post-residency (n = 1). Three individuals completed both questionnaires: current residency group (n = 1) and one-year post residency group (n = 2). Demographic responses were analyzed using descriptive statistics. Most respondents had a NURSE PRACTITIONER RESIDENCY PROGRAMS 18 master’s degree (87.50%), 0–5 years of nursing experience (43.75%), were age 31 – 40 (75.00%), female (93.75%), white (62.50%), and full-time employees (100%) (see Appendix E). Statistical Analysis and Procedures Satisfaction questions from the MNPJSS were measured on a six-point Likert scale, ranging from “6” (Very Satisfied), “5” (Satisfied), “4” (Minimally Satisfied), “3” (Minimally Dissatisfied), “2” (Dissatisfied), and “1” (Very Dissatisfied). Responses from each question were summed (258 = highest possible score) and averaged for all groups. A two-tailed independent ttest compared the overall satisfaction between December 2021 and February 2022. In December, the overall average satisfaction score was 158.09 (SD =36.05) and 164.92 (SD =49.2) in February. The cohen’s d = 0.16, meaning a small effect size. Increase in satisfaction rates were not statistically significant (alpha value of .05, t(17) = -0.35, p = 0.73) (see Appendix F). In both surveys, a participant outlier was identified. The outlier was the same in both surveys, one-year post-residency, with a satisfaction score of 84 for both surveys. A two-tailed independent sample t-test analysis was performed without the outlier's responses to determine the impact on overall satisfaction scores. As expected, average scores increased (see Appendix G). In December, the mean overall satisfaction was 165.50 (SD = 27.80) and in February, 176.48 (SD = 39.81), still not statistically significant (alpha value of .05, t(17) = -0.67, p = 0.512). The effect size increased but remained small with a cohen’s d = 0.32. The mean scores from December and February were averaged (M = 170.99), divided by the highest possible score (258), and multiplied by 6 (the highest score on the Likert scale). The average response M = 3.98, which indicated “minimally satisfied.” Descriptive Statistics by Subcategory and Cohort NURSE PRACTITIONER RESIDENCY PROGRAMS 19 Data was further analyzed using descriptive statistics by each cohorts' subscales of satisfaction (time, interactions, collegiality and partnership, professional growth, benefits, and challenge and autonomy). Current residents' average response on the Likert scale, M = 3.77, indicated scores trended towards "minimally satisfied" overall. They were most satisfied with challenge and autonomy (M = 4.28, SD = 0.34) and interactions with others (M = 4.00, SD = 0.86). They were least satisfied with collegiality and partnership (M = 3.26, SD = 0.67). Providers one-year post-residency had an overall response of M = 3.98, minimally satisfied. They were most satisfied with benefits (M = 4.53, SD = 1.07) and least satisfied with time (M = 3.04, SD = 1.30). Providers in their second-year post-residency had a response of M = 3.49, indicating a neutral response, neither "minimally satisfied" nor "minimally dissatisfied. They were most satisfied with benefits (M = 4.56, SD = 0.49). They were least satisfied with time (M = 2.90, SD = 1.43) see Appendix J. UDS Averages and Satisfaction Scores As a shared project, UDS performance averages were analyzed with overall satisfaction scores. Using Pearson Correlation, no significant correlations could be determined, r = 0.01, p = 0.968. Summary of Outcomes Surveys responses after two months showed increased satisfaction levels but were not statistically significant. The overall satisfaction did increase when analyzed without the outlier, and generally, the cohorts were "minimally satisfied." Further analysis showed that current residents had average responses that were "minimally satisfied." Their highest satisfaction was with questions related to challenge and autonomy. Whereas opportunities to improve were with collegiality and partnership. Providers that were one-year post-residency were the most satisfied NURSE PRACTITIONER RESIDENCY PROGRAMS 20 overall. The average response showed they were "minimally satisfied" even with the outlier. They were most satisfied with areas related to their benefits and least satisfied with time. Conversely, the second-year post-residency graduates were the least satisfied of the groups, which could threaten retention at the organization. However, their average responses were neutral, indicating neither job satisfaction nor dissatisfaction. Like the previous cohort, they were also most satisfied with benefits and were dissatisfied with questions related to time. Lastly, there was no correlation between performance on UDS metrics and overall job satisfaction among this population. Impact The impact of this project has implications for the organization, providers, and the system. Program improvements at the project site can improve provider satisfaction and increase retention. For the organization, retention is necessary to continually receive federal funding, reduce provider shortages, and provide continuity of care for underserved populations. Sustainability Plans are in place for this project to continue as a legacy project with a future Arizona State student. The next project would be a mixed-methods design, including administration of the MNPJSS surveys and a focus group session that elaborates on factors contributing to the satisfaction scores. The aim is to publish future projects to add information to the literature. Discussion Summary These project findings suggest that the APRN residents and graduates of this program are, for the most part, "minimally satisfied" in their job, and satisfaction was slightly higher in February compared to December. Current residents were most satisfied with Challenge and NURSE PRACTITIONER RESIDENCY PROGRAMS 21 Autonomy, which included expanding skill level and procedure score, delivering quality care, advancing scope of practice, and challenges at work (Misener & Cox, 2001). Satisfaction in this area suggests that providers believe they are growing in their competencies, which would be an expected outcome of a residency program. Analysis showed minimal dissatisfaction for current residents within the subgroup of collegiality and partnership, which included: opportunities to receive compensation for services outside regular duties, input into policy at the organization, and administrative support (Misener & Cox, 2001). Subcategories of dissatisfaction provided opportunities for program improvement and to reassess satisfaction. Finally, providers one-year post-residency and two-year post-residency were both "minimally dissatisfied" with time, including time to answer messages, time to see patients, and policies and practices around scheduling patients. Both cohorts were most satisfied with benefits, which pertained to benefit packages, the retirement plan, and vacation and leave policies. Limitations Several project limitations were identified, including the limited time between the two surveys. Initial surveys were postponed because eligible providers were learning a new Electronic Health Record (EHR), which interrupted normal flow. Stakeholders were concerned the providers may be overwhelmed by the changes at the project site and not respond to surveys. Therefore, initial surveys were delayed, and the time between the second survey could not be extended due to time constraints to complete the project. Despite these modifications, there were very few participants responded to both surveys. Additionally, because the cohort sizes and response rates were too small, statistical analysis of satisfaction could not be measured for individual cohorts. Furthermore, a higher response rate from the current residency groups would have provided greater insight into overall NURSE PRACTITIONER RESIDENCY PROGRAMS 22 satisfaction and program strengths and weaknesses. Another limitation is that results from project are specific to this organization and cannot be generalized to other residency programs. Finally, only 43 out of the 44-MNPJSS questions were administered to the participants. The deviation was due to human error when entering the survey questions into SurveySparrow. The specific question skipped related to provider satisfaction with time allotted to review labs and other test results. This error could have reduced the reliability of the survey scale, which was .89. Given that two cohorts were least satisfied with time, that could have changed depending on their responses to that question. Strengths There were also several strengths of this project. First, the average response time to the demographic questionnaire and MNPJSS was only 5-minutes, which did not take up much of their time and could be replicated on a larger scale. Additionally, the project was unique because it provided insights into the satisfaction of three cohorts during and after the 12-month residency program. Other Literature Participants at this project site were less satisfied overall compared to work by Bush & Lowery (2016), who found residency graduates' average response was a 5 (satisfied) on the MNPJSS. A factor worth considering is the timing of the project during a global pandemic. The coronavirus pandemic has put a strain on the health care system and led to higher turnover. The providers that finished postgraduate education two years ago were in the middle of their residency when the pandemic started, which may have contributed to high attrition rates during the program's first few years. Moreover, at the time of this manuscript, the healthcare system remains strained, which may have influenced overall satisfaction scores. NURSE PRACTITIONER RESIDENCY PROGRAMS 23 Future Recommendations Future QI projects at this site will likely include an additional qualitative component to understand satisfaction better. Meanwhile, future studies should examine provider satisfaction at several sites to better generalize provider satisfaction in NP residency programs. Finally, it would be beneficial to compare resident satisfaction to providers that did not enroll in a residency program. Conclusion National guidance from the IOM (2010) recommended increasing residency programs for APRNs to ease the transition into practice, reduce provider shortages, and create more confident and competent NP providers (Bryant & Parker, 2020). Unfortunately, there are not enough of these programs available for all the APRN graduates that would benefit from postgraduate education. More evidence is needed to show that they are effective and should be expanded. Project outcomes at the FQHC showed that FNP residents and residency graduates were overall "minimally satisfied." Program improvements would likely improve provider retention and increase federal funding for the program. On a larger scale, improving provider satisfaction in the FQHC will help reduce provider shortages and advance health equity. NURSE PRACTITIONER RESIDENCY PROGRAMS 24 References Brown, K., Poppe, A., Kaminetzky, C., Wipf, J., & Fugate Woods, N. (2015). Recommendations for nurse practitioner residency programs. Nurse Educator, 40(3), 148-151. https://doi.org/10.1097/NNE.0000000000000117 Bryant, S., & Parker, K. (2020). Participation in a nurse practitioner fellowship to instill greater confidence, job satisfaction, increased job retention. Journal of the American Board of Family Medicine, 32(10), 645-651. https://doi.org/10.1097/JXX.0000000000000313 Bush, C. T. & Lowery, B. (2016). Postgraduate nurse practitioner education: Impact on job satisfaction. The Journal for Nurse Practitioners, 12(4), 226-234. http://dx.doi.org/10.1016/j.nurpra.2015.11.018 Center for Disease Control and Prevention. (2015, December 8). CDC framework for program evaluation. Workplace health promotion. https://www.cdc.gov/workplacehealth promotion/model/evaluation/long-description3.html Dillon, D. L., Dolansky, M. A., Casey, K., & Kelley, C. (2016). Factors related to successful transition to practice for acute care nurse practitioners. American Association of CriticalCare Nurses, 27(2), 173-182. https://doi.org/10.4037/aacnacc2016619 Dumphy, D., DeSandre, C., & Thompson, J. (2019). Family nurse practitioner students’ perceptions of readiness and transition into advanced practice. Nursing Forum, 54(1), 352-357. https://doi-org.ezproxy1.lib.asu.edu/10.1111/nuf.12336 Faraz, A. (2019). Facilitators and barriers to the novice nurse practitioner workforce transition in primary care. Journal of the American Association of Nurse Practitioners, 31(6), 364-370. https://doi.org/10.1097/jxx.0000000000000158 Health Resources & Service Administration. (2019). Health center program data. NURSE PRACTITIONER RESIDENCY PROGRAMS 25 https://data.hrsa.gov/tools/data-reporting/program-data?grantNum=H80CS00230 Health Resources & Services Administration. (2020). Quality improvement awards. https://bphc.hrsa.gov/program-opportunities/quality Hicks, K. E., Rico, J., & Beauchesnes, M. (2018). Core curriculum and competencies: A multisite analysis of postgraduate training programs for primary care nurse practitioners. Journal for Nurses in Professional Development, 34(6), 454-462. https://doi.org/10.1016/j.profnurs.2017.12.012 Martsolf, G. R., Nguyen, P., Freund, D., & Poghosyan, L. (2017). What we know about postgraduate nurse practitioner residency and fellowship programs. The Journal for Nurse Practitioners, 13(1), 482-487. https://doi.org/10.1016/j.nurpra.2017.05.013 Misener, T. R. & Cox, D. L. (2001). Development of the misener nurse practitioner job satisfaction scale. Journal of Nursing Measurement, 9(1), 91-108. https://pubmed.ncbi.nlm.nih.gov/11469145/ National Nurse Practitioner Residency & Fellowship Training Consortium. (2021). Postgraduate Training.: Locations. https://www.nppostgradtraining.com/postgraduatetraining/locations/ Poghosyan, L., Norful, A. A., Jianfang, L., & Friedberg, M. W. (2018). Nurse practitioner practice environments in primary care and quality of care for chronic diseases. Medical Care, 56(9), 791-797. https://doi.org/10.1097/MLR.0000000000000961 Rugen, K. W., Dolansky, M. A., Dulay, M., King, S., & Harada, N. (2018a). Evaluation of Veterans Affairs primary care nurse practitioner residency: Achievement of competencies. Nursing Outlook, 66(1), 25-34. https://doi.org/10.1016/j.outlook.2017.06.004 NURSE PRACTITIONER RESIDENCY PROGRAMS 26 Rugen, K. W., Harada, N., Harrington, F., Dolansky, M. A., & Bowen, J. L. (2018b). Nurse practitioner residents’ perceptions of competency development during a year-long immersion in Veteran Affairs primary care. Nursing Outlook, 66(4), 352-364. https://doi.org/10.1016/j.outlook.2018.05.002 Scaglione, K. & Loyd, V. (2021). A game changer: Adult-geriatric acute care nurse practitioner fellowship for the veterans’ health administration. Advance online publication. https://dx.doi.org/10.1891/JDNP-D-20-00031 Speight, C., Firnhaber, G., Scotts, E. S., & Wei, H. (2019). Strategies to promote the professional transition of new graduate nurse practitioners: A systematic review. Nursing Forum, 54(4), 557-564. https://dx.doi.org/10.1111/nuf.12370 Spetz, J., Skillman, S. M., & Andrilla, H. A. (2017). Nurse practitioner autonomy and satisfaction in rural settings. Medical Care Research and Reviews, 74(2), 227-235. https://dx.doi.org/10.1177/1077558716629584 Syptak, J. M., Marsland, D. W., & Ulmer, D. (1999). Job Satisfaction: Putting theory into practice. Family Practice Management, 6(9), 26-30. https://www.aafp.org/fpm/1999/1000/p26.html Wiltse Nicely, K. L., & Fairman, J. (2015). Postgraduate nurse practitioner residency programs: Supporting transition to practice. Academic Medicine. 90(6), 707-709. https://dx.doi.org/10.1097/ACM.0000000000000567 POSTGRADUATE NURSE PRACTITIONER PROGRAMS 27 Appendix A Evaluation and Synthesis Tables Table A1 Qualitative Evaluation Table Citation Theoretical/ Conceptual Framework Design/ Method/ Sampling Sample/Setting Major Variables/ Research Questions Measurement/ Instrumentation Data Analysis Findings/ Themes Citation: Rugen et al. (2018b) Country: US Funding: VA’s Office of Academic Affiliations. Bias: Selfreport bias from participants. No reported bias for research stated. Framework: NS. Grounded Theory Inferred. Design: NS. Exploratory Qualitative Research Inferred. Purpose: To determine residents perceived development throughout the residency program. Sampling: Convenience Sampling. N= 38 Demographics: 84.2% Female, 50% had trained with the VA as NP students, 100% had a master’s level education, and average RN experience was 5.46 prior to pursing an APRN degree. Setting: Primary Care NP residents from the VA working in Boise, Cleveland, San Francisco, Seattle, and West Haven. From 2012-2015. Exclusion: None. Was part of the conditions of the program. Attrition: 2/38, 5.3%. IV1: Residency DV1: Clinical competence DV2: Leadership competence DV3: Interprofessional collaboration DV4: Patient-centered care DV5: Shared decision making DV6: Sustained relationships DV7: Performance improvement Research Questions: How do new NP’s perceive their strengths, learning needs, goals, opportunities at several stages while participating in the department of Veteran’s Affairs Primary Care NP Residency program? Definitions: Rugen et al. (2018b) defined clinical competence, leadership competence, interprofessional collaboration, patientcentered care, shared decision-making, sustained relationships, performance Measurement: NP Residency Competency Assessment Tool (69 competency questions Likert Scale), 7 openended questions. The results published in this paper were the 7 qualitative questions. Completed at 1 month, 6 months, and 12 months of the program. Data Analysis: 1. Coded and mapped openended responses 2. They used Atlas TI version 7.5.10 for qualitative analysis. 3. Conventional content analytic procedures. 4.Data was displayed using histograms. Findings Theme 1: Started program strong with patient-centered care. Strengths improved in other categories over time. Theme 2: Goals to improve at all times were in clinical and professional development. Theme 3: There was few reports of a desire to focus on leadership and performance improvement. Additional Findings: In the early stages of the residency, participants felt confident in more basic skills and procedures. By the end of their residency, they felt good at complex skills and complex care. Specific statistical tests were not stated or not used. Note. Key, DVx = Dependent Variable, IVx = Independent Variable, mo. = months, N = Sample size, NP = Nurse Practitioner, NS = None stated, PICOT = Population, Intervention, Comparison, Outcomes, & Timing, RN = Registered Nurse, US = United States, VA = Veteran’s Affairs Level of Evidence; Application to practice/ Generalization Level of Evidence: Level VI Strengths: Evaluated responses at three stages of the program Weakness: Limited samples size, NPs were not required to complete open ended questions. Questions provided limitations. Self-reported bias, no anonymity, short answers only, limited responses on last two questions. Feasibility: Not mentioned in research. However, most funding for residency programs comes from federal funds. Thus, program feasibility is limited based on funding. Utility to PICOT: Relevant to population, intervention, & time. POSTGRADUATE NURSE PRACTITIONER PROGRAMS Citation: Faraz (2018). Country: US Funding: NS. Bias: No conflicts of stated by researcher. Took measures to prevent participants from taking the survey more than once. Framework: Created by author. Factors influencing the successful transition of novice NPs in the primary care workforce. Design: Qualitative: Descriptive, cross-sectional design. Purpose: Identify factors contributing or detracting from a successful work transition. Sampling: Convenience, snowballing, & indirect methods. N = 177 Demographics: 92.9% women, 35 average age, 79.7% masters, 86.4% white, 66.1% married, 28.8% northeast, 65% Family focus, 71.8% no other nonRN healthcare job, 41.2% 6+ years RN experience, 32.8% working 9-12 months. Setting: National survey Exclusion: Not employed as NP, less than 3 mo. Employment or more than 12 mo., & age younger than 18. Inclusion: practicing in primary care for 312 mo. Attrition: 24/201 or 12% did not complete. improvement, and professional development (p. 355356). IV1: NP role transition DV1: Factors influencing satisfaction and dissatisfaction. Research Questions: What factors facilitate or detract from a successful transition into NP practice in primary care? Definitions: No definitions. 28 Measurement: online survey with 5 open-ended questions. Data Analysis: 1. Descriptive statistics for study demographics reported as number and percentage. 2. Aggregate themes using Krippendorf content analysis. Specific statistical tests were not stated or not used. Findings: Theme 1: Support/Mentorship to helped transition. Theme 2: More autonomy led to greater satisfaction. Less led to dissatisfaction. Theme 3: Learning and professional development. Feeling able to openly discuss questions. Theme 4: Work-life balance, flexibility in scheduling, and not staying late were important. Theme 5: Meaning in work was influential. Theme 6: Dissatisfaction for when lack of respect for NPs. Theme 7: Role ambiguity as the first NP is a dissatisfaction. Theme 8: Lack of support, isolation, or lack of resources leading to dissatisfaction. Theme 9: Workload and complexity of care in a short time led to dissatisfaction Theme 10: Lack of compensation, and time off, Note. Key, DVx = Dependent Variable, IVx = Independent Variable, mo. = months, N = Sample size, NP = Nurse Practitioner, NS = None stated, PICOT = Population, Intervention, Comparison, Outcomes, & Timing, RN = Registered Nurse, US = United States, VA = Veteran’s Affairs Level of Evidence: Level VI Strengths: Large sample size. Weaknesses: Demographics did not include residency enrollment. Feasibility: NS, but moderately feasible. Depended on others to forward e-mails on to recruit participants. Only took 10-15 minutes to complete, but most did not meet the inclusion criteria. Utility to PICOT: Relevant to population, outcomes, and timing. POSTGRADUATE NURSE PRACTITIONER PROGRAMS Citation Citation: Dillon et al. (2016) Country: US Funding: NS. Bias: NS. Theoretical/ Conceptual Framework Theoretical Framework: Meleis Transition Theory and Situational Transition Model Design/ Method/ Purpose Sample/Setting Design: Quantitative: Descriptive, correlational comparative design. Methods: Recruitment by a cover letter on website. Participants completed a 20minute survey. Purpose: 1. Determine relationships between personal resources, transition, and job retention during first 6 months. 2. Identify differences in experiences among RNs with more or less prior nursing experience. 3. Identify skills that were difficult to N= 34 Demographics: Mostly white women age 41-50. From 15 states including Puerto Rico. 75% with greater than 5 years of nursing experience in ICU or ED. 100% with a master’s degree. Setting: ACPNs in acute care setting Exclusion: Not stated. Inclusion: Must be an AG-ACNP or board certified ACNP, have 6 mo. to 3 years of experience in role. A member of social media ACNP network. 29 Table A2 Quantitative Evaluation Table Variables Measurement/ Instrumentation Data Analysis Results/ Findings IV1: Years of ED or ICU experience as an RN DV1: Skills DV2: Transition DV3: Community Resources DV4: Personal Resources Research Questions: Are there differences in role transition based on years of RN experience? Definitions: 1. PF: stress 2. CR: Organization support, communication, and leadership 3. ST: comfort, confidence, patient safety, professional satisfaction, job satisfaction 4. Retention: Length of time in first Measurement: The Casey-Fink Graduate NP Experience Survey. Includes demographics, skills/procedure performance, successful transition measurements, community resources, and personal resources. Validity & Reliability: Measurement tool that has been frequently used to evaluate role transition. Specifics to validity and reliability not mentioned. Statistical Analysis: 1. Descriptive statistics. 2. Bivariate correlations 3. Nonparametric tests. 4. Pearson correlational statistics. Results: Organizational support correlated: 1) with comfort & confidence r=0.49; P<.01 2) patient safety r=0.72; P<0.5 3) satisfaction r=0.72; P<0.5 4) Job satisfaction r=0.53, P<0.01 Communication and leadership correlated with comfort and confidence (r=0.68; P<0.1), patient safety (r=0.62, P<0.1), professional satisfaction (r=0.57; P <0.1) Level of Evidence; Application to practice/ Generalization Level of Evidence: Level IV Strengths: Not stated. Strong statistical evidence and display of results in concise tables. Weakness: Small sample size. Based on memory recall. Limited Feasibility: Very feasible study design. Participants completed a 20-minutelong survey. Utility to PICOT: Relevant to possible outcome Transition. No different between years of experience and transition, retention, or resources. Most difficult skills included: cricothyrotomies, Note. Key. ACPN = Acute Care Nurse Practitioner Network, AG = Adult-gerontology, AHRQ = Agency for Healthcare Research and Quality, AP = Acceptance by physicians, APRN = Advanced Practice Registered Nurse, BA = Bonuses available, CHC = Community Health Centers, CI = Confidence Interval, CO = Compensation, CR = Community Resources, CVD = Cardiovascular Disease, CXR = Chest X-ray, DEA# = Drug Enforcement Administration Number, DNP = Doctor of Nursing Practice, DQ = Ability to deliver quality of care, DVx = Dependent Variable, ECG = electrocardiogram, ED = Emergency Department, ES = Expand scope of practice, F = Fellowship, FNP = Family Nurse Practitioner, FMP = Family Practice, FORHP = Federal Office of Rural Health Policy, FP = Flexibility in Protocols, FQ = Freedom to Question Practice, GP = General Practice, HBC = Hospital-Based Clinic, HEDIS = Healthcare Effectiveness Data and Information Set measures, HRSA = Health Resources & Services Administration, hr = hour, ICU= Intensive Care Unit, IM = Internal Medicine, IP = Input in Organization, IPS = Independent Practice & Support, IRB = Institutional Review Board, ISRA = Isolated Small Rural Area, IVx = Independent Variable, LA = Level of autonomy, LRA = Large Rural Area, MA = Massachusetts, mo. = month, ME = Mean, MNPJSS = Misener Nurse Practitioner Job Satisfaction Scale, MPD = Massachusetts Provider Database, N = Sample size, NF = No Fellowship, NIH = National Institute of Health, NP = Nurse Practitioner, NP-AR = Nurse Practitioner–Administration Relationship, NP-PCOCQ = Nurse Practitioner-Primary Care Organizational Climate Questionnaire, NP-PR = Nurse Practitioner– Physician Relationship, NS = None stated, OB/GYN = Obstetrics & Gynecology, OC = Opportunity for change, OG = Opportunity for growth, OR = odds ratio, P = Statistical significance, PF = Personal factors, PICOT = Population, Intervention, Comparison, Outcomes, & Timing, PO = Physician Office, PR = panel ratio, Q = Question, QI = Quality Improvement, r = Pearson correlational coefficient, RN = Registered Nurse, RO = Respect for opinion, RS = Respect from superiors, SA= Sense of Accomplishment, SRA = Small Rural Area, ST = Successful transition, SV = Sense of value for what you do, UA = Urban Area, UP = University prepared, US = United States, USDHHS = U.S. Department of Health and Human Services, VA = Veteran’s Affairs, VA CoEPCE = Veteran’s Affairs Centers of Excellence in Primary Care Education, vs. = versus POSTGRADUATE NURSE PRACTITIONER PROGRAMS Citation: Rugen et al. (2018a) Country: US Funding: VA’s Office of Academic Affiliations Bias: Some selfevaluation. No other bias stated. Theoretical Framework: NS. Meleis Transition Theory inferred. perform Sampling: Convenience sampling recruited from LinkedIn. Design: NS. Quantitative: Descriptive, correlational comparative cohort study design inferred. Methods: Mentor and selfratings of competency measured through the 12-mo. long program. Items of highest and lowest ratings or greatest discrepancies between NP residents were then analyzed. Purpose: To describe NP residency outcomes over three time periods. Sampling: Convenience. Attrition: 0 position, thoughts of leaving N= 38 Demographics: 84.2% Female, 50% had trained with the VA as NP students, 100% had a master’s level education, and average RN experience was 5.46 prior to pursing an APRN degree. Setting: Primary care residents from five VA sites. Exclusion: Individuals not selected for the NP residency program. Inclusion: 1. Graduation from accredited master’s or DNP FNP or AG program. 2. Attained state IV1: VA CoEPCE residency DV1: Clinical competency DV2: Leadership DV3: Interprofessional Team Collaborations DV4: Patientcentered care DV5: Shared decision-making DV6: Sustained relationships DV7: Performance improvement; population management Research Questions: How do NP residency outcomes compare over time? Definitions: None. 30 documentation, and coding. Measurement: VA CoEPCE NP Residency Competency Assessment Tool. Made of 69 items and scored from 0-5 on level of supervision required for a skill. Validity & Reliability: Based on preliminary psychometric analysis, the domains were consistent. Statistical Analysis: Descriptive statistics. Twotailed standardized t tests. Generalized linear models. Results: All domains (clinical competency, leadership competency, interprofessional collaboration, patientcentered care, shared decision-making, sustained relationship, and quality improvement) increased over the 12-mo. period. P = <.0001 for residents and mentors. Level of Evidence: Level IV Strengths: Used selfreports and mentorreports. Measured at several intervals of the program. Weakness: Low response rate. Small sample size. Feasibility: NS. Fairly practical study design. Risk of incomplete surveys when program is finished. Utility to PICOT: Relevant to population, intervention (fellowship), & time period. Note. Key. ACPN = Acute Care Nurse Practitioner Network, AG = Adult-gerontology, AHRQ = Agency for Healthcare Research and Quality, AP = Acceptance by physicians, APRN = Advanced Practice Registered Nurse, BA = Bonuses available, CHC = Community Health Centers, CI = Confidence Interval, CO = Compensation, CR = Community Resources, CVD = Cardiovascular Disease, CXR = Chest X-ray, DEA# = Drug Enforcement Administration Number, DNP = Doctor of Nursing Practice, DQ = Ability to deliver quality of care, DVx = Dependent Variable, ECG = electrocardiogram, ED = Emergency Department, ES = Expand scope of practice, F = Fellowship, FNP = Family Nurse Practitioner, FMP = Family Practice, FORHP = Federal Office of Rural Health Policy, FP = Flexibility in Protocols, FQ = Freedom to Question Practice, GP = General Practice, HBC = Hospital-Based Clinic, HEDIS = Healthcare Effectiveness Data and Information Set measures, HRSA = Health Resources & Services Administration, hr = hour, ICU= Intensive Care Unit, IM = Internal Medicine, IP = Input in Organization, IPS = Independent Practice & Support, IRB = Institutional Review Board, ISRA = Isolated Small Rural Area, IVx = Independent Variable, LA = Level of autonomy, LRA = Large Rural Area, MA = Massachusetts, mo. = month, ME = Mean, MNPJSS = Misener Nurse Practitioner Job Satisfaction Scale, MPD = Massachusetts Provider Database, N = Sample size, NF = No Fellowship, NIH = National Institute of Health, NP = Nurse Practitioner, NP-AR = Nurse Practitioner–Administration Relationship, NP-PCOCQ = Nurse Practitioner-Primary Care Organizational Climate Questionnaire, NP-PR = Nurse Practitioner– Physician Relationship, NS = None stated, OB/GYN = Obstetrics & Gynecology, OC = Opportunity for change, OG = Opportunity for growth, OR = odds ratio, P = Statistical significance, PF = Personal factors, PICOT = Population, Intervention, Comparison, Outcomes, & Timing, PO = Physician Office, PR = panel ratio, Q = Question, QI = Quality Improvement, r = Pearson correlational coefficient, RN = Registered Nurse, RO = Respect for opinion, RS = Respect from superiors, SA= Sense of Accomplishment, SRA = Small Rural Area, ST = Successful transition, SV = Sense of value for what you do, UA = Urban Area, UP = University prepared, US = United States, USDHHS = U.S. Department of Health and Human Services, VA = Veteran’s Affairs, VA CoEPCE = Veteran’s Affairs Centers of Excellence in Primary Care Education, vs. = versus POSTGRADUATE NURSE PRACTITIONER PROGRAMS Citation: Bryant & Parker (2020) Country: US Funding: NS. Reports no conflicts of interest. Bias: Research was in a director role and in charge of hiring NPs & evaluating the process. Framework: NS. Maslow’s Human Needs Theory or Herzberg and Mausner’s motivationhygiene theory inferred. Design: NS. Quantitative: Exploratory CrossSectional Design inferred. Methods: Participants completed a surveymonkey with a 44 item 6-point Likert scale questionnaire. Purpose: Use the MNPJSS to determine NP job satisfaction, confidence, and retention between those who completed a fellowship versus those who did not. Sampling: Convenience and board certification within 90 days of the residency start date. 3. Passed a competitive interview process. Attrition: 2/38, 5.3%. N= 258 Demographics: 81% completed a fellowship, 19% did not. Nearly 90% were women, 45.63% in their 30s, and about 91% were Caucasian. 42.8% had 1-5 years of experience. Majority, 51.56% in current role between 1-5 years. Setting: NPs recruited from Emory Healthcare, MD Anderson, Cancer Center, DNP discussion IV: Fellowship programs DV: Likert Scale Items (SA, OG, ES, OC, IP, FQ, DQ, SV, FP, BA, CO, RO, AP, RS, LA) DV1: Job satisfaction DV2: confidence DV3: retention Research Question: How do NPs with fellowship experience compare on the MNPJSS compared to those who did not complete a fellowship? Definitions: MNPJSS: sense of accomplishment, professional growth, expansion of scope, 31 Measurement: MNPJSS, 6-point Likert scale, “1” very dissatisfied and “6” very satisfied. Validity & Reliability: Subscales have been shown to be consistent and valid. Statistical Analysis: Independentsample t-tests after failed Levene’s test. Descriptive statistics. A post hoc power analysis was performed, but there was a low power. Results: ME score: F vs. NF, P= x MESA: 5.20 vs. 5.01 MEOG: 4.69 vs. 4.38 MEES: 4.39 vs. 4.21, MEOC: 4.45 vs. 4.34 MEIP: 4.20 vs. 3.88 MEFQ: 4.47 vs. 4.34 MEDQ: 5.16 vs. 5.11 MESV: 5.02 vs. 4.65, P=<0.05 MEFP: 4.78 vs. 4.51 MEBA: 3.94 vs. 3.16, P=<0.01 MECO: 3.69 vs. 3.08, P=<0.05 MERO: 4.49 vs. 4.29 MEAP: 4.63 vs. 4.45 MERS: 4.31 vs. 4.33 MELA: 5.14 vs. 5.14 Level of Evidence: Level IV Strengths: Large sample size than most studies. However, needed large sample of persons who underwent a fellowship. Weakness: Unequal group sizes. Post hoc power analysis did not show statistical significance. Possible bias. Feasibility: Lengthy process obtaining permission from groups, finding participants, and obtaining IRB approval. Utility to PICOT: Relevant to the comparison. Note. Key. ACPN = Acute Care Nurse Practitioner Network, AG = Adult-gerontology, AHRQ = Agency for Healthcare Research and Quality, AP = Acceptance by physicians, APRN = Advanced Practice Registered Nurse, BA = Bonuses available, CHC = Community Health Centers, CI = Confidence Interval, CO = Compensation, CR = Community Resources, CVD = Cardiovascular Disease, CXR = Chest X-ray, DEA# = Drug Enforcement Administration Number, DNP = Doctor of Nursing Practice, DQ = Ability to deliver quality of care, DVx = Dependent Variable, ECG = electrocardiogram, ED = Emergency Department, ES = Expand scope of practice, F = Fellowship, FNP = Family Nurse Practitioner, FMP = Family Practice, FORHP = Federal Office of Rural Health Policy, FP = Flexibility in Protocols, FQ = Freedom to Question Practice, GP = General Practice, HBC = Hospital-Based Clinic, HEDIS = Healthcare Effectiveness Data and Information Set measures, HRSA = Health Resources & Services Administration, hr = hour, ICU= Intensive Care Unit, IM = Internal Medicine, IP = Input in Organization, IPS = Independent Practice & Support, IRB = Institutional Review Board, ISRA = Isolated Small Rural Area, IVx = Independent Variable, LA = Level of autonomy, LRA = Large Rural Area, MA = Massachusetts, mo. = month, ME = Mean, MNPJSS = Misener Nurse Practitioner Job Satisfaction Scale, MPD = Massachusetts Provider Database, N = Sample size, NF = No Fellowship, NIH = National Institute of Health, NP = Nurse Practitioner, NP-AR = Nurse Practitioner–Administration Relationship, NP-PCOCQ = Nurse Practitioner-Primary Care Organizational Climate Questionnaire, NP-PR = Nurse Practitioner– Physician Relationship, NS = None stated, OB/GYN = Obstetrics & Gynecology, OC = Opportunity for change, OG = Opportunity for growth, OR = odds ratio, P = Statistical significance, PF = Personal factors, PICOT = Population, Intervention, Comparison, Outcomes, & Timing, PO = Physician Office, PR = panel ratio, Q = Question, QI = Quality Improvement, r = Pearson correlational coefficient, RN = Registered Nurse, RO = Respect for opinion, RS = Respect from superiors, SA= Sense of Accomplishment, SRA = Small Rural Area, ST = Successful transition, SV = Sense of value for what you do, UA = Urban Area, UP = University prepared, US = United States, USDHHS = U.S. Department of Health and Human Services, VA = Veteran’s Affairs, VA CoEPCE = Veteran’s Affairs Centers of Excellence in Primary Care Education, vs. = versus POSTGRADUATE NURSE PRACTITIONER PROGRAMS 30% of individuals knew the researcher. Risk of response bias. Citation: Scaglione & Lloyd (2021) Country: US Funding: VA Bias: NS. Appears that authors are affiliated with the VA. Framework: Transition Stages Model. sample groups, and a social media site “show me your stethoscope.” Exclusion: NS. Attrition: None noted. Design: Quantitative: Prospective, descriptive design. Methods: 9 question survey pre and post intervention (6 weeks). Evaluation of performance on ECG & CXR scores pre and post fellowship. Purpose: To determine skills and confidence development after a VA fellowship program. Sampling: Convenience N= 3 Demographics: 30-59 years old, 100% female, 2/3rd white, 1/3rd African American, RN experience: 10-25 years. Setting: Midwestern VA hospital. Exclusion: Not stated. Inclusion: Senior AGNP students invited. Attrition: 0. opportunity for change, input in policy, ability to question, deliver quality care, recognition, feeling valued, having flexibility, bonuses, compensation, respected, acceptance by physicians, and autonomy. IV: A 6 week long fellowship DV1: Skills DV2: Confidence Research Question: Does a 6-week acute AGNP fellowship program enhance confidence and skills? Definitions: None of note. 32 Measurement: A Likert scale survey was developed. Validity & Reliability: NS. Likely not validated as it was newly developed. Statistical Analysis: Descriptive statistics with excel for demographics. Not used for outcomes, due to too small of a sample. Results: DV1: Confidence based on years of RN experience: (Pre = 2.67, Post = 2.67; 0%). DV2: UP for APRN (Pre = 2.33, Post = 2.00; 0.33%). DV3: UP for QI: (Pre = 2.00, Post = 2.00; 0%). DV4: UP for medically vulnerable: (Pre = 1.67, Post 2.00, +0.33%) DV5: UP to order & interpret diagnostics/labs (Pre = 3.00, Post = 2.33, 0.67%) Q7: Preparation for transition: (Pre = 2.33, Post = 2.67, +0.34%) Q9: UP for procedures: Level of Evidence:Level IV Strengths: 100% completion by participants, able to evaluate progression ofskills. Weaknesses: Very small sample size. Very experienced demographic. Feasibility: NS, but Initially planned to perform study on AGNP new hires, but none were hired. Recruitment was then changed to senior students. Difficult recruitment. Utility to PICOT: Relevant to the intervention & outcomes. Note. Key. ACPN = Acute Care Nurse Practitioner Network, AG = Adult-gerontology, AHRQ = Agency for Healthcare Research and Quality, AP = Acceptance by physicians, APRN = Advanced Practice Registered Nurse, BA = Bonuses available, CHC = Community Health Centers, CI = Confidence Interval, CO = Compensation, CR = Community Resources, CVD = Cardiovascular Disease, CXR = Chest X-ray, DEA# = Drug Enforcement Administration Number, DNP = Doctor of Nursing Practice, DQ = Ability to deliver quality of care, DVx = Dependent Variable, ECG = electrocardiogram, ED = Emergency Department, ES = Expand scope of practice, F = Fellowship, FNP = Family Nurse Practitioner, FMP = Family Practice, FORHP = Federal Office of Rural Health Policy, FP = Flexibility in Protocols, FQ = Freedom to Question Practice, GP = General Practice, HBC = Hospital-Based Clinic, HEDIS = Healthcare Effectiveness Data and Information Set measures, HRSA = Health Resources & Services Administration, hr = hour, ICU= Intensive Care Unit, IM = Internal Medicine, IP = Input in Organization, IPS = Independent Practice & Support, IRB = Institutional Review Board, ISRA = Isolated Small Rural Area, IVx = Independent Variable, LA = Level of autonomy, LRA = Large Rural Area, MA = Massachusetts, mo. = month, ME = Mean, MNPJSS = Misener Nurse Practitioner Job Satisfaction Scale, MPD = Massachusetts Provider Database, N = Sample size, NF = No Fellowship, NIH = National Institute of Health, NP = Nurse Practitioner, NP-AR = Nurse Practitioner–Administration Relationship, NP-PCOCQ = Nurse Practitioner-Primary Care Organizational Climate Questionnaire, NP-PR = Nurse Practitioner– Physician Relationship, NS = None stated, OB/GYN = Obstetrics & Gynecology, OC = Opportunity for change, OG = Opportunity for growth, OR = odds ratio, P = Statistical significance, PF = Personal factors, PICOT = Population, Intervention, Comparison, Outcomes, & Timing, PO = Physician Office, PR = panel ratio, Q = Question, QI = Quality Improvement, r = Pearson correlational coefficient, RN = Registered Nurse, RO = Respect for opinion, RS = Respect from superiors, SA= Sense of Accomplishment, SRA = Small Rural Area, ST = Successful transition, SV = Sense of value for what you do, UA = Urban Area, UP = University prepared, US = United States, USDHHS = U.S. Department of Health and Human Services, VA = Veteran’s Affairs, VA CoEPCE = Veteran’s Affairs Centers of Excellence in Primary Care Education, vs. = versus POSTGRADUATE NURSE PRACTITIONER PROGRAMS 33 sampling of senior AGNP students. (Pre = 2.67, Post = 2.00, 0.67%) Q15: Confidence in delivering care (Pre = 2.67, Post = 2.00, -0.67%). Q5, Q8, Q10=14: NS. ECG Confidence: Somewhat improved. ECG Scores: Decreased 66%. CXR Confidence: Not improved. CXR Scores: Decreased 33% Citation: Poghosyan et al. (2018) Country: US Funding: AHRQ, Robert Wood Johnson Foundation , & NIH Bias: Report no Framework: Donabedian’s quality of care model. Design: Quantitative: Crosssectional design. Methods: Linked survey data & performance on HEDIS scores. Purpose: To determine if the NP practice environment correlated with higher HEDIS scores. Sampling: Convenience, pulled from MPD. N= 221 NPs, 118 practices Demographics: ME age = 50, Masters = 88%, White = 90%, PO = 37%, CHC = 31% Urban area = 57% Setting: MA Exclusion: Lack of response to surveys, or HEDIS scores not obtainable. Inclusion: Listed IV: NP environment DV1: Asthma score DV2: CVD score DV3: Diabetes score Research Question: Does NP practice environment influence delivery of care for asthma, diabetes, or cardiovascular disease? Definitions: HEDIS: A measurement of management of care for chronic diseases. Measurement: 1. NP-PCOCQ 2. HEDIS scores Validity & Reliability: NPPCOCQ subscales have “high reliability with Cronbach a’s ranging from 0.87 to 0.95.” Reported valid too. Statistical Analysis: 1. Descriptive statistics with on demographics, entered into a multivariable regression model. 2. Fractional logistic regression models for HEDIS scores & NP-PCOCO subscales. Results: HBC: OR 0.47, P = 0.04 Other: OR 0.32, P = 0.01 IPS: OR 1.60, P = 0.035 PR: OR 0.61, P = 0.002 Level of Evidence: Level IV Strengths: Very organized, uses statistical analysis, reports framework, & discusses how demographics are representative of NPs in MA. Weaknesses: Does not mention that correlation is not necessarily causation. Asthma HEDIS reports did not differentiate pediatric vs. adult patients. Reliant on self-reports. Feasibility: Limited Note. Key. ACPN = Acute Care Nurse Practitioner Network, AG = Adult-gerontology, AHRQ = Agency for Healthcare Research and Quality, AP = Acceptance by physicians, APRN = Advanced Practice Registered Nurse, BA = Bonuses available, CHC = Community Health Centers, CI = Confidence Interval, CO = Compensation, CR = Community Resources, CVD = Cardiovascular Disease, CXR = Chest X-ray, DEA# = Drug Enforcement Administration Number, DNP = Doctor of Nursing Practice, DQ = Ability to deliver quality of care, DVx = Dependent Variable, ECG = electrocardiogram, ED = Emergency Department, ES = Expand scope of practice, F = Fellowship, FNP = Family Nurse Practitioner, FMP = Family Practice, FORHP = Federal Office of Rural Health Policy, FP = Flexibility in Protocols, FQ = Freedom to Question Practice, GP = General Practice, HBC = Hospital-Based Clinic, HEDIS = Healthcare Effectiveness Data and Information Set measures, HRSA = Health Resources & Services Administration, hr = hour, ICU= Intensive Care Unit, IM = Internal Medicine, IP = Input in Organization, IPS = Independent Practice & Support, IRB = Institutional Review Board, ISRA = Isolated Small Rural Area, IVx = Independent Variable, LA = Level of autonomy, LRA = Large Rural Area, MA = Massachusetts, mo. = month, ME = Mean, MNPJSS = Misener Nurse Practitioner Job Satisfaction Scale, MPD = Massachusetts Provider Database, N = Sample size, NF = No Fellowship, NIH = National Institute of Health, NP = Nurse Practitioner, NP-AR = Nurse Practitioner–Administration Relationship, NP-PCOCQ = Nurse Practitioner-Primary Care Organizational Climate Questionnaire, NP-PR = Nurse Practitioner– Physician Relationship, NS = None stated, OB/GYN = Obstetrics & Gynecology, OC = Opportunity for change, OG = Opportunity for growth, OR = odds ratio, P = Statistical significance, PF = Personal factors, PICOT = Population, Intervention, Comparison, Outcomes, & Timing, PO = Physician Office, PR = panel ratio, Q = Question, QI = Quality Improvement, r = Pearson correlational coefficient, RN = Registered Nurse, RO = Respect for opinion, RS = Respect from superiors, SA= Sense of Accomplishment, SRA = Small Rural Area, ST = Successful transition, SV = Sense of value for what you do, UA = Urban Area, UP = University prepared, US = United States, USDHHS = U.S. Department of Health and Human Services, VA = Veteran’s Affairs, VA CoEPCE = Veteran’s Affairs Centers of Excellence in Primary Care Education, vs. = versus POSTGRADUATE NURSE PRACTITIONER PROGRAMS conflict of interest. Nonrespon se bias could be an issue. Citation: Spetz et al. (2017) Country: US Funding: FORHP, HRSA, & USDHHS Bias: Report no conflict of interest. Framework: NS. Maslow’s Human Needs Theory or Herzberg and Mausner’s motivationhygiene theory inferred. Design: Quantitative: Crosssectional, correlational inferred. Methods: Data Analysis of a 2012 HRSA national NP survey. Purpose: To compare characteristics of NPs who provide primary care in rural vs. urban settings. Sampling: Survey was randomized. Analysis on convenience. in the MPD database. Attrition: 6/118 practices missing data, 5.1% NP-PCOCQ: A survey measuring domains of NP-PR, IPS, PV, & NP-AR. N = 13,000/22,000 Demographics: Setting: US national survey Exclusion: Did not complete the surveys. Inclusion: Employed in IM, FMP, Geriatrics, GP, Adolescents, OB/GYN, Women’s health, or school health. Attrition = Response rate 60% IV1: UA IV2: LRA IV3: SRA IV4: ISRA DV1: Hours worked DV2: Privileges DV3: Salaries & payment Research Question: How do NPs that practice in rural settings differ from NPs in urban settings? Definitions: National Sample Survey of NPs: demographics, licensure, education, clinical setting, title, field, independence vs. physician supervision, & satisfaction in practice areas. 34 OR & 95% CI reported. Measurement: National sample survey of NPs. Validity & Reliability: NS. Statistical Analysis: Rao-Scott chisquared tests and t tests. access to HEDIS measures. Utility to PICOT: Relevant to population, & outcomes. Results: 1. SRA & # of patients, P <.0001 2. ISRA & hospital admitting privileges, P <.0001 3. ISRA & DEA#, P<.0001 4. SRA & Weekly Patient #, p <.0001 5. SRA billed with NPI, P <.0001 6. UR paid by hr., p <.001 7. LRA paid by salary, P = .002 8. ISRA & Patient Panel, P =.0002 9. ISRA & ME hrs worked, P = 0.013 10. SRA and LRA NPs tend to work in states without physician oversight, p <.001 Level of Evidence: Level IV Strengths: Large, randomized sample. Weaknesses: Feasibility: Large study, only feasible if working with a large government agency. Utility to PICOT: Relevant to understanding the population. Other: Note. Key. ACPN = Acute Care Nurse Practitioner Network, AG = Adult-gerontology, AHRQ = Agency for Healthcare Research and Quality, AP = Acceptance by physicians, APRN = Advanced Practice Registered Nurse, BA = Bonuses available, CHC = Community Health Centers, CI = Confidence Interval, CO = Compensation, CR = Community Resources, CVD = Cardiovascular Disease, CXR = Chest X-ray, DEA# = Drug Enforcement Administration Number, DNP = Doctor of Nursing Practice, DQ = Ability to deliver quality of care, DVx = Dependent Variable, ECG = electrocardiogram, ED = Emergency Department, ES = Expand scope of practice, F = Fellowship, FNP = Family Nurse Practitioner, FMP = Family Practice, FORHP = Federal Office of Rural Health Policy, FP = Flexibility in Protocols, FQ = Freedom to Question Practice, GP = General Practice, HBC = Hospital-Based Clinic, HEDIS = Healthcare Effectiveness Data and Information Set measures, HRSA = Health Resources & Services Administration, hr = hour, ICU= Intensive Care Unit, IM = Internal Medicine, IP = Input in Organization, IPS = Independent Practice & Support, IRB = Institutional Review Board, ISRA = Isolated Small Rural Area, IVx = Independent Variable, LA = Level of autonomy, LRA = Large Rural Area, MA = Massachusetts, mo. = month, ME = Mean, MNPJSS = Misener Nurse Practitioner Job Satisfaction Scale, MPD = Massachusetts Provider Database, N = Sample size, NF = No Fellowship, NIH = National Institute of Health, NP = Nurse Practitioner, NP-AR = Nurse Practitioner–Administration Relationship, NP-PCOCQ = Nurse Practitioner-Primary Care Organizational Climate Questionnaire, NP-PR = Nurse Practitioner– Physician Relationship, NS = None stated, OB/GYN = Obstetrics & Gynecology, OC = Opportunity for change, OG = Opportunity for growth, OR = odds ratio, P = Statistical significance, PF = Personal factors, PICOT = Population, Intervention, Comparison, Outcomes, & Timing, PO = Physician Office, PR = panel ratio, Q = Question, QI = Quality Improvement, r = Pearson correlational coefficient, RN = Registered Nurse, RO = Respect for opinion, RS = Respect from superiors, SA= Sense of Accomplishment, SRA = Small Rural Area, ST = Successful transition, SV = Sense of value for what you do, UA = Urban Area, UP = University prepared, US = United States, USDHHS = U.S. Department of Health and Human Services, VA = Veteran’s Affairs, VA CoEPCE = Veteran’s Affairs Centers of Excellence in Primary Care Education, vs. = versus POSTGRADUATE NURSE PRACTITIONER PROGRAMS 35 Less diversity among NPs in rural areas. Note. Key. ACPN = Acute Care Nurse Practitioner Network, AG = Adult-gerontology, AHRQ = Agency for Healthcare Research and Quality, AP = Acceptance by physicians, APRN = Advanced Practice Registered Nurse, BA = Bonuses available, CHC = Community Health Centers, CI = Confidence Interval, CO = Compensation, CR = Community Resources, CVD = Cardiovascular Disease, CXR = Chest X-ray, DEA# = Drug Enforcement Administration Number, DNP = Doctor of Nursing Practice, DQ = Ability to deliver quality of care, DVx = Dependent Variable, ECG = electrocardiogram, ED = Emergency Department, ES = Expand scope of practice, F = Fellowship, FNP = Family Nurse Practitioner, FMP = Family Practice, FORHP = Federal Office of Rural Health Policy, FP = Flexibility in Protocols, FQ = Freedom to Question Practice, GP = General Practice, HBC = Hospital-Based Clinic, HEDIS = Healthcare Effectiveness Data and Information Set measures, HRSA = Health Resources & Services Administration, hr = hour, ICU= Intensive Care Unit, IM = Internal Medicine, IP = Input in Organization, IPS = Independent Practice & Support, IRB = Institutional Review Board, ISRA = Isolated Small Rural Area, IVx = Independent Variable, LA = Level of autonomy, LRA = Large Rural Area, MA = Massachusetts, mo. = month, ME = Mean, MNPJSS = Misener Nurse Practitioner Job Satisfaction Scale, MPD = Massachusetts Provider Database, N = Sample size, NF = No Fellowship, NIH = National Institute of Health, NP = Nurse Practitioner, NP-AR = Nurse Practitioner–Administration Relationship, NP-PCOCQ = Nurse Practitioner-Primary Care Organizational Climate Questionnaire, NP-PR = Nurse Practitioner– Physician Relationship, NS = None stated, OB/GYN = Obstetrics & Gynecology, OC = Opportunity for change, OG = Opportunity for growth, OR = odds ratio, P = Statistical significance, PF = Personal factors, PICOT = Population, Intervention, Comparison, Outcomes, & Timing, PO = Physician Office, PR = panel ratio, Q = Question, QI = Quality Improvement, r = Pearson correlational coefficient, RN = Registered Nurse, RO = Respect for opinion, RS = Respect from superiors, SA= Sense of Accomplishment, SRA = Small Rural Area, ST = Successful transition, SV = Sense of value for what you do, UA = Urban Area, UP = University prepared, US = United States, USDHHS = U.S. Department of Health and Human Services, VA = Veteran’s Affairs, VA CoEPCE = Veteran’s Affairs Centers of Excellence in Primary Care Education, vs. = versus POSTGRADUATE NURSE PRACTTIONER PROGRAMS 36 Table A3 Mixed Methods Approach Citation Theoretical/ Conceptual Framework Design/ Method/ Purpose Sample/Setting Variables Measurement/ Instrumentation Data Analysis Results/ Findings Citation: Dumphy et al. (2018) Country: United States Funding: NS. Bias: NS. Participation was voluntary & anonymous. Theoretical Framework : Bandura’s self-efficacy theory Design: Mixed Methods. NS. Exploratory Qualitative and Descriptive correlational comparative cohort study design inferred. Methods: FNP students were invited to participated in an anonymous online survey. A pre-test Barkley assessment was used to design curriculum for areas with lack of preparation. Purpose: 1. To determine how a strategically planned capstone course can prepare students for their role transition into practice. Sampling: Convenience sampling from N= 14 Demographics: Bachelors prepared, 85% female, 15% male, age 20-50 Setting: Southern university with individuals in a Master of Science in Nursing program. Exclusion: Less than 1 year of clinical experience. Inclusion: Senior FNP students in their final semester Attrition: 4/14, or 28.4% IV1: A practicum course DV1: Knowledge preparation DV2: Skills preparation Research Questions: How does a tailored course impact preparation for transition into practice? Definitions: 1. Skills: suturing, SOP, managing non-English speaking patients, mental health, coding & billing, x-ray interpretation, multiple concerns, DD, health assessment, managing chronic conditions, documentation, lab interpretation, chronic pediatric illness, pathophysiology, acute primary care, evidencebased practice, acute adult conditions, motivational interviewing, referral generation, pharmacotherapy, ECG interpretation, cultural competence Measurement: NP retrospective survey. 5-point Likert scale for 22 skills. 1 lowest prepared. 5highest prepared. 3 open-ended questions. Validity & Reliability: Not stated, but the tool has been used in the past. Statistical Analysis: 1. Quantitative: Mann-Whitney U tests, ShapiroWilks, and ordinaltype data 2. Qualitative: Thematic analysis Results: 1. Suturing increased from M=1 to M=3. P = 0.010. 2. SOP improved, P<0.10 3. 15 skills increased by P = 0.10 or more. 4. 3 skills declined, but not significantly. ECG (-0.01), pharmacology (-0.01), cultural competence (-0.16). Qualitative Themes: 1. Pre-survey students were prepared for assessments, diagnosis, and managing chronic conditions. 2. Post-survey most prepared participants: acute illness, chronic illness, and referrals. 3. Pre-survey unprepared for: coding and billing, ECG interpretation, X-ray interpretation, EMR documentation, and a lack of confidence entering practice. Level of Evidence; Application to practice/ Generalization Level of Evidence: Level IV Strengths: NS. Both qualitative and quantitative information reported. Weakness: Small sample size, Likert scale did not include a rating of preparedness. Feasibility: Not stated, but feasible design. Students took a pre- and postsurvey. Utility to PICOT: Relevant to population & outcomes. Note. Key: ACPN = Acute Care Nurse Practitioner Network, AG = Adult-gerontology, APRN = Advanced Practice Registered Nurse. CR = Community Resources, DVx = Dependent Variable, ED = Emergency Department, FNP = Family Nurse Practitioner, ICU= Intensive Care Unit, IVx = Independent Variable, IRB = Institutional Review Board, Mo.= months, N = Sample size, NP = Nurse Practitioner, NS = None stated, P = Statistical significance. PF = Personal factors, r = Pearson correlational coefficient, RN = Registered Nurse, ST = Successful transition, VA = Veteran’s Affairs. VA CoEPCE = Veteran’s Affairs Centers of Excellence in Primary Care Education POSTGRADUATE NURSE PRACTTIONER PROGRAMS 37 students enrolled in the university program. Citation: Brown et al. (2016) Country: US Funding: VA grant Bias: Authors do not declare. However, work for VA. Many participants known to authors. Framework NS. Constructive alignment theory inferred. Design: Mixed Methods. NS. Descriptive, cross sectional design inferred. Methods: Mixed Methods. Questionnaire emailed to participants and focus groups. Purpose: Identify and prioritize important aspects needed in the NP residency program. Sampling: Convenience sampling from a forum. N= 53 Demographics: 96% women, 37% master’s in nursing, 44% planned to develop a residency, 36% had an existing residency, 89% lived on the west coast. Setting: Round table discussion in Seattle. Exclusion: NS. Inclusion: Attendees of a NP residency forum. Attrition: 1/53, 98% completed IV1: Needs for a residency DV1: Framework DV2: Resources DV3: Vision IV2: Outcomes & Sustainability DV1: Desired outcome DV2: Impact Measures DV3: Cost & Benefit Research Questions: What features are necessary for a new nurse practitioner program? Definitions: 1. Focus groups: Participants divided into 7 groups. At least 2 groups would address a theme. Measurement: 1. Questionnaire using a 5-point Likert Scale. 2. Theme analysis from a discussion forum. Rated into 5 categories by Impact & Feasibility: 1. LI/LF 2. LI/HF 3. HI/LF 4. HI/HF 5. No Opinion Validity & Reliability: NS. Statistical Analysis: NS. Do not believe this was used. 4. Post-survey: Lack of preparation for suturing, charting in the EMR, but felt well prepared Results: 150 recommendations, 11% were viewed as HI/HF and leading to sustainability. Results not reported as p values but stated in conclusion. Top Results HI/HF: 1. Interprofessional training 2. Leadership & policy 3. Quality improvement & scholarship 4. Diagnostic skill & skill readiness 5. Mentorship & role development Requirements: 1. Compensation & Trained mentors 2. Funding 3. Accreditation 4. Space for training 5. Evaluation measures 6. University affiliation Level of Evidence: Level IV Strengths: NS. Both qualitative and quantitative information reported. Weakness: Small sample size, convenience sample, mostly west coast participation. Feasibility: NS, but clear design and simple to do. Utility to PICOT: Relevant to intervention. Note. Key: ACPN = Acute Care Nurse Practitioner Network, AG = Adult-gerontology, APRN = Advanced Practice Registered Nurse. CR = Community Resources, DVx = Dependent Variable, ED = Emergency Department, FNP = Family Nurse Practitioner, ICU= Intensive Care Unit, IVx = Independent Variable, IRB = Institutional Review Board, Mo.= months, N = Sample size, NP = Nurse Practitioner, NS = None stated, P = Statistical significance. PF = Personal factors, r = Pearson correlational coefficient, RN = Registered Nurse, ST = Successful transition, VA = Veteran’s Affairs. VA CoEPCE = Veteran’s Affairs Centers of Excellence in Primary Care Education POSTGRADUATE NURSE PRACTTIONER PROGRAMS 38 Appendix B Synthesis Table Author Rugen et al. Faraz Dillon et al. Rugen et al. Bryant & Parker Scaglione & Lloyd Poghosyan et al. Spetz et al. Dumphy et al. Brown et al. Year 2018b 2018 2016 2018a 2020 2021 2018 2017 2018 2016 Sample Size 38 177 34 38 258 3 221 13,000 14 53 Duration 12 mo. 3-12 mo. CS 12 mo. CS 6 weeks CS CS Semester CS Independent Variables Goals Residency, practicum course Role Transition X Competen cy Tool Survey Survey ⇧ ⇧ ⇧ ⇧ ⇧ ⇧ ⇧ ⇧ ⇧ --- ⇧ Patient Safety Professional Satisfaction Personal & Community Resources Successful Transition Retention Variability in scope & pay Clinical Competencies: ⇧ 4 4 ⇧ 5 Elements of Satisfaction ⇧ Elements of Dissatisfaction Themes ⇩ Leadership and performance Pre: Readiness for care, & referring Pre: Readiness for ECG, x-ray. EMR billi d di X Competency Tool MNPJSS Survey X X, X, X Outcomes Job Satisfaction Confidence X X Support, Experience, Communication & Leadership Practice Setting Measurement X Residency requirements --3 3 --⇧⇩ X X NP-PCOCQ, Competency Tool National sample survey of NPs Pre- and PostSurveys Survey, and Focus Groups ⇧ 5 ⇧ 2 ⇧ , --- ⇩ 2 ⇧ --- 3 X ⇧ X 11 3 ⇩ ⇧ ⇩ Note. Key. ACPN = Acute Care Nurse Practitioner Network, AG = Adult-gerontology, APRN = Advanced Practice Registered Nurse, CS = Cross-sectional, FNP = Family Nurse Practitioner, HEDIS = Healthcare Effectiveness Data and Information Set measures, HRSA = Health Resources & Services Administration, mo. = month, MI = motivational interviewing, MNPJSS = Misener Nurse Practitioner Job Satisfaction Scale, NP = Nurse Practitioner, NP-PCOCQ = Nurse Practitioner-Primary Care Organizational Climate Questionnaire, Qual. = Qualitative, Quan. = Quantitative, VA = Veteran’s Affairs, VA CoEPCE = Veteran’s Affairs Centers of Excellence in Primary Care Education, X = studied or identified, yr = year, ⇧ = Increased, ⇧ = Mildly increased, ⇩ = Decreased, --- POSTGRADUATE NURSE PRACTTIONER PROGRAMS = Not significant, Dark blue = correlated with organizational support, Pink = correlated with communication and leadership, Dark green = associated with residency 39 POSTGRADUATE NURSE PRACTTIONER PROGRAMS Appendix C Project Timeline Engage Stakeholders Fall 2020 Describe the program Fall 2020 Focus on the evaluation design Summer 2021 Gather credible evidence Fall 2021 and Winter 2022 Justify conclusions Spring 2022 Disseminate and share lessons learned Spring 2022 40 POSTGRADUATE NURSE PRACTTIONER PROGRAMS 41 Appendix D Budget: Postgraduate Nurse Practitioner Programs and Provider Satisfaction Phase Preparation Activities Direct Cost: SurveySparrow Subscription. Funding: personal funds by thestudent. Indirect Cost: Gas expenses for project leader and student assistant.Funding: personal funds by the student. Direct Cost: Cost to use evaluation tool Misener Nurse Practitioner JobSatisfaction Scale (MNPJSS). No cost per the co-author. Cost Subtotal Total $190 $90 $0 $280 Delivery $0 $0 Evaluation Direct Cost: Statistician analysis.Funding: Personal funds by the student. $100 $90 Dissemination Poster materials for presentations.Funding: by Arizona State University. $0 $0 $370 Budget Justification: This budget was created based on estimated direct and indirect costs. Potential revenue to the FQHC could come in federal grants if findings show high levels of provider job satisfaction. This project does not anticipate cost savings. However, if findings indicated areas of necessary improvement, a legacy project would likely decrease costs related to provider turnover. POSTGRADUATE NURSE PRACTTIONER PROGRAMS 42 Appendix E Table 1- Participant Demographics Variable Survey Completed PRE POST Number of Unique Participants Participants completed both PRE and POST Resident Status Current One-year post-residency Two-years post-residency Age Range 21 – 30 31 – 40 41 – 50 Gender Female Male Prefer Not to Answer Race White Hispanic Prefer Not to Answer Years as a Registered Nurse 0–5 6 – 10 11 – 15 16 – 20 Highest Level of Education Masters DNP Employment Status Full-Time n % 11 8 16 3 57.89 42.11 5 8 3 31.25 50.00 18.75 3 12 1 18.75 75.00 6.25 15 1 0 93.75 6.25 0 10 5 1 62.50 31.25 6.25 7 6 2 1 43.75 37.50 12.50 6.25 14 2 87.50 12.50 16 100.00 18.75 POSTGRADUATE NURSE PRACTTIONER PROGRAMS 43 Appendix F Table 2 - Two-Tailed Independent Samples t-Test for Total Satisfaction Scores with Outlier December 2021 February 2022 Variable M SD M SD t p Total 158.09 36.05 164.92 49.27 -0.35 .731 Note. N = 19. Degrees of Freedom for the t-statistic = 17. d represents Cohen's d. d 0.16 POSTGRADUATE NURSE PRACTTIONER PROGRAMS 44 Appendix G Table 3 - Two-Tailed Independent Samples t-Test for Total Satisfaction Scores Without Outlier December 2021 February 2022 Variable M SD M SD t p Total 165.50 27.80 176.48 39.81 -0.67 .512 Note. N = 19. Degrees of Freedom for the t-statistic = 17. d represents Cohen's d. d 0.32 POSTGRADUATE NURSE PRACTTIONER PROGRAMS 45 Appendix H Table H – 1 - Current Resident’s Mean MNPJSS Subcategory Score on the Likert Scale Variable Time Interactions Collegiality/Partnership Professional Growth Benefits Challenge/Autonomy Total Satisfaction M 3.88 4.00 3.26 3.71 3.77 4.28 3.77 5 Min 3.36 Max 4.32 5 3.00 5.10 5 2.52 5 5 3.42 3.36 5 3.90 5 3.33 3.90 4.02 4.32 4.80 4.23 SD 0.49 0.86 0.67 0.28 0.39 0.34 n 0.34 Table H – 2 - One-Year Post-residency Mean MNPJSS Subcategory Score on the Likert Scale Variable Time Interactions Collegiality/Partnership Professional Growth Benefits Challenge/Autonomy Total Satisfaction M 3.04 SD n Min Max 1.30 8 1.32 5.64 3.90 1.51 8 1.44 6.00 3.97 3.93 1.16 8 2.22 6.00 1.27 8 1.71 6.00 4.53 1.07 8 2.64 6.00 4.05 3.98 1.44 8 1.41 6.00 1.25 8 1.98 6.00 Table H – 3 – Second-Year Post-residency Mean MNPJSS Subcategory Score on the Likert Scale Variable Time Interactions Collegiality/Partnership Professional Growth Benefits Challenge/Autonomy Total Satisfaction M SD n Min Max 2.90 0.52 3 2.34 3.36 3.22 1.01 3 2.52 4.38 3.48 0.55 3 3.00 4.08 3.00 0.53 3 2.58 3.60 4.56 0.49 3 4.02 4.98 3.76 0.67 3 3.18 4.50 3.49 0.58 3 3.12 4.15 POSTGRADUATE NURSE PRACTTIONER PROGRAMS Appendix I Figure 1. 46