1 The Impact of Provider Education on Pediatric Palliative Care Referral Katelyn Newton, BSN, RN & Danielle Sebbens, DNP, CPNP-AC/PC DNP Student, Arizona State University Arizona State University, College of Nursing 5407 Texas Star Ln. Wichita Falls, TX 76310 602.770.8994 Disclosures: Katelyn Newton is an employee at Children’s Health in the Center for Cancer and Blood Disorders. Acknowledgments: Kimberly LaBronte, DNP Project Statistical Mentor Robyn Haynes, Nurse Practitioner at Children’s Medical Center Heather Patterson, Nurse Practitioner at Children’s Medical Center Key Words: pediatric palliative care, referral, provider education 2 1 Abstract 2 Introduction: Palliative care can significantly benefit children managing a life-limiting illness; 3 unfortunately, services are generally reserved for end of life. The aim of this project was to 4 demonstrate how established guidelines coupled with provider education could impact referrals. 5 Methods: Educational sessions developed using information processing theory and outlining 6 referral recommendations were offered to providers in the NICU, PICU, and Center for Cancer 7 and Blood Disorders at a tertiary care facility. Presurveys and postsurveys were administered at 8 the time of the intervention and referral numbers for the organization were collected for two 9 months prior and two months following. 10 Results: Descriptive statistics and paired t-tests were used to compare survey data and referral 11 rates. 12 Discussion: Palliative care is imperative for meeting patient goals and optimizing quality of life. 13 Provider knowledge of referral criteria ensures that patients receive this service early in their 14 disease trajectory and can benefit from its inclusion within their care team. 3 15 16 The Impact of Provider Education on Pediatric Palliative Care Referral Managing care for children with life-limiting illnesses is a complex and multifactorial 17 process that requires the collaboration of healthcare providers and multiple interprofessional 18 services to provide optimal care for the patient and family. Palliative care is a specific pediatric 19 subspecialty whose main goals are to relieve suffering, improve quality of life for both patients 20 and families, facilitate informed decision-making conversations, and provide care coordination 21 for children living with a life-limiting illness (American Academy of Pediatrics [AAP], 2013). 22 Not only is the provision of these services essential but also including palliative care early in the 23 disease trajectory ensures optimal patient and family outcomes (Zhukovsky, Herzog, Kaur, 24 Palmer, and Bruera, 2009). The Centers for Disease Control and Prevention most recent data on 25 vital statistics reported 41,881 deaths in children between the ages of zero to nineteen in 2015 26 (Kochanek, Murphy, Xu, Tejada-Vera, 2016). Of those 23,215 were children under the age of 27 one and 18,666 were children between the ages of one and nineteen (Kochanek et al., 2016). 28 While, the majority of these fatalities were the result of accidents, homicide, or suicide; a 29 significant number were from other causes such as malignancies, chromosomal abnormalities, 30 congenital malformations, and heart disease. These other causes of fatalities represent a 31 population of patients who could benefit from the early incorporation of a palliative care team. 32 The AAP recognizes the importance of pediatric palliative care and the desperate need for timely 33 referral. The most recent policy statement released by the organization included definitions for 34 core commitments for pediatric palliative care were defined. These included, the importance of 35 integrating a dedicated care team to help navigate complex decisionmaking and provide social 36 and spiritual support services early in the continuum of care (AAP, 2013). Furthermore, the AAP 37 (2013) explicitly states that referral to palliative care providers can occur at any point in the 4 38 disease process, including at diagnosis, and should be used throughout the course of the illness to 39 support the goals of care. These services should not be restricted to terminal patients, but rather 40 should supplement care even when goals are still focused on curative treatments (AAP, 2013). 41 The magnitude of these recommendations will undeniably be difficult to implement and will 42 require a massive overhaul in the way children living with these chronic conditions are managed; 43 however, the short and long-term benefits these families will see is irrefutable. 44 The project site is a freestanding children’s hospital with 490-licensed beds was utilized 45 as the project site. The palliative care department at this facility has automatic referral systems in 46 place for children undergoing hematopoietic stem cell transplant and heart transplant. However, 47 like most major children’s hospitals and palliative care programs, the department has difficulty 48 capturing patients with life-limiting illnesses earlier in their disease trajectory to be able to 49 provide more comprehensive services. 50 To develop an effective solution, a literature review was conducted using the PICOT 51 question: In pediatric patients with a life-threatening illness, how does a standardized approach 52 as compared to usual care impact early referral to palliative care? Databases searched included: 53 CINAHL, PubMed, Cochrane Library, and PsycINFO. Given the paucity of information 54 pertaining to this particular population, adult literature was also searched for information relative 55 to the topic. Keywords used in these searches included: pediatric, pediatric OR children, 56 palliative care, palliative care OR end of life, palliative care referral, referral, necessity of 57 referral, outcomes, referral criteria OR standard approach, early referral OR timing of referral, 58 timing of referral OR early referral OR referral, standard OR criteria OR guideline. Results were 59 limited to those in the English language and performed in humans. All reference lists of relevant 60 articles were also reviewed. This search resulted in ten high quality articles that were included 5 61 for synthesis. These articles were then graded using the Melnyk and Fineout-Overholt (2015) 62 criteria for hierarchy of evidence and included in the results was one level VI study, four level V 63 studies, and five level IV studies. Most of the studies were qualitative in nature, owing likely in 64 part to the sensitivity of the subject matter. 65 Multiple interventions for palliative care referral were examined across studies, 66 including: staff education and training, the formation of a dedicated palliative care team, the 67 development of guidelines or criteria for referral, procedures to increase awareness of palliative 68 care services, automatic referrals, routine symptom screening and assessment, and family 69 request. Most of the studies used a combination of these methods to achieve early referral. The 70 most common method demonstrated for efficacy throughout the evidence, however, was the 71 development of standardized guidelines or institutional criteria in conjunction with provider and 72 staff education on how to utilize these tools and the importance of early palliative care referral. 73 Based on a review of the literature, a project was developed to create standardized 74 referral criteria and subsequently providing staff education. Cognitive learning theory served as 75 the foundation for the project and the development of the intervention. Cognitive learning theory 76 is the interaction of perception, thought, reasoning, memory, development, and processing of 77 information within the learner (Butts & Rich, 2015). The theory is founded on five working 78 stages of learning: the attention stage, the sensory memory stage, short-term or working memory 79 stage, long-term memory stage, and the information retrieval stage; all of which were 80 incorporated while developing the intervention. Furthermore, The Iowa Model for Evidence- 81 Based Practice guided this project because it was created specifically for practitioners to 82 implement a practice change. This model assumes a team effort, a dedication to process rather 83 than an event, and includes evaluation as a crucial component of implementation (Rycroft- 6 84 Malone & Bucknall, 2010). This model also provides a framework for improving patient 85 outcomes and nursing practice, while simultaneously monitoring for cost containment (Taylor- 86 Piliae, 1999). Using the evidence, cognitive learning theory, and the Iowa Model for Evidence- 87 Based Practice it was concluded that the ultimate aim of this project was to demonstrate how 88 established guidelines coupled with provider education could impact referral rates. 89 90 Methods Design 91 Educational sessions were developed using information processing theory, which 92 outlined referral recommendations from the AAP, National Hospice and Palliative Care 93 Organization (NHPCO), and those included in the institutional policy. Presurveys and 94 postsurveys validated by two clinical experts in the field were administered to participants at the 95 time of the intervention. Referral rates for the organization were also collected for two months 96 prior to the intervention, and two months following the intervention. 97 Setting 98 99 This project was completed at a large, freestanding pediatric institution in Dallas, Texas. Participants were recruited through flyers and department educators to attend a short 100 informational session, each held twice a day for one week, in the Neonatal Intensive Care Unit 101 (NICU), Pediatric Intensive Care Unit (PICU), and Center for Cancer and Blood Disorders 102 (CCBD). The presentation was also given at a monthly meeting of Advanced Practice Providers. 103 Sample 104 Institutional Review Board (IRB) approval was obtained through the author’s affiliated 105 university prior to the start of the project. All of the educational sessions were open to 106 physicians, advanced practice providers, and registered nurses within the organization. Each 7 107 participant who attended an educational session was given a copy of the consent form and 108 completion of the presurvey and postsurvey served as their consent to participate in the project. 109 Subjects were notified that participation was voluntary. They were invited to attend the 110 presentation and informed that they could decline participation in the survey portion. All subjects 111 were required to be 18 years of age or older to participate, and there were no known risk factors 112 associated with the project. 113 Outcome Measures and Data Collection 114 Presurvey and Postsurvey 115 The presurvey and postsurveys were designed by the investigator in conjunction with 116 stakeholders from the palliative care department. The presurvey collected demographic data 117 including: role within the organization, number of years in current role, number of years within 118 the institution, current department employer, comfortability with palliative care referral on a 6- 119 point Likert scale, and an estimated number of personal palliative care referrals placed within the 120 last year. The postsurvey first assessed knowledge acquisition of the material presented by 121 posing four clinical questions related to the presentation. The postsurvey then reassessed 122 comfortability in referring to palliative care on the same 6-point Likert scale and also asked 123 participants to score the likelihood of placing a referral after listening to the presentation and 124 how valuable they thought the information was to their personal practice. Finally, the postsurvey 125 posed a qualitative question for participants asking what the biggest reason that affected their 126 decision to postpone placing a palliative care referral or deciding not to refer at all. 127 Referral Rates 8 128 The director for palliative care obtained referral rates collected through the electronic 129 medical record for the purposes of this project. Referral rates were collected for two months prior 130 to the intervention and compared with two months postintervention. 131 Data Analysis 132 Statistical analyses were completed using Statistical Package for the Social Sciences 133 (SPSS) version 25.0 software. Descriptive statistics were used to describe demographic variables 134 and postsurvey knowledge based question scores. Comfortability scores in the pretest and 135 posttest were analyzed using a paired t-test. 136 137 138 Results Demographics A total of 64 participants were recruited for the project and completed the questionnaires. 139 Key sample demographics for the group are displayed in Table 1. The majority of the 140 participants were advanced practice providers (57.8%; n= 37), followed by registered nurses 141 (39.1%, n= 25), and physicians (3.1%, n= 2). Of those participants, department sites included 142 the PICU (6.3%, n= 4), NICU (21.9%, n= 14), CCBD (26.6%, n= 17), and the majority from 143 other departments within the hospital (42.2%, n= 27); two participants were employed in 144 multiple departments (3.1%, n= 2). The majority of participants estimated that they had not 145 made a palliative care referral in the past year (48.4%, n= 31). 146 Knowledge Based Questions 147 148 The project was based on cognitive learning theory. To assess adequate knowledge acquisition immediately following the presentation, four questions were posed to participants 9 149 regarding the material presented. Overall, participants demonstrated a strong understanding of 150 the recommendations and guidelines used within the organization (see Table 2 and Figure 1). 151 Comfortability 152 Participants were asked to rate their comfortability with placing a referral to palliative 153 care using a Likert scale of 0 to 5 both before and after the intervention. Presurvey data was 154 compared to postsurvey data and there was a statistically significant difference in comfortability 155 scores (see Table 3 and Figure 2). Presurvey scores for participants who completed the entire 156 survey ranged from zero (very uncomfortable) to five (very comfortable) (n = 55; μ = 3.45). 157 Postsurvey scores ranged from three to five among participants who completed the entire survey 158 (n = 55; μ = 4.51). 159 Referral Rates 160 Preintervention referral rates were collected for two months prior to the intervention and 161 compared to postintervention referral rates in the two months following the intervention. Referral 162 rates were collected for the PICU, NICU, CCBD, and hospital-wide. The number of referral rates 163 varied between units, and while there was a clinically significant increase in the number of 164 hospital wide referrals postintervention, it was not considered enough to be statistically 165 significant (see Table 4 and Figure 3). 166 Value 167 In the postsurvey participants were asked to rank how valuable the information was to 168 their personal practice and the likelihood of making a referral to palliative care based on the 169 information provided from zero to five on Likert scale. Overall, participants did find the 170 information to be valuable to their personal practice (n = 55; μ = 4.65; σ = 0.552). When scoring 10 171 likelihood of referring to palliative care, zero (not likely to refer at all) and five (very likely to 172 refer). Overall, participants who answered the question rated that they were more likely to refer 173 to palliative care based on the information provided (n = 54; μ = 4.52; σ = 0.666). 174 Reasons for not referring 175 The final survey was: In your personal practice, what is the biggest reason you have 176 postponed referral to palliative care or decided not to refer at all? Qualitative data was evaluated 177 using grounded theory. The most common reasons given by participants included: “attending 178 provider refused to make the referral,” “parents requested providers not make a palliative care 179 referral,” “the provider did not know who qualified for a referral,” and “the provider did not 180 know how to place a palliative care referral.” 181 182 Discussion The aim of this project was to determine if standardized palliative care referral guidelines 183 and an educational initiative coupled with those guidelines would have an impact on referrals 184 within the department. Traditionally, children are not referred to palliative care until late in their 185 disease trajectory, so it is important to identify ways in which providers may be able to identify 186 them earlier. While this project did show that an educational initiative was effective in provider 187 comfortability with referral, there was no statistically significant increase in the overall number 188 of referrals the department received. 189 Referral Process 190 Presurvey and postsurvey scores demonstrated increased provider comfortability with the 191 referral process; and the increase prior to and after the intervention was considered statistically 192 significant. Furthermore, participants identified that they felt the informational sessions were 11 193 valuable to their personal practice and that they were more likely to place a referral based on the 194 information provided. 195 Referral Rates 196 Referral rates between the preintervention and postintervention period increased in the 197 PICU, CCBD, and hospital-wide; however, they decreased in the NICU. While this increase in 198 hospitalwide referrals is considered to be clinically significant, it was not considered statistically 199 significant. 200 Clinical Practice and Research Implications 201 The results of this project were consistent with the evidence that standardized referral 202 criteria and educational interventions are effective in impacting palliative care referrals. Within 203 the institution, further methods of disseminating these guidelines to point of care providers 204 continue to be implemented. For example, the presentation and accompanying materials have all 205 been uploaded to the hospital’s web-based platform and educators throughout the institution have 206 access to these materials so they can incorporate them in different unit-based educational 207 initiatives. 208 Future initiatives include the potential of expanding this educational opportunity to more 209 members of the various interprofessional teams present throughout the hospital. By reaching 210 more members of the healthcare team, the number of referrals will continue to be impacted. 211 Furthermore, some of the most common reasons given for not placing referrals included a lack of 212 knowledge regarding who qualified and how to place a referral, and parent refusal. Future 213 projects could explore the possibility of targeting these areas specifically with educational 214 initiatives. Initiatives aimed towards giving providers the tools regarding how to approach 215 families with this discussion could also impact referrals. 12 216 Limitations 217 There were several project limitations that warrant discussion. The most significant 218 limitation was the small sample size. The low physician attendance was another significant 219 limitation, especially because since qualitative data revealed that one of the most common 220 reasons not to refer was related to physician refusal. By targeting this population and finding 221 methods to reach more direct patient care providers, this project has the potential to be more 222 impactful within the institution. It is possible that the format for holding these educational 223 sessions was not convenient for the majority of providers. Future endeavors could explore using 224 computer-based training or making educational sessions mandatory. 225 Another limitation of the project was the short time frame for data collection. It is 226 possible that reviewing referral rates for a longer period of time preintervention and 227 postintervention could potentially yield more statistically significant results. 228 Conclusion 229 Multiple national initiatives have called for system-wide changes to impact early 230 palliative care referral and the benefits of involving this service early in the disease trajectory are 231 undeniable for both patients and their families. While this project demonstrated that institutional 232 guidelines and provider education are an effective method for impacting referrals, there is still 233 more work to do. Pediatric palliative care is an essential service for children with chronic and 234 life-limiting conditions. Considering the multiple national initiatives aimed at combating late 235 referral, and the evidence supporting better patient outcomes, effective and functional models for 236 impacting this process are needed for the field of palliative care to continue to grow and expand. 13 237 238 References American Academy of Pediatrics. (2013). Pediatric palliative care and hospice care 239 commitments, guidelines, and recommendations, Pediatrics, 132(5), 966-972. 240 doi:10.1542/peds.2013-2731 241 242 243 American Association of Colleges of Nursing. (2017). End-of-life nursing education consortium (ELNEC). Retrieved from http://www.aacn.nche.edu/elnec Baker, J.N., Levine, D.R., Hinds, P.S., Weaver, M.S., Cunningham, M.J., Johnson, L., . . . Kane, 244 J.R. (2015). Research priorities in pediatric palliative care. The Journal of Pediatrics, 245 167(2), 467-470.e3. doi: 10.1016/j.jpeds.2015.05. 246 247 248 Butts, J. B., & Rich, K.L (2015). Philosophies and theories for advanced nursing practice (2nd ed.). Sudbury, MA: Jones & Bartlett. Chang, E., MacLeod, R., & Drake, R. (2013). Characteristics influencing location of death for 249 children with life-limiting illness. Archives of Disease in Childhood, 98(6), 419-424. 250 Children’s Health. (2015). 2015 Key facts about children’s medical center Dallas. Retrieved 251 from https://www.childrens.com/wps/wcm/connect/childrenspublic/03b8a5a1-b137- 252 4c4a-adac-48a7208829ac/ChildrensDallas-Fact-Sheet2015.pdf?MOD=AJPERES&CVID 253 =llbem4H 254 255 Davis, K.G. (2016). Integrating pediatric palliative care into the school and community. Pediatric Clinics of North America, 63(5), 899-911. doi: 10.1016/j.pcl.2016.06.013 14 256 De Clercq, E., Rost, M., Pacurari, N., Elger, B.S., & Wangmo, T. (2017). Aligning guidelines 257 and medical practice: Literature review on pediatric palliative care guidelines. Palliative 258 and Supportive Care, 1-16. doi:10.1017/S1478951516000882 259 Doorenbos, A.Z., Starks, H., Bourget, E., McMullan, D.M., Lewis-Newby, M., Rue, T.C., . . . 260 Wilfond, B.S. (2013). Examining palliative care team involvement in automatic 261 consultations for children on extracorporeal life support in the pediatric intensive care 262 unit. Journal of Palliative Medicine, 16(5), 492-495. doi:10.1089/jpm.2012.0536 263 Feudtner, C., Kang, T. I., Hexem, K. R., Friedrichsdorf, S. J., Osenga, K., Siden, H., . . . Wolfe, 264 J. (2011). Pediatric palliative care patients: A prospective multicenter cohort 265 study. Pediatrics, 127(6), 1094-1101. doi:10.1542/peds.2010-3225 266 Hays, R.M., Adams, L., & Frost, M. (2011). Introducing home-based palliative care and hospice. 267 In J. Wolfe, P.S. Hinds, & B.M. Sourkes (Eds.), Textbook of interdisciplinary pediatric 268 palliative care (pp. 195-198). Philadelphia, PA: Elsevier Saunders. 269 Hui, D., Kim, Y.J., Park, J.C., Zhang, Y., Strasser, F., Cherny, N., . . . Bruera, E. (2015). 270 Integration of oncology and palliative care: A systematic review. The Oncologist, 20(1), 271 77-83. doi:10.1634/theoncologist.2014-0312 272 Hui, D., Meng, Y.C., Bruera, S., Geng, Y., Hutchins, R. Mori, M., . . . Bruera, E. (2016). 273 Referral criteria for outpatient specialty palliative cancer care: An international 274 consensus. The Lancet Oncology, 17(12), e552-e559. doi:10.1016/S1470- 275 2045(16)30577-0 15 276 Kirolos, I., Tamariz, L., Schultz, E. A., Diaz, Y., Wood, B. A., & Palacio, A. (2014). 277 Interventions to improve hospice and palliative care referral: A systematic 278 review. Journal of Palliative Medicine, 17(8), 957-964. doi:10.1089/jpm.2013.0503 279 280 281 Kochanek, K.D., Murphy, S.L., Xu, J. & Tejada-Vera, B. (2016). Deaths: Final data for 2014. National Vital Statistics Reports, 64(4), 1-121. Lafond, D.A., Kelly, K.P., Hinds, P.S., Sill, A., & Michael, M. (2015). Establishing feasibility of 282 early palliative care consultation in pediatric hematopoietic stem cell 283 transplantation. Journal of Pediatric Oncology Nursing, 32(5), 265-277. 284 doi:10.1177/1043454214563411 285 Lutmer, J. E., Humphrey, L., Kempton, T. M., Moore-Clingenpeel, M., & Ayad, O. (2016). 286 Screening criteria improve access to palliative care in the PICU. Pediatric Critical Care 287 Medicine : A Journal of the Society of Critical Care Medicine and the World Federation 288 of Pediatric Intensive and Critical Care Societies, 17(8), e335-42. doi:10.1097/PCC.0000 289 000000000848 290 Melnyk, B.M., & Fineout-Overholt, E. (2015). Evidence-based Practice in Nursing and 291 Healthcare: A Guide to Best Practice (3rd ed.). Lippincott, Williams & Wilkins. 292 Miller, E.G., Levy, C., Linebarger, J.S., Klick, J.C., & Carter, B.S. (2015). Pediatric palliative 293 care: Current evidence and evidence gaps. The Journal of Pediatrics, 166(6), 1536-1540. 294 doi: 10.1016/j.jpeds.2015.02.019 295 296 Moore, D., & Sheetz, J. (2014). Pediatric palliative care consultation. Pediatric Clinics of North America, 61(4), 735-747. doi: 10.1016/j.pcl.2014.04.007 16 297 National Hospice and Palliative Care Organization. (2015). NHPCO’s facts and figures: 298 Pediatric palliative and hospice care in America. Retrieved from http://www.nhp 299 co.org/sites/default/files/public/quality/Pediatric_Facts-Figures.pdf 300 301 National Hospice and Palliative Care Organization. (n.d.). Pediatric hospice and palliative care. Retrieved from http://www.nhpco.org/pediatric 302 Patient Protection and Affordable Care Act, 42 U.S.C. § 18001 (2010). 303 Reville, B., Reifsnyder, J., McGuire, D. B., Kaiser, K., & Santana, A. J. (2013). Education and 304 referral criteria: Impact on oncology referrals to palliative care. Journal of Palliative 305 Medicine, 16(7), 786-789. doi:10.1089/jpm.2012.0487 306 307 308 309 310 311 312 Rycroft-Malone, J., & Bucknall, T. (2010). Models and frameworks for implementing evidencebased practice: Linking evidence to action. Chichester, West Sussex: Wiley-Blackwell. Thadani, Rahul. (2016). A well-illustrated overview on the information processing theory. Retrieved from http://www.buzzle.com/articles/information-processing-theory.html Taylor-Piliae, R.E. (1999). Utilization of the Iowa model in establishing evidence-based nursing practice. Intensive and Critical Care Nursing, 15, 357-362. Thompson, L.A., Knapp, C., Madden, V., & Shenkman, E. (2009). Pediatricians' perceptions of 313 and preferred timing for pediatric palliative care. Pediatrics, 123(5), e777-82. 314 doi:10.1542/peds.2008-2721 315 Titler, M.G., Kleiber, C., Steelman, V.J., Rakel, B.A., Budreau, G., Everett, L.Q, Buckwalter, 316 K.C., Tripp-Reimer, T., & Goode, C. (2001). The Iowa model of evidence-based practice 317 to promote quality care. Critical Care Nursing Clinics of North America, 13(4), 497-509. 17 318 Twamley, K., Craig, F., Kelly, P., Hollowell, D.R., Mendoza, P., & Bluebond-Langner, M. 319 (2014). Underlying barriers to referral to paediatric palliative care services: Knowledge 320 and attitudes of health care professionals in a paediatric tertiary care centre in the United 321 Kingdom. Journal of Child Health Care, 18(1), 19-30. doi:10.1177/1367493512468363 322 Weaver, M. S., Heinze, K. E., Kelly, K. P., Wiener, L., Casey, R. L., Bell, C. J., Wolfe, J., Garee, 323 A.M., Watson, A., & Hinds, P. S. (2015). Palliative care as a standard of care in pediatric 324 oncology. Pediatric Blood & Cancer, 62(S5), S829-S833. doi:10.1002/pbc.25695 325 326 327 World Health Organization. (2017). WHO definition of palliative care. Retrieved from http://www.who.int/cancer/palliative/definition/en/ Zhukovsky, D. S., Herzog, C. E., Kaur, G., Palmer, J. L., & Bruera, E. (2009). The impact of 328 palliative care consultation on symptom assessment, communication needs, and palliative 329 interventions in pediatric patients with cancer. Journal of Palliative Medicine, 12(4), 343- 330 349. doi:10.1089/jpm.2008.01 18 Table 1. Demographics for Study Participants Characteristic N Professional Role Physician 2 Advanced Practice Provider 37 Registered Nurse 25 Number of Years Practiced in Current Role Less than 1 year 1 1 to 5 years 18 6 to 10 years 17 11 to 20 years 16 More than 20 years 12 Number of Years Practiced at this Institution Less than 1 year 4 1 to 5 years 21 6 to 10 years 14 11 to 20 years 20 More than 20 years 5 Department Pediatric Intensive Care Unit 4 Neonatal Intensive Care Unit 14 Center for Cancer and Blood Disorders 17 Other 27 Missing* 2 Self-Estimation of the Number of Referrals Placed in the Past Year 0 31 1 to 5 21 6 to 10 10 More than 10 1 Missing 1 Percentage 3.1 57.8 39.1 1.6 28.1 26.6 25.0 18.8 6.3 32.8 21.9 31.3 7.8 6.3 21.9 26.6 42.2 3.1 48.4% 32.8% 15.6% 1.6% 1.6% * 2 participants were employed in multiple departments. Employee 1 was employed in the PICU and NICU and employee 2 held employment in the PICU and other. 19 Table 2. Knowledge Based Acquisition Questions Question Question 1 Correct Incorrect Question 2 Correct Incorrect Question 3 Correct Incorrect Missing Question 4 Correct Incorrect Missing N Percentage 58 6 90.5 9.4 63 1 98.4 1.6 45 18 1 70.3 28.1 1.6 55 1 8 85.9 1.6 12.5 20 Figure 1. Knowledge Based Acquisition Questions Correct Question 1 90.6 Question 2 Question 3 Question 4 Incorrect 9.4 98.4 70.3 1.6 28.1 85.9 1.6 21 Table 3. Comfortability Scores Paired Samples Statistics Pre-survey Comfortability in Referring to Palliative Care Post-Survey Comfortability in Referring to Palliative Care Mean N Std. Deviation 3.45 55 1.358 4.51 55 0.635 N Correlation Significance 55 0.436 <.01 Paired Samples Correlation Pre-survey Comfortability and PostSurvey Comfortability Paired Samples T-Test Pre-survey Comfortability and PostSurvey Comfortability Mean Std. Deviation Std. Error Mean -1.055 1.224 .165 95% Confidence Interval of the Difference Lower Upper -1.385 -.724 t df Sig. (2tailed) -6.392 54 <.01 22 Figure 2. Comfortability in Referring to Palliative Care 23 Table 4. Referral Rates Group Statistics PICU Referrals NICU Referrals CCBD Referrals Hospital-Wide Referrals Pre or Post Intervention Pre-Intervention Post-Intervention Pre-Intervention Post-Intervention Pre-Intervention Post-Intervention Pre-Intervention Post-Intervention N 3 12 9 6 4 7 27 44 Mean (per month) 1.5 6.0 4.5 3.0 2.0 3.5 13.5 22.0 Std. Deviation 2.121 1.414 0.707 1.414 0.000 2.121 2.121 5.657 Independent Samples Test t-test for Equality of Means t PICU Referrals NICU Referrals Equal Variances Assumed Equal Variances Not Assumed Equal Variances Assumed Equal Variances Not Assumed df 95% Confidence Interval of the Difference Sig. (2tailed) Mean Difference Std. Error Difference Lower Upper -2.496 2 .130 -4.500 1.803 -12.257 3.257 -2.496 1.742 .148 -4.500 1.803 -13.467 4.467 1.342 2 .312 1.500 1.118 -3.311 6.311 1.342 1.471 .350 1.500 1.118 -5.419 8.419 24 CCBD Referrals Hospitalwide Referrals Equal Variances Assumed Equal Variances Not Assumed Equal Variances Assumed Equal Variances Not Assumed -1.000 2 .423 -1.500 1.500 -7.954 4.954 -1.000 1.000 .500 -1.500 1.500 -20.559 17.559 -1.990 2 .185 -8.500 4.272 -26.881 9.881 -1.990 1.276 .252 -8.500 4.272 -41.589 24.589 25 Figure 3. Referral Rates Hospital-Wide