Healthy Families Arizona Evaluation Report 2003 November 2003 Prepared by: LeCroy & Milligan Associates, Inc. 620 N. Country Club Tucson, AZ 85716 (520) 326-5154 FAX (520) 326-5155 Prepared for: The Arizona Department of Economic Security Division of Children, Youth & Families Office of Prevention and Family Support 1789 W. Jefferson, Site Code 940A Phoenix, Arizona 85007 Acknowledgements This evaluation report represents the efforts of many individuals and many collaborating agencies. Craig W. LeCroy, PhD, and Pat Beauchamp, MSW, were the primary authors of this report. Kerry Milligan, MSSW provided guidance to and oversight of both the evaluation and the creation of this report. The evaluation team was instrumental in the creation of this report. Cindy Jones conducted statistical analyses. Olga Valenzuela and Veronica Urcadez performed data entry and data cleaning. Connie Warren edited and formatted the report. We are grateful to Rachel Whyte, Coordinator for the Child Abuse Prevention Fund and Healthy Families Arizona for her guidance and support. Valerie Roberson continues to provide leadership and vision for the program. Kate Whitaker, Pauline Haas-Vaughn, Barbara Griffin and Kathy Van Meter provide quality assurance and training and helped the sites to collect valid information. Bob Hoekstra and Penny Swenson provided administrative assistance for quality assurance for all the sites. Thank you to the program managers who have spent their time collecting and sending data. Staff at the sites have dutifully collected the data and answered endless questions to increase the accuracy of the data. Lastly, we acknowledge the families who have received Healthy Families Arizona services. Suggested Citation: LeCroy & Milligan Associates, Inc. (2003). Healthy Families Arizona Evaluation Report 2003. Tucson, AZ: LeCroy & Milligan Associates, Inc. ii Healthy Families Evaluation Report 2003 Healthy Families Arizona 2003 Evaluation Report Highlights Program Participants n n n n 69% of mothers were single 85% of families were on AHCCCS 15% of infants were born <37 weeks gestation (state average 10%) 14% of infants had low birth weight (state average 7%) Service Delivery n n n n n n 2,278 families were served 90% (2,043) engaged with the program (4 or more home visits) 63% remained in the program 1 year or longer Average length of time in program grew to 698 days (595 in 2002) 14% terminated due to program completion (10% in 2002) Team developed Problem Situation Inventory Outcomes n n n n n n n 99.03% of families did not have a substantiated CPS match (comparison group 98.72%) Average Parenting Satisfaction Index score improved 62% of parents’ total stress score improved Those with highest stress (above 85th percentile) had most gain Immunization rate for infants was 89% (state average 71%) 97% of children were linked to a medical doctor 14% of mothers had subsequent pregnancies (31% 18 or younger) Critical Elements n Evaluation data indicates that the program adheres to Critical Elements Recommendations n n 2-Month Program Satisfaction Survey should be reviewed Continue to emphasize family planning, especially with young mothers Future Directions n n Longitudinal study should be pursued The supervisory relationship should be evaluated as an extension of the Problem Situation Inventory iii Healthy Families Evaluation Report 2003 Table of Contents Acknowledgements ...................................................................................................ii Healthy Families Arizona 2003 Evaluation Report Highlights..........................iii List of Exhibits and Tables........................................................................................ v Executive Summary..................................................................................................vi The Healthy Families Arizona Program............................................................vi The Families........................................................................................................ viii Service Delivery..................................................................................................... ix Program Outcomes............................................................................................... ix Critical Elements ................................................................................................... xi Conclusions and Recommendations .................................................................. xi The Healthy Families Arizona Program...........................................................xii In This Report ............................................................................................................. 1 Program Participants................................................................................................. 2 Service Delivery.......................................................................................................... 6 Program Outcomes.................................................................................................. 21 Critical Elements ...................................................................................................... 31 Conclusions and Recommendations ..................................................................... 36 References.................................................................................................................. 41 Appendix A............................................................................................................... 43 Appendix B ............................................................................................................... 55 Appendix C............................................................................................................... 60 iv Healthy Families Evaluation Report 2003 List of Exhibits and Tables Exhibit 1: Selected risk factors for Health Families Arizona mothers at intake...................2 Exhibit 2: Ethnicity of Healthy Families Arizona mothers. ....................................................3 Exhibit 3: Percentage of parents rated severe on the Family Stress Checklist.....................4 Exhibit 4: Percentage of infants with high-risk characteristics...............................................4 Exhibit 5: Healthy Families Arizona families served by site..................................................7 Exhibit 6: Average length of time in program at termination for engaged families...........8 Exhibit 7: Reasons for program termination—engaged families...........................................9 Exhibit 8: Participant satisfaction responses...........................................................................10 Table 1. Demographic characteristics of the home visitors .................................................15 Table 2. Fifteen most difficult situations for home visitors .................................................16 Table 3. Fifteen most frequent situations for home visitors.................................................17 Table 4. Percent and mean number of home visitors addressing serious difficulties...............................................................................................................................18 Exhibit 9: Percent of child abuse and neglect incidences in program and comparison groups. ..............................................................................................................22 Exhibit 10: Parenting Stress Index Findings............................................................................23 Exhibit 11: Percent of safety practices implemented. ............................................................25 Exhibit 12: Immunization rate of Healthy Families Arizona children................................25 Exhibit 13: Percentage of children linked to a medical doctor.............................................26 Exhibit 14: Percent of Healthy Families Arizona participants who make appropriate use of the emergency room............................................................................27 Exhibit 15: Developmental delay from 4 to 36 months. ........................................................28 Exhibit 16: Percentage of families who screened positive for alcohol and drug problems.................................................................................................................................28 Exhibit 17: Mother’s employment status.................................................................................29 v Healthy Families Evaluation Report 2003 Executive Summary Children belong in families, which, ideally, serve as a sanctuary and a cushion from the world at large. Parents belong to society and are a part of that greater world. Sometimes parents are a channel to the larger society, sometimes they are a shield from it. Ideally, they act as filters, guiding their children and teaching them to avoid the tempting trash. Louise Hart, a noted psychologist and author, in her book On the Wings of Self-Esteem As much as parents seek to guide their children as they attempt to grow into healthy, productive adults, society on occasion has the task to help support the parents in that endeavor. Healthy Families Arizona is a program designed to provide that support, and through its efforts reduce the incidence of child abuse and neglect, provide stability for at-risk families, and grow a new generation of healthy families in the state. The Healthy Families Arizona Program Healthy Families Arizona is a home visitation program designed to provide supportive services and education to parents of newborns who might benefit from support to strengthen their families at this crucial time. The goals of the program include: § § § Promote positive parent/child interaction Improve child health and development Prevent child abuse and neglect All services are voluntary and assistance is typically provided for 12 to 18 months but may be provided for up to five years. Families enter the program based on a two level screening and assessment process. In the hospital after a child’s birth, the family can consent to an initial screening, which identifies family, parental, child and community risk factors associated with child abuse and neglect. If the screening is assessed as positive (indicating potential increased needs), the family is referred to a Family Assessment vi Healthy Families Evaluation Report 2003 Worker who conducts a more detailed interview and assessment with the family. If the assessment is positive (family may be in need based on risk), the family is offered intensive home visiting services through the Healthy Families Arizona program. Any family who has had or receives a substantiated report of child abuse and/or neglect from Child Protective Services in Arizona will be excluded from the program, as required by law. Since the program is voluntary, the family can withdraw from the program at any time. After the family is referred to the program and accepts home visitation services, a Family Support Specialist visits the family in their home on a regular basis to provide supportive services and education. The Family Support Specialist seeks to develop a trusting, open, and constructive relationship with the family to meet their individual needs. The core Healthy Families Arizona services are: § § § § § § § Emotional support Assistance in developing positive parenting skills Education on child development and nutrition Education and assistance in problem solving and coping skills Education on preventive health care (immunizations, links to medical doctor) Linkages to preschool resources Referrals related to education, employment, mental health, and substance abuse services This report focuses on aggregate data that are summarized across the 23 sites that make up the Healthy Families Arizona program. This report presents the evaluation data for the cohort of participants who received services in the Healthy Families Arizona program between the period of July 1, 2002 and June 30, 2003. This includes all families who received services at any time during the study period regardless of when they entered the program. In this year’s report, more extensive site level data can be obtained in the Appendices. vii Healthy Families Evaluation Report 2003 The Families Healthy Families Arizona served a diverse set of Asian Native African American program participants who American American 0.3% 9.0% 4.9% displayed one or more of Other the risk factors positively 3.0% associated with potential for child neglect or abuse. Hispanic Prominent among these 55.3% Caucasian were single motherhood, 27.5% mothers without a high school education, and mothers with late, inadequate, or no prenatal care. A high number of parents scored “severe” on several of the Family Stress Checklist Scales, including coping with a history of child abuse, having low self-esteem and feeling isolated, and dealing with current life stress, including low income, poor housing, and relationship difficulties. Finally, a larger than average number of infants were born early (less than 37 weeks gestation) and with low birth weight—risk factors associated with potential child abuse and neglect. Risk Factor Rate Teen Births (<19) 21.3% Births to single parents 69.2% Less than high school education 61.9% Not employed 85.3% No health insurance 3.4% Late or no prenatal care 37.6% Median yearly income $9,600 viii Healthy Families Evaluation Report 2003 Service Delivery During the study year, 2,278 families were served and 2,043 remained engaged for at least three months. For those who did become engaged in the program, factors such as marital status, depression, and infant birth weight seem to have had some bearing on the decision. The majority of the engaged families stayed with the Up to 3 3-6 months months program for at least one 10% 3% year, with the average time between enrollment and termination increasing this 7-12 year to nearly two years months 24% (698 days). While the 1 year and majority of the families beyond terminate because they 63% move or cannot be contacted, an increasing number terminate due to program completion—13.8 percent this year compared to 10.5 percent the previous year. In regard to program satisfaction, over 95 percent of all participants were satisfied with the program. Additionally, the program team (both evaluation and Technical Assistance/Quality Assurance) worked with the sites to develop a Problem Situation Inventory to use in training and staff development. This effort identified 77 unique situations encountered by home visitors, which were then scored in terms of frequency and difficulty. Finally, analyses were done that yielded five clusters of problems. The goal of this effort is to use these data to develop training modules to prepare the home visitors to better respond to these real life difficulties. Program Outcomes Healthy Families Arizona participants had a lower rate of substantiated reports of child neglect and abuse when compared to a group of families eligible for the program but who received less than four home visits. ix Healthy Families Evaluation Report 2003 Group Percent without substantiated report Healthy Families Arizona Participants 99.03% (n=1554) Comparison Group Participants 98.72% (n=234) The participant families showed significant improvement on all but one subscale on the Parenting Stress Index, and demonstrated greater safety practices during the time they spent in the program. On health-related measures, the infants of participating families were immunized at a rate that exceeds the state standard, were linked to a doctor at a high rate, and most did not use the emergency room in an inappropriate manner. Immunization period Percent immunized 2 month 95.1% 4 month 92.8% 6 month 85.1% 12 month 92.0% Received all 4 in the series 88.8% Immunization rate for 2-year-olds in Arizona 71.3% For this study period, an increased percentage of infants were screened using the Ages and Stages Questionnaire, and those identified with developmental delays were appropriately referred. In regard to maternal life course outcomes, 14.2 percent of the mothers had subsequent pregnancies, with 31.4 percent of those 18 years or younger. In addition, during the period of time they were involved in the program, a progressively higher percentage of mothers became employed, and over one-quarter of those who did not finish high school or obtain a GED did so or were in the process of doing so. x Healthy Families Evaluation Report 2003 Critical Elements A review of the data indicates that the Healthy Families Arizona program adheres to the 14 Critical Elements mandated by the state. This is based on a review of the available evaluation data and an interview with the Healthy Families Arizona TA/QA staff. Healthy Families America sees adherence to the critical elements as one way to ensure that the program is operating in accordance with best practices. This review indicates that Healthy Families Arizona continues to operate in this manner. Conclusions and Recommendations This year’s study indicates that the Healthy Families Arizona program continues to meet the needs of many of the state’s at risk families in an effective manner. The program is recruiting the target population, providing services in a manner that results in a 90 percent engagement rate, and continuing to show positive results in multiple indicators. There are two recommendations that result from this evaluation. § The Initial Parent Satisfaction Questionnaire, administered after the parent has been in the program approximately two months, should be reviewed. Currently, the survey does not provide any differentiation between families who remain engaged and those who do not. § The program should continue to emphasize family planning. Fourteen percent of the mothers had a subsequent pregnancy while in the program—30 percent of those were 18 or younger. These two recommendations, along with continued adherence to the Critical Elements, should offer the program the opportunity to continue providing effective services to a population clearly in need. xi Healthy Families Evaluation Report 2003 Introduction Children belong in families, which, ideally, serve as a sanctuary and a cushion from the world at large. Parents belong to society and are a part of that greater world. Sometimes parents are a channel to the larger society, sometimes they are a shield from it. Ideally, they act as filters, guiding their children and teaching them to avoid the tempting trash. Louise Hart, a noted psychologist and author, in her book On the Wings of Self-Esteem In some ways, this statement exemplifies the role parents play in creating healthy families in our society, and ultimately producing productive young adults. Unfortunately, fulfilling that role can be a vexing challenge for some parents. As Kamerman and Kahn (1995) observe: Parents who are stressed or disturbed will have more difficulty in meeting their children’s needs. Parents who have little support—from friends, relatives, neighbors, or the community—are more likely to be overburdened by the demands of their babies and to be unable to respond to them adequately. Parents who experience severe poverty or economic insecurity, who cannot satisfy their own basic needs, are likely to have difficulty in responding to their children’s needs. Stress, isolation, and poverty are prominent risk factors for child abuse and neglect. As much as parents seek to guide their children as they attempt to grow into healthy, productive adults, society on occasion has the task to help support the parents in that endeavor. Healthy Families Arizona is a program designed to provide that support, and through its efforts reduce the incidence of child abuse and neglect, provide stability for at-risk families, and grow a new generation of healthy families in the state. The Healthy Families Arizona Program Healthy Families Arizona is a home visitation program designed to provide supportive services and education to parents of newborns who might benefit xii Healthy Families Evaluation Report 2003 from support to strengthen their families at this crucial time. The goals of the program include: § § § Promoting positive parent/child interaction Improving child health and development Preventing child abuse and neglect All services are voluntary and assistance is typically provided for 12 to 18 months but may be provided for up to five years. Families enter the program based on a two-level screening and assessment process. In the hospital after a child’s birth, the family can consent to an initial screening, which identifies family, parental, child and community risk factors associated with child abuse and neglect. If the screening is assessed as positive (indicating potential increased needs), the family is referred to a Family Assessment Worker who conducts a more detailed interview and assessment with the family. If the assessment is positive (family may be in need based on risk), the family is offered intensive home visiting services through the Healthy Families Arizona program. Any family who has had or receives a substantiated report of child abuse and/or neglect from Child Protective Services in Arizona will be excluded from the program, as required by law. Since the program is voluntary, the family can withdraw from the program at any time. After the family is referred to the program and accepts home visitation services, a Family Support Specialist visits the family in their home on a regular basis to provide supportive services and education. The Family Support Specialist seeks to develop a trusting, open, and constructive relationship with the family to meet their individual needs. The core Healthy Families Arizona services are: § § § § § Emotional support Assistance in developing positive parenting skills Education on child development and nutrition Education and assistance in problem solving and coping skills Education on preventive health care (immunizations, links to medical doctor) xiii Healthy Families Evaluation Report 2003 § § Linkages to preschool resources Referrals related to education, employment, and mental health and substance abuse services. This report focuses on aggregate data that is summarized across the 23 sites that make up the Healthy Families Arizona program. This report presents the evaluation data for the cohort of participants who received services in the Healthy Families Arizona program between the period of July 1, 2002 and June 30, 2003. This includes all families who received services at any time during the study period regardless of when they entered the program. In this year’s report, more extensive site level data can be obtained in the Appendices.1 1 Separate site reports are produced quarterly and provided to each site to provide feedback to the site as a way of enhancing program quality. xiv Healthy Families Evaluation Report 2003 In This Report The Healthy Families Arizona Evaluation Report 2002 (published November 2002) summarized the cumulative evidence from 10 years of evaluation. That summary concluded the convergent data suggest the program was effective based on a number of findings, including: § § § § § § replicated evaluations showing improvement from baseline to post assessment periods, positive results when using a comparison group on the Parenting Stress Index, replication of positive gains and positive results from a comparison group using the Child Abuse Potential Inventory, findings showing the comparison group getting worse on most measures while the Healthy Families participants were showing improvements, findings showing immunization rates higher than the statewide average, and findings that consistently show the Healthy Families participants had lower rates of child abuse and neglect when contrasted to a comparison group not receiving the program. Following a review of the Fiscal Year 2002 data, the report concluded with a series of recommendations based on Gutterman’s (2001) practice principles. This report builds on the previous work by reviewing the Fiscal Year 2002 data in a manner similar to last year. Family demographic data are reviewed in order to assess whether the program is reaching the families it was designed for. Service delivery is discussed in regard to family engagement and retention, and participant satisfaction. In the same section, work completed over the last year regarding development of a problems situation inventory/tool is reviewed. Finally, selected outcome data is reviewed. Building upon the previous year’s discussion regarding best practices, the overall program is assessed against the Healthy Families Arizona Critical Elements in the discussion section. These critical elements are seen as a framework for ensuring adherence to best practices. 1 Healthy Families Evaluation Report 2003 Program Participants The Healthy Families Arizona program is designed to reach out to families with multiple stressors. These risk factors are positively associated with poor child health and development outcomes as well as increased risk for child abuse and neglect (LeCroy & Milligan Associates, 2001). Exhibit 1 highlights the risk factor data for the Fiscal Year 2003 program participants. Exhibit 1: Selected risk factors for Health Families Arizona mothers at intake. Risk Factor Rate Teen births (<19) 21.3% Births to single parents 69.2% Less than high school education 61.9% Not employed 85.3% No health insurance 3.4% Late or no prenatal care 37.6% Median yearly income $9,600 These data illustrate that the screening process is recruiting (and engaging) the population targeted by Healthy Families Arizona. The most notable factors are the high rates of births to single parents and parents without a high school education. While the number of parents without health insurance is low, 85 percent of those with insurance are on Arizona Health Care Cost Containment System (AHCCCS). The Healthy Families Arizona program also seeks to serve a culturally diverse population. Exhibit 2 details the population served by the various sites across the state. 2 Healthy Families Evaluation Report 2003 Exhibit 2: Ethnicity of Healthy Families Arizona mothers. Asian Native American American 0.3% 9.0% Other 3.0% African American 4.9% Hispanic 55.3% Caucasian 27.5% These data are collected by the site and used to ensure that the staff is representative of the population they serve. Site level data regarding program participant ethnicity is found in Appendix A. In addition to collecting basic demographic data during the initial screening period, the mother and the father—when they are involved—are each assessed using the Family Stress Checklist. This checklist evaluates each parent’s level of stress in 10 domains. The data for the parents scoring severe on each of the scales are presented in Exhibit 3. 3 Healthy Families Evaluation Report 2003 Exhibit 3: Percentage of parents rated severe on the Family Stress Checklist. Childhood Abuse Crime, Substance Abse, Mental Illness Self-esteem, Isolation CPS Involvement Current Life Stresses Violence Potential Expectations of Infant Discipline Attitudes Male Difficult Child Female Parental Attachment 0 10 20 30 40 50 60 70 Percent with Severe Rating As in previous years, the three most significant stressors are coping with a history of child abuse, having low self-esteem and feeling isolated, and dealing with current life stress, including low income, poor housing, and relationship difficulties. During the screening process, high-risk characteristics of infants entering the program are also assessed (see Exhibit 4). Exhibit 4: Percentage of infants with high-risk characteristics. Risk Factor Born <37 weeks gestation Birth defects Low birth weight Positive alcohol screen HFAz Rate State Rate (2001 Data) 15.3% 9.9% 1.3% NA 14.0% 7.0% 0.4% NA 4 Healthy Families Evaluation Report 2003 The percentage of Healthy Families Arizona infants born early (less than 37 weeks gestation) was noticeably higher—15.3 percent—than the statewide average—9.9 percent.2 The percentage of low birth weight infants in the program is double the state average—14 percent versus seven percent. These risk factors are known to be associated with increased potential for child abuse and neglect. The Healthy Families Arizona home visitors support the needs of these families with high-risk infants by providing support, assessment, and referral from the time of birth onward. Healthy Families Arizona served a diverse set of program participants who displayed one or more of the risk factors positively associated with potential for child neglect or abuse. Prominent among these were single motherhood; mothers without a high school education; and mothers with late, inadequate, or no prenatal care. A high number of parents scored severe on several of the Family Stress Checklist Scales that included: coping with a history of child abuse; having low self-esteem; feeling isolated; and dealing with current life stress, including low income, poor housing, and relationship difficulties. Finally, a larger than average number of infants were born early (less than 37 weeks gestation) and with low birth weight—risk factors associated with potential child abuse and neglect. 2 Based on 2001 data from the Department of Health Services Vital Statistics website. 5 Healthy Families Evaluation Report 2003 Service Delivery Two aspects of service delivery will be reviewed in this report. First, data regarding program engagement and retention are reviewed. These data can be viewed as a barometer of the success of the program in meeting its objective of supporting high risk families and reducing the potential for child neglect and abuse. These data form the output of the service delivery effort— they tell the story of what happened. Equally important to the success of the program is the “why” of what happened—why did the parents stay engaged in the program, what did the home visitors do or not do to help keep the families in the program? Part of the answer comes from the families themselves, based on program satisfaction data. That program satisfaction data will be reviewed to help answer the question. One of the issues that effects program satisfaction and, consequently, program retention, is the ability of the home visitors to deal with problem situations that arise in the course of program delivery. These problem situations often limit the ability of the home visitors to provide the services the families need. Over the last year, there has been an effort to develop a tool to help prepare home visitors to deal with the most serious and frequent of these problem situations. This tool will also be reviewed in this section. The total number of families served by all Healthy Families Arizona sites during the study period (July 1, 2002 through June 30, 2003) was 2,278. The distribution by site is shown in Exhibit 5. The number of families served by the sites is based on the funding level and number of Family Support Specialists at each site. The broad range of services offered by each site continues to be a strength of the program. These services are designed to meet needs ranging from child safety education and infant health needs (immunizations), to basic parenting skills. This strength is evident in the large number of families who stay engaged with the program despite its volunteer nature. 6 Healthy Families Evaluation Report 2003 Exhibit 5: Healthy Families Arizona families served by site. Site and Participants Served Site and Participants Served Cochise County Douglas/Bisbee Sierra Vista Coconino County Flagstaff Page Tuba City 87 52 58 Mohave County Lake Havasu City 127 105 87 Maricopa County Central Phoenix East Valley Phoenix Maryvale Mesa South Phoenix Southeast Phoenix Sunnyslope 74 80 105 115 88 97 105 Pinal County Pinal County Department of Public Health 109 Yuma County Yuma Pima County Casa de los Niños CODAC La Frontera Pascua Yaqui Child & Family Resources Santa Cruz County Nogales Yavapai County 107 Prescott Verde Valley TOTAL ALL SITES = 2,278 159 102 152 42 63 124 162 78 Engagement is a critical factor in the success of home visitation programs. Families may enroll in the program, but they are not considered actively engaged until four home visits have been completed. In the current study year, 90 percent (2,043) of the families served by the program became engaged—a slight increase from the previous year. 7 Healthy Families Evaluation Report 2003 Looking at the difference between those families who engage and those that do not, there are three factors that seem to differentiate between the two groups. First, approximately 16 percent more of those who did not engage were single parents. Secondly, more of the families who did not engage had a history of or were currently suffering from depression (a 9% difference). Finally, families with low birth weight infants engaged at a higher rate than families with normal weight infants (92.4% versus 89.5%). The average length of time in the program at termination for engaged families is shown in Exhibit 6. The data illustrate a continuing trend toward increased time in the program by families. Only three percent terminated within the first three months—compared to 4 percent previously; 62.8 percent participated for a year or longer—compared to 56 percent previously; and the average length of time in the program prior to termination grew from 595 days to 698 days. Exhibit 6: Average length of time in program at termination for engaged families. 3-6 months 10% Up to 3 months 3% 7-12 months 24% 1 year and beyond 63% 8 Healthy Families Evaluation Report 2003 Understanding why families leave the program is also important. This allows the program providers to better tailor engagement and retention strategies. The data for the families terminating are shown in Exhibit 7. As can be seen, the two main reasons for termination are the family moved away or the family could not be contacted. The data also indicate that the programs continue to show improvement, with 13.8 percent of the families terminated because they completed the program, an increase from 10.5 percent the previous year. Exhibit 7: Reasons for program termination—engaged families. Moved away 29.7% Unable to contact 27.3% Completed program Family refused further services All other reasons Refused a change in worker Self-reported self-sufficiency 0.0% 13.8% 10.0% 7.5% 5.6% 5.1% 20.0% 40.0% 60.0% 80.0% 100.0% Percent of Families Enrollment in Healthy Families Arizona is voluntary. As such, an important factor in regard to retention is participant satisfaction with the program. In order to monitor this, each of the families is asked to fill out a short program satisfaction survey after two months in the program. In the current year, 1649 parents filled out the 2-month questionnaire—1584 program families and 65 comparison group families.3 In general, both program families and comparison families were satisfied with the program at the point they responded to the survey. The majority of all respondents felt they were 3 The comparison group families were those enrolled families who terminated from the program before four home visits were completed. 9 Healthy Families Evaluation Report 2003 treated with respect, and although the comparison group’s averages were slightly higher on most measures, there were no significant differences between the groups in regard to their perception of the home visitors’ politeness, friendliness, concern, or patience. The inability of this instrument to detect any differences between those who engage and those who do not suggests that the instrument should be reviewed to determine if any changes could be made to make it more useful to program staff. Exhibit 8: Participant satisfaction responses from the two-month Initial Parent Satisfaction Questionnaire. Politeness Concern Patience Friendliness Satisfaction so far Treated with Respect 0 20 Program 40 60 80 100 Comparison Engagement, retention, and program satisfaction are all indicators of the effectiveness of service delivery. In a sense, they provide information on the output of the home visitors’ efforts in program delivery—good delivery 10 Healthy Families Evaluation Report 2003 yields higher levels of engagement, retention, and program satisfaction. Much of the success of the Arizona Healthy Families program along these lines lies in the efforts to enhance the capabilities of the home visitors through a strong quality assurance/technical assistance (QA/TA) process. To further enhance that process, the evaluation team, working with the QA/TA team, developed a tool to help home visitors better handle problematic situations they encounter with the families assigned to them. A home visitor is likely to face a number of critical decisions during the course of program delivery. She goes to the home and finds the mother is simply not motivated to participate in the child development curriculum that day. What does she do? Should she continue with the delivery of the curriculum or set aside the work and address the mother’s lack of motivation? These day-to-day issues have received scant attention. Home visiting programs rely on the home visitors to build relationships and work effectively with families. If the home visitors can’t deliver the program because of a problematic situation, then there is no reason to believe the program would have an effect. Yet, staffing is recognized as a critical part of the success of any home visitation program. Gomby (1999) notes that a home visitor’s role is critical: “Home visitors must have the personal skills to establish rapport with families, the organizational skills to deliver the home visiting curriculum while still responding to family crises that may arise, the problem-solving skills to address issues that families present in the moment when they are presented, and the cognitive skills to do the paperwork that is required.” Many reviews have, in fact, discussed and recommended staffing issues be addressed by home visiting programs. One of Gutterman’s critical practice principles is, “to effectively service families in their homes, workers must structure their work to bound and clarify their focus with families.” In this practice principle, he discusses clarifying their role, addressing timing and 11 Healthy Families Evaluation Report 2003 scheduling issues, negotiating day-to-day events that happen in the home. In a similar manner, Gomby says programs need to “explore training of workers to increase the quality of services.” A beginning point for improving the training and quality of services is to try to understand the difficulties of doing home visitation and the competencies needed to address those difficulties. The skills and abilities of home visitors are frequently taught on the basis of face validity alone. Indeed, few efforts have been made to empirically identify the skills and competencies of home visitation. To develop the content of supervision and training programs without first analyzing the performance problems of a population defeats the intent of helping home visitors interact effectively with their environments. Lewin (1939) argued a long time ago about the importance of the situation in understanding human behavior. He believed that any attempt to understand a person’s behavior by studying the individual and not the environment in which he or she functions was necessarily incomplete. Within this framework that acknowledges the environment, Mischel (1968, p. 10) has observed, “the emphasis is on what a person does in situations rather than on inferences about what attributes he has more globally.” From this conceptualization, one’s environment places demands on a person that are experienced as problematic situations. The extent to which a person can effectively address those situations is determined by the skills and competencies he or she has to meet those environmental demands. Problems occur when there is an imbalance between abilities or competencies and demands in the person-in-environment system. Therefore, the task is to match the person’s competencies with the situational demands of the environment by establishing a balance in the system either through promotion of competencies to meet the demands, or through decreasing or eliminating the environmental demands. The implication for supervision and training is clear; identified problematic situations provide a framework for designing training and supervision to teach the skills needed to competently interact in those difficult situations. 12 Healthy Families Evaluation Report 2003 LeCroy & Milligan Associates elected to address this issue by developing a Problem Situation Inventory for use by the program. A first step in understanding the role of situational variables is the development of a taxonomy of difficult situations. The situational or environmental context is considered important because situations can contain subtle and complex factors that elicit poor performance. While the situations themselves may contain critical information needed to understand how to develop competent responses, a taxonomy may also be useful. Two types of potential taxonomies that can be helpful include the frequency of problem situations and the difficulty of problem situations. Home visitors ought to know how to respond to problem situations that come up more frequently and home visitors ought to be able to respond to difficult situations since it is here that the demands of the situation may exceed their skills and abilities. The initial step in an effort to develop such a taxonomy is to identify a large, representative sample of common problem situations that are relevant and genuinely problematic for home visitors. These situations should include those specific situations with which individuals in that environment must respond to effectively to be considered “competent” (Goldfried & D’Zurilla, 1969). Furthermore, these situations need to be “problematical” in that how to respond is not immediately apparent. A large pool of problem situations was generated using a sample of 114 home visitors. Approximately 20 focus groups were conducted to gather the situations. In groups of about 5-8, home visitors were given instructions to identify problem situations. Specifically they were told to: “Make a list of difficult or challenging situations you’ve encountered, situations where you weren’t sure what to do, situations that didn’t go well.” These situations were sorted and reviewed so that redundant items were eliminated and similar situations were combined into one. This produced a final list of 77 problem situations. An inventory of these items was created in order to obtain ratings of the frequency and difficulty associated with each problem situation. Five-point Likert scales were used to rate the frequency 13 Healthy Families Evaluation Report 2003 and difficulty of the situations. For example, situations were rated from very frequent to rarely, and very difficult to very easy. This inventory of situations was then administered to a sample of 91 home visitors who completed the questionnaire, rating each situation and answering some demographic questions. The background characteristics of the 91 home visitors are presented in Table 1. All home visitors were female and most were Caucasian (42%), although a large percentage was Hispanic (28.6%). The average age was 35.4 years old and most of the home visitors had some college education. They were fairly experienced, having done home visiting for an average of 3.8 years. The majority (70.4%) of home visitors were themselves mothers. 14 Healthy Families Evaluation Report 2003 Table 1. Demographic characteristics of the home visitors. Characteristic N Percent Ethnicity Caucasian Hispanic African Am Asian Am Native Am Mixed race Other 38 26 4 1 6 12 4 42.9 28.6 4.4 1.1 6.6 13.2 4.4 5 33 39 8 5.9 38.8 45.9 9.4 57 24 70.4 29.6 Age Mean age SD 35.4 10.4 Schooling HS graduate Some college college degree grad degree Length of time in position Mean months 46.2 SD 48.1 Have children Yes No Table 2 presents the top 15 problem situations rated to be difficult based on the mean ratings by home visitors. Perhaps surprisingly, the situation rated as most difficult was “working with limited resources to help parents.” Home 15 Healthy Families Evaluation Report 2003 visitors are clearly frustrated in their attempts to provide the kinds of services they believe families need. The other situations suggest clear difficulties with parents where substance use is present. Working with uncommitted or unmotivated families also rated high on difficultly. In a similar manner, situations where it seems difficult for them to do their job rated highly such as changing parenting styles and contacting parents. Lastly, a lot of the situations represent some of the more “clinical” aspects of their work with families: substance use, suicide, domestic violence, and crisis situations. Table 2. Fifteen most difficult situations for home visitors. Mean4 SD 5 Working with limited resources to help parents 3.58 1.14 Helping parents that threaten to commit suicide Working with families when one person is u nder the influence 3.34 3.34 1.23 1.33 Working in the homes during the summer heat 3.31 1.26 Working with families when someone gives drugs or alcohol to children 3.31 1.26 Responding to threats or dangerous behavior directed at you 3.19 1.46 Working with uncommitted families Working with families that aren’t motivated 3.09 3.08 1.10 1.13 Dealing with family members who show up under the influence 3.00 1.27 Inability to contact parents Working with parents to change their parenting style 2.99 2.98 1.13 1.13 2.96 1.12 2.92 2.89 2.87 1.00 1.23 1.08 Most Difficult Situation Working with family members who are not motivated because of alcohol or drug problems Working with families who are in constant crisis Providing services in unsafe homes Addressing domestic violence 4 The “average” score for that item across all workers who took the survey. 5 The Standard Deviation, an indicator of the range of scores on that particular item. 16 Healthy Families Evaluation Report 2003 Table 3 presents the top 15 problem situations rated to occur most frequently by home visitors. The number one most frequent difficulty, “working in homes during the summer heat” is a problem situation most likely to be unique to Arizona. Many of the situations that were rated as frequent reflect some of the fundamental aspects of doing home visitation such as work with teenage mothers, keeping the environment confidential, selecting activities for the home visit, working with parents whose decisions you don’t agree with, working with unmotivated families, and working with parents’ emotional feelings like sadness. In many respects, these situations represent a broad diversity of problem situations that occur with a high level of frequency. Table 3. Fifteen most frequent situations for home visitors. Mean SD Working in homes during the summer heat Working with limited resources to help parents 3.97 3.52 1.26 1.21 Working with teenage mothers 3.48 1.11 Trying to create a confidential environment Knowing what activities to do in a home visit 3.22 3.22 1.42 1.6 Working with parents whose decisions you don’t agree with 3.19 1.18 Working with families that aren’t motivated Working with parents’ emotional feelings like sadness 3.19 3.18 1.18 1.10 Helping families when they are experiencing a crisis 3.08 1.08 Working with uncommitted family members Working with parents who have different values 3.07 3.04 1.23 1.28 Working with immature clients 3.04 1.16 Working with parents who are in denial about their problems Trying to collaborate with other agencies 2.98 2.98 1.24 1.24 Inability to contact clients to set appointments 2.98 1.30 Most Frequent Situation 17 Healthy Families Evaluation Report 2003 It is also noteworthy to examine items that are both rated high on difficulty and rated high on frequency. These problem situations include: § § § § § working with families who are in a constant crisis, working with limited resources to help parents, working with uncommitted family members, working with families that aren’t motivated, and working in the homes during the summer heat. After having conducted the focus groups and refining the situations into the inventory, the level of truly “difficult” situations that home visitors faced became more prominent. Perhaps because during the focus group process we heard very specific examples of difficulties that home visitors had faced. Because of this it was decided to ask home visitors direct questions about their experience with three critical problems home visitors face: domestic violence, substance abuse, and mental illness. Table 4 presents data on the percent of home visitors who have experienced these problems in the last year and in the last 30 days. As the table reveals, these serious issues are fairly common, with over 80% of the home visitors having seen families with these issues in the last year. When asked about the occurrence of these issues in the last 30 days, over 60% of the home visitors had experienced working with families on these issues. On average, the home visitors worked with approximately five families with these problems in the last year, and two in the last 30 days. Table 4. Percent of home visitors addressing serious difficulties. In the last year Domestic Violence Substance Abuse Mental Illness In the last 30 days 81.8% 82.7% 86.7% 64.6% 67.5% 78.5% 18 Healthy Families Evaluation Report 2003 Additional analyses were used to look at the various situations to determine if the situations could be grouped into common themes or clusters. This resulted in five identified clusters: § § § § § lack of clinical skill, addressing family difficulties, addressing parenting difficulties, personal issues, and lack of experience. These clusters could be used as a framework for training modules that address the problem situations identified by this effort. They could also be used to help home visitors better understand their own training needs and enable tailored training to address those needs. The taxonomy developed during this effort provides a structure for designing supervision and training to teach the skills and competencies needed to satisfactorily interact in a variety of difficult situations. (Too often training and supervision focuses on characteristics deemed important to home visitation rather then empirically derived contextual situations). The difficulty ratings could be used to conceptualize training with more of a focus on: working with difficult family issues; addressing domestic violence, substance abuse, and mental illness; and motivating families. The frequency ratings provide a very direct agenda for training, for example: § § § § § § § § working with limited resources to help parents, working with teenage mothers, trying to create a confidential environment, knowing what activities to do in a home visit, working with parents whose decisions you don’t agree with, working with families that aren’t motivated, working with parent’s emotional feelings like sadness, and helping families when they are experiencing a crisis. 19 Healthy Families Evaluation Report 2003 A training program based on the data developed during this effort has the potential to yield significant benefits to the Healthy Families Arizona program. Enabling the home visitors to deal effectively with the problem situations can reduce their own stress. The family gets the help it needs in dealing with the problem, reducing stress and allowing them to move forward in the more routine aspects of parenting. In the end, the quality of services provided by the home visitors will be enhanced by this collective effort. During the study year, 2,278 families were served and 2,043 remained engaged for at least three months. For those who did become engaged in the program, factors such as marital status, depression, and infant birth weight seem to have had some bearing on the decision. The majority of the engaged families stayed with the program for at least one year, with the average time between enrollment and termination increasing this year to nearly two years (698 days). While the majority of the families terminate because they move or cannot be contacted, an increasing number terminate due to program completion—13.8 percent this year compared to 10.5 percent the previous year. In regard to program satisfaction, over 95 percent of all participants were satisfied with the program. Additionally, the program team (both evaluation and TA/QA) worked with the sites to develop a Problem Situation Inventory. This effort identified 77 unique situations, which were then scored in terms of frequency and difficulty. Finally, analyses were done that yielded five clusters of problems. The goal of this effort is to use these data to develop training modules to prepare the home visitors to better able to respond to these real life difficulties. 20 Healthy Families Evaluation Report 2003 Program Outcomes Healthy Families Arizona continues to collect outcome data on a number of outcome indicators in order to evaluate program effectiveness. These indicators include program impact on child abuse and neglect, parental stress and competence, safety practices in the home, medical and social service use, and employment and educational attainment. Many practitioners look to the incidence of child abuse and neglect as an indicator of program impact. However, reports of child abuse and neglect are unlikely to be good measures of program impact for several reasons, including: § § § child abuse and neglect are low occurring events, and even small changes can appear to be significant when they are not; many incidents of child abuse and neglect go unreported, calling into question the reliability of the measure; increased surveillance of families who are involved in community programs such as Healthy Families Arizona may lead to increased reporting. For these reasons, caution must be used when coming to conclusions regarding program impact based on child abuse and neglect data. Exhibit 9 presents the data regarding child abuse and neglect reports for the families participating in the Healthy Families Arizona program. Data for two groups are presented—the Healthy Families Arizona program families and a comparison group. The program group consists of families who have had at least four or more home visits and the comparison group consists of families who dropped out prior to completing at least four visits. The results are based on all families who were active in the program during the study period of July 1, 2002 to June 30, 2003 with at least six months time in the program. As seen in the exhibit, both groups had high rates of families with no substantiated reports. There is no practical difference between the two rates. 21 Healthy Families Evaluation Report 2003 Exhibit 9: Percent of child abuse and neglect incidences in program and comparison groups. Group Percent without substantiated report Healthy Families Arizona Participants 99.03% (n=1554) Comparison Group Participants 98.72% (n=234) Reducing parental stress is one of the primary indicators for the success of the Healthy Families Arizona program due to the relationship between parental stress and child abuse and neglect. Healthy Families Arizona uses the Parenting Stress Index (Abidin, 1995) to assess total stress and data regarding seven subscales—sense of competence, parental attachment, feeling restricted in one’s role, depression, isolation, distractibility, and mood. This instrument is a reliable and valid measure used extensively in research and evaluation. Exhibit 10 shows the scores for each of the subscales and the total stress score for baseline data (pretest) and subsequent tests at six, 12, and 18 months. As illustrated in the exhibit, the total parenting stress score shows significant change across all time intervals. The sense of competence, feeling restricted in role, depression, isolation, and mood subscales all showed increases across all time intervals, and the parental attachment subscale showed increases in the first two intervals—baseline to six months and baseline to 12 months.6 Only the distractibility showed no significant differences, which may be due to the low reliability of the scale. Definitions of the subscales and additional statistical details can be found in Appendix B. 6 For the group that took the 18-month PSI there was a small positive change in scores from baseline to 18 months, but the change was not large enough to be statistically significant. 22 Healthy Families Evaluation Report 2003 Exhibit 10: Parenting Stress Index Findings. Time Period Subscale Baseline to 6 months Baseline to 12 months Baseline to 18 months Sense of Competence Significant Improvement Significant Improvement Significant Improvement t=7.72, p<.000 t=7.07, p<.000 t=4.60, p<.0001 Significant Improvement Significant Improvement No Significant Improvement t=3.62, p<.000 t=4.29, p<.000 t=1.26, p>0.05 Significant Improvement Significant Improvement Significant Improvement t=3.13, p<.002 t=3.27, p<.001 t=4.02, p<.000 Significant Improvement Significant Improvement Significant Improvement t=4.82, p<.000 t=569, p<.000 t=2.78, p<.01 Significant Improvement Significant Improvement Significant Improvement t=3.56, p<.000 t=3.97, p<.000 t=3.49, p<.001 No Significant Improvement No Significant Improvement No Significant Improvement t=4.99, p>0.05 t=1.15, p>0.05 t=1.15, p>0.05 Significant Improvement Significant Improvement Significant Improvement t=9.00, p<.000 t=6.36, p<.000 t=2.61, p=0.01 Significant Improvement Significant Improvement Significant Improvement t=7.25, p<.000 t=7.55, p<.000 t=4.74, p<.000 Parental Attachment Feeling Restricted in Role Depression Isolation Distractibility Mood Total Stress Score During the study year, parents were administered the baseline (2-month) Parent Stress Index (PSI) and a 6-month PSI. A review of the distribution of the scores shows that the average score on the 2-month PSI was 135.5. For 23 Healthy Families Evaluation Report 2003 those parents taking both the 2-month PSI and the 6-month PSI, 62 percent showed improvement in the total stress score on the 6-month PSI. However, looking at those parents who scored above the 85th percentile on the 2-month PSI—those considered to be at highest stress —70 percent showed improvement at the 6-month point. In addition, the average improvement for that group was 17 points compared to only 4 points for those scoring below the 85th percentile. This indicates that the program has a greater effect in regard to reducing stress for those scoring highest on the instrument and, according to the author of the instrument, at greater risk to develop dysfunctional parenting patterns. On a more basic level, improved safety practices in the home provide a direct indication of the program impact. By being in the parents’ home environment, home visitors have an advantage over other parenting programs. Home visitors can demonstrate good safety practices in the home, and then directly observe the parents’ use of these practices over time, rather than depend on parental self-report. Based on the child safety checklist, the data show that most families follow sound safety procedures and that in some cases, increase use of safety practices over time. Exhibit 11 details two of the items tracked on the child safety checklist— outlets covered and poisons being locked. Increases in use of safety practices can be seen across all three assessment periods—2, 6, and 12 months. Other indicators include appropriate use of car seats, the securing of scissors, knives, lighters and matches, water safety, outside supervision, food storage, and a listing of emergency phone numbers. At the two-month assessment, these other safety practices were all being actively used by over 90 percent of the participants. 24 Healthy Families Evaluation Report 2003 Exhibit 11: Percent of safety practices implemented. 2 Month 6 Month 12 Month Outlets Covered 49.8% 61.1% 75.5% Poisons Locked 84.8% 87.4% 93.1% One of the goals of Healthy Families Arizona is to ensure that all the families receive adequate medical care. There are two measures that can be used to assess this goal. The immunization rate for the children is the first of these. Exhibit 12 shows this rate for the infants of families enrolled in Healthy Families Arizona in the current study year. Exhibit 12: Immunization rate of Healthy Families Arizona children Immunization period Percent immunized 2 month 95.1% 4 month 92.8% 6 month 85.1% 12 month 92.0% Received all 4 in the series 88.8% Immunization rate for 2-year-olds in Arizona 71.3% In comparison with the overall state rate, the families in the Healthy Families Arizona program are doing well, in spite of the fact that they represent a high-risk group perceived as less likely to receive immunizations. There is also an improvement in the immunization rate when compared to the 2001 Healthy Families Arizona families. For example, the percentage of infants receiving all four immunizations in the series increased from 83.9 percent to 88.8 percent. All the monthly measures showed similar increases. 25 Healthy Families Evaluation Report 2003 A second way to look at the goal of ensuring the families receive adequate medical care is to look at the percentage of children linked to a medical doctor. The data are shown in Exhibit 13. While there is no comparison data available, this does indicate that the program is meeting one of its critical goals—ensuring adequate medical care for its participants. The Healthy Families Arizona program has emphasized timely immunization of the infants in the program. Data regarding the immunization rate and the linkage to a doctor are reported regularly to the sites via the Healthy Families Arizona Quarterly Family Data Report, which is used by the sites as an ongoing quality assurance tool. These data allow the sites to work with families not yet achieving these milestones to do so—an effort that can have a positive effect on healthy families and healthy children. Exhibit 13: Percentage of children linked to a medical doctor. 6 months 97.1% 12 months 97.9% 24 months 97.7% 0.0% 20.0% 40.0% 60.0% 80.0% 100.0% Percentage of Children Linked to Medical Doctor A related outcome addresses one concern from the health providers regarding program participants—inappropriate use of the emergency room. As can be seen in the exhibit below, a significant majority of participants use the emergency room only after obtaining a doctor’s referral. 26 Healthy Families Evaluation Report 2003 Exhibit 14: Percent of Healthy Families Arizona participants who make appropriate use of the emergency room. 6 months 12 months 24 months 30 months 36 months 42 months 48 months 54 months 60 months 0 10 20 30 40 50 60 70 80 90 100 Percent Using Emergency Room Appropriately Another goal of the Healthy Families Arizona program is the promotion of healthy child development. Home visitors attempt to administer the Ages and Stages Questionnaire to all of their families in order to assess the children’s developmental status. The information obtained allows the home visitors to help parents learn new ways in which they can encourage proper stimulation for growth and development of their child. At the 4-month point, 66.4 percent of the families completed the questionnaire—a 10-percentage point increase from the previous year. At the 6-month point, 75.5 percent of the families completed the questionnaire—a 15-percentage point increase from the previous year. In a manner similar to the medical issues, the quarterly reports to the sites help supervisors and staff focus on the need to administer the Ages and Stages Questionnaire with all families on a timely basis. In addition to encouraging healthy child development, the questionnaire provides for the early detection of developmental delays. This early 27 Healthy Families Evaluation Report 2003 detection allows for referral to appropriate follow-up services such as the Arizona Early Intervention Program (AzEIP), or another intervention program. Exhibit 15 displays the information regarding the percentage of children screened positive for developmental delays. Almost all of the children identified were referred to follow-up services. Exhibit 15: Developmental delay from 4 to 36 months. Developmental delay at 4 months 12.3% Developmental delay at 12 months 4.1% Developmental delay at 24 months 19.5% Developmental delay at 36 months 21.2% Another risk for these families is the potential for alcohol and drug problems, which is strongly linked to child abuse and neglect. The Healthy Families Arizona program provides an initial screening to families in an effort to help determine who may need to seek alcohol or drug treatment. Those who are identified are referred to appropriate treatment programs when possible, but availability of treatment programs in many Arizona communities is limited. While only a small number of those screen positive7 (Exhibit 16), a referral to treatment could be instrumental in reducing the risk for child neglect and abuse. Exhibit 16: Percentage of families who screened positive for alcohol and drug problems. 7 2 months (N=35) 6.3% 6 months (N=16) 7.8% 12 months (N=22) 4.6% 18 months (N=12) 4.6% The screen is a self-report instrument that may result in under-reporting. 28 Healthy Families Evaluation Report 2003 While the Healthy Families Arizona program focuses on parent-child interactions, the program can also benefit maternal life outcomes such as subsequent pregnancies, education, and employment. During the study period, 14.2 percent of the participants reported subsequent pregnancies. Of these, 31.4 percent were 18 years or younger. In terms of how quickly they became pregnant, 33.0 percent did so within one year, 45.9 percent became pregnant within 1 to 2 years, and 21.1 percent became pregnant over two years later. Exhibit 17 shows the percentage employment status of mothers actively engaged in the program at various points in the program. Exhibit 17: Mother’s employment status. 100.0% Percent 80.0% 60.0% 20.0% 39.4% 34.6% 40.0% 14.7% 0.0% Baseline 6 months 12 months Another positive outcome in this area is the number of mothers who continued to be involved in educational programs while being served by Healthy Families Arizona. For mothers that were involved in the program at the six month point, 10 percent were enrolled full-time in school, 6 percent were enrolled part-time, 9 percent had obtained their high school diploma or GED during the six months they were enrolled, and 1 percent had obtained a college degree. 29 Healthy Families Evaluation Report 2003 Healthy Families Arizona participants had a lower rate of substantiated reports of child neglect and abuse when compared to a group of families eligible for the program but who received less than four home visits. The participant families showed significant improvement at the 12 month point on all but one subscale on the Parenting Stress Index, and demonstrated greater safety practices during the time they spent in the program. On health related measures, the infants of participating families were immunized at a rate that exceeds the state standard, they were linked to a doctor at a high rate, and most did not use the emergency room in an inappropriate manner. For this study period, an increased percentage of infants were screened using the Ages and Stages Questionnaire, and those identified with developmental delays were appropriately referred. In regard to maternal life course outcomes, while only 14.2 percent of the mothers had subsequent pregnancies, 31.4 percent of those were 18 years or younger. In addition, during the period of time they were involved in the program, a progressively higher percentage of mothers became employed, and one-quarter of those who did not finish high school or obtain a GED did so or were in the process of doing so. 30 Healthy Families Evaluation Report 2003 Critical Elements The previous year’s report focused on Gutterman’s (2001) best practices as an organizing tool for the findings and recommendations. An alternative way of viewing the program is to look at adherence to the Critical Factors mandated by the state. Healthy Families America takes the position that this framework of critical elements “represent quality standards and ensure the program’s adherence to best practices in home visitation” (PCA, 2002). In effect, the critical elements are a tool to be used to ensure that the best practices are adhered to. In the following review, those elements for which there is applicable evaluation data are assessed. The rationale cited for each of the critical elements is based on Prevent Child Abuse America’s literature review of their critical elements (2001). Intake at birth (3 month maximum chronological age of infants at intake) This element addresses the benefits of early intervention with families at risk. These benefits include early linkage with appropriate medical care, the ability to enhance the bonding between the child and parents, and facilitating a strong bond and respectful relationship between the parents and the home visitor. Of the 2043 families actively engaged in Healthy Families Arizona during the study period, 99.5 percent were enrolled within three months. The Healthy Families Arizona Screen and the Parent Survey (Family Stress Checklist), a standardized risk assessment tool, will be used to systematically identify families at risk for poor outcomes from defined areas. The use of a standard instrument ensures that the program is reaching the families for which it is designed. In addition, it provides the home visitor with insight into strengths, weaknesses, and risk factors of the families they are working with. All of the families enrolled in Healthy Families Arizona had both the screen and the Family Stress Checklist administered. The assessment specifically identifies issues known to be risk factors for potential child neglect and abuse. The data indicate that the families involved in the 31 Healthy Families Evaluation Report 2003 program exhibit high percentages of single parent families, families with low income, parents with less than a high school education, and unemployed parents. Program participation is voluntary and services will be offered to eligible families in a positive and supportive manner. Outreach services will be provided up to three months for the purpose of re-engaging an enrolled family to an appropriate service level. If a family requests closure, their case will be closed and no further outreach will be attempted. Programs based on voluntary acceptance of services are generally more effective. In addition, voluntary programs empower the parents—they feel they are part of the solution rather than feeling a solution is being forced on them. By providing extended outreach, families who are initially reluctant to accept services may later enroll as they see the benefits of the program and recognize the increased stress associated with having a newborn. Healthy Families Arizona is a voluntary program. The data indicate that the families who enroll see the home visitors in a positive, supportive light. For example, in a satisfaction survey administered two months following enrollment, 95 percent of the families saw the worker as concerned about their welfare and 99 percent of the families viewed the home visitor as friendly, patient, and polite. This positive assessment offers evidence that the program is provided in a positive, supportive manner. Home visitation is the core service for all families. Intensity of services will vary based on family needs, moving gradually from weekly to quarterly home visits as families become more self sufficient. Program services are available for up to five years. The provision of intensive services allow for establishment of trust between the program families and the home visitors. At birth, the families’ needs are greatest. As time passes and parents become more comfortable with their role, the need lessens, although continued service allows the program to address new challenges as they arise. The Healthy Families Arizona program 32 Healthy Families Evaluation Report 2003 uses a multi-level system to address this critical element. As families come into the program, they are visited weekly. Over time, the visits are reduced to bi-monthly and quarterly. For example, for families in the current year, 70 percent of the families who had been in the program nine months were at Level 1 (weekly visits), and 18 percent were at level two (twice monthly visits).8 For families in the program 18 months, only 24 percent were on Level 1, 58 percent were on Level 2, and five percent were on Level 3 (quarterly visits). At 36 months, only 8 percent remained on Level 1, 46 percent were on Level 2, and 38 percent were on Level 3. This illustrates not only that the program complies with the element of basing intensity on need, but also the ability of the program to provide services over an extended length of time. Services shall be culturally sensitive and materials should reflect the cultural, linguistic, geographic, racial and ethnic diversity of the population served. Cultural context is one of the determining factors in regard to needs, beliefs, and practices. Failure to address cultural context may inhibit the home visitor’s ability to establish relationships with the family, limiting their ability to work together to address the problems the family faces. There are two approaches to reviewing adherence to this element. One is to look at diversity among the home visitors. While there is not a one-to-one match of ethnicities between participant families and home visitors, the home visitors do reflect the overall population diversity within the state. Importantly, when surveyed, virtually 100 percent of the families indicated that the home visitor spoke their language, services were provided based on their language and culture, and that materials were provided in their own language. Clearly, the families’ opinion is that the program does provide services in a culturally relevant manner. 8 The remaining families were either on outreach or at pre-designated special levels. 33 Healthy Families Evaluation Report 2003 Services shall be family centered, culturally sensitive, and focus on enhancing parent-child interaction, child development, and family functioning. Enhancing parent-child interaction, providing information on normal child development and helping the family function in an effective manner all help reduce the stress the parent feels with a new infant. This in turn helps decrease the potential for child neglect and abuse. As mention previously, the families feel that the program is respectful of their culture, and delivered in a culturally relevant manner. In addition, many of the services used by the families are specific to this critical element. For example, looking at the service data for the 12-month point: § § § § 84% of the families received the Model Bonding/Parent Child curriculum, 95% received child development education, 81% were provided emotional support, and 62% were given the Life Coping Skills curriculum. The program provides a variety of services that focus on the family and helps prepare the parents to deal appropriately with their children. Services shall address linking families to health care systems with an emphasis on prevention (immunizations, primary care physicians and well baby care). Depending on need, the families may be linked to services such as financial, food and housing assistance programs, child care, job training programs, family support centers, substance abuse treatment programs, domestic violence shelters, and school readiness programs (Head Start, at-risk preschool, family literacy). Many of the families involved in the Healthy Families Arizona program have wide-ranging needs. Focusing on health care early in the program has numerous benefits, including prevention of future health related problems, increased comfort levels when working with other social support systems, 34 Healthy Families Evaluation Report 2003 and increased potential for a good educational experience on the part of the child. As reported earlier in the report, the rate of immunizations was higher than the state average and over 97 percent of the infants were linked to a primary care physician at the 6, 12, and 18 month periods. The families enrolled in the program used a large array of services. These included: § § § § § § § crisis intervention (30% of families at the 12 month point), case management services (45%), transportation (15%), information and referral (74%), parent groups (29%), literacy programs (28%), and playgroups (20%). In addition, for those families with substance abuse issues/concerns, educational materials were provided and referrals were made to outside support services. All of these services assist the families in dealing with the stress of dealing with unfortunate life situations while at the same time trying to raise an infant. The evaluation does not collect information on the remaining critical elements, which are largely associated with staffing, training, and quality assurance. However, the Healthy Families Arizona’s last Credentialing process revealed that the programs also adhere to Critical Elements 8 through 14. Continued adherence was confirmed in an interview with the Healthy Families Arizona TA/QA staff. A review of the data indicates that the Healthy Families Arizona program adheres to the 14 Critical Elements mandated by the state. This is based on a review of the available evaluation data and an interview with the Healthy Families Arizona TA/QA staff. Healthy Families America sees adherence to the critical elements as one way to ensure that the program is operating in accordance with best practices. This review indicates that Healthy Families Arizona continues to operate in this manner. 35 Healthy Families Evaluation Report 2003 Conclusions and Recommendations The Healthy Families Arizona program continues to meet the needs of many of the state’s at risk families in an effective manner. The program served a population reflective of the state’s diversity. These families entered the program with a number of risk factors, including a high percentage of teenage parents, many single parent families, high rates of unemployment, low income, and many cases where pre-natal care was non-existent or late. Cases of low-weight births and early births exceeded the state averages. Many of these families also had histories of childhood abuse, substance abuse, and other mental health problems, and suffered from low self-esteem, isolation, and stress from daily living. In summary, the program is serving its target population—families at-risk. In regard to service delivery, two important indicators are the engagement rate—the percentage of families who receive at least four home visits—and the average length of time in the program at termination. Both of these indicators showed increases from the previous year—engagement rose to 90 percent and the average length of time in the program grew from 595 days to 698 days. This is part of a continuing trend over the last several years, and reflects the emphasis the program staff has placed on engagement and retention. Another indicator of effective program delivery is participant satisfaction. Based on results of a survey given after the family has been in the program two months, program satisfaction is high—exceeding 95 percent on all the questions asked—for engaged families and a small group of families who did not remain enrolled in the program. The fact that there was no difference between the engaged group and the non-engaged group suggests that the survey may need to be revised to better capture why the non-engaged group did not stay with the program. Looking at outcomes, the Healthy Families Arizona program continues to have very low rates of substantiated reports of child abuse or neglect, although this measure has limited utility due the uncertain nature of child abuse and neglect reporting data. The parents in the program are given the 36 Healthy Families Evaluation Report 2003 Parenting Stress Index at a number of points in the program. The total stress scores showed significant improvement from the baseline score across the 6month, 12-month, and 18-month intervals. Of the parents who took both a baseline and 6-month PSI, 62 percent showed an improvement in their total stress score. Importantly, of those parents scoring at or above the 85th percentile on the baseline measure, 70 percent showed improvement at the six-month point. This differential effect provides some evidence that the program is increasingly effective for those parents at the greatest risk as measured by the PSI. There are a number of other outcome measures that indicate that the program is working. Increased child safety is seen in the gains in the child safety checklist over time. The overall immunization rate is substantially higher than the state rate, and reflects an increase from previous years. A very high rate of the children are linked to a medical doctor, and a substantial number are screened for developmental delays by the Family Support Specialists. Maternal life outcomes are also tracked. During the study period, 14.2 percent of the mothers had subsequent pregnancies. Of those who did become pregnant, 31 percent were 18 or younger, and 33 percent became pregnant within one year. These data suggest a need to continue to emphasize family planning, especially with the teenage mothers. Finally, a review of the data indicates that the Healthy Families Arizona program adheres to the 14 Critical Elements mandated by the state. This is based on a review of the available evaluation data and an interview with the Healthy Families Arizona TA/QA staff. Adherence to the critical elements is an indicator that the program is operating along best practice lines. Based on this evaluation, two recommendations are made. First, the program satisfaction survey should be reviewed. Ideally, the survey should provide information that allows the program staff to differentiate between those who are likely to engage and those who are not, and 37 Healthy Families Evaluation Report 2003 information on the reasons why. The current instrument does not provide that kind of data. Second, the program should continue to emphasize family planning, especially with young mothers. Fourteen percent of the program participants had subsequent pregnancies while enrolled in the program. One-third of them were by mothers 18 or younger. These two recommendations, along with continued adherence to the Critical Elements, should offer the program the opportunity to continue providing effective services to a population clearly in need. Future Considerations for Program Improvement and Program Evaluation Looking to the future, there are several avenues to explore to continue improving the evaluation and consequently, strengthening the program. Greater understanding of outcomes and the program components leading to outcomes would be gained from a longitudinal study. In the recent Emerging Practices in the Prevention of Child Abuse (Caliber, 2003), the need to expand existing knowledge about the effectiveness of prevention is the overriding theme. The report specifically states, “Existing knowledge about the efficacy of prevention in the field of child maltreatment is limited; clearly, all the major prevention models and strategies could benefit from more rigorous study” (p. 1). Healthy Families Arizona was selected by the Emerging Practices in the Prevention of Child Abuse and Neglect in 2003 (Caliber Associates, 2003) in the “Effective Programs Category for Programs with Noteworthy Aspects” for its independent comprehensive evaluation, its management information system, and for involving and retaining fathers in the program. Healthy Families Arizona was selected by National Center for Children in Poverty, 38 Healthy Families Evaluation Report 2003 Columbia University as one of 25 noteworthy programs for its use of standardized instrumentation to monitor child development, parental stressors, and coping skills, incorporating a Training and Quality Assurance Team, and developing an extensive data tracking system to ensure effectiveness and quality. Because of these independent reviews, past evaluations, successful credentialing from Healthy Families America, its well developed quality assurance program, and its long term implementation, the Healthy Families Arizona program is an ideal program to implement the type of rigorous study called for by the Caliber Associates report. For example, readiness to learn is emerging as a key variable in positive youth development. The Healthy Families Arizona program would be ideal for the exploration of the contribution to learning readiness by home visitation programs. Evaluations of other Healthy Families programs that have used more rigorous designs have been in states (Hawaii and Alaska) with very different cultural and geographic characteristics making generalization difficult. Arizona has a large Hispanic population, the fastest growing ethnic group in the United States, and is an ideal setting for a rigorous evaluation. The Healthy Families Arizona program shows strong readiness for a longitudinal study and great potential for utilizing the results for program improvement, policy development and knowledge for the field. Further examination of the supervisory relationship would yield important information to guide training of home visitors, and a deeper understanding of the impact of the home visitor behaviors on positive outcomes for families. Increased attention has been paid to home visitor qualities and qualifications. A recent study (Korfmacher, et al., 1999) found higher rates of program retention for nurse home visitors than for non-college degreed paraprofessional home visitors. However, as McGuigan et al. (2003) noted, it remains unclear how home visitors with other educational degrees (Masters and Bachelors), might influence retention rates. Healthy Families Arizona is 39 Healthy Families Evaluation Report 2003 an excellent site to contribute to this knowledge because it has about 50% of its workers with college degrees, and 50% without degrees. As Duggan et al. (2000) notes, supervision of the home visitor has been absent from most studies and could be an important factor (Wallach & Lister, 1995). A recent study (McGuigan et al., 2003) examined retention and found that the amount of time of supervision predicted client retention. A deeper examination of the qualities, approach and challenges of the supervisory relationship would complement the evaluation information gained during the 2003 evaluation regarding problematic and challenging situations faced by home visitors. 40 Healthy Families Evaluation Report 2003 References Abdin, R. L. (1995). The parenting stress index. Odessa, FL: Psychological Assessment Resources. Caliber Associates (2003). Emerging Practices In the Prevention of Child Abuse. Washington, DC: Author. Duggan, A., Windham, A., McFarlane, E., Fuddy, L., Rohde, C., Buchbinder, S., & Sia, C. (2000). Hawaii’s Healthy Start Program of home visiting for atrisk families: Evaluation of family identification, family engagement, and service delivery. Pediatrics, 105 (1), 250-259. Goldfried M. R., & d'Zurilla, T.J. (1969). A behavioral-analytic model for assessing competence. In: Current Topics in Clinical and Community Psychology, Vol 1, Spielberger, C., ed. New York: Academic Press, pp 151-196. Gomby, D. (1999). Home visiting: Recent program evaluations—analysis and recommendations. The Future of Children, 9 (2), 4-26. Gutterman, N. (2001). Stopping child maltreatment before it starts. Thousand Oaks, CA: SAGE Publications. Hart, L. (1994). On the wings of self-esteem. Berkeley, CA: Celestial Arts. Kamerman, S., & Kahn, A. (1995). Starting right: How America neglects its youngest children and what we can do about it. New York: Oxford University Press. Korfmacher, J., O’Brien, R., Hiatt, S., & Olds, D. (1999). Differences in program implementation between nurses and paraprofessionals providing home visits during pregnancy and infancy: A randomized trial. American Journal of Public Health, 89, 1847-1851. 41 Healthy Families Evaluation Report 2003 LeCroy & Milligan Associates, Inc. (2002). Healthy Families Arizona Evaluation Report 2002. Tucson, AZ: LeCroy & Milligan Associates, Inc. McGuigan, W.M., Katzev, A.R., & Pratt, C.C. (2003). Multi-level determinants of retention in a home-visiting child abuse prevention program. Child Abuse & Neglect, 27, 363-380. Mischel, W. (1968). Personality and assessment. New York: Wiley & Sons. 42 Healthy Families Evaluation Report 2003 Appendix A Site Level Data l Age of Child at Entry l Days to Termination l Reason for Termination l Health Insurance at Intake l Late or No Prenatal Care or Poor Compliance at Intake l Ethnicity of Mother l Gestational Age l Low Birth Weight l Yearly Income l Family Stress Checklist Score 43 Healthy Families Evaluation Report 2003 Age of Child at Entry by Site (Age in days) Mean (Age in Days) Standard Deviation Number Douglas/Bisbee 15.74 15.92 93 Central Phoenix 28.48 22.78 73 Maryvale (Phoenix) 19.54 13.98 92 South Phoenix 23.46 22.74 76 East Valley (Phoenix) 22.26 22.33 68 Nogales 13.75 17.13 112 Page 19.89 19.46 45 Casa de los Niños (Tucson) 21.06 18.25 138 CODAC (Tucson) 18.91 18.88 90 La Frontera (Tucson) 18.86 19.85 145 La Hacienda (Tucson) 20.76 23.22 59 Sierra Vista 12.72 14.78 75 Tuba City 14.66 18.38 53 Verde Valley 10.46 12.54 70 Yuma 18.89 16.77 93 Pascua Yaqui 38.88 28.10 40 Lake Havasu City 24.29 17.59 120 Flagstaff 17.28 21.22 78 Sunnyslope (Phoenix) 25.87 20.42 91 Prescott 20.99 19.06 145 Casa Grande 17.91 22.33 85 Mesa 19.55 14.88 97 Southeast Phoenix 20.97 17.56 91 Total 19.91 19.42 2029 Site 44 Healthy Families Evaluation Report 2003 Days to Termination by Site (For terminated clients) Mean (Days to termination) Standard Deviation Number Douglas/Bisbee 1266.65 704.08 20 Central Phoenix 896.33 512.21 24 Maryvale (Phoenix) 815.43 523.38 35 South Phoenix 675.92 564.92 28 East Valley (Phoenix) 592.53 388.07 19 Nogales 976.32 772.55 31 Page 956.47 769.06 17 Casa de los Niños (Tucson) 540.79 390.05 47 CODAC (Tucson) 696.42 475.19 19 La Frontera (Tucson) 768.53 497.36 47 La Hacienda (Tucson) 894.14 389.16 22 Sierra Vista 552.65 505.96 31 Tuba City 870.83 598.06 23 Verde Valley 469.00 549.57 17 Yuma 640.88 570.26 32 Pascua Yaqui 833.20 808.61 5 Lake Havasu City 772.57 612.36 54 Flagstaff 591.56 451.78 16 Sunnyslope (Phoenix) 748.64 405.16 33 Prescott 362.19 187.91 42 Casa Grande 356.59 295.86 22 Mesa 667.00 428.71 23 Southeast Phoenix 494.91 254.69 32 Total 698.69 541.65 639 Site 45 Healthy Families Evaluation Report 2003 Reason for Termination by Site (Number and Percent within Site) Moved Away Unable to contact Completed Program Douglas/Bisbee 45.0% (9) 15.0% (3) 35.0% (7) Central Phoenix 20.8% (5) 25% (6) 12.5% (3) Maryvale (Phoenix) 28.1% (9) 31.3% (10) 9.4% (3) South Phoenix 24.0% (6) 48.0% (12) 12.0% (3) East Valley (Phoenix) 33.3% (6) 44.4% (8) 5.6% (1) Nogales 25.8% (8) 12.9% (4) 41.9% (13) Page 18.8% (3) 25.0% (4) 18.8% (3) Casa de los Niños (Tucson) 8.5% (4) 46.8% (22) 4.3% (2) CODAC (Tucson) 26.3% (5) 31.6% (6) 15.8% (3) La Frontera (Tucson) 19.6% (9) 34.8% (16) 21.7% (10) La Hacienda (Tucson) 31.8% (7) 18.2% (4) 18.2% (4) Sierra Vista 48.3% (14) 6.9% (2) 13.8% (4) Tuba City 22.7% (5) 18.2% (4) 22.7% (5) Verde Valley 41.2% (7) 29.4% (9) 11.8% (2) Yuma 34.4% (11) 18.8% (6) 21.9% (7) Pascua Yaqui 60.0% (3) 0 20.0% (1) Lake Havasu City 37.0% (20) 14.8% (8) 16.7% (9) Flagstaff 71.4% (10) 0 0 Sunnyslope (Phoenix) 24.2% (8) 21.2% (7) 6.1% (2) Prescott 35.7% (15) 31.0% (13) 0 Casa Grande 54.5% (12) 27.3% (6) 0 Mesa 19.0% (4) 47.6% (10) 19.0% (4) Southeast Phoenix 16.1% (5) 45.2% (14) 0 29.7% (185) 27.3% (170) 13.8% (86) Site Total 46 Healthy Families Evaluation Report 2003 Health Insurance by Site at Intake (Number and Percent within Site) Site None AHCCCS Private Douglas/Bisbee 2.9% (3) 92.2% (95) 4.9% (5) Central Phoenix 4.5% (4) 83% (73) 11.4% (10) Maryvale (Phoenix) 3.7% (3) 84.1% (69) 9.8% (8) 0 64% (87.7) 9.6% (7) 4.6% (3) 80% (52) 15.4% (10) 15.2% (16) 81% (85) 1% (1) Page 6.1% (3) 91.8% (45) 2% (1) Casa de los Niños (Tucson) 1.7% (2) 74.8% (89) 18.5% (22) CODAC (Tucson) 2.9% (3) 83.3% (85) 11.8% (12) La Frontera (Tucson) 5.4% (7) 82.3% (107) 10.8% (14) Devereux (Tucson) 1.7% (2) 78.6% (92) 13.7% (16) Sierra Vista 2.1% (2) 76.8% (73) 16.8% (16) Tuba City 25% (14) 71.4% (40) 3.6% (2) 0 94.7% (71) 5.3% (4) 7.5% (7) 86% (80) 2.2% (2) 0 76.9% (30) 7.7% (3) Lake Havasu City 3.3% (4) 85% (102) 10.8% (13) Flagstaff 6.8% (5) 89.2% (66) 4.1% (3) Sunnyslope (Phoenix) 6.3% (6) 75.8% (72) 16.8% (16) 5% (6) 79.8% (95) 7.6% (9) Casa Grande 2.1% (2) 83% (78) 14.9% (14) Mesa 4.3% (4) 75.3% (70) 16.1% (15) Southeast Phoenix 4.8% (5) 83.7% (87) 11.5% (12) 4.8% (101) 82.3% (1720) 10.3% (215) South Phoenix East Valley (Phoenix) Nogales Verde Valley Yuma Pascua Yaqui Prescott Total 47 Healthy Families Evaluation Report 2003 Late or No Prenatal Care or Poor Compliance at Intake by Site (Number and Percent within Site) The participant received no or late prenatal care or showed poor compliance with prenatal care Site True False Unknown Douglas/Bisbee 40.9% (38) 54.8% (51) 4.3% (4) Central Phoenix 26.4% (19) 72.2% (52) 1.4% (1) Maryvale (Phoenix) 34.4% (32) 62.4% (58) 3.2% (3) South Phoenix 32.9% (24) 67.1% (51) 0 East Valley (Phoenix) 35.8% (24) 58.2% (39) 6.0% (4) Nogales 62.8% (71) 31.9% (36) 5.3% (6) Page 38.6% (17) 61.4% (27) 0 Casa de los Niños (Tucson) 27.9% (39) 63.6% (89) 8.6% (12) CODAC (Tucson) 32.2% (29) 62.2% (56) 5.6% (5) La Frontera (Tucson) 32.9% (48) 62.3% (91) 4.8% (7) La Hacienda (Tucson) 41.4% (24) 55.2% (32) 3.4% (2) Sierra Vista 36.5% (27) 62.2% (91) 1.4% (1) Tuba City 38.9% (21) 59.3% (32) 1.9% (1) Verde Valley 55.1% (38) 44.9% (31) 0 Yuma 45.7% (42) 54.3% (50) 0 Pascua Yaqui 20.0% (8) 77.5% (31) 2.5% (1) Lake Havasu City 40.0% (48) 60.0% (72) 0 Flagstaff 34.6% (27) 65.4% (51) 0 Sunnyslope (Phoenix) 41.8% (38) 57.1% (52) 1.1% (1) Prescott 36.8% (53) 59.0% (85) 4.2% (6) Casa Grande 39.5% (34) 60.5% (52) 0 Mesa 28.9% (28) 68.0% (66) 3.1% (3) Southeast Phoenix 35.2% (32) 64.8% (59) 0 Total 37.6% (762) 59.6% (1209) 2.8% (57) 48 Healthy Families Evaluation Report 2003 Ethnicity of Mother by Site (Number and Percent within Site) Site African Asian American American Native American Caucasian Hispanic Douglas/Bisbee 12.9% (12) 87.1% (81) 0 0 0 0 Central Phoenix 27.4% (20) 45.2% (33) 16.4% (12) 1.4% (1) 2.7% (2) 6.8% (5) Maryvale (Phoenix) 25.0% (23) 65.2% (60) 5.4% (5) 0 1.1% (1) 3.3% (3) South Phoenix 21.1% (16) 57.9% (44) 17.1% (13) 1.3% (1) 2.6% (2) 0 East Valley (Phoenix) 39.7% (27) 41.2% (28) 13.2% (9) 0 0 5.9% (4) 0 100% (113) 0 0 0 0 4.4% (2) 2.2% (1) 0 0 91.1% (41) 2.2% (1) Casa de los Niños (Tucson) 26.1% (36) 60.1% (83) 5.1% (7) 0.7% (1) 4.3% (6) 3.6% (5) CODAC (Tucson) 10.0% (9) 82.2% (74) 3.3% (3) 0 0 4.4% (4) La Frontera (Tucson) 13.7% (20) 76.7% (112) 4.1% (6) 0.7% (1) 3.4% (5) 1.4% (2) La Hacienda (Tucson) 13.6% (8) 81.4% (48) 1.7% (1) 0 1.7% (1) 1.7% (1) Nogales Page Other 49 Healthy Families Evaluation Report 2003 Sierra Vista 45.3% (34) 41.3% (31) 6.7% (5) 0 0 6.7% (5) 0 0 0 0 100% (54) 0 61.4% (43) 34.3% (24) 0 0 4.3% (3) 0 6.6% (6) 92.3% (84) 1.1% (1) 0 0 0 0 12.5% (5) 0 0 57.5% (23) 30.0% (12) Lake Havasu City 51.7% (62) 45.0% (54) 0 0 0.8% (1) 2.5% (3) Flagstaff 18.8% (15) 41.3% (33) 1.3% (1) 0 38.8% (31) 0 Sunnyslope (Phoenix) 35.9% (33) 47.8% (44) 10.9% (10) 0 2.2% (2) 3.3% (3) 69.0% (100) 29.7% (43) 0 0.7% (1) 0.7% (1) 0 Casa Grande 27.9% (24) 55.8% (48) 8.1% (7) 0 3.5% (3) 4.7% (4) Mesa 51.0% (50) 31.6% (31) 6.1% (6) 1.0% (1) 6.1% (6) 4.1% (4) Southeast Phoenix 20.9% (19) 57.1% (52) 15.4% (14) 0 1.1% (1) 5.5% (5) 27.5% (559) 55.3% (1126) 4.9% (100) 0.3% (6) 9.0% (183) 3.6% (61) Tuba City Verde Valley Yuma Pascua Yaqui Prescott Total 50 Healthy Families Evaluation Report 2003 Gestational Age by Site (Number and Percent within Site) Was the gestational age less than 37 weeks? Site No Yes Douglas/Bisbee 90.1% (82) 9.9% (9) Central Phoenix 65.7% (44) 34.3% (23) Maryvale (Phoenix) 75.0% (63) 25.0% (21) South Phoenix 89.7% (61) 10.3% (7) East Valley (Phoenix) 81.8% (54) 18.2% (12) Nogales 91.1% (102) 8.9% (10) Page 88.6% (39) 11.4% (5) Casa de los Niños (Tucson) 82.3% (102) 17.7% (22) CODAC (Tucson) 88.1% (74) 11.9% (10) La Frontera (Tucson) 78.4% (105) 21.6% (29) La Hacienda (Tucson) 85.2% (46) 14.8% (8) Sierra Vista 95.4% (62) 4.6% (3) Tuba City 85.1% (40) 14.9% (7) Verde Valley 87.1% (61) 12.9% (9) Yuma 91.1% (82) 8.9% (8) Pascua Yaqui 97.2% (35) 2.8% (1) Lake Havasu City 82.8% (82) 17.2% (17) Flagstaff 85.1% (63) 14.9% (11) Sunnyslope (Phoenix) 80.5% (70) 19.5% (17) Prescott 91.5% (129) 8.5% (12) Casa Grande 88.0 % (73) 12.0% (10) Mesa 73.7% (70) 26.3% (25) Southeast Phoenix 82.9% (68) 17.1% (14) 84.7% (1607) 15.3% (290) Total 51 Healthy Families Evaluation Report 2003 Low Birth Weight by Site (Number and Percent within Site) Did the child have low birth weight (less than 2500 grams or 88 ounces)? Site No Yes Douglas/Bisbee 81.7% (76) 18.3% (17) Central Phoenix 70.8% (51) 29.2% (21) Maryvale (Phoenix) 76.3% (71) 23.7% (22) South Phoenix 86.8% (66) 13.2% (10) East Valley (Phoenix) 77.9% (53) 22.1% (15) Nogales 89.4% (101) 10.6% (12) Page 93.3% (42) 6.7% (3) Casa de los Niños (Tucson) 87.1% (122) 12.9% (18) CODAC (Tucson) 86.8% (79) 13.2% (12) La Frontera (Tucson) 82.8% (120) 17.2 (25) La Hacienda (Tucson) 93.2% (55) 6.8% (4) Sierra Vista 90.7% (68) 9.3% (7) Tuba City 86.8% (46) 13.2% (7) Verde Valley 89.9% (62) 10.1% (7) Yuma 93.5% (86) 6.5% (6) Pascua Yaqui 97.6% (40) 2.4% (1) Lake Havasu City 84.0% (100) 16.0% (19) Flagstaff 79.7% (63) 20.3% (16) Sunnyslope (Phoenix) 84.8% (78) 15.2% (14) Prescott 95.1% (137) 4.9% (7) Casa Grande 94.2% (81) 5.8% (5) Mesa 73.5% (72) 26.5% (26) Southeast Phoenix 87.9% (80) 12.1% (11) 86.0% (1749) 14.0% (285) Total 52 Healthy Families Evaluation Report 2003 Yearly Income by Site Mean Yearly Income Standard Deviation Number Douglas/Bisbee $9453.89 6426.13 85 Central Phoenix $11237.29 12201.30 52 Maryvale (Phoenix) $11367.06 9158.79 72 South Phoenix $8558.14 9434.05 58 East Valley (Phoenix) $14763.78 20265.16 50 Nogales $10499.85 6831.93 110 Page $7383.18 8971.13 39 Casa de los Niños (Tucson) $11062.21 10707.66 108 CODAC (Tucson) $12732.06 13416.96 69 La Frontera (Tucson) $10425.05 6585.07 114 La Hacienda (Tucson) $11843.39 7076.46 41 Sierra Vista $6824.41 7694.68 68 Tuba City $12822.11 15404.26 46 Verde Valley $9730.98 8138.32 63 Yuma $7071.26 5616.16 81 Pascua Yaqui $8506.76 6625.17 37 Lake Havasu City $14382.88 13605.80 113 Flagstaff $10885.31 11309.17 72 Sunnyslope (Phoenix) $12519.22 19398.12 65 Prescott $15761.51 13548.98 45 Casa Grande $9846.61 8284.20 51 Mesa $10133.74 10504.70 68 Southeast Phoenix $11162.83 11591.96 65 Total $10836.15 11100.17 1572 Site 53 Healthy Families Evaluation Report 2003 Family Stress Checklist Score by Site Mean Score Percent of mothers whose FSC score was greater than 40 Number of mothers whose FSC score was greater than 40 Douglas/Bisbee 39.02 53.8% 50 Central Phoenix 37.88 42.5% 31 Maryvale (Phoenix) 37.85 51.6% 48 South Phoenix 38.75 52.6% 40 East Valley (Phoenix) 37.94 47.1% 32 Nogales 34.03 25.7% 29 Page 35.56 40.0% 18 Casa de los Niños (Tucson) 38.29 53.9% 76 CODAC (Tucson) 35.00 34.1% 31 La Frontera (Tucson) 37.81 46.6% 68 La Hacienda (Tucson) 36.12 35.6% 21 Sierra Vista 39.93 50.7% 38 Tuba City 30.65 11.1% 6 Verde Valley 34.07 30.0% 21 Yuma 37.37 43.0% 40 Pascua Yaqui 34.02 31.7% 13 Lake Havasu City 36.68 40.8% 49 Flagstaff 37.75 46.3% 37 Sunnyslope (Phoenix) 39.24 48.9% 45 Prescott 43.79 62.8% 91 Casa Grande 33.55 33.7% 29 Mesa 36.96 44.9% 44 Southeast Phoenix 35.77 42.9% 39 Total 37.28 43.9% 896 Site 54 Healthy Families Evaluation Report 2003 Appendix B Parenting Stress Index Information Reliabilities for Current Study Subscale Competence Attachment Restricted Role Depression Isolation Distractibility Mood Alpha .70 .60 .74 .78 .73 .50 .67 Change in Parenting Stress Index Subscales Scores from baseline to 6 months Baseline Subscale 6 months Significance Mean SD Mean SD t Competence 31.5 6.2 29.7 6.2 7.17*** Attachment 12.9 3.6 12.4 3.7 3.62*** Restricted role 19.6 5.3 18.9 5.5 3.13* Depression 20.8 6.1 19.7 6.1 4.82*** Isolation 14.5 4.7 13.8 4.8 3.56*** Mood 10.4 3.0 9.1 2.9 9.01*** Note: * p<.01, ** p<.001, *** p<.000, dependent t-tests, SD=Standard Deviation. Test is significant when applying a Bonferroni correction. 55 Healthy Families Evaluation Report 2003 Change in Total Parenting Index Scores from baseline to 6 months Baseline 6 months Significance Subscale Mean SD Mean SD t Total Stress Score 137.0 25.5 128.9 24.9 7.25*** *** p<.000 Change in Parenting Stress Index from baseline to 12 months Baseline Subscale 12 months Significance Mean SD Mean SD t Competence 31.6 6.2 29.3 6.0 7.07*** Attachment 12.8 3.6 12.0 3.4 4.29*** Restricted role 19.5 5.2 18.6 5.2 3.27** Depression 20.9 6.1 19.2 5.9 5.69*** Isolation 14.5 4.5 13.6 4.5 3.97*** Mood 10.5 3.0 9.4 2.9 6.36*** Note: * p<.01, ** p<.001, *** p<.000, dependent t-tests, SD=Standard Deviation. Test is significant when applying a Bonferroni correction. Change in Total Parenting Index Scores from baseline to 12 months. Baseline 12 months Significance Subscale Mean SD Mean SD t Total Stress Score 135.5 23.4 127.3 23.5 7.55*** *** p<.000 56 Healthy Families Evaluation Report 2003 Change in Parenting Stress Index from baseline to 18 months Baseline Subscale 18 months Significance Mean SD Mean SD t Competence 31.6 6.5 29.4 6.4 4.60*** Attachment 12.8 3.8 12.4 3.6 1.26 Restricted role 19.4 5.2 17.9 5.6 4.02*** Depression 20.6 6.5 19.5 6.4 2.78* Isolation 14.7 4.6 13.6 4.7 3.49** Mood 10.3 3.1 9.6 3.0 2.60* Note: * p<.01, ** p<.001, *** p<.000, dependent t-tests, SD=Standard Deviation. Tests are significant when applying a Bonferroni correction. Change in Total Parenting Index Scores from baseline to 18 months. Baseline 18 months Significance Subscale Mean SD Mean SD t Total Stress Score 135.0 24.5 126.8 26.9 4.74*** *** p<.000 57 Healthy Families Evaluation Report 2003 Range and Reliability of the Parenting Stress Index (PSI) (Selected subscales for original reliabilities analysis) Alpha Administration Subscales Rangea Coefficient Sense of Competence Parental Attachment 13 - 65 7 - 35 .77 .64 Administered at 3 weeks, 6 months, and 18 months Role Restriction 7 - 35 .74 Depression 9 - 45 .75 Social Isolation 6 - 30 .69 Mood 5 - 25 .70 Distractibility 9 - 45 .82 Total Scoreb 78-390 .85 a A higher score on each of the subscales represents a higher degree of stress in that area. b The total score on the Parenting Stress Index is computed by summing all of the subscales, with a higher score indicating more stress. 58 Healthy Families Evaluation Report 2003 Description of Parenting Stress Index Subscales Sense of Competence Subscale: Assesses the parent’s sense of competence in relation to his or her role as parent. It relates to knowledge of how to manage the child’s behavior and comfort in making decisions such as when and how to discipline the child. Parental Attachment Subscale: Assesses the intrinsic investment the parent has in the role of parent. This subscale was expected to determine the parent’s motivation level to fulfill the role of parent. Restrictive Role Subscale: Assesses the negative impact, losses, and sense of resentment associated with the parent’s perceptions of loss of important life roles. Depression Subscale: Assesses the extent to which the parent’s emotional availability to the child is impaired and the extent to which the parent’s emotional and physical energy is compromised. Isolation Subscale: Examines the parent’s social isolation and the availability of social support for the role of parent. Distractibility Subscale: Assesses the degree to which the child displays many of the behaviors associated with Attention Deficit Disorder with Hyperactivity and other behaviors which result in a continuous drain on the parents’ energy, which requires not only active parental management but also sustained high states of vigilance. Mood Subscale: Assesses child characteristics related to excessive crying, withdrawal, and depression. The parent usually experiences these behaviors as anxiety or anger provoking. 59 Healthy Families Evaluation Report 2003 Appendix C Family Stress Checklist Family Stress Checklist Problem Areas and Interpretation (Mother & Father) Problem Areas I. Childhood history of physical abuse and deprivation. II. Substance abuse, mental illness, or criminal history. III. Previous or current CPS involvement. IV. Self-esteem, available lifelines, possible depression. V. Stresses, concerns. Range Interpretation/ Administration The FSC is a 10 item rating scale. 0, 5, or 10 A score of 0 represents normal, 5 represents a mild degree of the problem, and a 10 represents severe, on both the Mother and 0, 5, or 10 Father Family Stress Checklist items. The FSC is an assessment tool and is administered to the 0, 5, or 10 mother through an interview by a Family Assessment Worker from 0, 5, or 10 the Healthy Families Arizona Program. The interview takes 0, 5, or 10 place shortly after birth, or as near VI. Potential for violence. 0, 5, or 10 to that time as possible. VII. Expectations of infants’ milestones, behaviors. 0, 5, or 10 VIII. Discipline of infant, toddler, and child. 0, 5, or 10 IX. Perception of new infant. X. Bonding, attachment issues. 0, 5, or 10 0, 5, or 10 A score over 25 is considered medium risk for child abuse and Total Score 0 - 100 neglect, and a score over 40 is considered high-risk for child abuse. 60 Healthy Families Evaluation Report 2003