Healthy Families Arizona Evaluation Report 2004 November 2004 Prepared by: LeCroy & Milligan Associates, Inc. 620 N. Country Club Tucson, AZ 85716 (520) 326-5154 FAX (520) 326-5155 Prepared for: The Arizona Department of Economic Security Division of Children, Youth & Families Office of Prevention and Family Support 1789 W. Jefferson, Site Code 940A Phoenix, Arizona 85007 Acknowledgements This evaluation report represents the efforts of many individuals and many collaborating agencies. The evaluation team for Healthy Families Arizona includes Craig W. LeCroy, PhD, Pat Beauchamp, LCSW, Kerry Milligan, MSSW, Judy Krysik, Ph.D., evaluators, and Cindy Jones, BS-MIS, Olga Valenzuela, BA, Veronica Urcadez, and Delcia Cardenas, data management staff. Connie Warren formatted the report. We are grateful to Rachel Whyte, LCSW, Coordinator for the Child Abuse Prevention Fund and Healthy Families Arizona for her guidance and support. Valerie Roberson continues to provide leadership and vision for the program. Kate Whitaker, Pauline Haas-Vaughn, and Kathy Van Meter provide quality assurance and training, and helped the sites to collect valid information. Penny Swenson provided administrative assistance for quality assurance for all the sites. Thank you to the program managers who have spent their time collecting and sending data. Staff at the sites have dutifully collected the data and answered endless questions to increase the accuracy of the data. Lastly, we acknowledge the families who have received Healthy Families Arizona services. Suggested Citation: LeCroy & Milligan Associates, Inc. (2004). Healthy Families Arizona Evaluation Report 2004. Tucson, AZ: LeCroy & Milligan Associates, Inc. ii Healthy Families Evaluation Report 2004 Healthy Families Arizona 2004 Evaluation Report Highlights Program Participants 70% of mothers were single 87% of families were on AHCCCS 14% of infants were born <37 weeks gestation (state average 10%) 13% of infants had low birth weight (state average 7%) Service Delivery 2,356 families were served 91% (2,137) engaged with the program (4 or more home visits) 63% remained in the program 1 year or longer Average length of time in program grew to 804 days (698 in 2002) Outcomes 98.4% of families did not have a substantiated CPS match (comparison group 98.6%) Average Parenting Stress Index score improved 62% of parents’ total stress score improved Immunization rate for infants was 94% (state average 77%) 97% of children were linked to a medical doctor 15% of mothers had subsequent pregnancies, with 30% of those 18 or younger. Recommendations Examine and develop more best practices protocols for addressing substance abuse, domestic violence and mental illness Continue to emphasize careful planning of subsequent pregnancies, especially with young mothers The substance abuse screen should be re-evaluated and replaced and more training should occur in how to discuss substance use and abuse Re-emphasize the use of the home safety checklist Ages and Stages Questionnaire needs to be consistently administered The criteria for “successful completion” of the program should be reexamined Expand evaluation to include examination of implementation of prenatal services and development of relevant measures for these families Future Directions Newly developed Healthy Parenting Inventory being implemented Longitudinal study underway iii Healthy Families Evaluation Report 2004 Table of Contents Acknowledgements ......................................................................................................... ii Healthy Families Arizona 2004 Evaluation Report Highlights................................iii List of Exhibits and Tables.............................................................................................. v Executive Summary........................................................................................................ vi Introduction ...................................................................................................................... 1 In this Report .................................................................................................................... 2 Program and Policy Updates ......................................................................................... 3 The Evaluation Review ................................................................................................... 5 Overview of the Healthy Families Arizona Program............................................... 11 Program Participants..................................................................................................... 11 Service Delivery ............................................................................................................. 15 Program Outcomes ........................................................................................................ 21 Conclusions and Recommendations ........................................................................... 33 References ....................................................................................................................... 37 Appendix A..................................................................................................................... 38 Appendix B ..................................................................................................................... 50 Appendix C..................................................................................................................... 54 Appendix D..................................................................................................................... 55 iv Healthy Families Evaluation Report 2004 List of Exhibits Exhibit 1: Developments in the Healthy Families Arizona program in 2003-04 ..............3 Exhibit 2: Healthy Families Parenting Inventory Theoretical Model................................9 Exhibit 3: Selected Risk Factors For Healthy Families Arizona Mothers At Intake. ......12 Exhibit 4: Ethnicity Of Healthy Families Arizona Mothers. ..............................................13 Exhibit 5: Percentage Of Parents Rated Severe On The Family Stress Checklist............14 Exhibit 6: Percentage Of Infants With High-Risk Characteristics.....................................14 Exhibit 7: Healthy Families Arizona Participants Served By Site.....................................16 Exhibit 8: Average Length Of Time In Program At Termination For Engaged Families..............................................................................................................................18 Exhibit 9: Reasons For Program Termination—Engaged Families. .................................19 Exhibit 10: Healthy Family Arizona Service Patterns.........................................................20 Exhibit 11: Percent Of Child Abuse And Neglect Incidences In Program And Comparison Groups. .......................................................................................................22 Exhibit 12. Substantiated Reports Trend. ............................................................................23 Exhibit 13: Parenting Stress Index Findings.........................................................................25 Exhibit 14: Percent Of Safety Practices Implemented.........................................................26 Exhibit 15: Immunization Rate of Healthy Families Arizona Children...........................27 Exhibit 16: Percentage of Children Linked to a Medical Doctor.......................................28 Exhibit 17: ASQ Screening ......................................................................................................29 Exhibit 18: ASQ Referral Status. ............................................................................................30 Exhibit 19: Percentage Of Families Who Screened Positive For Alcohol And Drug Problems............................................................................................................................31 Exhibit 20: Length Of Time To Subsequent Pregnancy......................................................32 Exhibit 21. Mother’s Employment Status. ...........................................................................32 v Healthy Families Evaluation Report 2004 Executive Summary "In the sheltered simplicity of the first days after a baby is born, one sees again the magical closed circle, the miraculous sense of two people existing only for each other, the tranquil sky reflected on the face of the mother nursing her child." Anne Morrow Lindbergh in Gift of the Sea "Healthy parenting is especially challenging when our own childhoods weren't healthy. It requires energy, attention, and constant restraint. Realize that you need healing. Take time out to nurture yourself." Vimala McClure in The Tao of Mothering The two quotes above capture in many ways the challenge and the hope of Healthy Families Arizona. The first talks of the special joy found in the relationship between the mother and the child. The second reflects the challenge faced by many of the program families, and the need to find the strength within themselves to overcome their own history. As much as parents seek to guide their children as they attempt to grow into healthy, productive adults, society on occasion has the task to help support the parents in that endeavor. Healthy Families Arizona is a program designed to provide that support, and through its efforts reduce the incidence of child abuse and neglect, provide stability for at-risk families, and grow a new generation of healthy families in the state. The Healthy Families Arizona Program Healthy Families Arizona is a home visitation program designed to provide supportive services and education to parents of newborns who might benefit from support to strengthen their families at this crucial time. The goals of the program include: To promote positive parent/child interaction To improve child health and development To prevent child abuse and neglect vi Healthy Families Evaluation Report 2004 All services are voluntary and may be provided for up to five years. Families enter the program based on a screening and assessment process. In the hospital after a child’s birth, the family can consent to an initial screening that identifies family, parental, child, and community risk factors associated with child abuse and neglect. If the screening is assessed as positive (indicating potential increased needs), the family is offered intensive home visiting services through the Healthy Families Arizona program. Since the program is voluntary, the family can withdraw from the program at any time. After the family accepts home visitation services, a Family Support Specialist visits the family in their home on a regular basis to provide supportive services and education. The Family Support Specialist seeks to develop a trusting, open, and constructive relationship with the family to meet their individual needs. The core Healthy Families Arizona services are: Emotional support Assistance in developing positive parenting skills Education on child development and nutrition Education and assistance in problem solving and coping skills Education on preventive health care (immunizations, links to medical doctor) Linkages to preschool resources Referrals related to education, employment, mental health, and substance abuse services This report focuses on aggregate data that are summarized across the 23 sites that make up the Healthy Families Arizona program. This report presents the evaluation data for the cohort of participants who received services in the Healthy Families Arizona program between the period of July 1, 2003 and June 30, 2004. This includes all families who received services at any time during the study period regardless of when they entered the program. In this year’s report, more extensive site level data can be obtained in the Appendices. vii Healthy Families Evaluation Report 2004 The Families Healthy Families Arizona served an ethnically diverse Asian Native American group of program American African 0.5% 9.4% Other American participants who displayed 2.8% 4.8% one or more of the risk factors associated with potential for child neglect or Caucasian abuse. Prominent among 26.5% Hispanic these were single 56.0% motherhood, mothers Mother’s Ethnicity without a high school education, and mothers with late, inadequate, or no prenatal care. A high number of parents scored “severe” on several of the Parent Survey scales, including coping with a history of child abuse, having low self-esteem and feeling isolated, and dealing with current life stress, including low income, poor housing, and relationship difficulties. Finally, a larger than average number of infants were born early (less than 37 weeks gestation) and with low birth weight—risk factors associated with potential child abuse and neglect. Because Healthy Families Arizona has just begun to deliver prenatal services (as of fall 2004) as part of the program’s continuum of services, there is an opportunity to potentially impact the number of premature births and low birth weight babies. Risk factors for mothers at intake Rate Teen Births (<19) 22.7% Births to single parents 69.9% Less than high school education 61.2% Not employed 83.7% No health insurance 2.8% Late or no prenatal care 35.8% Median yearly income $7,540 viii Healthy Families Evaluation Report 2004 Service Delivery During the study year, 2,356 families were served and 2,137 remained with the program for four or more visits (referred to as “engaged families”). Families who did not engage in the program shared some common characteristics, including single marital status, a history of depression, and a history of criminal behavior. The majority of the engaged families stayed with the Up to 3 3-6 months months 7-12 program for at least one 9% 4% months 24% year, with 31% of families receiving services for three or more years. The average time between enrollment and termination increased 1 year and from 2003 to 2004 to over beyond 63% two years (804 days). Time in the Program Program Outcomes The Healthy Families Arizona program exceeded the goal of fewer than five percent of the active families with a substantiated case of child abuse or neglect with an overall rate of 98.4 percent with no substantiated reports. This percent is similar when compared to a group of families eligible for the program but who received less than four home visits. Percent Without Substantiated Report Group Healthy Families Arizona Participants 98.4% (n=1568) Comparison Group Participants 98.6% (n=213) The participant families showed significant improvement on all reliable subscales of the Parenting Stress Index. Families demonstrated increased competence and attachments, and decreased feelings of depression, isolation, negative mood, and feelings of restricted role as a parent. (p<.05). Parents ix Healthy Families Evaluation Report 2004 demonstrated greater use of safety practices as the child grew older and they spent more time in the program. However, three areas could use additional attention—the need to have electrical outlets covered, keeping poisons locked away, and having working smoke alarms installed. On health-related measures, the infants of participating families were immunized at a rate that exceeds the state standard and were linked to a doctor at a high rate. Overall, the percent of Healthy Families children who were immunized rose 6% from the previous year, indicating a strong and consistent effort by home visitors to support families in this area. Immunization period Percent immunized 2 month 96.7% 4 month 94.3% 6 month 87.1% 12 month 95.9% Received all 4 in the series 94.0% Immunization rate for 2-year-olds in Arizona 77.0% For this study period, an increased percentage of infants were screened using the Ages and Stages Questionnaire, and those identified with developmental delays were appropriately referred. In regard to maternal life course outcomes, 15 percent of the mothers had subsequent pregnancies, with 30 percent of those 18 years or younger. In addition, during the period of time they were involved in the program, a progressively higher percentage of mothers became employed. Conclusions and Recommendations This year’s study indicates that the Healthy Families Arizona program continues to meet the needs of many of the state’s at-risk families in an effective manner. The program is recruiting the target population, providing services in a manner that results in a 90 percent engagement rate, and continuing to show positive results in multiple indicators. Recommendations that result from this evaluation include: x Healthy Families Evaluation Report 2004 The program should examine and develop more best practices protocols for appropriately addressing substance abuse, domestic violence and mental illness The program should continue to emphasize careful planning of subsequent pregnancies, especially with young mothers. The substance abuse screen should be re-evaluated and replaced and more training in how to discuss substance use and abuse is needed The use of the safety checklist to identify and help families implement safety practices in their homes should be re-emphasized. More emphasis needs to be placed on administering the Ages and Stages Questionnaire consistently through the program. The criteria for “successful completion” of the program should be reexamined. With the implementation of prenatal services, the evaluation will expand to include an examination of the implementation of prenatal services and development of relevant measures for these families. xi Healthy Families Evaluation Report 2004 Introduction “In the sheltered simplicity of the first days after a baby is born, one sees again the magical closed circle, the miraculous sense of two people existing only for each other, the tranquil sky reflected on the face of the mother nursing her child.” Anne Morrow Lindbergh in Gift of the Sea “Healthy parenting is especially challenging when our own childhoods weren't healthy. It requires energy, attention, and constant restraint. Realize that you need healing. Take time out to nurture yourself.” Vimala McClure in The Tao of Mothering The two quotes above capture in many ways the hope and the challenge of Healthy Families Arizona. The first talks of the special joy found in the relationship between the mother and the child. The second reflects the challenge faced by many of the program families, and the need to find the strength within themselves to overcome their own history. As much as parents seek to guide their children as they attempt to grow into healthy, productive adults, society on occasion has the task to help support the parents in that endeavor. Healthy Families Arizona is a program designed to provide that support, and through its efforts reduce the incidence of child abuse and neglect, provide stability for at-risk families, and grow a new generation of healthy families in the state. In this Report Home visitation programs continue to be refined as research and experience provide insights into effective practices and critical concerns. As the Healthy Families Arizona program matures, the program’s policies, practice, and evaluation have been refined to address emerging issues and needs. Over the past several years, the evaluation reports focused on an annual review of selected implementation issues, services provided and participant outcomes, a review of the previous studies and research, an examination of the program’s adoption of commonly accepted practice principles, and a review of the program’s adherence to the Healthy Families America Critical Elements. In collaboration with Healthy Families Arizona Steering Committee members, and the quality assurance and training team and program staff, the evaluation team has examined the overall evaluation approach. A number of recommendations are in the process of being implemented. The first portion of this year’s report will provide an overview of changes in policy, program delivery, and evaluation. Subsequent sections will review the Fiscal Year 2004 data in comparison to data reported last year in order to examine the trends that may be found in the data over time. Family demographic data are reviewed in order to assess whether the program is reaching the families for which it was designed. Service delivery is discussed in regard to family engagement and retention, and participant satisfaction. Finally, selected outcome data is reviewed. 1 This report focuses on aggregate data that is summarized across the 23 sites that made up the Healthy Families Arizona program in FY 04. This report presents the evaluation data for the cohort of participants who received services in the Healthy Families Arizona program between the period of July 1, 2003 and June 30, 2004. This includes all families who received services at any time during the study period regardless of when they entered the program. In this year’s report, more extensive site level data can be obtained in the Appendices. 1 2 Healthy Families Evaluation Report 2004 Program and Policy Updates Exhibit 1 depicts some of the program and policy changes that have occurred in the past year. Exhibit 1: Developments in the Healthy Families Arizona program in 2003-04 Healthy Families Statewide Credential received The Re-accreditation process was completed and Healthy Families Arizona received a Statewide Credential (November 2004), with no need for a deferment period (i.e., all standards were met successfully without correction needed). In early 2004, the entire state system completed a self-assessment, and 60% of the sites received a site visit. The HFAz evaluation system was lauded by the peers and the entire state system was called “a gem” of a program by objective peer reviewers who are part of other state systems. Implementation of the Social/Emotional Ages and Stages Questionnaire An ongoing goal of home visitors is to find ways to effectively initiate conversations with parents about child development and to identify referral needs. In April 2003, the HFAz program implemented use of the Social/Emotional scales of Ages and Stages Questionnaire (ASQ-SE) to assess social and emotional development among children and identify needs for referrals. Training in the Ages and Stages Questionnaire is required within 6 months to one year of hire of new Family Support Specialists. HFAz program staff have found it helpful in identifying needs for infant mental health services, but report a lack of available services to meet the needs. Arizona is the only statewide program utilizing the ASQ-SE in a systematic way. Revision of the Individual Family Service Plan The Individual Family Service Plan (IFSP “Goal Plan”) was revised to increase its utility as a method to address challenging family issues. The goals will have more depth and be based around the family’s regular routines—so that incorporating these goals into the family’s life will be more easily accomplished. An additional training for the IFSP process was developed and will become a regular part of Healthy Families training services along with the ASQ and ASQSE. State Policy/procedures updated The policies and procedures manual was revised to give sites a clearer understanding of statewide policy. 3 Healthy Families Evaluation Report 2004 Technical Assistance/Quality Assurance (TA/QA) process revised The Technical Assistance/Quality Assurance (TA/QA) site visit system was revised to offer a “menu” type approach to training. Two different types of site visits are designed to better meet site needs—one is a “credentialing site visit” in which standards of best practice are reviewed once per year; the other is a “technical assistance” site visit in which TA/QA staff will observe practice and respond to the sites’ needs and issues. This emphasis on effective practice enables the TA/QA staff to provide training and mentoring on the specific difficult situations and challenges encountered by home visitors, a need that was illuminated in last year’s evaluation report. Prenatal services to begin Beginning in August, 2004, program policy supports provision of services to families during the prenatal period. Sites are working with their “Baby Arizona” doctors, prenatal health clinics, Women Infants and Children program (WIC), County health departments, Health Start programs, and local OB/GYNs to develop systematic ways for reaching out to families during the prenatal period. Prenatal policies, procedures, and appropriate forms were developed and implemented statewide. Healthy Families TA/QA staff members have been trained as certified Healthy Families America Prenatal Core trainers and were able to train all of the “seasoned staff” across the state during 2004. A training schedule has been established to train all new staff. Healthy Families eligibility policy revised Program policy was changed as of August 2004 so that HFAz can serve families who have been involved with child protective services. The appropriate policies and protocols have been developed with the Department of Economic Security to determine how to provide services in a voluntary way and work closely at the local level to support healthy parenting. 4 Healthy Families Evaluation Report 2004 The Evaluation Review One of the particular strengths of Healthy Families Arizona is the collaborative nature of all elements of the program. The Department of Economic Security staff, the program providers, the Technical Assistance/Quality Assurance (TA/QA) staff and the evaluation staff have a close, on-going relationship that enables open examination of the program when needed, and allows change to take place in an efficient manner. In the last year, evaluation activities have focused on further explicating the program theory through logic model development, research, and development of new measures to more accurately capture outcomes of interest, and beginning preparation for conducting a longitudinal study. Logic Model Development Accurately measuring the true impact of the Arizona Healthy Families program is an important goal that the evaluation team has worked on since the program’s inception. During this year, the evaluation team led the Evaluation Subcommittee (comprised of several providers, TA/QA staff, and others) in reexamining the theory and logic forming the basis of the Healthy Families program. The first step was clarifying outcomes for the program aligned along five major goals. These were: Decreased Child Abuse and Neglect Increased Family Economic Self-Sufficiency Increased Family Social Competence Increased Positive Parent/Child Relationship Optimized Child Development These goals form the basis for further development of a program logic model, which in turn will help the program further refine the outcomes of interest for future evaluations. The evaluation team will also be assisting the program staff in developing a logic model specific to the prenatal program. That logic model will be used to select outcomes for the prenatal portion of the program, as well as choose an appropriate measurement tool for those outcomes. 5 Healthy Families Evaluation Report 2004 Research and Development of New Tools Two new tools have been under development during this year—the Healthy Families Parenting Inventory (described below), and an “embedded” evaluation tool. The embedded evaluation tool grew out of a desire to 1) provide the Family Support Specialists an improved means of targeting goals and areas for support, and highlighting progress; and 2) track family outcomes related to those goals. Longitudinal Study Plans Finally, LeCroy & Milligan Associates will begin a longitudinal study of the program in late 2004. This study will provide valuable information on the longterm effects of the program, and will be used to further refine the program and the evaluation. Development of the Healthy Families Parenting Inventory Research and development of the new Healthy Families Parenting Inventory was completed during this year. The process used to develop the inventory is outlined below. While various instruments have been used to evaluate home visitation programs like Healthy Families, it is becoming apparent that many of the instruments may not be equipped to capture the changes that take place in families. In the HFAz program various instruments have been used including the Parenting Stress Index and the Child Abuse Potential Inventory. Although these instruments are considered acceptable in terms of the standard scientific criteria of reliability and validity, these instruments were not necessarily designed to capture outcome data, and clearly were not designed as measures specifically related to the Healthy Families Arizona program. The intent of the Parenting Stress Index is to measure stress in the family; the intent of the Child Abuse Potential Inventory is to measure the likelihood of abuse. Nevertheless, this is not the same thing as measuring change that might occur within families while participating in the program. For example, consider these items in the Child Abuse Potential Inventory: 6 Healthy Families Evaluation Report 2004 I have a child who is clumsy My telephone number is unlisted I have a physical handicap As a child I was abused Right now, I am deeply in love. It is clear that these items are not amenable to change and would not do a good job in evaluating the outcomes of the Healthy Families Arizona program. In this respect, many of the existing instruments used in evaluations are not sensitive to change specifically related to Healthy Families Arizona. For example, the Parenting Stress Index measures constructs (and changes to those constructs) related to stress—including temperament which is considered an inherited trait and likely resistant to change. It is not sensitive to the changes occurring in the families that lead to reduced stress. In order to better assess the changes that are occurring within these families that are directly related to the outcomes of the HFAz program, the evaluation team began the process of designing an outcome instrument specific to the Healthy Families program and that meets the criteria of being sensitive to program-specific change. Process Of Developing The Measure The evaluation team used four basic principles to guide the effort: Develop a measure grounded in the practice of Healthy Families Develop the measure with input from home visitors, supervisors, and experts Use information from previous studies of Healthy Families Use existing measures to build on To accomplish the goal of building an instrument from “the ground up” we gathered data from multiple contexts including focus groups with home visitors, the development of a logic model, and an extensive review of relevant literature. For example, in the focus groups, one of the critical questions we asked was: “If you had the opportunity to tell us what we should be measuring to capture the 7 Healthy Families Evaluation Report 2004 changes in these families, what do you think we should be looking for?” A rich source of data was gathered. An example of one response to this question was: You can see a difference in how they are taking care of themselves. In the past we might have seen the parent being attentive to the family but we didn’t really see the parent attend to themselves. So now you not only see them taking care of the child or children, but you see them also taking care of themselves, and realizing they need some mental health services. As we gathered multiple sources of data, we identified key themes that represent the changes happening in the families. The observable, positive changes included such things as: Obtaining support Reducing stress Using problem solving skills Having better abilities to parent Creating a better home environment Increasing one’s commitment as a parent Improving child/parent interactions Reducing depression Taking care of oneself Having a more positive outlook on life We compared these themes with the various instruments that are available for review. From this information, we drafted a model that captures the unique home visitor behaviors that are intended to help parents achieve positive outcomes related to the ultimate goal of reduced child abuse and neglect. The model that forms the theoretical basis for the new Healthy Families Parenting Inventory is shown is Exhibit 2. 8 Healthy Families Evaluation Report 2004 Exhibit 2: Healthy Families Parenting Inventory Theoretical Model Model for Healthy Families Parenting Inventory Practitioner Goals Parenting Outcomes Program Goals Ultimate Outcome Parent groups Parent support Teach problem solving Change perceptions Increased social support Reframing stressful situations Improved problem solving/Coping Reduced Emotional support Referral for counseling Environmental assessment Parenting ability Parenting self efficacy Enhanced home environment Parenting Competence Reduced child abuse and neglect. Parent-child bonding Parent awareness of child’s needs Child development curriculum Acceptance of parent role Parenting commitment Parent-child interactions Parent-child Optimized child development Mental health counseling Physical health promotion Emotional support Planning for self Reduced depression Positive outlook on life Personal care Mobilizing resources Stress Attachment Self Care The final result of this research and input was the new Healthy Families Parenting Inventory that includes 10 subscales: Social support Problem solving and coping Depression Personal care Mobilizing resources Acceptance/commitment of parent role Parent child behaviors 9 Healthy Families Evaluation Report 2004 Home environment Parenting competence Parenting efficacy After questions or items were developed for each of the subscales, the instrument was pilot tested on over 100 families not enrolled in the Healthy Families program. This data allowed us to establish the reliability of the instrument. Further analysis was also conducted to establish the factorial validity of the instrument. This data also confirmed that the Healthy Families Parenting Inventory was measuring what we intended it to measure. (See Appendix D for the final Healthy Families Parenting Inventory.) The Healthy Families Parenting Inventory was designed to be maximally useful to program providers. It will be used for service planning with the program participants. In particular, it can measure overall severity of difficulties by creating a “cutting score” that warrants immediate action by the home visitor. In addition, the measure will contain information on critical needs that will help the home visitors to establish priorities. The measurement of support targets will focus on areas that home visitors can easily target with various intervention strategies. For example, an intervention target might be social support, problem solving, or depression. Lastly, the Healthy Families Parenting Inventory is unique in its inclusion of strengths as an important aspect to many of the subscales. 10 Healthy Families Evaluation Report 2004 Overview of the Healthy Families Arizona Program Healthy Families Arizona is a home visitation program designed to provide supportive services and education to parents of newborns who might benefit from support to strengthen their families at this crucial time. The goals of the program include: To promote positive parent/child interaction To improve child health and development To prevent child abuse and neglect Families are selected via a screening process that begins in the hospital. If the parent is found to be at risk (based on factors known to be associated with child abuse and neglect), they are offered program services. The program is voluntary, and the families may remain in the program for up to five years. For the period covered by this report, families with any substantiated report of child abuse or neglect were excluded from the program. However, legislation was passed in 2004 that will allow the program to serve all at-risk families beginning in August 2004. In addition, the legislation allows the program to serve families prenatally. Program Participants The families that participate in the Healthy Families Arizona program enter the program because they have many stresses in their lives. The stressors constitute risk factors that have been associated with increased risk for child abuse and neglect, as well as poor child health and developmental outcomes (LeCroy & Milligan Associates, 2001). Exhibit 3 highlights the risk factor data for the Fiscal Year 2003 program participants and Fiscal Year 2004 participants. 11 Healthy Families Evaluation Report 2004 Exhibit 3: Selected Risk Factors For Healthy Families Arizona Mothers At Intake. Risk Factors of Mothers at Intake Rate-2003 Rate-2004 Teen births (<19) 21.3% 22.7% Births to single parents 69.2% 69.9% Less than high school education 61.9% 61.2% Not employed 85.3% 83.7% 3.4% 2.8% Late or no prenatal care 37.6% 35.8% Median yearly income $9,600 $7,540 No health insurance These data illustrate that the screening process is recruiting the population targeted by Healthy Families Arizona. The most notable factors are the continued high rates of births to single parents and parents without a high school education. The proportion of all mothers in Arizona who are single is 41.2 percent of the total—as compared to the 69.9 percent of mothers in Healthy Families Arizona. While the number of parents without health insurance is lower than in 2003, a greater percentage are insured by Arizona Health Care Cost Containment System (AHCCCS) in 2004 (87.3%) than in 2003 (82.3%). The percentage of parents with late or no prenatal care remains high, although it is lower in 2004 than in 2003. However, it is considerably higher than the state rate of 7.2 percent. Since the program is able to work with pregnant women beginning in August 2004, the change in the percentage of mothers receiving prenatal care in the next report will be of particular interest. An additional change this year is the shift in median income of the participant families, which fell from $9,600 in 2003 to $7,540, a change of $2,060. Also, 88% of all HFAz families report incomes of $20,000 or less per year, and the average household size among participants is 4.9 persons. This highlights the continued economic stress faced by the families in the program. 12 Healthy Families Evaluation Report 2004 Healthy Families Arizona continues to serve a culturally diverse population. The ethnic makeup of the families served in 2004 (Exhibit 4) is show below. Exhibit 4: Ethnicity Of Healthy Families Arizona Mothers. Caucasian 26.5% Other 2.8% Native American 9.4% Asian American 0.5% African American 4.8% Hispanic 56.0% Families (mothers, and fathers when they are involved) are assessed using the Parent Survey. At intake, the Family Assessment Worker evaluates each parent’s level of stress in 10 domains. The percentage of parents scoring severe on each of the scales are presented in Exhibit 5. As in previous years, the three most significant stressors are coping with a history of child abuse, having low self-esteem and feeling isolated, and dealing with current life stress, including low income, poor housing, and relationship difficulties. Criminal involvement, having a history of substance abuse, and mental illness are more significant problems for the fathers than for the mothers. Also, fathers show more history of violence, and report more difficulties with parental attachment than do mothers. 13 Healthy Families Evaluation Report 2004 Exhibit 5: Percentage Of Parents Rated Severe On The Family Stress Checklist. Childhood Abuse Crime, Substance Abuse, Mental Illness Self-esteem, Isolation CPS Involvement Current Life Stresses Violence Potential Expectations of Infant Discipline Attitudes Father Mother Difficult Child Parental Attachment 0 10 20 30 40 50 60 70 Percent with Severe Rating Data are also collected regarding high-risk characteristics of the newborns (see Exhibit 6). Exhibit 6: Percentage Of Infants With High-Risk Characteristics. Risk Factors for infants at intake Born <37 weeks gestation Birth defects Low birth weight Positive alcohol screen State Rate HFAz Rate HFAz Rate 2004 2003 9.9%* 13.9% 15.3% NA 1.3% 1.3% 7.1%** 12.8% 14.0% NA 0.6% 0.4% * 2001 Data from the Arizona Department of Health Services ** 2003 Data from the Arizona Department of Health Services The percentage of Healthy Families Arizona infants born early (less than 37 weeks gestation) was less than the 2003 percentage, although it is still high when compared to the state rate. The percentage of low birth weight infants in the program is also less than previous years, but again remains high in comparison 14 Healthy Families Evaluation Report 2004 to the state rates. These risk factors are known to be associated with increased potential for child abuse and neglect and other unhealthy outcomes. The Healthy Families Arizona home visitors support the needs of these families with high-risk infants by providing support, assessment, and referral from the time of birth onward. Service Delivery Three aspects of service delivery will be reviewed in this report. First, data regarding program engagement and retention are reviewed to give insight into characteristics of families who voluntarily join and stay in the program. Second, a review of types of services provided to the family provides insight into the key components of the program as the child grows. Third, program satisfaction data is provided to gain insight into what aspects of the program are most worthwhile from the families’ point of view. Number of families served The total number of families served (enrolled) by all Healthy Families Arizona sites during the study period (July 1, 2003 through June 30, 2004) was 2,356. Families may enroll in the program, but they are not considered actively engaged until four home visits have been completed. In the current study year, 2,137 families (90.7%), who were enrolled in the program became engaged—a slight increase from last year. The distribution of engaged families for each site is shown in Exhibit 7. The number of families served by the sites is based on the funding level and number of Family Support Specialists at each site. 15 Healthy Families Evaluation Report 2004 Exhibit 7: Healthy Families Arizona Participants Served By Site. Site and Participants Served Site and Participants Served Cochise County Coconino County Douglas/Bisbee 100 Sierra Vista 77 Maricopa County Flagstaff 96 Page 47 Tuba City 51 Mohave County Central Phoenix 104 East Valley Phoenix 64 Maryvale 88 Mesa Lake Havasu City Pima County 113 South Phoenix 73 Southeast Phoenix 101 Sunnyslope 106 Pinal County Pinal County Department of Public Health 103 Casa de los Niños 131 CODAC 113 La Frontera 146 Pascua Yaqui Child & Family Resources 54 48 Santa Cruz County Nogales 93 Yuma County 107 Yavapai County Yuma 93 Prescott Verde Valley 147 82 TOTAL ALL SITES = 2,137 The consistent characteristics of families less likely to engage in the program include: More often single parents More likely to be living alone or with non-parental relative More history of depression (an 8% difference from families who engage) More likely to have history of criminal behavior, substance abuse or mental illness Greater potential for violence Greater difficulty with daily life stresses 16 Healthy Families Evaluation Report 2004 The risk factors present in these families highlight the importance of reaching out to engage them in the program. Because they are likely to be single parents often living alone, they may have increased needs for support and outreach. Understanding this pattern enables the program to develop ways to creatively engage mothers and fathers who may be hesitant to become involved. As the Healthy Families Arizona program matures, the average length of time in the program at termination for engaged families is increasing. A greater proportion of families are staying in the program for three, four and five years. Therefore, the average length of time families are in the program increased from 698 days to 804 days—an increase of over three months. The length of time families stay in the program is shown in Exhibit 8. As can be seen, 19% of the families who terminated in 2004 remained in the program at 1-2 years, 22% stayed 2-4 years, and 22% remained over 4 years. It is noteworthy that in a recent evaluation of a Healthy Families program in Hawaii (Duggan, et.al., 2004), only about half of the families stayed with the program for more than one year. Healthy Families Arizona, with its emphasis on engagement and retention, has had success in retaining families that exceeds many other programs. The success at program implementation may be one of the reasons why program outcomes for the HFAz program have been more positive than for other programs like the one evaluated in Hawaii. 17 Healthy Families Evaluation Report 2004 Exhibit 8: Average Length Of Time In Program At Termination For Engaged Families. 3-6 months 9% Up to 3 months 4% Over 4 years 22% 3-4 years 9% 7-12 months 24% 1-2 years 19% 2-3 years 13% Staff retention appears to contribute to success in retaining families in the program. Healthy Families Arizona also gathers data on staff retention of Supervisors, Family Support Specialists and Family Assessment Workers. In examining those sites with the longest retention rates, program data reveals that those sites also have high retention of their staff. For example, in four of the sites with the longest average length of time to termination, the supervisors have been employed for at least 8 years, and nearly all of the Family Support Specialists have been employed for at least 5 years. These supervisors and home visitors are seasoned professionals with experience and expertise in building relationships, dealing with difficult issues, and effectively conducting outreach to retain families. The number of families who have successfully completed the program according to the criteria established by Healthy Families Arizona has doubled over the past year. In the current evaluation year, the program completion rate was 20.5 percent. The previous two years’ completion rates were 10.5 percent and 13.8 respectively. While many of the criteria for “successful program completion” focus on family stability, aspects of self-sufficiency, consistent child development, and decreased parental stress, several of the exit criteria should be examined. Criteria such as the need for completion of three years of program service may be less meaningful than the outcomes of the family in achieving its goals and exhibiting stability. 18 Healthy Families Evaluation Report 2004 Similar to last year, over half of the families leaving the program either moved away (24.8%) or could not be contacted (26.6%). The transience of families in Arizona poses particular challenges to retention. With the expansion of Healthy Families programs throughout Arizona, programs may be able to develop better strategies for transferring families between program sites. The data for the families terminating are shown in Exhibit 9. Exhibit 9: Reasons For Program Termination—Engaged Families. Moved away 24.8% 29.7% 27.3% 26.6% Unable to contact 13.8% Completed program Family refused further services 6.0% 20.5% 10.0% 5.6% 8.1% Refused a change in worker 5.1% 5.8% Self-reported self-sufficiency 7.5% 8.1% All other reasons 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Percent of Families 2004 Report 2003 Report Types of Services provided As the Healthy Families Arizona program has grown, program staff and evaluators have increasingly examined the types of service provided to the families engaged in the program. Understanding the particular mix of services that are most beneficial at different points in the developing families’ lives helps to focus the intervention. As stated earlier in this report, the core Healthy Families Arizona services are: Emotional support Assistance in developing positive parenting skills Education on child development and nutrition 19 Healthy Families Evaluation Report 2004 Education and assistance in problem solving and coping skills Education on preventive health care (immunizations, links to medical doctor) Linkages to preschool resources Referrals related to education, employment, and mental health and substance abuse services. The chart below illustrates the pattern of services provided parents at the six, twelve, and eighteen month points for active families in the program. The first five services are the ones most used in implementing the program. Exhibit 10: Healthy Family Arizona Service Patterns 6-Months 12-Months 18-Months No. of families No. of families No. of families who used service who used service who used service (Total N=593) (Total N=399) (Total N=276) Child Development Education 575 379 266 Emotional Support 484 334 244 Model Bonding/Parent-Child 508 331 214 Information and Referral 428 308 200 Model Life Coping Skills 314 244 161 Literacy Training 202 123 66 Crisis Intervention 148 111 95 Case Management Services 197 147 113 Linkage to MD or Other Services 130 83 55 Parent Groups 162 112 83 Playgroups 109 81 53 Transportation 89 64 36 Discussions Regarding 75 60 41 Service Curriculum Domestic Violence 20 Healthy Families Evaluation Report 2004 As can be seen in the table, during the families’ first 18 months in the program, Family Support Specialists are focusing most heavily on child development education, modeling parenting skills, providing emotional support and providing referrals to community resources. Discussions about sensitive and difficult issues such as domestic violence are much less frequent, although many home visitors have reported that domestic violence, substance abuse and mental health problems are often evident in families they serve. As reported in the 2003 annual report, over two thirds of all home visitors reported they had experienced dealing with these issues with families “in the last 30 days” (LeCroy & Milligan Associates, 2003). The HFAz Quality Assurance and Training team and program supervisors have focused training and supervision on building skills and knowledge in domestic violence, substance abuse, and mental health issues, so that home visitors are more comfortable and prepared to address these areas and help families get into services. The development of the embedded evaluation instrument is also designed to provide a tool for targeting goals and service needs of families and measuring progress in family outcomes. Program Outcomes Healthy Families Arizona continues to collect outcome data on a number of outcome indicators in order to evaluate program effectiveness. These indicators include program impact on child abuse and neglect, child health outcomes, parental stress and competence, safety practices in the home, medical and social service use, and employment and educational attainment. Earlier in this report, there was a discussion regarding measurement, and the sometimes difficult choices that must be made in regard to what measure to use to accurately capture the kind of change that is occurring. With the implementation of several new measures in the fall of 2004, new data will be available in next year’s report. Child Abuse and Neglect Exhibit 11 presents the data regarding substantiated child abuse and neglect reports for two groups—the Healthy Families Arizona program families and a comparison group. The program group consists of families who have had at least four or more home visits and the comparison group consists of families who 21 Healthy Families Evaluation Report 2004 dropped out prior to completing at least four visits. The results are based on all families who were active in the program during the study period of July 1, 2003 to June 30, 2004 with at least six months time in the program. Exhibit 11: Percent Of Child Abuse And Neglect Incidences In Program And Comparison Groups. Group Healthy Families Arizona Participants Comparison Group Participants Percent without Percent without substantiated report substantiated report 2003 2004 99.0 % 98.4 % (n=1554) (n=1568) 98.7 % 98.6 % (n=234) (n=213) Exhibit 12 shows the rate of substantiated child abuse and child neglect for all of the years Healthy Families Arizona has reported data.2 Prior to 1999, the program was allowed to work with families with a history of substantiated child abuse or neglect reports. Beginning in 1999, the program no longer could enroll or support families with substantiated reports. As might be expected, the data indicate that rate of substantiation for enrolled families was greater when working with families with a history of abuse and neglect (1992 through 1995). Beginning in August 2004, the program will once again be allowed to support families with a history of substantiated reports, and therefore the overall rate of substantiated cases for the program families is likely to rise. 2 Data were not available in 1996, 1997 or 1998 because the state CPS data reporting system was being revised. Comparison group data were not collected until 1994. 22 Healthy Families Evaluation Report 2004 Exhibit 12. Substantiated Reports Trend. Substantiated Percentage 10% 8% 6% 4% 2% 0% 1992 1993 1994 HF Families Comparison Group 1995 1999-2000 2001 2002 2003 2004 Report Year While the percent of families without a substantiated report has gone down, the incidence of child abuse and neglect may not be good measures of short-term program impact for several reasons, including: child abuse and neglect are low occurring events, and small changes over short periods of time may not represent the longer term effect of the program; many incidents of child abuse and neglect go unreported, calling into question the reliability of the measure; increased surveillance of families who are involved in community programs such as Healthy Families Arizona may lead to increased reporting. For these reasons, caution must be used when coming to conclusions regarding program impact based on child abuse and neglect data. Having said that, there may be more useful data regarding longer term changes, and the permanency of 23 Healthy Families Evaluation Report 2004 those longer term changes which will be investigated with the longitudinal study now underway. Parenting Stress Reducing parental stress is one of the key measures used to look at the success of the Healthy Families Arizona program. This is because of the relationship between parental stress and child abuse and neglect. Healthy Families Arizona used the Parenting Stress Index (Abidin, 1995) to assess total stress and data regarding seven subscales—sense of competence, parental attachment, feeling restricted in one’s role, depression, isolation, distractibility, and mood. While this instrument is a reliable and valid measure used extensively in research and evaluation, use of the instrument was discontinued in December 2003 as a newly developed measure is being introduced to the program (as explained earlier in this report). However, there was sufficient data to analyze the baseline to sixmonth and baseline to twelve-month changes. Exhibit 13 summarizes the analytic results for the total stress score and each subscale for baseline data (pretest) and subsequent tests at six and twelve months. As illustrated in the exhibit, the total parenting stress score shows significant change across both time intervals. The sense of competence, feeling restricted in role, depression, isolation, parental attachment and mood subscales all showed improvement across all time intervals.3 3 Definitions of the subscales and additional statistical details can be found in Appendix B. The Distractibility scale is not reported as the scale has been found to be unreliable. 24 Healthy Families Evaluation Report 2004 Exhibit 13: Parenting Stress Index Findings. Subscale Significant Improvement Baseline to 6 months Significant Improvement Baseline to 12 months Sense of Competence Parental Attachment Feeling Restricted in Role Depression Isolation Mood Total Stress Score The data also reveal that 67 percent of the families taking both the baseline and 6month PSI showed improvement in the total stress score. This is an increase over the last year. Safety Practices Healthy Families home visitors report great success in using the “Child Safety Checklist” with families, as families are very receptive to learning how to make their homes safe for their children. Because accidents are a leading cause of injury and death among young children, improved safety practices in the home has long been a key objective of the program. At the 2-month assessment point, home visitors are developing relationships with the family and setting the stage for discussing safety practices as the family’s infant grows. Exhibit 14 details two of the items tracked on the child safety checklist—outlets covered and poisons locked away. Families are showing increases in use of these safety practices as the child gets older— at 2, 6, and 12 months—for last year and the current year.4 4 Other indicators include appropriate use of car seats, the securing of scissors, knives, lighters and matches, water safety, outside supervision, food storage, and a listing of emergency phone 25 Healthy Families Evaluation Report 2004 However, the overall percentage of families employing the practice is not increasing. As children become more mobile through crawling and walking, program staff should re-emphasize the need to help families adopt safe practices in these two areas at 6 and 12 months and beyond. Exhibit 14: Percent Of Safety Practices Implemented. 2 Month 6 Month 12 Month 2003 2004 2003 2004 2003 2004 Outlets Covered 49.8% 46.5% 61.1% 56.4% 75.5% 75.7% Poisons Locked 84.8% 83.3% 87.4% 86.2% 93.1% 93.8% One other safety practice that is independent of child mobility that needs attention is the availability of a working smoke alarm in the home. At the twomonth interval, 84 percent of homes have working alarms. At 12 months, 87 percent have working alarms. This could also be an area of emphasis for the home visitors in the future. Adequate medical care for all family members is another key goal of Healthy Families Arizona. The immunization rate for the children is one of the indicators used to measure this objective. Exhibit 15 shows this rate for the infants of families enrolled in Healthy Families Arizona in the current study year and the previous year. numbers. At the two-month assessment, these other safety practices were all being actively used by over 98 percent of the participants. 26 Healthy Families Evaluation Report 2004 Exhibit 15: Immunization Rate of Healthy Families Arizona Children Percent Percent Immunization Rate for 2- Immunized Immunized year-olds in Arizona 2003 2004 (July 2004) 2 month 95.1% 96.7% 4 month 92.8% 94.3% 6 month 85.1% 87.1% 12 month 92.0% 95.9% Received all 4 in the series 88.8% 94.0% Immunization Period 77.0% The program continues to see growth in this important area, and the rate of immunization continues to outpace the state rate by a considerable margin, in spite of the fact that the families served represent a high-risk group perceived as less likely to receive immunizations. A second way to look at the goal of ensuring the families receive adequate medical care is to look at the percentage of children linked to a medical doctor. As can be seen in Exhibit 16, over 96% of children have a medical home at each of the data intervals. There is very little change from last year to this year on this variable, showing that the program continues to meet the challenge of having a high percentage of children linked to a primary care physician. 27 Healthy Families Evaluation Report 2004 Exhibit 16: Percentage of Children Linked to a Medical Doctor. 6 months 12 months 24 months 0.0% 20.0% 40.0% 60.0% 80.0% 100.0% Percentage of Children Linked to Medical Doctor 2003 2004 Equally important to the need for quality care of the child is the need to ensure the parent receives appropriate health care. Health care for parents can contribute to better family planning and early identification of problems such as depression or domestic violence—all problems that affect the health and wellbeing of the entire family. This year, more than 80 percent of the parents report they have a primary care physician after they’ve been in the program eighteen months. One of the keys to achieving the high rate of success in both immunizations and linkage to physicians is continued emphasis by supervisors and regular feedback to the sites. Data regarding the immunization rates and the linkage to a doctor are reported regularly to the sites via the Healthy Families Arizona Quarterly Family Data Report, which is used by the sites as an on-going quality assurance tool. The quarterly reports provide timely information about which families need support in medical and health care. Child Development The promotion of healthy child development is a central goal of Healthy Families Arizona. One tool used by the Family Support Specialists to achieve this goal is the Ages and Stages Questionnaire (ASQ). By working with the parents to fill out the questionnaire every 6 months, the home visitors have an opportunity to teach about the developmental tasks and milestones of their children as they grow. 28 Healthy Families Evaluation Report 2004 Additionally, the questionnaire can indicate the need to do additional assessment for potential developmental delays. This early detection allows for referral to appropriate follow-up services such as the Arizona Early Intervention Program (AzEIP), or another intervention program. Exhibit 17 displays the information regarding the percentage of children in the program screened, and the percentage who screened positive for developmental delays. The data suggest that the overall percentage of children screened goes down over time, while the number of children scoring as having a potential delay is going up. Exhibit 17: ASQ Screening Percent Screened as Interval ASQ Percent of Children Administered Screened with ASQ 6-Month 81.0 % 5.3 % (22) 12-Month 80.6 % 10.9 % (29) 18-Month 73.3 % 17.7 %(36) 24-Month 76.1 % 21.8 % (43) 30-Month 75.1 % 15.2 %(28) Delayed Percent (N) Many of the children are bilingual which can result in temporary “delays” at the two-year period. If the ASQ indicates a potential delay, it is important to ensure further assessment is made to determine the proper course of action. Continued assessment often indicates no delay is evident. In other cases, further assessment indicates that additional intervention is needed, and appropriate referrals need to be made. Exhibit 18 provides the data regarding assessment status and referral. 29 Healthy Families Evaluation Report 2004 Exhibit 18: ASQ Referral Status. Reassessment Shows “No “Delay” Referred to AzEIP Referred to Early Intervention Provided Developmental Intervention Referred to Therapy Parent Declined Referral % (N) % (N) % ( N) % (N) % (N) % (N) 6month Screen 40.9% (9) 13.6% (3) 18.2% (4) 54.5% (12) 0% (0) 4.5% (1) 12month Screen 24.1% (7) 27.6% (8) 13.8% (4) 37.9% (11) 10.3% (3) 10.3% (3) 18month Screen 13.9%(5) 27.8% (10) 19.4% (7) 41.7% (15) 0% (0) 11.1% (4) 24month Screen 7.0% (3) 46.5% (20) 7.0% (3) 41.9% (18) 4.7% (2) 11.6% (5) 36month Screen 7.1% (2) 50.0% (14) 21.4% (6) 50.0% (14) 10.7% (3) 3.6% (1) Note: percents do not equal 100% as multiple referrals can happen for a single child In all cases, other than those where the re-assessment indicated that the child was not delayed or where the parent refuses further action, the children screened as developmentally delayed receive follow-up services. These services include referral to the Arizona Early Intervention Program, other developmental intervention, or other therapy. Substance Abuse Services Another risk for many families is the potential for alcohol and drug problems, which is strongly linked to child abuse and neglect. The Healthy Families Arizona program provides screening to families in an effort to help determine who may need to seek alcohol or drug treatment. While a positive screen doesn’t necessarily indicate a substance abuse problem or alcoholism, it may be an 30 Healthy Families Evaluation Report 2004 important signal to Healthy Families staff about the need for further discussion or referral. Routine use of an appropriate screen may reduce the stigma associated with asking questions about substance use and in turn, help families seek help more readily. Exhibit 19 shows the results of the screening at the 2, 6, 12, and 18-month intervals. Only a small number of mothers revealed positive screens. However, this rate is very similar to the 3% rate of positive screens found in a study of women in the general population. Data at 12 months shows that the home visitors provided information or educational materials about substance abuse to 82% of families and made referrals to a community service, AA or Alanon with 11% of families. Exhibit 19: Percentage Of Families Who Screened Positive For Alcohol And Drug Problems. Screening Interval Positive Screens Positive Screens 2004 2003 2 months (N=315) 5.7% 6.3% 6 months (N=223) 3.6% 7.8% 12 months (N=119) 2.5% 4.6% 18 months (N=123) 5.7% 4.6% Both Healthy Families home visitors and the evaluation team have not been satisfied with the CAGE as a tool for substance abuse screening and discussion. Furthermore, the CAGE has been found to be less sensitive and reliable with women and teens than with men (Allen & Anthenelli 2003). As part of the current evaluation review, the evaluation team is researching and recommending a more appropriate instrument that will screen effectively for substance abuse problems while at the same time promoting communications. The TA/QA team is also working with DES staff and providers to provide additional training to the home visitors in skills such as motivational interviewing, so they are better equipped to communicate with the families on these kinds of sensitive issues. 31 Healthy Families Evaluation Report 2004 Mother’s Health and Employment The Healthy Families Arizona program focuses on the health and well-being of the parents as well as children. Each year, the evaluation examines maternal life outcomes such as subsequent pregnancies, education, and employment. During the study period, 15 percent of the mothers reported subsequent pregnancies. Of these, 29.7 percent were 18 years or younger. Exhibit 20 shows the length of time to subsequent pregnancy. As shown, the time between births is lengthening. The trend, albeit minor, indicates that Healthy Families Arizona mothers are waiting longer to have another child. Exhibit 20: Length Of Time To Subsequent Pregnancy Length of Time to 2004 2003 Subsequent Pregnancy Percent of Mothers Percent of Mothers 1 to 12 mos. 31.6 % 33.0 % 13 to 24 mos. 42.3 % 45.9 % Over 24 mos. 26.1 % 21.1 % Exhibit 21 shows the employment status of mothers actively engaged in the program at various points in the program as compared to 2003. While the mothers employed at baseline shows a slight increase (16.3% in 2004), those employed at six (31.8% in 2004) and twelve months (36.5% in 2004) show slight decreases. Exhibit 21. Mother’s Employment Status. Percent 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 2003 Baseline 6 months 2004 12 months 32 Healthy Families Evaluation Report 2004 Conclusions and Recommendations Healthy Families Arizona is a growing and vital program for delivering prevention services to at-risk families. Many important changes are taking place in the program. All families, regardless of any previous history with Child Protective Services, are now eligible to be screened and assessed into the program. Families can begin to be served by the program before the birth of the child. All of these changes can have a positive impact on a vulnerable part of the population—the newborn children of families at risk. In response to these and other changes, the evaluation is also changing. A new instrument, specifically developed to track the positive changes possible with the Healthy Families Arizona program is being implemented. The program is in the process of determining how to include the home visitors’ perception of change as part of an embedded evaluation instrument. The team is beginning the process of choosing the appropriate tool for measuring change with prenatal children. At the same time, a more rigorous longitudinal study is under way. All of this, combined with the changes in the program itself, create an exciting period, and future, for the program. The program continues to undergo evaluation and quality improvement. This report describes a program that continues to serve a diverse population with clear indicators of risk. The number of participants who have had little or no prenatal care and low birth weight babies continues to be higher than average, indicating a potential to positively impact this area with prenatal services. Almost all sites continue to serve families to their capacity. Although the program served approximately the same number of families as last year, an increasing number (91 percent) elected to stay in the program for an increasingly longer period—the average length of time in the program grew from 698 days to 804 days. This is part of a continuing trend over the last several years, and reflects the emphasis the program staff has placed on engagement and retention of program families. 33 Healthy Families Evaluation Report 2004 Program outcomes continue to document the program’s effectiveness. The number of parents without a substantiated report of child abuse and neglect continues to meet the state goal of over 95 percent. Multiple measures of parental stress show significant improvements over time. A larger percent of total families reported positive changes compared to last year. The child safety checklist continues to be a valuable tool, although some increased attention could be paid to some items. The immunization rate for Healthy Families Arizona babies continues to increase, well outpacing the overall state rate. The children are screened for developmental delays at a high rate. As more families remain longer in the program, it is important for family support specialists to continue to achieve a high rate of developmental screening as the children grow nearer to school-age. Families, particularly young mothers, continue to need strong support to carefully plan subsequent pregnancies. During the study period, 15 percent of the mothers had subsequent pregnancies. Of those who did become pregnant, 30 percent were 18 or younger, and 32 percent became pregnant within one year. Based on this evaluation, the following recommendations are made. 1) To effectively address some of the most difficult family problems such as substance abuse, domestic violence and mental illness, the program should continue to find the most appropriate role for home visitors in addressing these issues. Further clarification of the Healthy Families logic model will help define the most appropriate services to be provided by home visitors. The program should review and refine existing protocols to ensure they are consistent with emerging best practices, appropriately address the significant issues faced by the families, and address the developmental needs of the child. Step-by-step practice protocols, grounded in best practice research, will clarify and strengthen the fidelity of the Healthy Families program model. 2) The program should continue to emphasize careful planning of subsequent pregnancies, especially with young mothers. Fifteen percent of the program participants had subsequent pregnancies while enrolled in the program. Nearly one-third of them were by teenage mothers. 34 Healthy Families Evaluation Report 2004 3) The substance abuse screen has been re-evaluated and replaced. More training in how to discuss substance use and abuse is needed. A screen which is more sensitive and accurate for use with women has been recommended; both research and practical experience of home visitors supports this recommendation. In addition, because of the high risks associated with prenatal alcohol and other drug use, a prenatal screen is needed that opens the door for discussion around the dangers unique to drug use during this period. A welldone screening represents a learning opportunity for the parent and leads to better assurance that the most appropriate services will be recommended to the family. However, home visitors must be comfortable and trained to effectively deal with these issues. 4) The use of the safety checklist to identify and help families implement safety practices in their homes should be re-emphasized. Three areas in particular could be easily resolved with minimal investment—covers on electrical outlets, keeping poisons locked up, and installing working smoke alarms. 5) More emphasis needs to be placed on administering the Ages and Stages Questionnaire consistently through the program. The percent of children screened using the instrument declines over time, while the number of children screening positive for developmental delays is increasing. 6) The criteria for “successful completion” of the program should be re-examined. There may be additional ways to examine the extent to which a family is “successful” that are based on specific outcomes related to their goals. Current criteria related to a specified length of time in the program may be a less useful and valid measure of their progress. 7) With the implementation of prenatal services, the evaluation will expand to include an examination of the implementation of prenatal services. As part of the expansion, relevant measures for these families should be developed. Development of a prenatal logic model will aid in determining the appropriate outcomes for the program and serve as a guide in developing appropriate measures. 35 Healthy Families Evaluation Report 2004 Future Considerations for Program Improvement and Program Evaluation Looking to the future, there are several initiatives underway that will improve the evaluation and consequently, strengthen the program. The longitudinal study has begun. In the recent Emerging Practices in the Prevention of Child Abuse (Caliber, 2003), the need to expand existing knowledge about the effectiveness of prevention is the overriding theme. The report specifically states, “Existing knowledge about the efficacy of prevention in the field of child maltreatment is limited; clearly, all the major prevention models and strategies could benefit from more rigorous study” (p. 1). Chaffin (2004) echoed the same theme in a recent commentary on the Hawaii Healthy Start program. The Healthy Families Arizona program is uniquely positioned to address many of the criticisms levied against previous home visitation studies. The program has a centralized, strong quality assurance/technical assistance component; it has over 10 years of evaluation experience and a database with 14 years of data. The new Healthy Families Parenting Inventory has been implemented. One of the consistent criticisms of home visitation programs has been that there is insufficient data regarding the specific outcomes related to the program model. The Healthy Families Parenting Inventory was developed in a manner that grounded the instrument in the specific outcomes of the Healthy Families Arizona program. Work is underway to develop an additional measure that will be filled out by the Family Support Specialists to provide information regarding the type and scope of change in the program. 36 Healthy Families Evaluation Report 2004 References Abdin, R. L. (1995). The parenting stress index. Odessa, FL: Psychological Assessment Resources. Allen, J. P. & Anthenelli, R. M. (2003). Getting to the bottom of problem drinking: The case for routine screening. Current Psychiatry Online, Vol 2, No. 6. www.currentpsychiatry.com Caliber Associates (2003). Emerging Practices in the Prevention of Child Abuse. Washington, DC: Author. Chaffin, Mark (2004). Is it time to rethink Healthy Start/Healthy Families? Child Abuse and Neglect, 28, 589-595. Duggan, A., McFarlane, E., Fuddy, L., Burrell, L., Higman, S. M., Windham, A., & Sia, C. (2004). Randomized trial of a statewide home visiting program: Impact in preventing child abuse and neglect. Child Abuse and Neglect, 28, 597-622. LeCroy & Milligan Associates, Inc. (2001). Healthy Families Arizona Evaluation Report 2001. Tucson, AZ: LeCroy & Milligan Associates, Inc. LeCroy & Milligan Associates, Inc. (2002). Healthy Families Arizona Evaluation Report 2002. Tucson, AZ: LeCroy & Milligan Associates, Inc. LeCroy & Milligan Associates, Inc. (2003). Healthy Families Arizona Evaluation Report 2003. Tucson, AZ: LeCroy & Milligan Associates, Inc. Lindberg, Anne M. (1997). Gift from the Sea. New York: Pantheon Books. McClure, Vimala (1997). The Tao of Motherhood. Novato, CA: New World Library. 37 Healthy Families Evaluation Report 2004 Appendix A Site Level Data Age of Child at Entry Days to Termination Reason for Termination Health Insurance at Intake Late or No Prenatal Care or Poor Compliance at Intake Ethnicity of Mother Gestational Age Low Birth Weight Yearly Income Family Stress Checklist Score 38 Healthy Families Evaluation Report 2004 Age of Child at Entry by Site - 2004 (Age in days) Mean (Age in Days) Standard Deviation Number Douglas/Bisbee 14.92 15.97 99 Central Phoenix 23.64 20.99 101 Maryvale (Phoenix) 22.90 20.74 87 South Phoenix 23.29 21.88 72 East Valley (Phoenix) 21.92 18.40 62 Nogales 14.65 18.82 106 Page 27.55 22.98 47 Casa de los Niños (Tucson) 21.40 17.39 129 CODAC (Tucson) 23.88 22.55 112 La Frontera (Tucson) 20.87 21.40 145 Child & Family Resources (Tucson) 21.75 22.15 48 Sierra Vista 14.26 16.58 74 Tuba City 17.32 20.09 50 Verde Valley 11.49 15.59 82 Yuma 19.52 17.14 93 Pascua Yaqui 33.92 25.21 53 Lake Havasu City 23.17 19.70 103 Flagstaff 18.21 21.92 95 Sunnyslope (Phoenix) 24.86 18.36 104 Prescott 21.67 18.93 147 Casa Grande 17.86 22.35 90 Mesa 22.06 17.37 111 Southeast Phoenix 19.32 16.23 98 Total 20.66 19.94 21.08 Site 39 Healthy Families Evaluation Report 2004 Days to Termination by Site - 2004 (For terminated clients) Mean (Days to termination) Standard Deviation Number Douglas/Bisbee 1157.54 607.51 28 Central Phoenix 817.96 701.93 25 Maryvale (Phoenix) 721.50 642.95 26 South Phoenix 755.30 560.96 20 East Valley (Phoenix) 692.32 586.18 19 Nogales 1162.08 771.27 37 Page 726.82 525.21 17 Casa de los Niños (Tucson) 855.35 577.91 40 CODAC (Tucson) 853.05 579.99 22 La Frontera (Tucson) 876.35 567.70 37 Child & Family Resources (Tucson) 964.13 565.09 16 Sierra Vista 761.52 652.61 23 Tuba City 910.26 732.31 19 Verde Valley 732.27 578.71 26 Yuma 766.81 622.12 31 Pascua Yaqui 1390.35 658.00 17 Lake Havasu City 509.03 374.22 34 Flagstaff 823.16 531.80 32 Sunnyslope (Phoenix) 577.30 631.49 33 Prescott 682.24 552.43 58 Casa Grande 664.71 552.26 24 Mesa 642.83 512.09 24 Southeast Phoenix 729.19 564.24 21 Total 804.90 615.70 629 Site 40 Healthy Families Evaluation Report 2004 Top Three Reasons for Termination by Site - 2004 (Percent and number within Site) Moved Away Unable to contact Completed Program Douglas/Bisbee 18.5% (5) 14.8% (4) 29.6% (8) Central Phoenix 24.0% (6) 24% (6) 20.0% (5) Maryvale (Phoenix) 34.6% (9) 23.1% (6) 19.2% (5) South Phoenix 15.8% (3) 31.6% (6) 10.5% (2) East Valley (Phoenix) 27.8% (5) 44.4% (8) 16.7% (3) Nogales 24.3% (9) 13.5% (5) 48.6% (18) Page 17.6% (3) 29.4% (5) 11.8% (2) Casa de los Niños (Tucson) 5.1% (2) 43.6% (17) 15.4% (6) CODAC (Tucson) 13.6% (3) 31.8% (7) 27.3% (6) La Frontera (Tucson) 35.1% (13) 24.3% (9) 27.0% (10) Child & Family Resources (Tucson) 18.8% (3) 12.5% (2) 37.5% (6) Sierra Vista 17.4% (4) 34.8% (8) 21.7% (5) Tuba City 31.6% (6) 21.1% (4) 31.6% (6) Verde Valley 40.0% (10) 28.0% (7) 12.0% (3) Yuma 13.8% (4) 48.3% (14) 24.1% (7) Pascua Yaqui 11.8% (2) 5.9 (1) 52.9% (9) Lake Havasu City 29.4% (10) 14.7% (5) 2.9% (1) Flagstaff 38.7% (12) 6.5 (2) 3.2 (1) Sunnyslope (Phoenix) 27.3% (9) 6.1% (2) 15.2% (5) Prescott 31.6% (18) 28.1% (16) 15.8 (9) Casa Grande 25.0% (6) 45.8% (11) 20.8 (5) Mesa 33.3% (8) 29.2% (7) 8.3% (2) Southeast Phoenix 19.0% (4) 61.9% (13) 14.3 (3) 24.8% (154) 26.6% (165) 20.5% (127) Site Total 41 Healthy Families Evaluation Report 2004 Health Insurance by Site at Intake - 2004 (Percent and number within Site) Site None AHCCCS Private Douglas/Bisbee 3.0% (3) 91.9% (91) 5.1% (5) Central Phoenix 1.0% (1) 81.6% (84) 15.5% (16) Maryvale (Phoenix) 4.6% (4) 83.9% (73) 11.5% (10) South Phoenix 1.4% (1) 84.9% (62) 12.3% (9) East Valley (Phoenix) 4.9% (3) 82.0% (50) 11.5% (7) Nogales 7.6% (8) 88.6% (93) 1.9% (2) 0 91.5% (43) 8.5% (4) Casa de los Niños (Tucson) 1.5% (2) 84.6% (110) 12.3% (16) CODAC (Tucson) 0.9% (1) 93.7% (104) 3.6% (4) La Frontera (Tucson) 2.1% (3) 90.4% (132) 6.2% (9) Child & Family Resources (Tucson) 2.1% (1) 89.4% (42) 8.5% (4) 0 74.6% (53) 21.1% (15) Tuba City 15.7% (8) 84.3% (43) 0 Verde Valley 1.3% (1) 88.6% (70) 10.1% (8) Yuma 4.3% (4) 94.6% (87) 0 0 88.7% (47) 3.8% (2) Lake Havasu City 2.9% (3) 87.4% (90) 8.7% (9) Flagstaff 3.1% (3) 89.6% (86) 7.3% (7) Sunnyslope (Phoenix) 2.9% (3) 85.7% (90) 9.5% (10) Prescott 1.4% (2) 86.2% (125) 6.9% (10) Casa Grande 3.3% (3) 89.1% (82) 7.6% (7) Mesa 0.9% (1) 88.3% (98) 4.5% (5) Southeast Phoenix 3.0% (3) 84.0% (84) 10.0% (10) Total 2.8% (58) 87.3% (1839) 8.0% (169) Page Sierra Vista Pascua Yaqui 42 Healthy Families Evaluation Report 2004 Late or No Prenatal Care or Poor Compliance at Intake - 2004 by Site (Percent and number within Site) The participant received no or late prenatal care or showed poor compliance with prenatal care Site True False Unknown Douglas/Bisbee 43.0% (43) 53.0% (53) 4.0% (4) Central Phoenix 30.8% (32) 69.2% (72) 0 Maryvale (Phoenix) 31.8% (28) 63.6% (56) 4.5% (4) South Phoenix 37.0% (27) 61.6% (45) 1.4 (1) East Valley (Phoenix) 33.3% (21) 61.9% (39) 4.8% (3) Nogales 54.2% (58) 40.2% (43) 5.6% (6) Page 39.1% (18) 60.9% (28) 0 Casa de los Niños (Tucson) 26.2% (34) 63.8% (83) 10.0% (13) CODAC (Tucson) 32.1% (36) 63.4% (71) 4.5% (5) La Frontera (Tucson) 29.5% (43) 65.8% (96) 4.8% (7) Child & Family Resources (Tucson) 35.4% (17) 60.4% (29) 4.2% (2) Sierra Vista 29.9% (23) 68.8% (53) 1.3% (1) Tuba City 29.4% (15) 62.7% (32) 7.8% (4) Verde Valley 57.3% (47) 42.7% (35) 0 Yuma 45.2% (42) 53.8% (50) 1.1 (1) Pascua Yaqui 18.9% (10) 79.2% (42) 1.9% (1) Lake Havasu City 28.2% (29) 70.9% (73) 1.0% (1) Flagstaff 30.9% (29) 69.1% (65) 0 Sunnyslope (Phoenix) 28.6% (30) 69.5% (73) 1.9% (2) Prescott 42.5% (62) 54.8% (80) 2.7% (4) Casa Grande 36.6% (34) 62.4% (58) 1.1% (1) Mesa 35.4% (40) 59.3% (67) 5.3% (6) Southeast Phoenix 42.6% (43) 56.4% (57) 1.0% (1) Total 25.8% (761) 61.1% (1300) 3.1% (67) 43 Healthy Families Evaluation Report 2004 Ethnicity of Mother by Site - 2004 (Percent and number within Site) Caucasian Hispanic African American Asian American Native American Other Douglas/Bisbee 10.0% (10) 89.0% (89) 0 0 0 1.1% (1) Central Phoenix 26% (27) 57.7% (60) 9.6% (10) 1.9% (2) 1.0% (1) 3.8% (4) Maryvale (Phoenix) 20.7% (18) 70.1% (61) 6.9% (6) 0 1.1% (1) 1.1% (1) South Phoenix 12.3% (9) 68.5% (50) 17.8% (13) 0 0 1.4% (1) East Valley (Phoenix) 35.9% (23) 42.2% (27) 17.2% (11) 1.6% (1) 0 3.1% (2) 0 100% (105) 0 0 0 0 2.1% (1) 4.3% (2) 0 0 91.5% (43) 2.1% (1) Casa de los Niños (Tucson) 26.7% (35) 59.5% (78) 5.3% (7) 0.8% (1) 3.1% (4) 4.6% (6) CODAC (Tucson) 14.3% (16) 74.1% (83) 6.3% (7) 0 3.6% (4) 1.8% (2) La Frontera (Tucson) 13.1% (19) 75.9% (110) 2.1% (3) 1.4% (2) 5.5% (8) 2.1% (3) Child & Family Resources (Tucson) 14.6% (7) 79.2% (38) 4.2% (2) 0 0 2.1% (1) Site Nogales Page 44 Healthy Families Evaluation Report 2004 Caucasian Hispanic African American Asian American Native American Other 40.3% (31) 48.1% (37) 3.9% (3) 0 1.3% (1) 6.5% (5) 0 0 0 0 100% (51) 0 59.8% (49) 36.6% (30) 0 0 3.7% (3) 0 Yuma 5.6% (5) 93.3% (84) 1.1% (1) 0 0 0 Pascua Yaqui 1.9% (1) 9.4% (5) 0 0 64.2% (34) 24.5% (13) Lake Havasu City 53.4% (55) 37.9% (39) 1.0% (1) 1.9% (2) 2.9% (3) 2.9% (3) Flagstaff 16.7% (16) 44.8% (43) 3.1% (3) 0 35.4% (34) 0 Sunnyslope (Phoenix) 50.0% (53) 29.2% (31) 10.4% (11) 0.9% (1) 2.8% (3) 6.6% (7) Prescott 62.6% (92) 36.7% (54) 0 0.7% (1) 0 0 Casa Grande 25.3% (23) 63.7% (58) 6.6% (6) 0 2.2% (2) 2.2% (2) Mesa 46.0% (52) 41.6% (47) 6.2% (7) 0 3.5% (4) 2.7% (3) Southeast Phoenix 21.8% (22) 58.4% (59) 11.9% (12) 0 3.0% (3) 5.0% (5) Total 26.5% (564) 56.0% (1190) 4.8% (103) 0.5% (10) 9.4% (199) 2.8% (60) Site Sierra Vista Tuba City Verde Valley 45 Healthy Families Evaluation Report 2004 Gestational Age by Site - 2004 (Number and Percent within Site) Was the gestational age less than 37 weeks? Site No Yes Douglas/Bisbee 89.8% (88) 10.2% (10) Central Phoenix 79.3% (73) 20.7% (19) Maryvale (Phoenix) 85.9% (67) 14.1% (11) South Phoenix 89.2% (58) 10.8% (7) East Valley (Phoenix) 80.0% (48) 20.0% (12) Nogales 88.6% (93) 11.4% (12) Page 91.3% (42) 8.7% (4) Casa de los Niños (Tucson) 81.5% (97) 18.5% (22) CODAC (Tucson) 85.0% (85) 15.0% (15) La Frontera (Tucson) 82.1% (110) 17.9% (24) Child & Family Resources (Tucson) 84.8% (39) 15.2% (7) Sierra Vista 88.7% (63) 11.3% (8) Tuba City 87.2% (34) 12.8% (5) Verde Valley 85.4% (70) 14.6% (12) Yuma 92.2% (83) 7.8% (7) Pascua Yaqui 95.9% (47) 4.1% (2) Lake Havasu City 89.2% (83) 10.8% (10) Flagstaff 89.7% (78) 10.3% (9) Sunnyslope (Phoenix) 80.2% (77) 19.8% (19) Prescott 94.4% (136) 5.6% (8) Casa Grande 91.1% (82) 8.9% (8) Mesa 71.3% (72) 28.7% (29) Southeast Phoenix 83.7% (77) 16.3% (15) 86.1% (1702) 13.9% (275) Total 46 Healthy Families Evaluation Report 2004 Low Birth Weight by Site (Number and Percent within Site) Did the child have low birth weight (less than 2500 grams or 88 ounces)? Site No Yes Douglas/Bisbee 86.0% (86) 14.0% (14) Central Phoenix 83.5% (86) 16.5% (17) Maryvale (Phoenix) 81.8% (72) 18.2% (16) South Phoenix 87.7% (64) 12.3% (9) East Valley (Phoenix) 78.1% (50) 21.9% (14) Nogales 88.8% (95) 11.2% (12) Page 93.6% (44) 6.4% (3) Casa de los Niños (Tucson) 88.5% (116) 11.5% (15) CODAC (Tucson) 89.4% (101) 10.6% (12) La Frontera (Tucson) 83.4% (121) 16.6 (24) Child & Family Resources (Tucson) 89.6% (43) 10.4% (5) Sierra Vista 88.3% (68) 11.7% (9) Tuba City 86.3% (44) 13.7% (7) Verde Valley 87.8% (72) 12.2% (10) Yuma 94.5% (86) 5.5% (5) Pascua Yaqui 96.2% (51) 3.8% (2) Lake Havasu City 87.4% (90) 12.6% (13) Flagstaff 81.3% (78) 18.8% (18) Sunnyslope (Phoenix) 87.7% (93) 12.3% (13) Prescott 94.5% (138) 5.5% (8) Casa Grande 91.4% (85) 8.6% (8) Mesa 76.6% (85) 23.4% (26) Southeast Phoenix 88.1% (89) 11.9% (12) Total 87.2% (1857) 12.8% (272) 47 Healthy Families Evaluation Report 2004 Yearly Income by Site Median Yearly Income Number Douglas/Bisbee $ 7,200 95 Central Phoenix $ 5,664 77 Maryvale (Phoenix) $ 9,600 65 South Phoenix $ 5,448 55 East Valley (Phoenix) $ 7,500 44 Nogales $ 9,600 100 Page $ 4,800 45 Casa de los Niños (Tucson) $ 7,540 101 CODAC (Tucson) $10,300 94 La Frontera (Tucson) $ 7,500 122 Child & Family Resources (Tucson) $ 9,600 40 Sierra Vista $ 3,300 68 Tuba City $ 6,900 36 Verde Valley $ 7,200 76 Yuma $ 7,200 80 Pascua Yaqui $ 5,124 51 Lake Havasu City $10,800 102 Flagstaff $ 7,200 89 Sunnyslope (Phoenix) $ 5,268 79 Prescott $12,000 44 Casa Grande $ 9,300 52 Mesa $ 6,000 87 Southeast Phoenix $ 5,160 77 Total $ 7,540 1679 Site 48 Healthy Families Evaluation Report 2004 Family Stress Checklist Score by Site - 2004 Mean Score Percent of mothers whose FSC score was greater than 40 Number of mothers whose FSC score was greater than 40 Douglas/Bisbee 39.39 54.0% 54 Central Phoenix 38.80 46.2% 48 Maryvale (Phoenix) 36.31 42.0% 37 South Phoenix 38.49 49.3% 36 East Valley (Phoenix) 36.95 48.4% 31 Nogales 32.71 21.5% 23 Page 35.53 36.2% 17 Casa de los Niños (Tucson) 39.27 49.6% 65 CODAC (Tucson) 36.19 43.4% 49 La Frontera (Tucson) 39.25 53.4% 78 Child & Family Resources (Tucson) 36.49 41.7% 20 Sierra Vista 35.78 35.1% 27 Tuba City 31.18 19.6% 10 Verde Valley 34.76 32.9% 27 Yuma 35.00 34.4% 32 Pascua Yaqui 31.57 25.9% 14 Lake Havasu City 37.86 44.7% 46 Flagstaff 38.49 42.7% 41 Sunnyslope (Phoenix) 39.86 52.8% 56 Prescott 42.07 58.5% 86 Casa Grande 31.24 25.8% 24 Mesa 37.23 43.4% 49 Southeast Phoenix 36.34 45.5% 46 Total 37.05 42.9% 916 Site 49 Healthy Families Evaluation Report 2004 Appendix B Parenting Stress Index Information Reliabilities for Current Study Subscale Competence Attachment Restricted Role Depression Isolation Distractibility Mood Alpha .70 .60 .74 .78 .73 .50 .67 Change in Parenting Stress Index Subscales Scores from baseline to 6 months Baseline Subscale 6 months Significance Mean SD Mean SD t Competence 31.5 6.2 29.7 6.2 7.17*** Attachment 12.9 3.6 12.4 3.7 3.62*** Restricted role 19.6 5.3 18.9 5.5 3.13* Depression 20.8 6.1 19.7 6.1 4.82*** Isolation 14.5 4.7 13.8 4.8 3.56*** Mood 10.4 3.0 9.1 2.9 9.01*** Note: * p<.01, ** p<.001, *** p<.000, dependent t-tests, SD=Standard Deviation. 50 Healthy Families Evaluation Report 2004 Change in Total Parenting Index Scores from baseline to 6 months Baseline 6 months Significance Subscale Mean SD Mean SD t Total Stress Score 137.0 25.5 128.9 24.9 7.25*** *** p<.000 Change in Parenting Stress Index from baseline to 12 months Baseline Subscale 12 months Significance Mean SD Mean SD t Competence 31.6 6.2 29.3 6.0 7.07*** Attachment 12.8 3.6 12.0 3.4 4.29*** Restricted role 19.5 5.2 18.6 5.2 3.27** Depression 20.9 6.1 19.2 5.9 5.69*** Isolation 14.5 4.5 13.6 4.5 3.97*** Mood 10.5 3.0 9.4 2.9 6.36*** Note: * p<.01, ** p<.001, *** p<.000, dependent t-tests, SD=Standard Deviation. Change in Total Parenting Index Scores from baseline to 12 months. Baseline 12 months Significance Subscale Mean SD Mean SD t Total Stress Score 135.5 23.4 127.3 23.5 7.55*** *** p<.000 51 Healthy Families Evaluation Report 2004 Range and Reliability of the Parenting Stress Index (PSI) (Selected subscales for original reliabilities analysis) Subscales Sense of Competence Parental Attachment Rangea Alpha Coefficient 13 - 65 7 - 35 .77 .64 Administration Administered at 3 weeks, 6 months, and 18 months Role Restriction 7 - 35 .74 Depression 9 - 45 .75 Social Isolation 6 - 30 .69 Mood 5 - 25 .70 Distractibility 9 - 45 .82 Total Scoreb 78-390 .85 a A higher score on each of the subscales represents a higher degree of stress in that area. b The total score on the Parenting Stress Index is computed by summing all of the subscales, with a higher score indicating more stress. 52 Healthy Families Evaluation Report 2004 Description of Parenting Stress Index Subscales Sense of Competence Subscale: Assesses the parent’s sense of competence in relation to his or her role as parent. It relates to knowledge of how to manage the child’s behavior and comfort in making decisions such as when and how to discipline the child. Parental Attachment Subscale: Assesses the intrinsic investment the parent has in the role of parent. This subscale was expected to determine the parent’s motivation level to fulfill the role of parent. Restrictive Role Subscale: Assesses the negative impact, losses, and sense of resentment associated with the parent’s perceptions of loss of important life roles. Depression Subscale: Assesses the extent to which the parent’s emotional availability to the child is impaired and the extent to which the parent’s emotional and physical energy is compromised. Isolation Subscale: Examines the parent’s social isolation and the availability of social support for the role of parent. Distractibility Subscale: Assesses the degree to which the child displays many of the behaviors associated with Attention Deficit Disorder with Hyperactivity and other behaviors which result in a continuous drain on the parents’ energy, which requires not only active parental management but also sustained high states of vigilance. Mood Subscale: Assesses child characteristics related to excessive crying, withdrawal, and depression. The parent usually experiences these behaviors as anxiety or anger provoking. 53 Healthy Families Evaluation Report 2004 Appendix C Family Stress Checklist Family Stress Checklist Problem Areas and Interpretation (Mother & Father) Problem Areas I. Childhood history of physical abuse and deprivation. II. Substance abuse, mental illness, or criminal history. III. Previous or current CPS involvement. IV. Self-esteem, available lifelines, possible depression. V. Stresses, concerns. Range Interpretation/ Administration The FSC is a 10 item rating scale. 0, 5, or 10 A score of 0 represents normal, 5 represents a mild degree of the problem, and a 10 represents severe, on both the Mother and 0, 5, or 10 Father Family Stress Checklist items. The FSC is an assessment tool and is administered to the 0, 5, or 10 mother through an interview by a Family Assessment Worker from 0, 5, or 10 the Healthy Families Arizona Program. The interview takes 0, 5, or 10 place shortly after birth, or as near VI. Potential for violence. 0, 5, or 10 to that time as possible. VII. Expectations of infants’ milestones, behaviors. 0, 5, or 10 VIII. Discipline of infant, toddler, and child. 0, 5, or 10 IX. Perception of new infant. 0, 5, or 10 X. Bonding, attachment issues. 0, 5, or 10 A score over 25 is considered medium risk for child abuse and Total Score neglect, and a score over 40 is 0 - 100 considered high-risk for child abuse. 54 Healthy Families Evaluation Report 2004 Appendix D Healthy Families Parenting Inventory Healthy Families Parenting Inventory Alpha Scores Subscale Alpha* Social support r=.87 Problem solving r=.92 Depression r=.79 Personal care r=.69 Mobilizing resources r=.83 Accepting the parent role r=.72 Parent child behavior r=.87 Home environment r=.84 Parent competence r=.86 Parenting efficacy r=.89 *Alpha score represents the correlation of items on a scale, and is an indication of how well the items in a subscale relate to each other. 55 Healthy Families Evaluation Report 2004