Healthy Families Arizona Longitudinal Evaluation 4th Annual Report 2008 Prepared by: LeCroy & Milligan Associates, Inc. 4911 E. Broadway, Suite 100 Tucson, Arizona 85711 (520) 326­5154 FAX (520) 326­5155 www.lecroymilligan.com Prepared for: Arizona Department of Economic Security Division of Children, Youth & Families Office of Prevention and Family Support 1789 W. Jefferson, Site Code 940A Phoenix, Arizona 85007 Table of Contents Executive Summary........................................................................................................4 Findings from the Child’s First Year ............................................................................11 I. Child Maltreatment................................................................................................11 II. Parenting Practices ...............................................................................................16 III. Substance Use .....................................................................................................18 IV. Mental Health .....................................................................................................20 V. Parenting Knowledge and Attitudes ....................................................................22 VI. Participation in Community Services .................................................................23 VII. Family Violence ................................................................................................24 VIII. Income and Financial Hardship .......................................................................27 IX. Child Health........................................................................................................28 X. Participation in Healthy Families Arizona...........................................................30 Conclusion ....................................................................................................................31 Recommendations.....................................................................................................34 Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 1 List of Tables Table 1. Percentage of Mothers Reporting Never Using Select Disciplinary Practices with their Infants from Six Months to One Year-of-Age ................13 Table 2. Mothers Self-Reported History of Childhood Maltreatment and Prior CPS Involvement.............................................................................................16 Table 3. Substance Use among Mothers.......................................................................19 Table 4. Average Parenting Attitude Scores.................................................................23 Table 5. Indicators of Mother and Partner Violence.....................................................25 Table 6. Perceived Financial Hardship Over the First Year .........................................28 Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 2 Acknowledgements Our sincere thanks are extended to the Arizona Department of Economic Security (ADES) for their commitment to this evaluation, with a special thank you to Karen Bulkeley, Manager, Office of Prevention and Family Support. Thank you to Dr. Alyson Shapiro from the ASU School of Social and Family Dynamics, who has shared her expertise on couple relationships and early child development. We would also like to acknowledge the support of the staff of the participating Healthy Families Arizona sites, and the leadership of Julie Dale, Lupe Ricardez, Pauline Haas­Vaughn, and Merielle Robinson of Child and Family Resources, Inc., in Tucson. Thank you also to the families who have shared their time and their stories so that we might learn more about effective strategies in the prevention of child maltreatment. Members of the LeCroy & Milligan Associates, Inc., evaluation team who were primary contributors to this report include Judy Krysik, PhD, Craig LeCroy, PhD, Allyson Baehr, BA, Olga Valenzuela, BA, Melissa Nelson, and Cindy Jones, BSBA. Suggested Citation: LeCroy & Milligan Associates, Inc. (2008). Healthy Families Arizona Longitudinal Evaluation – 4th Annual Evaluation Report. LeCroy & Milligan Associates, Inc. Tucson, AZ. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 3 Executive Summary Healthy Families Arizona is a voluntary, home visitation program, aimed at the prevention of child abuse and neglect. This secondary prevention program is targeted to the parents of newborns with risk factors that make them vulnerable to child abuse and neglect, and at risk of parent/child relationship and child development problems. The risk factors that qualify parents for the program include parental history of abuse and neglect; substance abuse; mental health problems; poor coping skills; a lack of social support; unrealistic developmental expectations of infants and toddlers, difficulty with bonding and attachment; attitudes favorable toward harsh discipline; anger management issues; and a lack of resources to meet basic needs. Modeled on the Healthy Families America program, Healthy Families began in Arizona in 1991 with two sites. Administered by the Arizona Department of Economic Security, Healthy Families Arizona has realized considerable growth over the years, and is now available to families statewide. As with any considerable investment in cost and effort, the question of return is central to ongoing support of the Healthy Families Arizona program. Does the Healthy Families Arizona program (a) reduce the risk of child abuse and neglect; (b) lead to better health and development for children, and (c) does it enhance parent/child relationships? The longitudinal evaluation of Healthy Families Arizona was designed to answer these questions. In the longitudinal evaluation, 98 families were randomly assigned to the Healthy Families Arizona program, and 97 families were randomly assigned to a control group that does not receive Healthy Families Arizona services. All 195 participating families voluntarily agreed to be involved in the longitudinal evaluation for a period of five years, with the option to withdraw at any time without consequence. Data were collected from the families upon entry to the evaluation, and was thereafter scheduled to follow the age of the child at 6­months, one, two, three, four, and five years. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 4 The longitudinal evaluation is now collecting third­year data and second­year data collection is scheduled for completion in November 2008. The data are gathered in the participants’ homes. All of the questions are read to the participants by one of two trained research assistants who have worked with the longitudinal evaluation since it began. The participant interviews are conducted in English or Spanish, according to the participant’s preference. In this way, the longitudinal evaluation is representative of the families that the Healthy Families Arizona program serves, unlike some evaluations of home visitation programs that have excluded participants who did not speak English well (e.g., the evaluation of Healthy Families Alaska by Duggan, 2005). The longitudinal evaluation differs from the annual evaluations of the Healthy Families Arizona program in the following three ways: 1. The annual evaluation of Healthy Families Arizona does not follow­up with families once they leave the program and therefore cannot assess long­term outcomes. Participation in the longitudinal evaluation for the group receiving Healthy Families Arizona was designed to continue for five years, regardless of whether or not participating families terminate their involvement in Healthy Families Arizona. In this way, the longitudinal evaluation was designed to focus on children from birth to five years of age, the period that children are known to be the most vulnerable to child abuse and neglect. As of August 16, 2007, 49 families had terminated their involvement in the Healthy Families Arizona program. The average time to termination was just under 8 months, 233 days, and ranged from a minimum of 74 days to a maximum of 533 days (1.5 years). 2. The annual evaluation does not make any comparisons to a control group, a group that does not receive Healthy Families Arizona services. The annual evaluation, therefore, does not allow for an assessment of what the outcome would have been in the absence of Healthy Families Arizona. The longitudinal evaluation, in contrast, used random assignment to groups to create a control group. In theory, random assignment creates groups that are equivalent in important ways that might impact the outcomes. Random assignment to groups is the only research design that allows for an assessment of the effects of the program. Similar to the annual evaluation, the longitudinal evaluation assesses changes within the families over time, however it adds a comparison Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 5 across the two randomly assigned groups, those receiving Healthy Families Arizona services and those in the control group. Thus, the longitudinal evaluation offers information on what changes in families can be attributed to the Healthy Families Arizona program. 3. The longitudinal evaluation includes a variety of measures that examine factors not addressed in the scope of the annual evaluation: e.g., mental health, domestic violence, discipline, parent­child attachment, child behavior, quality of the home environment, and cognitive development. The longitudinal evaluation assesses school readiness, beginning in the third year of data collection and continuing to age five. The purpose of these additional measures that are not used in the annual evaluation is to assess the full­range of risk and protective factors related to child abuse and neglect, and to examine potential program outcomes that are not otherwise considered. The outcome of the longitudinal evaluation, which is restricted to participating Healthy Families Arizona sites in Pima County, has local as well as statewide significance. Pima County was chosen over other sites in Arizona for its stability. When the longitudinal evaluation began in 2005, Healthy Families Arizona was in a period of rapid expansion. Even though Pima County added new sites, there was a transfer of experienced supervisors and home visitors to the new sites. Although transition poses a challenge to evaluation, the assessment of outcomes is based in a realistic community context and therefore the results have greater generalizability than outcomes obtained in a highly controlled setting. As Arizona has long been recognized as a leader in the Healthy Families America model of home visitation, this evaluation has national as well as local relevance. Update on the Status of the Longitudinal Evaluation In December 2006, a final sample of 97 families was recruited to the control group and 98 families were recruited to the Healthy Families Arizona group. The 2007 evaluation report described the evaluation participants based on information collected at enrollment to the evaluation or ‘baseline.’ Analysis of the baseline data suggested that the two groups were statistically similar on several measures; however, there were also some important differences. The Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 6 98 mothers in the Healthy Families Arizona group are on average about two years younger than the 97 mothers in the control group. Significantly fewer of the Healthy Families Arizona mothers compared to mothers in the control group reported receiving prenatal care prior to the birth of the infant that made them eligible for the evaluation, and more were covered by AHCCCS, the Arizona Medicaid program, than were covered by other types of health insurance. Overall, the control group participants had significantly higher incomes than the Healthy Families Arizona participants, and appeared to have greater financial potential. Indicators such as increased rates of employment, vehicle ownership, driver licenses, and monthly income evidenced the existence of increased protective factors in the control group. Interestingly, however, there were no significant differences in parenting attitudes at baseline. The group differences imply that the Healthy Families group is at greater risk than the control group for child abuse and neglect, and parent/child relationship and child development problems. Summary of the Findings This is the first of the Healthy Families Arizona longitudinal evaluation reports to address outcomes. The purpose of this report is to describe differences in outcomes between the Healthy Families Arizona group and the control group at 6 and 12 months. Statistical significance suggests that the difference is of sufficient magnitude that it is unlikely to have occurred by chance, and can therefore be considered a program effect. Sample size impacts the likelihood of finding statistically significant results; small differences will be statistically significant in large samples, whereas small samples require relatively large differences to achieve significance. Thus, some program impacts of a conservative magnitude are likely to be missed in this evaluation because of the relatively small sample size of 195 families. The items reported below were shown to be statistically significant across groups or over time from baseline to 6 or 12 months. The following differences are attributable to the Healthy Families Arizona program. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 7 1. Healthy Families Arizona mothers reported a significant decrease in violent behaviors (such as shouting, yelling, or screaming at their infants) between 6 and 12 months of age compared to the control group mothers. 2. A significant decrease in strong parental belief in corporal punishment was realized in the first 6 months of the Healthy Families Arizona group, while there was no like decrease at any time period in the control group. Furthermore, attitudes favorable toward corporal punishment increased from 6 to 12 months in the control group, although not significantly. 3. Healthy Families Arizona mothers experienced a significant increase in satisfaction with parenting from baseline to 6 months compared to a non­significant decrease in satisfaction with parenting in the control group over the same period. 4. Healthy Families Arizona mothers had significantly greater acceptance of the infant’s behavior at six months than control group mothers, indicating the degree to which the parent expresses frustration with the infant, as well as employs physical discipline and criticism of the infant. 5. Significantly fewer Healthy Families Arizona mothers reported drinking beer or alcohol at 6 months than control group mothers. 6. Healthy Families Arizona mothers experienced a significant decrease in emotional loneliness from baseline to six months, emotional loneliness increased during this period for the control group, but not significantly. 7. Healthy Families Arizona mothers experienced a significant increase from baseline to 12 months in hope­related agency, i.e., the belief that they themselves are capable of pursuing their goals successfully. Control group mothers experienced a significant decrease from baseline to 12 months in hope­related pathway, i.e., the ability to imagine or plan ways to achieve their goals. A significant decrease in pathway in the control group was also observed from 6 to 12 months. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 8 8. Significantly more Healthy Families Arizona mothers than control group mothers used the services of a public health nurse, mental health treatment counseling, and free diapers during the period from baseline to six months. In the 6 to 12 month period, more Healthy Families Arizona mothers than control group mothers used the food bank, mental health treatment counseling, and free diapers. The average number of community services used by Healthy Families mothers was significantly greater than the average number of services used by control group mothers in both 6­month and 12­month time periods. 9. Although the control group had significantly higher income than the Healthy Families Arizona group, perceived difficulty in paying rent or mortgage decreased significantly in the Healthy Families Arizona group from baseline to 12 months, whereas the perceived difficulty of paying rent/mortgage increased significantly in the control group over the same time period. 10. Significantly more Healthy Families Arizona infants had siblings with health insurance at 12 months compared to the control group. 11. The safety practices of posting the telephone number for poison control near or on the phone and turning down the water heater to less than 120 degrees were greater in the Healthy Families group than the control group at 6 months. Only one between­group difference favored the control group. This was significantly higher scores on the emotional and verbal responsivity subscale at 6 months. This subscale refers to the manner in which the mother interacts with the child through verbalization, praise, and non­verbal communication such as caressing and kissing. Although the control group scored significantly higher on this subscale than the Healthy Families group, the between­group difference was not significant once baseline differences in income were controlled. It should also be noted that this subscale was scored based on observing the mother and child during the interview that lasted approximately one hour. The interactions observed during this time may not have been typical for the family. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 9 Healthy Families Arizona home visitors are trained to promote nonviolent discipline practices, teach families coping and stress reduction techniques, provide social support, connect families with community resources, and are a source of child development and child safety information. The outcomes achieved by the Healthy Families Arizona program are consistent with the intent and practices of the program, and bode well for the three overarching goals of decreased child abuse and neglect, improved parent/child relationships, and enhanced child development. It is important to note that the statistically significant differences were found in light of the small sample size and greater pre­intervention (baseline) risk for child abuse and neglect in the Healthy Families Arizona group compared to the control group. The Healthy Families Arizona group had significantly higher rates of CPS involvement as parents prior to the evaluation, scored higher on history of childhood abuse and neglect, and had lower incomes prior to entering the longitudinal evaluation when compared to the control group. Maintenance of the first­year outcomes, as well as the achievement of additional outcomes will be examined in subsequent longitudinal evaluation reports. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 10 Findings from the Child’s First Year This report organizes findings from the longitudinal evaluation into 10 domains: child maltreatment, parenting practices, substance use, mental health, parenting knowledge and attitudes, participation in community services, family violence, income and financial hardship, child health, and participation in Healthy Families Arizona. I. Child Maltreatment Previous evaluations of Healthy Families Arizona, and of child serving programs for at­risk children in general, have tended to rely on child protective services (CPS) substantiated reports of child abuse and neglect to represent child maltreatment. The use of this measure to represent child abuse and neglect is limited in several ways. Perhaps most important to the assessment of child maltreatment is that many children who experience abuse and neglect do not come to the attention of CPS, and when they do, there is often a lack of evidence to substantiate the reports. This is especially true for infants and toddlers who do not have the verbal ability to communicate abuse, and who are relatively isolated in comparison to older children who attend school, and who have greater exposure outside of their homes. To provide a broader assessment of child maltreatment, mothers in the longitudinal evaluation were asked about their use of various discipline practices, as well as asked about the methods of discipline that other caretakers of their infants employ. Other caretakers were defined as mother’s husband or partner and other adults living in the home. Mothers were also asked about their involvement with CPS and law enforcement regarding the care of their infants. The trained research assistants also administered measures regarding the quality of the child’s home environment and home safety practices. Official CPS reports of child abuse and neglect as well as substantiated incidents will be examined in the coming year. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 11 Disciplinary Practices Table 1 provides information on maternal reports of aggressive verbal and corporal disciplinary practices. The data in the table represent the percentage of mothers who reported ‘never using’ select disciplinary practices with their infants from 6 months to one year­of­age. The information presented in Table 1 shows that the control group mothers were more likely to use each type of violent discipline than those mothers enrolled in Healthy Families Arizona. The group differences were statistically significant for verbal aggression including shouting, yelling and screaming at the infant, and for minor corporal aggression such as slapping the child’s hand. There were also group differences in the number of mothers’ reporting shouting, yelling and screaming practices used by caretakers, with a statistically lower proportion of the Healthy Families Arizona group caretakers never using the practice compared to caretakers in the control group. Acts of major physical aggression were rare in both groups. For instance, none of the mothers reported shaking their infants; shoving or pushing; or hitting infants with an object such as a belt, ruler, etc. The data were also analyzed for frequency of disciplinary practices used. The assessment of frequency is based on a four­point scale including ‘0 = never;’ ‘1 = rarely;” “2 = sometimes;” and ‘3 = often.’ The overall frequency of aggressive discipline practices with infants from 6 months to 1 year of age was significantly less in the Healthy Families Arizona group than in the control group (an average of 1.56 for Healthy Families Arizona compared to 2.19 for the control group). The overall frequency of aggressive discipline administered by caretakers was also higher in the control group, however the difference was not statistically significant (1.04 compared to 1.29). Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 12 Table 1. Percentage of Mothers Reporting Never Using Select Disciplinary Practices with their Infants from Six Months to One Year-of-Age Disciplinary Practices Healthy Families Control Shouted, yelled, or screamed at child* 50.6% 34.1% Called name or cursed at child 96.4% 94.1% Threatened to smack or hit, but did not 69.5% 63.5% Withheld food or water from child 100% 98.8% Left child alone in house or car 100% 98.8% Slapped his or her hand* 56.6% 38.8% Spanked on bottom with bare, open hand 71.1% 65.9% Hit child elsewhere on body with hand 98.8% 96.5% Slapped his or her face 100% 97.6% Threw an object at child 100% 98.8% Pinched child 98.8% 94.1% Note. * Represents a statistically significant difference at p < .05, indicating a positive Healthy Families Arizona program effect. The longitudinal evaluation also asked mothers about their use of three non­ aggressive disciplinary practices: time out, taking away toys or privileges, and explaining why something was wrong. It was clear that most mothers did not use these tactics with their infants. There is likely to be an increase in these non­aggressive types of disciplinary methods at the 2, 3, 4, and 5­year old data collection points. When asked for other forms of disciplinary practices used that were not included on the list, eight Healthy Families Arizona mothers and two control group mothers reported the practice of “saying no and redirecting the infant. “ The findings on maternal discipline are consistent with the philosophy and curriculum of the Healthy Families America model that advocates the use of nonviolent discipline. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 13 Parental Stress Healthy Families Arizona home visitors offer parents techniques to reduce stress. Parental stress can materialize as frustration and is a risk factor for child abuse and neglect. Interestingly, Healthy Families Arizona mothers reported feeling less frustrated with their children than did control group mothers at 12­months. Parental frustration was measured with a single item “parenting often makes me feel frustrated,” and mothers were asked to report their degree of agreement with the item on a four–point response scale. Another factor that can increase parental stress and leave children vulnerable for child abuse and neglect is rapid, repeat pregnancies and births. Equally high proportions in both groups, 73%, of women reported that the birth of the infant that made them eligible for the longitudinal evaluation was not planned. In terms of repeat pregnancies, 10 Healthy Families Arizona mothers and 10 control group mothers reported another pregnancy during the first year of the study. Of the 10 Healthy Families mothers, 8 of the 10 were experiencing their second consecutive unplanned pregnancy. Satisfaction with the Parenting Role Satisfaction with the parental role was measured by the Being a Parent scale. This is an adaptation of the Parenting Sense of Competence Scale (Gibaud­ Wallston & Wandersman, 1978), which assesses parenting satisfaction and parenting efficacy. The parenting efficacy scale did not have sufficient reliabilities to be useful in the analysis; however the satisfaction scale performed better. Healthy Families mothers significantly increased their satisfaction with parenting from baseline until the child reached 6­months of age. In comparison, control group mothers’ satisfaction with parenting decreased from baseline to six months, although not by a significant amount. Satisfaction with parenting increased for mothers in both groups from birth until the child was 12­months of age. What this suggests is that critical change in satisfaction with the parenting role can occur early in the Healthy Families Arizona program. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 14 Involvement with Child Protective Services and Law Enforcement Although referral to Healthy Families Arizona by CPS was an exclusion criterion for enrollment in the longitudinal evaluation, it was learned through interviews with some of the mothers who terminated their involvement in Healthy Families Arizona prior to the child’s first birthday, that some did not perceive their participation as voluntary, but rather as required by CPS. This was unfortunate in terms of the longitudinal evaluation, as it biased the Healthy Families Arizona group in terms of increased risk for continued CPS involvement. Approximately 25% of Healthy Families mothers reported prior involvement with CPS as a parent, compared to only 12% of the control group. As shown in Table 2, mothers in the Healthy Families Arizona group reported more abuse in childhood than did mothers in the control group, especially related to emotional and physical types of abuse. Additionally, more Healthy Families mothers reported involvement with Child Protective Services (CPS) as a parent, an important predictor of subsequent CPS reports of child abuse and neglect. A history of childhood maltreatment has been shown to be a risk factor for abusive and neglectful behaviors toward children (Belsky, 1993; Murphy, Orkow, & Nicola, 1985; Renner & Slack­Shook, 2004). Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 15 Table 2. Mothers Self-Reported History of Childhood Maltreatment and Prior CPS Involvement Healthy Families Control (n = 98) (n = 97) Neglected by parents or caretakers 24.7% 21.6% Experience emotional abuse 33.0% 19.6% Experienced physical abuse 30.9% 18.6% Experienced sexual abuse 24.7% 21.6% Received therapy to deal with history of abuse 25.8% 19.8% Any involvement with CPS as a parent 24.7% 11.3% Characteristics of Abuse Similar proportions of each group reportedly came into contact with law enforcement as a result of concerns for the well­being of their children between 6­months and 1­year of age. Most of these contacts involved a partner or ex­partner (7% of the Healthy Families Arizona group and 6% of the control group). Only one mother, a Healthy Families Arizona participant, had contact with law enforcement over the care of her child. Child Safety Practices Mothers were asked about their use of 25 safety practices at 6 and 12 months. The reported use of child safety precautions were similar between the two groups with the exception of having the telephone number for poison control posted near or on the phone, and turning the water heater down to less than 120 degrees. The Healthy Families Arizona group had significantly more families taking these safety precautions at six months (73% compared to 42% for poison control; and 40% compared to 26% for hot water). II. Parenting Practices Although the previous section examined parenting behaviors that are considered potentially harmful to the infant, this section examines positive parenting behaviors that assist in developing a positive parent/child Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 16 relationship. A healthy relationship between the parent and their new child is a desirable characteristic that is associated with school readiness and healthy child development. The first measure reported in this section is the Infant/Toddler Home Inventory (HOME). The HOME includes six subscales that relate to quality parenting practices. The HOME was administered to the participants of the longitudinal evaluation when the infants were 6 months of age and is based on a one hour observation of the family during the interview. Not all of the subscales were demonstrated reliable, even after deleting some of the poorer performing items. Typically, researchers consider an Alpha of .70 and above as acceptable. 1. Emotional and verbal responsivity relates to the manner in which the parent interacts with the child through verbalization, praise, and non­ verbal communication such as caressing and kissing. Cronbach’s alpha for this scale was .78. when controlling for income differences between the groups there was no significant difference between the Healthy Families group and the control group. The control group scored significantly higher on this subscale than the Healthy Families group indicating greater maternal verbal responsivity, however, the between­ group difference was not significant when income was controlled. 2. The acceptance of child’s behavior subscale relates to the degree that the parent expresses frustration with the child, and includes items on physical discipline and criticism. The alpha for this scale was .61. The Healthy Families group scored significantly higher than the control group at 6 months, indicating greater acceptance of the child’s behavior. 3. Organization of the environment relates to the safety of the play environment, appropriate substitute care, and space for toys and treasures. The alpha for this scale was .15. The group difference for this subscale is not reported due to poor reliability. 4. Provision of play material relates to the age appropriateness of the child’s toys. The alpha for this scale was .68. There was no significant between­group difference on this subscale. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 17 5. Parental involvement with the child relates to the extent to which the parent involves his or her self with the child. The alpha for this scale was .80. No significant between­group difference was found on this subscale. 6. Opportunities for variety include father involvement, reading to child and reading materials, eating together, family visits. The alpha on this scale was .61. No significant between­group difference was found on this subscale. The percentage of mothers who read to their infant daily was considerably higher in the Healthy Families Arizona group than in the control group at 6 months (35.2% compared to 25.3% control). This difference was not statistically significant, but would likely be significant with a larger sample size. III. Substance Use Research suggests that parents who abuse drugs and alcohol are also more likely to have children who experience child abuse and neglect (National Clearinghouse on Child Abuse and Neglect, 2004; Windham et al., 2004). There are various ways that parental substance abuse can impact the safety and health of children (Chaffin, Kelleher, & Hollenberg, 1996; Dubowitz & Black, 2002; Tanner & Turney, 2003). According to Donohue (2004), mothers who abuse substances spend less time with their children, are inconsistent with discipline, are more likely to be socially isolated, and are more likely to leave their children unsupervised. Substance abusing parents may be emotionally or physically unavailable to their children, increasing the risk for accidental injuries and abuse by others. Heavy drug use can interfere with the parent’s ability to provide consistent and nurturing care, and can interfere with limit setting that promotes children’s development and protects against behavior problems. Substance­abusing parents may also divert money for basic needs such as housing, food, and utilities away from the family to support their substance use (Munkel, 1996). Parental substance abuse can also interfere with the parent’s ability to maintain employment and may increase the parent’s involvement with the criminal justice system, further limiting his or her ability to provide support for the family (Magura & Laudet, 1996). Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 18 Finally, children living with substance abusing parents are more likely to become intoxicated or ingest harmful chemicals, either deliberately or by passive inhalation or accidental ingestion, and are more likely to be exposed to criminal behavior and weapons (Munkel, 1996). Table 3 displays information on mothers’ self­reported substance use characteristics at baseline, 6, and 12 months. About five percent of the Healthy Families Arizona group was involved in drug or alcohol treatment at baseline, and similar proportions of each group had received drug or alcohol treatment in the past. The difference at baseline might be attributable to Healthy Families Arizona, as the home visitor met with the mothers prior to the mothers’ first meetings with the research assistants for the longitudinal evaluation. The one significant between­group difference found a greater proportion of control group mothers reporting beer and alcohol use at 6 months (35.2% in the control group compared to 16.5% in the Healthy Families Arizona group). Table 3. Substance Use among Mothers Substance use Healthy Families Control Entry 6 mo. 12 mo. Entry 6 mo. 12 mo. Drink beer or alcohol 14.4% 16.5%* 20.5% 19.6% 35.2%* 29.1% Smoke marijuana 2.1% 4.4% 2.4% 1.0% 1.1% 2.3% Use tobacco 22.7% 28.6% 28.9% 15.5% 26.1% 22.1% Currently receiving drug/alcohol treatment 5.2% 3.3% 2.4% 0 1.1% 0 Ever received drug/alcohol treatment 9.4% NA NA 10.3% NA NA Perceived need for drug/alcohol treatment 3.2% 3.3% 1.2% 1.0% 1.1% 0 Note. * Represents a statistically significant difference at p < .05. No mothers in either group reported current drug use other than marijuana at any time period. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 19 IV. Mental Health Two scales were administered at baseline to describe maternal depression: these were the Center for Epidemiologic Studies Depression Scale (CES­D), and the Mental Health Index. The CES­D is a commonly used measure of depression, with scores ranging from 0 through 60 for the 20­item version. Higher scores represent greater levels of depression, and scores greater than or equal to 16, suggest clinically significant levels of psychological distress. Scoring above 16 does not necessarily equate to a clinical diagnosis of depression. In the general population, about 20% would be expected to score in this range. The average CES­D score for the Healthy Families Arizona group was 14.6, compared to a slightly lower average score of 12.5 for the control group. The between­group differences were not statistically significant. It is important to note, however, that 35% of the Healthy Families Arizona group and 30% of the control group scored above the clinical cutoff, suggesting a clinically significant problem with maternal depression at baseline. The first year outcome report was intended to answer the question ‘Did maternal depression scores change overtime, and in what direction?’ however; the Mental Health Index that was used at baseline, 6 and 12­months did not prove to be reliable. The CES­D will be administered again at 24 months. The baseline scores on the CES­D and the Emotional Social Loneliness Inventory (ESLI) emotional loneliness subscale were highly correlated (r = .52, p. < .001), suggesting that maternal depression and feelings of emotional loneliness go hand­in­hand. Although the Healthy Families Arizona group scored significantly higher on emotional loneliness at baseline than the control group, the between­group difference was not significant once income was statistically controlled. No between­group difference in emotional loneliness was observed at 6 months. The Healthy Families Arizona group experienced a statistically significant decrease in emotional loneliness from baseline to 6 months. For the control group, emotional loneliness increased during this time period, although not significantly. The Emotional Social Loneliness Inventory was not administered at 12 months. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 20 The Adult Dispositional Hope Scale (a.k.a. the Goals Scale) was administered at baseline, 6­ and 12­months. The Goals scale measures attitudes about goal setting and problem solving, asking the individual’s extent of agreement with statements such as ‘I meet the goals I set for myself,” “I can think of many ways to get out of a jam, “ and “I can think of many ways to get things in life that are most important to me.” The goals scale is divided into two subscales, pathway and agency. Hope theory contends that individuals who can imagine or plan ways to achieve their goals, i.e., pathway, have greater hope. Agency, in contrast, refers to individuals who think that they themselves are capable of pursuing goals successfully, those who believe in their own capacity to get what they want and are more hopeful. Reliabilities for the two subscales, pathway and agency, at each of the three data collection periods, and the total scale score ranged from .72 to .85, indicating sufficient to good reliability. Analysis of the two subscales and the total scale score revealed no statistically significant between­group differences at any point. As expected, given the group differences in baseline characteristics such as income and prior CPS involvement that favored the control group, the control group scored higher than the Healthy Families Arizona group on both subscales and the total score at baseline indicating greater hope. What was observed over the three time periods, however, was a significant decrease in pathway for the control group from baseline to 12 months, and from 6 to 12 months, with a like decrease in the total scale score. The Healthy Families Arizona group, in contrast, experienced a significant increase in agency from baseline to 12 months. All of the Healthy Family Arizona subscale and total scale scores increased at each time period, although not significantly, with the one exception in agency. At 12 months, the average Healthy Families Arizona score was higher than the average control group score on subscales pathway and agency, and the total scale score that represents the concept of hope. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 21 V. Parenting Attitudes The Adult­Adolescent Parenting Inventory­2 (AAPI­2) is an inventory designed to assess the parenting and child rearing attitudes of adult and adolescent parents. The five subscales of the AAPI­2 represent attitudes that are reflective of parental behaviors known to be associated with risk for, and protection against, child abuse and neglect. For instance, a parent scoring low on the subscale titled inappropriate expectations of child is demonstrating attitudes consistent with positive parenting behaviors. In contrast, a high score on the subscale represents attitudes associated with risk of parental behaviors that are related to child abuse and neglect. Average scores can range from one to five, and higher scores on each subscale represent greater problem levels. The five AAPI­2 subscales are as follows: • • • • • Inappropriate Expectations of Children Parental Lack of Empathy towards Children’s Needs Strong Parental Belief in Corporal Punishment Reversing Parent­Child Family Roles Oppressing Children’s Power and Independence There were no statistically significant differences between the two groups on any of the five AAPI­2 average subscale scores at any of the three points, as shown in Table 4. There was, however, a significant decrease at each time point on three of the subscales – expectations of child, empathy, and role reversal. A significant decrease in strong parental belief in corporal punishment was realized in the first 6 months of the Healthy Families Arizona group; however, there was no like decrease at any time period in the control group. The trend in the control group was for attitudes toward corporal punishment to increase from 6 to 12 months. These findings are consistent with those on satisfaction with parenting, in which Healthy Families Arizona mothers also realized a significant increase from baseline to 6 months compared to a non­significant decrease in satisfaction with parenting in the control group over the same period. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 22 Table 4. Average Parenting Attitude Scores Healthy Families Control Subscale Entry 6 mo. 12 mo. entry 6 mo. 12 mo. Expectations of child 3.33 3.05* 2.77* 3.33 2.88* 2.77* Lack of empathy to child’s needs 2.45 1.95* 1.84* 2.37 1.94* 1.78* Belief in corporal punishment 2.49 2.27* 2.22 2.36 2.15 2.23 Role reversal 2.98 2.60* 2.43* 2.80 2.48* 2.26* Note. * Represents a statistically significant difference at p < .05. Higher scores reveal greater problem levels. Scores represent the average on a 5-point scale. The Power and Independence subscale score was not calculated due to extremely low reliability. VI. Participation in Community Services Mothers were questioned on their perceived ability to ask for assistance and meet their needs through external resources. The reliabilities for the Mobilizing Resources subscale of the Healthy Families Parenting Inventory was .80 at baseline and .76 at 6 months, indicating very good reliability. Although there were no between­group differences on perceived ability to mobilize resources, there were considerable group differences in the use of community services. From baseline to six months, significantly more mothers in the Healthy Families Arizona group compared to the control group had used the services of a public health nurse (13% compared to 3.4%); mental health treatment counseling (10.9% compared to 3.4%); and free diapers (29.3% compared to 2.3%). There was no between­group differences at 6 months on any of the other community resources surveyed. Between 6 and 12 months of age, significantly more mothers of infants in the Healthy Families Arizona group compared to the control group had used the services of a food bank (19.3% compared to 7%); mental health treatment counseling (8.4% compared to 1.2%); and free diapers (21.7% compared to 3.5%). There were no between­ group differences at 12 months on any of the other resources surveyed. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 23 The number of services used ranged from zero to nine at six months, excluding Healthy Families Arizona and income resources such as TANF and child support. At 12 months, the number of services used ranged from zero to six with the same exclusions. During both periods, the average number of services used per family in the Healthy Families Arizona group was significantly greater than those used in the control group. The respective averages at six months were 2.7 for Healthy Families Arizona compared to 1.7 for the control group, and at 12 months the respective difference was 1.0 compared to .48. This suggests Healthy Families Arizona is working effectively within the community and that helping families access needed resources may impact long­term outcomes. VII. Family Violence Conflict between parental figures is also associated with risk for child maltreatment (Brown et al., 1998). Over the past few decades there has been a growing awareness of the co­occurrence of domestic violence and child maltreatment (Appel & Holden, 1998). Research suggests that in 30% to 60% of families where either domestic violence or child maltreatment is identified, it is likely that both forms of abuse exist (Appel & Holden, 1998). In a national survey of over 6,000 American families, 50% of men who frequently assaulted their wives also abused their children (Edelson, 1999). Table 5 presents 15 indicators of violence, similar to those included on the Conflict Tactics Scale, the most common measure used to assess domestic violence. The items were asked in reference to partner perpetrated and mother­perpetrated violence. The items in the table are arranged from least to most severe. The questions were administered at baseline, 6 and 12 months. At baseline the mothers were asked to reflect on the last year, at 6 months they were asked to reflect on the last 3 months, and at 12 months they were asked about the last 6 months. As seen by the information in Table 5, verbal acts of violence were common among the partners in each group, with about three­ quarters of the mothers and their partners engaging in such acts. More severe forms of violence were less common; however, both mothers and their partners reportedly engaged in such acts including destroying property, threatening behavior, pushing and shoving, and slapping. Very few mothers Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 24 or their partners engaged in violence involving weapons. The 12­month findings show a decrease in the proportions of mothers and partners engaging in the various acts of violence. Both groups realized a significant decrease in their frequency of acts of psychological aggression for mother and father. When examining the data across all the indicators, the Healthy Families participants had a greater decrease (2 times greater) in mother and partner violence when compared to the control group. In other words, the HF group had a stronger trend of decreasing violence when compared with the control group. Using a non­parametric sign test on the indicators of violence, the difference between the Healthy Families group and the control group was significant. Table 5. Indicators of Mother and Partner Violence Indicators of violence present Healthy Families at entry and 12­months Entry 12 mo. Cursing or swearing at other Partner to mother 69.5% 47.6% Mother to partner 73.4% 62.5% Yelling or shouting at other Partner to mother 77.9% 66.7% Mother to partner 84.0% 73.6% Stomping off during a disagreement Partner to mother 64.2% 58.3% Mother to partner 50.0% 41.7% Said something to hurt other Partner to mother 68.1% 50.0% Mother to partner 59.1% 44.4% Called other fat, ugly, or unattractive Partner to mother 24.2% 16.7%% Mother to partner 20.2% 15.3%% Deliberately destroyed belonging of other Partner to mother 26.3% 18.1% Mother to partner 13.8% 11.1% Control Entry 12 mo. 65.3% 68.4% 56.5% 58.0% 75.8% 76.8% 66.7% 73.9% 61.1% 53.7% 46.4% 46.4% 61.1% 61.1% 44.9% 40.6% 23.2% 15.8% 11.6% 10.1% 24.2% 11.6% 13.0% 7.2% Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 25 Indicators of violence present at entry and 12­months Threatened to hit or throw something at Partner to mother Mother to partner Pushed or shoved other Partner to mother Mother to partner Slapped other Partner to mother Mother to partner Forced sex on other Partner to mother Mother to partner Threw or tried to throw other Partner to mother Mother to partner Threw an object at other Partner to mother Mother to partner Chocked, kicked or punched other Partner to mother Mother to partner Threatened other with a knife or gun Partner to mother Mother to partner Used a knife or gun on other Partner to mother Mother to partner Healthy Families Entry 12 mo. Control Entry 12 mo. 20.0% 19.1% 13.9% 13.9%% 17.9% 18.9% 13.0% 7.2% 28.4% 24.5% 16.7% 19.4% 20.0% 13.7% 11.6% 8.7% 12.6% 25.5% 4.2% 11.1% 9.5% 9.5% 5.8% 4.3% 7.4% ­ 1.4% ­ 4.2% ­ ­ ­ 11.6% 4.3% 2.8% 2.8% 8.4% 1.1% 2.9% 1.4% 8.4% 18.1% 5.6% 9.7% 12.6% 15.8% 5.8% 11.6% 11.6% 10.6% 1.4% 9.7% 7.4% 4.2% 2.8% 2.8% 3.2% 3.2% ­ ­ 3.2% ­ 1.4% ­ ­ 2.1% ­ ­ ­ ­ ­ ­ Note. The truthfulness of the responses may be dependent on whether or not the partner was in the room during the interview. These percentages should be considered lower-bound estimates. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 26 VIII. Income and Financial Hardship Low socioeconomic status has been identified as a major contributing factor for child maltreatment, particularly neglect (Chaffin et al., 1996; Cicchetti, 2004; Dubowitz & Black, 2002; Erickson & Egeland, 2002; Garbarino & Collins, 1999). Low socioeconomic status is related to a wide range of factors including unemployment, limited education, social isolation, large number of children, and childbirth to unmarried adolescents (Crittenden, 1999). It is important to note, however, that child maltreatment also occurs in affluent families and that only some families living in poverty neglect and abuse their children. Nevertheless, a wealth of research has found that poverty has a strong association with substantiated child maltreatment, and thus the socioeconomic status of families should not be overlooked. The average reported typical monthly income was lower in the Healthy Families Arizona group than in the control group at baseline, 6 months, and 12 months. The difference in typical monthly income between the groups was statistically significant at baseline and 12 months. Table 6 shows that at baseline, significantly more mothers in the Healthy Families Arizona group than the control group reported difficulty paying for the cost of shelter, and although not significant, there was also considerable between­group difference in reported difficulty buying food. Perceived difficulty was recorded on a 4­point scale from ‘none’ to ‘a lot.’ The percentages in Table 6 show that the reported difficulty in covering basic needs such as food, utilities, and rent or mortgage, and even the incidence of eviction, consistently decreased over time in the Healthy Families Arizona group. The decrease in perceived difficulty paying for rent/mortgage from baseline to 12 months was significant in the Healthy Families Arizona group. This same overall trend in financial hardship was not noted in the control group, whereas there was increased hardship reported in the area of paying for food and utilities, and the same rate of eviction as in the Healthy Families Arizona group at 12 months. The perceived difficulty in paying for rent or mortgage significantly increased in the control group from baseline to 12 months. These differences reflect a considerable Healthy Families Arizona program effect, notable given the significantly higher earnings in the control group at baseline and 12 months compared to the Healthy Families Arizona group. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 27 Table 6. Perceived Financial Hardship Over the First Year Characteristics of financial hardship in a typical month Healthy Families Control Entry 6 mo. 12 mo. Entry 6 mo. 12 mo. Difficulty buying food 52.6% 47.3% 45.8% 40.2% 34.1% 44.2% Difficulty paying for utilities 45.4% 43.5% 42.2% 45.4% 37.5% 50.0% Difficulty paying rent/mortgage 44.3%* 37.0% 30.1% 30.9% 27.3% 26.7% Evicted in past 12 months 5.2% 3.3% 2.4% 6.2% 1.1% 2.3% Average typical monthly income from all sources $969* $1,096 $1,042* $1,366 $1,323 $1,487 (S = $756) (SD=$601) (SD=$745) (SD=$1,034) (SD=$965) (SD=$1,069) Note. Percentages represent mothers who responded that they have a little, some, or a lot of difficulty. * Represents a statistically significant group difference at p < .05. IX. Child Health Regular medical visits are not only important in preventing major childhood diseases, but problems like ear infections, hearing, and vision problems can create irreversible effects if not treated early. Furthermore, unhealthy children have a greater risk for child maltreatment as their temperament is often affected by how well they feel. A sick infant may be more likely to cry, creating a situation whereby the parent resorts to abusive behavior out of frustration at not being able to quiet the child. Until 6 months of age, 100% of Healthy Families Arizona infants and 99% of control group infants attended well baby checks. Although the overall proportion decreased, the groups remained equivalent at 6 to 12 months with 95.2% of families attending well baby checks. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 28 Health Insurance A pattern of health insurance coverage emerged at 6 and 12 months with higher rates of coverage for infants and sibling children in the Healthy Families Arizona group compared to the control group. At 6 months about 91% of Healthy Families Arizona infants had health insurance, compared to a similar percentage, 92%, of the control infants. At 12 months, the balance changed to 92.8% of Healthy Families Arizona infants compared to 89.5% of the control infants. Sibling health insurance coverage was higher in the Healthy Families Arizona group than in the control group at both time periods (90% at 6 months compared to 83% control; and 95.5% at 12 months compared to 84.5% control). The between­group difference in health insurance coverage for siblings at 12 months was statistically significant. Use of Health Services Mothers were asked about 17 infant health concerns that may cause a parent to visit a doctor. For 16 of these concerns, mothers in the Healthy Families Arizona group were no more likely to seek medical attention during either data collection period than were mothers in the control group. Only one of the 17 problem areas revealed a between­group difference at six to 12 months. The number of families with doctor visits for difficulties in breathing was significantly higher in the control group than in the Healthy Families Arizona group. There were no between­group differences in injuries or hospitalizations at either time period. About 12% of infants in the Healthy Families Arizona group were hospitalized at some point from zero to six months of age, compared to 11% in the control group. The groups were even from six to 12 months with 6% of infants hospitalized. Immunization Self­reported immunization rates at baseline and 6 months were high in both the Healthy Families Arizona group and the control group. Reported completion was no less than 97.6% for each of the 14 required shots. There were no statistically significant between­group differences given the high rates of reported immunization completion in both groups. Twelve­month immunizations will be reported with the 24­month data because often times Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 29 the 12­month interview was completed with the family only days before a scheduled checkup and immunization appointment. During the 24 and 36­ month interviews, the Research Assistants are asking to see the shot/immunization records to get more reliable data. X. Participation in Healthy Families Arizona Of the 98 families in the Healthy Families Arizona group, exactly half had terminated their involvement with the program as of August 16, 2007. Reasons for the terminations recorded by Healthy Families Arizona were: moved (20%); unable to contact (12%); family refused further services (16%); self sufficiency (6%); duplication of services (2%); didn’t respond to outreach (40%); and supervisor discretion (2%). Of the 49 families who had terminated their involvement in the program as of August 16, 2007, the average time to termination was just under 8 months, 233 days, and ranged from a minimum of 74 days to a maximum of 533 days (1.5 years). Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 30 Conclusion Recruitment to the longitudinal evaluation of Healthy Families Arizona began in November 2005. As of December 2007, 97 families had been recruited to the control group and 98 families to the experimental group. This report describes the differences in outcomes between the control group and Healthy Families Arizona group on data collected during the first year. Data collection for the outcome evaluation continues with a few outstanding 24­month interviews to be scheduled in the end of 2008, and administration of the 36­month interviews that began in September 2008. Over the next year, the 36­month interviews are scheduled to conclude in November 2009, and the 48­month interviews are scheduled to begin in September 2009. A final fifth year of data collection is scheduled to begin in September 2010 and to conclude in November 2011. The outcomes achieved by the Healthy Families Arizona program are consistent with the intent and practices of the program, and bode well for achievement of the three overarching goals: decreased child abuse and neglect, improved parent/child relationships, and enhanced child development. The following outcomes are attributable to the Healthy Families Arizona program. 1. Healthy Families Arizona mothers reported a significant decrease in violent behaviors (such as shouting, yelling, or screaming at their infants) between 6 and 12 months of age compared to the control group mothers. 2. A significant decrease in strong parental belief in corporal punishment was realized in the first 6 months of the Healthy Families Arizona group, while there was no like decrease at any time period in the control group. Furthermore, attitudes favorable toward corporal punishment increased from 6 to 12 months in the control group, although not significantly. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 31 3. Healthy Families Arizona mothers experienced a significant increase in satisfaction with parenting from baseline to 6 months compared to a non­significant decrease in satisfaction with parenting in the control group over the same period. 4. Healthy Families Arizona mothers had significantly greater acceptance of the infant’s behavior at six months than control group mothers, indicating the degree to which the parent expresses frustration with the infant, as well as employs physical discipline and criticism of the infant. 5. Significantly fewer Healthy Families Arizona mothers reported drinking beer or alcohol at 6 months than control group mothers. 6. Healthy Families Arizona mothers experienced a significant decrease in emotional loneliness from baseline to six months, emotional loneliness increased during this period for the control group, but not significantly. 7. Healthy Families Arizona mothers experienced a significant increase from baseline to 12 months in hope­related agency, i.e., the belief that they themselves are capable of pursuing their goals successfully. Control group mothers experienced a significant decrease from baseline to 12 months in hope­related pathway, i.e., the ability to imagine or plan ways to achieve their goals. A significant decrease in pathway in the control group was also observed from 6 to 12 months. 8. Significantly more Healthy Families Arizona mothers than control group mothers used the services of a public health nurse, mental health treatment counseling, and free diapers during the period from baseline to six months. In the 6 to 12 month period, more Healthy Families Arizona mothers than control group mothers used the food bank, mental health treatment counseling, and free diapers. The average number of community services used by Healthy Families mothers was significantly greater than the average number of services used by control group mothers in both 6­month and 12­month time periods. 9. Although the control group had significantly higher income than the Healthy Families Arizona group, perceived difficulty in paying rent or mortgage decreased significantly in the Healthy Families Arizona group from baseline to 12 months, whereas the perceived difficulty of Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 32 paying rent/mortgage increased significantly in the control group over the same time period. 10. Significantly more Healthy Families Arizona infants had siblings with health insurance at 12 months compared to the control group. 11. The safety practices of posting the telephone number for poison control near or on the phone and turning down the water heater to less than 120 degrees were greater in the Healthy Families group than the control group at 6 months. It is important to note that these statistically significant differences in outcomes were found in light of the small sample size, and despite the greater pre­intervention (baseline) risk for child abuse and neglect in the Healthy Families Arizona group compared to the control group. The Healthy Families Arizona group had significantly higher rates of CPS involvement as parents prior to the evaluation, scored higher on history of childhood abuse and neglect, and had lower incomes prior to entering the longitudinal evaluation when compared to the control group. The findings demonstrate that some important gains unique to the Healthy Families Arizona participants in child safety practices, satisfaction with parenting, use of community resources, as well as decreased emotional loneliness and beliefs in corporal punishment can occur in the first six months of the child’s life, the period in which children are most vulnerable to child abuse and neglect. These successes should be celebrated, and the program should continue to emphasize current practices that are working to produce positive gains in these areas. The evaluation also contributes to a better understanding of the issues facing Healthy Families Arizona. Apart from those participants referred to Healthy Families Arizona by CPS, one­third of enrollees had prior CPS involvement as a parent. For some, CPS involvement is long standing and has been in relation to multiple children. These parents are at increased risk of child welfare involvement, and this should not be interpreted as a failing of Healthy Families Arizona. Testimonials from two of the three CPS involved families who maintained care of their children reflected comments consistent with achieving positive outcomes that the mothers attributed to Healthy Families Arizona. Further research in the coming year examines official CPS records on child abuse and neglect and will incorporate Healthy Families Arizona administrative data on services. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 33 Recommendations The findings described in this report point to several areas that require examination and which may result in further tailoring the Healthy Families Arizona approach, as well as informing the approach of other community service providers. 1. Family planning. Rapid, repeat pregnancies and births can place considerable stress on already at­risk families. Seventy­three percent of the 195 women in the evaluation reported that the birth of the infant that made them eligible for Healthy Families Arizona was not planned. Twenty of these women became pregnant again within one year of their infant’s birth, ten from each group. Of the 10 Healthy Families mothers experiencing a rapid, repeat pregnancy, 8 of the 10 were experiencing their second consecutive unplanned pregnancy. Family planning is one area that Healthy Families Arizona should examine their practice, as it is strongly associated with child abuse and neglect and other poor developmental outcomes. Other home visitation programs have been shown to decrease rapid, repeat pregnancies, and best practices in this area should be considered for adoption. 2. Domestic violence. Domestic violence occurred frequently in these families. Home visitors can be instrumental in identifying domestic violence, informing women of resources in the community to deal with domestic violence, and ensuring that women experiencing domestic violence have a safety plan. Domestic violence is a problem that is perpetrated by women as well as men. Healthy Families Arizona is successful in reducing beliefs and behavior in psychological and physical discipline toward infants. Should Healthy Families Arizona home visitors address domestic violence with mothers and their partners – the answer to this question involves consideration of resources, training, and capability. Nevertheless, this is a question that should be considered. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 34 References Appel, A. E. & Holden, G. W. (1998). Co­occurring spouse and child abuse: Implications for CPS practice. APSAC Advisor, 11, 11­14. Belsky, J. (1993). Etiology of child maltreatment: A developmental­ecological analysis. Psychological Bulletin, 114, 413­434. Brown, J., Cohen, P., Johnson, J. G., & Salzinger, S. (1998). A longitudinal analysis of risk factors for child maltreatment: Findings of a 17­year prospective study of officially recorded and self­reported child abuse and neglect. Child Abuse and Neglect, 22, 1065­1078. Carlson, B. E. (2000). Children exposed to intimate partner violence: Research findings and implications for intervention. Trauma, Violence, and Abuse, 1, 321­340. Chaffin, M., Kelleher, K., & Hollenberg, J. (1996). Onset of physical abuse and neglect: Psychiatric, substance abuse, and social factors from prospective community data. Child Abuse and Neglect, 20, 191­203. Cicchetti, D. (2004). An odyssey of discovery: Lessons learned through three decades of research on child maltreatment. American Psychologist, 731­ 740. Crittendon, P. M. (1999). Child neglect: Causes and contributors. In H. Dubowitz (Ed), Neglected Children: Research, Practice, and Policy (pp.47­68). Thousand Oaks, CA: Sage Publications. Donohue, B. (2004). Coexisting child neglect and drug abuse in young mothers: Specific recommendations for treatment based on a review of the outcome literature. Behavior Modification, 28, 206­233. Dubowitz, H., & Black, M. M. (2002). Neglect of children’s health. In J. E. B. Myers, L. Berliner, J. Briere, C. T. Hendrix, C. Jenny & T. A. Reid (Eds), The APSAC Handbook on Child Maltreatment (2nd ed.), (pp.269­292). Thousand Oaks, CA: Sage Publications. Duggan, A. (2005, January). Evaluation of the Healthy Families Alaska Program: Final report. Author. Edleson, J. L. (1999). Children’s witnessing of adult domestic violence. Journal of Interpersonal Violence, 14, 839­870. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 35 Erickson, M. F., & Egeland, B. (2002). Child neglect. In Briere, J & Berliner, J. & Bulkley, J., & Bulkley, C., & Reid, T. (Eds.). The APSAC Handbook on Child Maltreatment (2nd ed.), (pp. 3­20). Thousand Oaks: Sage Publications. Garbarino, J., & Collins, C. C. (1999). Child neglect: The family with a hole in the middle. In H. Dubowitz (Ed.), Neglected Children: Research, Practice and Policy (pp.1­23). Thousand Oaks, CA: Sage Publications. Gibaud­Wallston, J., & Wandersman, L. P. (1978). Development and utility of the Parenting Sense of Completence Scale. Paper presented at the meeting of the American Psychological Association,Toronto, Ontario, Canada. Krysik, J., Ashford, J. B., & LeCroy, C. W. (2008). Participants’ perceptions of Healthy Families: A home visitation program to prevent child abuse and neglect. Children and Youth Services Review, 30, 45­61. Magura, S., & Laudet, A. B. (1996). Parental substance abuse and child maltreatment: Review and implications for intervention. Children and Youth Services Review, 18, 193­220. Munkel, W. I. (1996). Neglect and abandonment. In J. A. Monteleone (Ed). Recognition of child abuse for the mandated reporter, 2nd ed. St. Louis, MO: G.W. Medical Publishing, Inc. Murphy, S., Orkow, B., & Nicola, R. M. (1985). Prenatal prediction of child abuse and neglect: A prospective study. Child Abuse and Neglect, 9, 225­ 235. National Clearinghouse on Child Abuse and Neglect Information (April, 2004). Child abuse and neglect fatalities: Statistics and interventions. Online: https://nccanch.acf.hhs.gov. Renner, L., & Slack­Shook, K. (2004). Intimate partner violence and child maltreatment: Understanding the co­occurrence and intergenerational connections. Retrieved on September 6, 2004 from www.ssc.wisc.edu/irp. Tanner, K., & Turney, D. (2003). What do we know about child neglect? A critical review of the literature and its application to social work practice. Child and Family Social Work, 8, 25­34. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 36 Windham, A. M., Rosenberg, L., Fuddy, L., McFarlane, E., Sia, C., & Duggan, A. K. (2004). Risk of mother­reported child abuse in the first 3 years of life. Child Abuse and Neglect, 28, 645­667. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 37 Appendix A Participant Recruitment, Retention, and Data Collection Participating Healthy Families Arizona Sites The evaluation team established a set of criteria to target evaluation site selection. In 2004 the Healthy Families Arizona Quality Assurance team provided data on the 24 established Healthy Families Arizona sites. Based on stability of staff and number of participant openings it was decided that the best location for the evaluation in a metro area would be Pima County. Oversight for all nine Pima County sites occurs through Child and Family Resources, Inc. Target Participants Five exclusion criteria related to recruitment were specified in collaboration with the administrative staff of Healthy Families Arizona in Pima County and in consultation with the Family Assessment Workers in Pima County. The five exclusion criteria included: 1. Families referred to Healthy Families Arizona by CPS 2. Families who self­referred 3. Families for which the hospital social worker made a referral to Healthy Families Arizona 4. Families that were particularly crisis ridden as determined by the FAW staff in consultation with their supervisor 5. Families who enrolled in Healthy Families Arizona prenatally, except for those who enrolled in the 8th month of pregnancy or later, in which case they were not enrolled in the evaluation until after they had the baby. These five exclusion criteria were specified to ensure a sample representative of the most typical Healthy Families Arizona participants in Pima County. In addition, the family had to meet two standard criteria for inclusion in Healthy Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 38 Families Arizona. First, the score on the Parent Survey had to be equal to or greater than 25 for either parent, and second, the child had to be no more than 3 months of age at the time of enrollment to the evaluation. This is important with regard to the information that is collected at baseline. Recruitment to the Longitudinal Evaluation Recruitment to the longitudinal evaluation followed the standard Healthy Families Arizona recruitment process. Following the birth of a child at one of the referral hospitals, the Family Assessment Worker (FAW) conducted the Healthy Families Arizona 15­item screen. The FAWs work in local hospitals to screen and recruit new mothers for participation in Healthy Families Arizona. Over the course of recruitment for the longitudinal evaluation and in addition to the standard recruitment process, the FAWs gave mothers a brochure about the longitudinal evaluation and asked if they would be interested in participating in a randomized evaluation referred to as the Arizona Child Development Project. If the parent was not interested they were provided with information on Healthy Families Arizona without the longitudinal evaluation. If the family was interested in participating in the longitudinal evaluation they were randomly assigned to one of two groups. To simplify the process of random assignment, group assignment followed days of the week. For instance, if the family was screened on Monday, Tuesday, Friday or Saturday, they were assigned to the Healthy Families Arizona group. If the family was screened on Wednesday or Thursday and agreed to participate in the evaluation, they were assigned to the control group. The control group participants have no involvement with Healthy Families Arizona. If they are offered and accept Healthy Families Arizona Services with a subsequent birth, they are at that point dropped from the evaluation. The control group participants did, however, complete the Parent Survey to establish if they met the eligibility requirements of Healthy Families Arizona. In total, three of the mothers interviewed for the control group did not score 25­points or greater on the Parent Survey, nor did the fathers, so these families were not included in the group of 97 control families because they would not have been otherwise eligible for Healthy Families Arizona. To increase the pool of families eligible for recruitment, an extra FAW was hired by LeCroy & Milligan Associates, Inc., to screen families on Sundays at two local hospitals. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 39 The extra FAW alternated recruitment for each group, control and Healthy Families Arizona, each Sunday. Two research assistants employed by LeCroy & Milligan Associates, Inc. received the referrals for the longitudinal evaluation from the Family Assessment Workers (FAWs) in the Pima County Healthy Families Arizona program. Families assigned to the experimental group were first enrolled into Healthy Families Arizona by the FAWs before the research assistants received the referral. The research assistants did not contact families that did not enroll in Healthy Families Arizona. Families assigned to the control group were referred directly to the research assistants and were not contacted any further by Healthy Families. Once the research assistants received a referral for either group, they initiated contact with the mother by telephone to give her additional information about the longitudinal evaluation (called the Arizona Child Development Project), to share the benefits of participating in the evaluation, and to set up the first interview. At the baseline interview, a detailed consent form outlining the evaluation was reviewed, contact information was collected, and the baseline interview schedule administered. Participants who agreed to participate in the Arizona Child Development Project were asked to sign an informed consent form outlining a description of the longitudinal evaluation and any potential benefits and risks. The consent form also outlined the incentives for participation and the responsibility of the participant and research assistants. One copy of the signed consent was left with the participant and a second copy is kept on file at LeCroy & Milligan Associates, Inc. Families were informed that participation in the evaluation included free developmental screenings of their children at regularly scheduled intervals. Healthy Families parents receive developmental screening as a part of regular service. The research assistants do not duplicate the developmental screening conducted by Healthy Families Arizona home visitors. If, however, a family terminates involvement with Healthy Families Arizona, the research assistants provide equivalent developmental screenings consistent with control group families. Families are also provided with information on community resources if requested, and monetary incentives that increase in value on an annual basis. Participants were also advised of the time commitment of the Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 40 evaluation ­ a maximum of 90­minutes per interview, and a total of seven interviews over a five­year period. Parents were told that if they moved or decided not to continue with Healthy Families Arizona they could still participate in the evaluation and receive monetary incentives as promised (i.e., if the family moves out of state their participation can continue by telephone or mail). Recruitment for the longitudinal evaluation began November 1, 2005. Recruitment was originally scheduled to end in the summer of 2006, but was delayed due to the low number of openings in the participating Healthy Families Arizona sites. Recruitment for the evaluation was completed in December 2006, seven months longer than originally anticipated. There were several reasons for the delay that include: � the program began enrolling a greater number of families involved at the prenatal phase who were not eligible for participation in the evaluation unless they were in their eighth month of pregnancy or beyond � incomplete information on the referral that led to failure to contact � receiving fewer referrals than anticipated from the FAWs � periods of time when many of the Healthy Families sites were at capacity, so new families could not be enrolled � several of the families enrolled in Healthy Families Arizona were not eligible due to the exclusion criteria. Retention Strategy Two efforts specific to this evaluation are important for retention. The first is removing all possible barriers to keep in contact with participants. The second is establishing a positive relationship between the research assistants and the participants. Maintaining contact with the participants and not losing them before their next scheduled interview poses a significant challenge for the research assistants. However, the following information collected at baseline and updated at each contact has been extremely helpful in retaining Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 41 participants over time. The information collected has been successfully used to reach participants when primary information has changed and initial attempts to contact have failed: • current contact information (address, phone, cell phone, alternative phone, email) • partner’s contact information (boyfriend, father of baby, or husband) • any plans to move in the next 6 months and any information they have about their new address • employment and/or school information • contact information for two other people in case the participant cannot be reached. To offset the long­term commitment, the project reciprocates by providing incentives for participation. The participants have been very pleased with the incentives and most comment that the time they spend with the research assistants is well worth it. These incentives include: • Information about local resources for basic needs, child care, domestic violence, Arizona Early Intervention Program, etc. as requested • Administration of a developmental screening tool (ASQ) at 6 months, and at each birthday until the child reaches five years of age. This tool is used to identify any potential developmental delays. A referral to a local service provider is given if a delay is found and if requested by the parent • Monetary incentives are given at each interview1 Monetary incentives are given if the parent provides any change of contact information between interviews. Four cash drawings will be held throughout the 5 years for current participants. 1 Participants receive $60 for Year 1 ($20 for each interview including baseline, 6 and 12 months), $30 for Year 2 (24 months), $40 for Year 3 (36 months), $50 for Year 4 (48 months), and $60 for Year 5 (60 months). A $10 incentive is provided to anyone who informs the research assistants of changes in contact information between interviews (i.e., relocation or change in telephone number). Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 42 Most important to establishing a positive relationship with the participants is making sure they have ongoing and frequent contact with the same research assistant throughout the evaluation. To date, the two research assistants that started with the evaluation have been able to continue interviews with the participants they recruited. Additional retention efforts that help with establishing a positive relationship include: • Providing a self­addressed, stamped postcard for the mothers to submit if their contact information changes • Providing a magnet and business card with contact information for the research assistants, including a 1­800 number, work phone number, cell phone number, email address, and mailing address • Sending thank you cards following each interview • Sending birthday cards each year for the mother and the child • Sending reminder letters to participants about the next interview and the importance of their continued participation • A project identity (the Arizona Child Development Project) was created for the evaluation and promoted through the use of a project logo that can reduce concerns about the credibility of the project and help facilitate recognition of correspondence related to the project. • Reminder phone calls before each interview • Research assistants are available to the participants throughout the evaluation if assistance is needed. Update on Retention The success of any longitudinal evaluation is reliant upon successful recruitment and retention. Evaluations with high dropout rates can yield biased findings regarding the impact of program services. Retention efforts are critical to the success of this evaluation and will continue to be important in maintaining contact with the evaluation participants for the remainder of the evaluation. The goal of the Healthy Families Arizona longitudinal Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 43 evaluation is to retain at least 80% of participating families over the life of the evaluation. The recruitment efforts are particularly crucial here for several reasons: the long­term commitment (e.g., seven 60­to­90 minute interviews over a period of 5 years), the sensitive nature of the questions, and the location of the interview process in the participants’ homes. This population tends to have characteristics that make retention difficult. For instance, they often move, change phone numbers and jobs. Table A­1 shows the number of baseline, 6­month, 1­year, and 2­year interviews completed as of October 2008. Table A­1. Data Collection Completion Summary as of October 2008 Control Group Experimental Total # of families 97 98 6­month interviews complete 88 (91%) 92 (94%) 1­year interviews complete 86 (89%) 85 (88%) 2­year interviews complete 76* 65* * 2­year interviews are not yet complete and the retention rate cannot be determined at this point. Of the nine 6­month control group interviews not completed as of December 2007, four participants had moved with no forwarding address, four participants did not respond to multiple attempts to contact, and one participant refused further participation. The 6­month control group retention rate was 91%. Of the six 6­month interviews not completed for the Healthy Families Arizona group, one participant moved out of state due to domestic violence issues and contact was lost, four participants moved and have not responded although collateral contacts have been made, and one participant ended her involvement after severance of parental rights was granted by the court. The six­month retention rate in the Healthy Families Arizona group was 94%. The one­year attrition rate has yet to be determined, although eight of Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 44 the Healthy Families Arizona group families have now moved out of state. As of August 16, 2007, 50 participants had ended their involvement with the Healthy Families Arizona program. Data Collection The outcome portion of the longitudinal evaluation involves the collection of data separate from the ongoing data collection that occurs for Healthy Families Arizona. Maternal demographic data and risk factor data are collected on an interview schedule administered by the research assistants. The interview schedule was designed specifically for the longitudinal evaluation and asks about the mother’s living arrangements, employment, education, and mothers’ perception of the child, relationship with the father, etc. Data collection occurs in the home or at a place convenient to the mother and the baseline interviews averaged 71 minutes in each group. The questions vary somewhat at each data collection period, although some questions remain the same to measure change over time. The first 12­month interview occurred in September 2006. Although the original plan was for the research assistants to not know the participants’ treatment assignments, this has not been possible as there are only two research assistants. The research assistants have been responsible for recruitment as well as data collection, and this has necessitated that the research assistants know the participant’s group assignment so they can determine which form to use for data collection. For instance, the family support specialist administers the Ages and Stages Questionnaire (ASQ) to the Healthy Families Arizona participants, whereas the research assistants administer the ASQ to the control group participants. If and when experimental families leave the Healthy Families Arizona program, the research assistants administer the ASQ. To ensure that the participants properly understand each item on the interview schedule, the research assistants read all items out loud and record the participants’ responses on the interview schedule. Visual charts that depict the response categories for questions with ordinal level responses (e.g., strongly disagree, disagree, neutral, agree, strongly agree) are used as visual aids to assist the participant in answering the questions. Furthermore, the Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 45 research assistants have never been involved in delivering or managing the Healthy Families Arizona program or any other type of home visitation program. The research assistants are young females who each have a young child, one is Hispanic and Spanish speaking, and the other Caucasian, and thus they mirror some important characteristics of the participants. Schedule of Standardized Measures Table A­2 presents the standardized measures that are implemented at the different observation points in the longitudinal evaluation. The standardized measures are integrated into the overall interview schedules that have been developed for each data collection point. Note that this table is subject to revision as new measures are added. For instance, three measures that were not originally planned were included in the 24­month interview schedule in 2007. Table A­2. Schedule of Standardized Measures by Child’s Age in Months Measure Baseline 6 12 24 36 Mental Health Inventory x x x x Center for Epidemiologic Studies Depression Scale (CES­D) x Parent Survey Control only Being a Parent x x x x Adult­Adolescent Parenting Inventory 2 (AAPI­2) x x x x 48 60 x Eyberg (child’s behavior) x Bracken (school readiness) x x x x Goals Scale x x Emotional Social Loneliness Scale (ESLI) x x x Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 x x 46 Mobilizing Resources x x Safety checklist x HOME x ASQ Ages and Stages developmental screen x x x x x x x x x x x x TAS­45 Toddler Attachment Sort­45 x (BITSEA)­parent version Brief Infant Toddler Social and Emotional Assessment x Revised Dyadic Adjustment Scale x x x In addition to these standardized measures, the interview schedules include questions related to a number of domains such as child abuse and neglect history of the parent, prenatal care, father involvement, child’s health, parent’s health, medical care, housing stability, education, employment, family violence, parenting practices, finances and financial hardships, criminal involvement, transportation, subsequent births, substance use, child care arrangements, and service use. The entire longitudinal evaluation interview schedule, consent forms, etc., are available in Spanish and English and have been approved by an Internal Review Board (IRB). Protection – Data Security, Storage, and Confidentiality A separate database from the ongoing Healthy Families Arizona evaluation was developed for the longitudinal evaluation. In order to preserve confidentiality, each family was assigned a unique identification number. Each interview schedule is coded with the family’s ID number rather than their name to protect confidentiality. The research assistants and data entry staff enters the data and file the hard copy records. The hard copy data are stored in a locked file cabinet used exclusively for the Healthy Families Arizona longitudinal evaluation. Only the staff members involved with the longitudinal evaluation have access to the data and the list of names Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 47 associated with the unique identifiers. As an additional precaution, the research assistants do not store data in their cars or briefcases. An independent ethics review committee, ARGUS IRB, reviewed the protocol for the evaluation and a one­year renewal was applied for and granted in 2007 and 2008. Healthy Families Arizona Longitudinal Evaluation Annual Report 2008 48