Following the 1996 welfare reform legislation requiring states to spend 4% of Child Care and Development Fund (CCDF) allocations on initiatives to improve child care quality, the interest in evaluating the effectiveness of quality-improvement initiatives has grown significantly. The mounting concern with documenting program results and conducting evidence-based research has placed a new emphasis on holding policymakers accountable for how funds are spent. Evaluation offers state administrators, legislators, and program managers the means to assess the effectiveness of initiatives and expend their CCDF allocations more efficiently.
In 2000, Bank Street College of Education, Abt Associates Inc., and the National Center for Children in Poverty of the Mailman School of Public Health at Columbia University began a joint study, Assessing Child Care Development Fund (CCDF) Investments in Child Care Quality. A three-year effort funded by the Administration for Children and Families (ACF) Child Care Bureau, it is intended to provide information for policy makers about how to make more efficient use of CCDF quality set-aside funding. Our initial product, A Study of Selected State Initiatives, describes 104 initiatives that use a variety of strategies to improve child care for different populations of providers and different populations of children. The companion volume, Program Profiles, includes information about the start date, geographic scope, budget, outreach strategies, implementation and evaluation of each initiative.
A Toolkit for Evaluating Initiatives to Improve Child Care Quality is the second-phase product. Developed with an Advisory Committee of state child care administrators and child care experts, the Toolkit is intended to help state child care administrators design and conduct evaluations of quality improvement initiatives. Unlike general evaluation manuals, it focuses specifically on efforts to improve child care, using examples of current initiatives to illustrate the evaluation process. It is intended to supplement step-by-step evaluation guides such as the ACF Program Manager's Guide to Evaluation (1997) for conducting evaluations of social programs.
The Toolkit consists of a brief guide to evaluation and a set of 14 user-friendly sample instruments that can be used for both formative and summative evaluations of child care quality improvement initiatives. The Toolkit approaches child care evaluation from the perspective of a theory of change model, which can be used to plan initiatives, understand program operations and to explain outcomes.
The Toolkit is divided into two parts. The first part provides a framework for understanding and conducting evaluation of child care quality improvement initiatives. Chapter II offers a general perspective on the two types of evaluation-- formative and summative. The following three chapters discuss evaluation in greater detail. Chapter III discusses the theory of change model as a basis for planning and evaluating child care quality improvement initiatives and Chapter IV illustrates how to use a theory of change model to design an evaluation. Chapter V presents some issues related to summative evaluation. Chapter VI provides practical advice on conducting evaluation. It describes specific steps for preparing an evaluation plan based on the ACF Guide and the Institute for Women's Policy Research report, Building a Stronger Child Care Workforce (2002). The Appendices include other resources for evaluation.
The second part of the Toolkit consists of a set of instruments that can be used to evaluate child care quality improvement initiatives. The instruments, which are easy to use, are samples that can be adapted to evaluate these efforts. There is also a table that identifies outcomes for practitioners, programs, and children and families that are linked to specific quality improvement strategies. The Appendix includes an annotated bibliography of selected observation instruments that can be used to assess child care quality.
Click here to download Part I: The Guide (in PDF format)
Click here to learn more about Part II: The Instruments