Skip to main content
back to search page

Evaluate Program Effectiveness

STRATEGY: Are the expenses in line with what was forecasted, or are they substantially lower or higher?

Districts and schools are faced with the challenge of having to implement, sustain, and evaluate many different innovations, initiatives, and programs simultaneously, with limited resources. As districts and schools seek ways to enhance the performance of all students while reducing instructional and performance gaps, there has been greater focus on ensuring that resources are channeled to where they are most needed. Educational efficiency aligns achievement with cost. Resources need to be efficiently allocated and well used to ensure the effectiveness of programs to the greatest possible extent; however, schools continuously face competing demands for available resources that often involve complex trade-offs. Ongoing program evaluation facilitates a feedback loop that supports efficient resource allocation.

Details

Many schools have so many programs, strategies, and practices being implemented that the prospect of assessing their effectiveness is overwhelming. Yet evaluating impact is a key way to avoid wasting time and money and to identify possible barriers to learning. As Fiona Hollands and her coauthors wrote in 2015, “Combining cost and effectiveness data on alternative interventions can indicate which interventions are likely to provide the best results for a given level of investment.” Schools and districts need to determine the return on investment (ROI) for the time, energy, and money they are devoting to initiatives and programs—foremost when these calculations will result in increasing student achievement.

A detailed accounting of all the initiatives and programs in a school building is critically important to this process. Once that is established, school personnel need to have a deep understanding of these initiatives and program—their purpose, the students impacted, the staff involved, the data being generated. The school’s data team (or an analogous group) is the best group to undertake this deep review. If that team has not done so already, it can catalog all initiatives and programs that aim to drive, supplement, or remediate learning. Armed with this data, the team can begin to identify programs that are effective, ineffective, and potentially unaligned with current education goals, all in an effort to close the achievement gap and eliminate inefficient resource allocation.

After this is done, the rest of the school, in some thoughtful forum, should carefully evaluate the comprehensive program list, to discuss and make further determinations.

First Steps to Consider

  • Combine or eliminate programs with the same or similar goals.
  • Identify current initiatives and programs across educational units and determine human and financial allocations for them.
  • Quantify the impact on student learning of current initiatives and programs.
  • Consider the extent to which current practices are implemented with fidelity, and assess the cost of implementation.
  • Take a baseline measure of a problem before implementing new efforts to address that problem.

Complexities & Pitfalls

  • One potential pitfall is not having a baseline measure of a problem before implementing new efforts to address it. If possible, conduct one or more baseline (pretest) measurements as well as several posttest measurements spaced over time.
  • Various programs and initiatives sometimes produce different data or outcomes and are hard to compare. Identify the appropriate methods of measurement and work to align them across programs and initiatives.
  • Measurements at the population level evaluate efforts that reach the entire school. However, population level measurements tend to be done less frequently (e.g., annually or every other year) compared to measurements at other levels.

Guiding Questions

  • How does your school or district manage and deploy available resources at different levels within the system?
  • What is the alignment between the school’s three-tiered classification system used in the Response to Intervention (RTI) and Positive Behavioral Interventions and Supports (PBIS) frameworks?
  • Are there programs that share a common goal that can be combined or eliminated?
  • When is the optimal data collection based on your school schedule? Determine how data from different time points will be compared.