STRATEGY: Develop easy-to-use data reports for various users of the data to interpret and consistently use it for improvements in learning outcomes; this should evaluate root causes with qualitative data measures.
According to the AASA report Using Data to Improve Schools, “Data provide quantifiable proof, taking the emotion and rancor out of what can be tough calls for superintendents and school boards (e.g., dismantling a popular but ineffective program or closing a school). Data also provide the substance for meaningful, ongoing dialogue within the educational community.” Data reports are, by design and intent, meant to objectively communicate information that is to be interpreted, understood, and lead to a plan of action. Creating an evidence-based or data-driven decision-making culture means that anyone of any background can look at a data report and draw similar conclusions to others. This common-lens approach helps to create a shared language of action and understanding, and through this shared language meaningful and sustainable education reforms can take place. Recognizing that data is both quantitative and qualitative and that student learning is often better understood in a qualitative format due to the personalized and contextual nature of student learning, the data presented should provide a mixture of quantitative and qualitative data, all to provide a more complete picture that represents the whole child and his or her learning experience.
A root cause is an underlying factor or condition that creates a problem and that, if addressed sufficiently, would remove the problem. Analysis of a root cause is common in a school or district setting, as personnel examine a potentially wide range of data, use that data to develop a set of potential explanations for the problem, and, ultimately, trace that set of explanations down to the root cause. To do this analysis, school or district data teams use a variety of data to explore the issue—from community feedback to student achievement data—and can employ a variety of protocols. Critical to the success of any protocol, however, are data reports that are clear and understandable to all involved with the analysis process.
To aid this work, the school or district data team often develops templates, language, and common procedures to be used when various personnel (teachers, administrators, etc.) use data to make improvements in learning outcomes. The templates, language, and procedures should be field-tested in some manner to determine their overall effectiveness, and the results of the effectiveness should be given to the data team to adjust, iterate, and support data-driven decisionmaking in the school.
First Steps to Consider
Developing and integrating easy-to-use data reports as part of root cause analysis requires input and understanding from those who read and interpret the data. Here are a few initial steps that a school or district can take:
- Identify key stakeholders to be a part of a school-wide data team.
- Identify all sources of data, including what data sources are already in use.
- Collect input from various users on how data can be easily and consistently interpreted.
- Identify current structures and systems in place that use data to make improvements in learning outcomes and ask, “How do these structures and systems work? Do they provide sufficient information to make valuable interpretations and improvements?”
- Work with the staff to uncover what works with the analysis and interpretations of those already existing data sources, and apply these findings to new data sources to build the foundation of an organically grown system of data-driven decision-making.
Complexities & Pitfalls
Data are all too often relegated to numbers, percentages, and scores, often because this information is relatively easy to access, easy to develop reports about, and easy to discuss in group settings. However, quantitative data often only shows a small fraction of the whole child/school experience. Schools that are working to develop comprehensive data systems also need to develop the means to collect, analyze, and interpret qualitative data, including but not limited to open-ended survey questions, interviews, and observations. Without taking this holistic approach to data analysis, any decisions and/or reports will only provide fractional information, which could make decisionmaking and implementation potentially faulty.
- To what extent are all stakeholders (teachers, administrators, students, etc.) involved in the collection, analysis, interpretation, and implementation of information from the data?
- To what extent are data measures balanced between the quantitative and the qualitative?
- How might the school or district better understand the readability and usability of the data reports that are developed?
- How does any data team determine the effectiveness of its data interpretation?
- To what extent has any data team examined its data holistically?