In the last few posts, this blog series has dispelled some myths about test difficulty as a function of item type and explained how a test’s intended purpose affects the information that can be gleaned from its results. A test’s difficulty depends mostly on the test design (not item type), which stems from the intended purpose of the test and intended use of the results. This post expands on this idea, delving into how laws can affect a test’s intended purpose and ultimately hinder teachers, parents, and administrators from using the test results to improve teaching and learning.
First, here’s a reason why there is so much angst and discussion about testing, which begins with the universal equation in measurement:
This equation is fairly universal in measurement. Most constructs that we measure about humans — even the measurements or test we receive from a medical doctor during a physical examination — are subject to this equation because we cannot directly measure a person’s intelligence, depressive mood, or personality. We need to develop a test, or measurement, that attempts to measure these hidden aspects of the person.
In a later post, I’ll explain the concept of measurement error in more detail but for now, there are things that happen during a test or measurement that impact the accuracy of that measurement. That is the error part of the equation. For example, if a student were not feeling well the day of a test, that would add to the error. Hence, the observed score is an estimate of what the person’s true score is for any construct being measured.
For example, the latest reauthorization of the Elementary and Secondary Education Act of 1965 (ESEA), known as the No Child Left Behind Act of 2001 (NCLB), explicitly mandated accountability testing. It also enacted penalties for failure to achieve success as determined by Adequate Yearly Progress (AYP). This set up a poor set of dynamics for improving teaching and learning. The focus of educators and administrators changed from teaching students a set of content to focusing on getting as many students as possible over the bar of proficiency. This has sometimes been called a focus on the bubble students, or those students who just missed making it over the threshold of proficient.
One cannot blame educators for such behavior when money and resources are removed from a school or district because of failure to attain the goal. However, because of the punitive nature of the current law for failing, critics have taken to blaming the test or suggesting that different test types will fix the problem. All this chatter is simply adding to the error side of the equation. By focusing on the test as the problem, critics miss the import fact that it is not the test that is in error but rather the use of the test score for such high stakes and punitive circumstances legislated in NCLB.
Diagnostic, formative, interim, and summative tests all play in role in helping improve teaching and learning. However, all the noise being generated about testing has masked the true issue, which is that the real problem isn’t testing. The real problem is a law that had good intentions for using accountability as a way to positively impact achievement for all students, yet it had other mandates that inevitably steered the focus and goal in a different direction.
Money, teachers, activities, breakfast and lunch programs, or even more tests have not been found to help students achieve their highest learning potential. Instead, research finds that the following criteria have the biggest impact on student learning (see Dr.John Hattie, Visible Learning):
- high expectations for students by teachers, family, and friends
- teacher credibility (in the eyes of the student)
- good, formative evaluations of teachers
- positive teacher-student relationships
- constructive feedback
- reciprocal teaching
One way to reduce the noise that steers the focus away from improving teaching and learning in the classroom is for Congress to pass a new authorization of ESEA. The law should allow for innovation in the use of measurement that helps drive the things that actually improve student learning (e.g., the items in the list above) and ultimately removes the punitive nature of the current law. Teachers might worry less about the test and more about setting high expectations, developing solid relationships with their students, driving both surface and deep understanding into their lessons, and becoming great teachers. Passing a law that helps using tests as they should be use, as information, as a tool to help guide instruction, as a tool to understand strengths and areas in need of improvement will reduce the error of using test information.