According to the “Standards for Educational and Psychological Testing,” alignment is “the degree to which the content and cognitive demands of test questions match targeted content and cognitive demands described in the test specifications” (AERA, APA, & NCME, 2014, p. 216).[1]
In other words, alignment is a process by which the content standards, curriculum, instruction, and assessments are all targeted to a specific measurement purpose. The goal is an assessment program with educational components that all focus on the same instructional and content objectives in order to better serve the teachers and students as they work toward success in the classroom and beyond. Evidence of alignment provides the starting point for content validity, that the assessment measures the content or standards it is intended to measure. As such, ensuring alignment is an important aspect of ensuring that the assessment is a valid instrument for measuring student proficiency.
Alignment became a major focus in the assessment industry when it became a part of the U.S. Department of Education’s peer review process for state assessment programs. The following is the list of sections in the “Standards and Assessments Peer Review Guidance,” with alignment being section 5.
- Academic content standards
- Academic achievement standards
- Statewide assessment system
- Technical quality
- Alignment
- Inclusion
- Reporting
The peer review process is currently being updated, and the form of peer review may change. When it does, I will write a blog post and/or assessment brief explaining the changes.
The No Child Left Behind Act (NCLB) requires all states to align their state assessments in terms of complexity, depth of learning, balance, and the range of content standards. The Webb Alignment Tool seems to be the most commonly used and accepted alignment method by the U.S. Department of Education as part of the peer review process. However, other methods are also available, such as Surveys of Enacted Curriculum (SEC) from the Council of Chief State School Officers (CCSSO); the Toolkit for Evaluating the Alignment of Instructional and Assessment Materials from Achieve, CCSSO, and Student Achievement Partners; and Links to Academic Learning for alternate assessments.
The basic process for all of these models is to convene a group of subject matter experts to determine how well an assessment’s items align to the academic standards. Participants also evaluate how complex the items are and how deeply the items assess student knowledge. Evaluating the item against taxonomy of knowledge or learning typically does this.
Taxonomies of learning have been proposed in education long before the concept of aligning test items to standards became a requirement of NCLB. They were developed as a means of understanding the levels of knowledge and skill in order to help educators understand the different levels of knowledge and how to improve a student’s learning. They also allow for the differentiation of content and instruction that help scaffold instructional practice.
One of the most well-known taxonomies is Bloom’s Taxonomy of Learning. This taxonomy classifies levels of learning into six categories, with evaluation being the highest level:
- Evaluation
- Synthesis
- Analysis
- Application
- Comprehension
- Knowledge
The Webb Alignment Tool has its own taxonomy called Depth-of-Knowledge (DOK). This taxonomy has four levels, with extended thinking being the highest level:
- Extended Thinking
- Strategic Thinking
- Skills & Concepts
- Recall & Reproduction
Regardless of the taxonomy or alignment method used, the basic outcome is that each item is evaluated for how closely it aligns to a content standard and at what level of complexity or depth the item assesses that content.
Assessments that do not align well to the standards or only measure the most basic level of learning as described by a given taxonomy may be of little use in making decisions about student learning or the effectiveness of a school or district in teaching its students. Therefore, without evidence of alignment, or content validity, it is nearly impossible to build a validity argument for the use of a test’s results. This is especially true in high-stakes accountability testing and particularly true when test results are used for promotion or graduation.
Thus, alignment is an important factor of testing. It is also built into the development of a good assessment through test blueprints, test and item specifications, and instructions to item writers, as well as throughout the various stages of item review. Alignment, therefore, is a starting point for building the validity evidence for the use of test results.
[1] American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. Washington, D.C.: AERA.