For many reasons, this time of year reminds me of spending time in the kitchen. Whether I’m roasting a turkey or baking pies and cookies, the activity always causes me to reflect on the amount of time it takes to actually make something from scratch. Sure, I could pick up a box mix, throw an egg and some oil in there and maybe come up with a dessert that tastes so good that most people wouldn’t be able to tell that I did the baking equivalent of phoning it in. (Seriously, have you tried those mixes lately? They aren’t like the ones our moms counted on in a pinch!) But during the holidays that short-cut approach just seems wrong. Special times call for extra effort.
I remember the first time my daughter wanted to bake something on her own. She accidentally put two cups of sugar, instead of two tablespoons, into her mix — and then asked me what she could do to fix it. As I told her, usually it’s easy to correct a measuring error by increasing the rest of the ingredients in equal proportion and making a bigger batch. However, sometimes there is just no way to save a “bad” batch. When that happens — when you simply know you won’t end up with a good end product — you’re better off throwing the whole thing out and starting over from scratch.
This same rule applies to standardized testing. The design for the test — or recipe — might give you great results. Or it might end in a complete disaster.
From standardized test perspective, designing a test using sound methodology ensures the integrity of the results. If you have too much of one item type or too little of another, it can sour the overall results and defeat the purpose of the test. Creating tests with a variety of items such as multiple choice, technology-enhanced, and open-ended responses with specific results in mind, is an art — and a virtual recipe for success.
From my perspective as a scoring services manager, I love that my company has so many levels of participation in our test designs — including that of the teachers whose students’ tests we will ultimately score. With the input we receive from them in our Item Writer Workshops, as well as guidance other professionals, from subject matter experts and graphic designers to technology wizards and hand scoring experts to weigh in on what makes a “good” question, we can develop highly customized tests that provide the results that matter most to our students, parents, and educators. And as we continue to encourage teacher involvement in our item development, we not only tap into the direct line to the students, but also emphasize the importance of real-time classroom activities.
Like that homemade pumpkin pie that your whole family looks forward to all year, a good test design is more than the sum of its individual ingredients. It’s in the measure of those ingredients. It’s in the timing of the bake. But most of all, it’s the care, precision, and personal touch that ultimately turns simple ingredients into something truly extraordinary.