January 24, 2009
Far more important to me than the inauguration of our first African-American president, we had our state E/LA exam this week; which– excuse me while I spit up some blood here– did not seem so bad.
In particular every consecutive year, barring the usual one or two mortifyingly badly written multiple choice questions, there seems to be one more move towards background-appropriate text. Is someone over at McGraw-Hill finally listening to E.D. Hirsch?
Or are the tests merely being “dumbed down”, as stats guru Eduwonkette wonders last year, writing in response to our state’s astronomical gains from 2007 to 2008? (Having done an unofficial scoring of the multiple choice, I’ll wager something similar will pop up this year.)
Not that I actually know what “dumbing down” a state exam means, really, in a world where it would take 22 years to cover the national average of 274 (often developmentally inappropriate) benchmarks in E/LA. Perhaps this year’s exam only covers 86 of those.
On the desert island where I can be wrecked with only three other justifications for my concerns about standardized exams, you’d find the following stashed in my flotation device.
1) Their treatment as hard science due to our culture’s Nine 1/2-Weeks-type love affair with numbers, when in fact they are anything but. Read this fascinating tidbit from the NYTimes. (Gosh. Maybe the inauguration is important.)
Similarly, James Popham does a lovely job here and here of summarizing for the lay person how standardized exams do very little other than measure students’ socio-economic status, as well as talking more graciously about my flippant-but-true 86-standards remark (thanks, James).
2) If you have a few more moments, check out Andrew Ho’s video on how easy it is to manipulate standardized exam data. While you digest that, keep in mind that 14 out of 50 states require teachers to take a course in educational testing; 2 out of 50 for administrators.
3) And finally, a microcosmic statement on the macrocosm: consider the ridiculousness, in New York State, of administering the exam which is supposed to measure my students’ grasp of (and my ability to teach) our 7th grade E/LA content– in January. Isn’t this all I have to say?
Now, don’t get me wrong. There are better and worse standardized exams out there, as there are better and worses uses and interpretations for their data. Nothing perturbs my sense of fair play more than hopping on the bandwagon of knee-jerk vilification.
But for the record, I do think I am riding on a unicycle a few miles back.
January 24th, 2009 at 8:19 pm
Popham claims simultaneously that achievement tests measure nothing by socioeconomic status and that test bias was fixed in less than a decade. So which is it? Are the tests still purely SES bias or has the bias problem been fixed?
I think that the SES bias in the tests i much less than Popham claims—the problem is that the achievement test measures the cumulative effect of years of education, and students in low-SES districts have been shortchanged on education for years.
January 24th, 2009 at 8:32 pm
Good point, Kevin. He’s not very clear about that in the Ed Week article. I assumed he was talking about some other type of egregious test bias from the 70s or earlier, such as racial or cultural; his claim really doesn’t make sense otherwise, as you say.
January 24th, 2009 at 11:51 pm
Egregious test bias still creeps in. I had 2nd students in Oakland in a predominantly black school given questions about green salad, which most of them had NEVER had. You can NEVER assume what a typical experience is for folks. First graders in our ELA program have a whole selection on dogsled racing with NO background information on life in the arctic or a place with snow (something many of them have never experienced). When I look at release questions, there are tons of references to things that are very foreign to students, and could help slip them up.