The Annual Standardized Testing Rant: First in a series!

Welcome back to my favorite topic: how standardized tests are killing American education. I've tackled this topic before, so this year I'm going to go for a series of the main reasons I detest standardized testing so much in the form of a "top ten" list." Here we go (drum roll, please!):

10) Sampling error makes it impossible to get accurate results: "Sampling Error" refers to the inherent error that exists when you choose a small sample of all possible items to evaluate mastery of the entire set. For standardized tests, there are millions of possible questions that could be asked to assess students' mastery of the standards that students are supposed to learn in a given year. To create a usable test, a small number of those possible questions must be chosen. The assumption is that the questions are chosen carefully enough so that they are representative of all possible questions. In other words, if a student answers 70% of the sample questions correctly, the assumption is that they would have answered 70% of all possible questions correctly.
"Sampling error" is a mathematical term that refers to the probability that the sample score is close (usually 90% or 95% accuracy is checked for) to the actual score the student would have received if tested on all questions. You see this number when political polls results are reported, it's called the "margin of error." So if candidate A is poled at 40% and candidate B is polled at 45% but the margin of error is + or - 7%, you would say that they are in a statistical tie. The margin of sampling error is greater than the difference between the results, meaning that the poll doesn't really indicate a clear advantage for either candidate.
For the California standards tests, students scores are grouped into "quintiles," where a student in the bottom 20% is in quintile 1, students in the next 20% (21% to 40%) are in quintile 2, etc. A student who is in the 3rd percentile is in quintile 1 and receives a score of 200. A student in the 19th percentile is also in the 1st quintile and also receives a score of 200. A student in the 22nd percentile would be in quintile 2 and receives a score of 400. Quintile 3 gets 600, 4 gets 800 and 5 gets 1000. The problem is, if you look at the average number of questions correct of a student in quintile 3 and the average number of questions correct of a student in quintile 4, the difference is less than the margin of error due to sampling error! Students could go up or down 1 quintile just by choosing different questions to include in the test, without any additional learning or skills on the students' part.
It seems immoral to me to attach such high stakes to tests that suffer from this tragic flaw from the outset. I think that we can use these tests as long as we acknowledge their limited ability to give us accurate data. When we make major funding decisions as if these results are objective fact and not broadly fallible approximations, we are playing Russian Roulette with our kids' education and future. Our kids deserve better than that.

(Resources: 1 2)

-"Definition of Statistics: The science of producing unreliable facts from reliable figures."