Wednesday, November 16, 2005

The Impossible Dream

So, I was just looking at my school's API report for last year. If you don't know what an API report is, I envy you. Let me ruin you now: API stands for "Academic Performance Index." It's an average of the scores on a statewide standardized test for all of the students in your school. The way a student gets a score is not based on how many questions they got right... not directly, anyway. What happens is that all students are ranked based on the number of questions they got right. The bottom 20% of students get a score of "200." The next 20% get a score of "400," and so on. This divides the students into five groups of 20% each. These groups are called quintiles. Everyone in the quintile gets the same score, no matter what the range of the quintile is. It's entirely possible for a student in the bottom quintile to have answered 85% of the questions correctly if the test was really easy, or for a student in the top quintile to have answered only 20% of the questions correctly if the test was very difficult.
So, our school had an API of 575 last year, up from 528 the year before. We gained 47 points, which is pretty impressive when you consider that the goal the state set for us was just 14 points. We also surpassed our targets in all of the key demographics, which for our school were black, latino, and socio-economically disadvantaged students.
What does an API of 575 mean? Well since individual students can only get scores of 200, 400, 600, 800, or 1000, it means that we probably had most kids in the 3rd quintile, with a few in the second quintile. That makes us statistically average, since a student right in the middle of the state would score 600. (for statisticians reading this: I know this is confusing the meaning of "mean" and "median," but the whole system's screwed up anyway, so give me a break!)
This is good news, isn't it? I mean, if you're average, you're not... below average. You're not above average, either, but come on! Isn't it enough to ask of a significantly poor, minority, urban school that they do as well as the average students in the state? It seems so to me.
But not to the California Department of Education and the state government in Sacramento. If you read the legend below the initial score report, it says that an "A" "means the school scored at or above the statewide performance target of 800 in 2004."
Did you get that? The statewide performance target is 800. Do you remember what 800 means? It doesn't mean that you got 80% of the questions right. It means that you were among the top 40% of students in the state. No matter how many questions you got right. Or wrong.
So, the "performance target" that the state is setting up is for every school to be in the top 40%. If you're not a mathematician, let me say that another way: ... no, I won't. Because you don't have to be a mathematician to figure out that it's IMPOSSIBLE for every school to be in the top 40%! Do you know how many schools were in the top 40% last year? 40%. The year before that? 40%. Let me do a little prognosticating and tell you that I will bet 100 million dollars that next year, 40% of the schools will be in the top 40%!
I can just picture the geniuses at the department of ed. scratching their heads and saying, "gee, student performance went way up this year, but there's still the same number of schools meeting the statewide performance target of 800. How is that possible?" or, "Wow, this year we forgot to mail out the test books and every student just randomly guessed, and we STILL had exactly the same number of schools meeting the statewide performance target of 800! How did they do it?"
There's nothing better for a bureaucracy than making it's reason for existing a goal that is by definition impossible to achieve. The standardized testing racket will continue to be given millions of your tax dollars to give you the same meaningless statistics year after year: Once again, 40% of the schools in the state met their performance target and - surprise! - they're the schools that are predominantly white and asian non-socio-economically disadvantaged students with university educated parents. The other 60% of the schools "failed."
We are doomed to have 50% of our students perform below average year after year. Not doomed by fate or market forces or social and cultural disadvantages. Doomed by the nature of numbers and the definitions we don't understand. This is not a doom we should fear, it is a reality that we should dismiss as inherent to math and nothing to worry about. Being below average isn't a bad thing, it's inevitable that half of any given group are below average by definition! Half of the geniuses in the world are "below-average" geniuses!
Anyway, I look forward to a future where all of our children are above average, human flatulence is found to be a low-cost source of simian organisms, and the number one cause of airline crashes is mid-flight porcine collision. It's an impossible future, but when your governor is a time-traveling killer robot, no dream is too big, or too stupid to throw millions of tax dollars at... especially if its money that could have been used to actually educate children.

- "'You'll be on the run with no friends! You'll live in constant danger of betrayal!' 'I live that way now!'"

2 comments:

Sean said...

Catch-22

Mr. Mac said...

It's from the movie... but the title of the movie is apropos of the topic on which I write... hence the choice of quote!