Friday, March 16, 2007

How Standardized Testing is Killing American Education: Reason #7

Fourth in a series.

7) English Learners unfairly penalized: One would expect someone who is learning English to score lower on a test of English proficiency than a native English speaker. Test scores bear that out, and it's not surprising or unexpected.
There is no reason to assume, however, that someone who is learning English will necessarily be less proficient in math, or science, or general social studies (apart from US History) than native English speakers. Indeed, a student who received more education in their home country before coming to the US than a native speaker would be expected to be more proficient. At the very least, we should assume that the scores in these non-English subjects should be roughly equivalent. A more realistic assumption based on international research would be that foreign-educated students might actually score better than native English speakers educated in the American public-education system.
So why do English learners consistently score lower in math than native speakers? A cursory glance at the test will reveal this immediately: the format of the questions require a level of English proficiency just to understand what the question is; a level of English proficiency that many English learners have yet to attain.
Math questions are almost all contextualized word problems. A problem given as "4+7=___" could be accessed by anyone, regardless of English proficiency. When it is given instead as "Farmer Brown has 4 chickens and 7 ducks. How many birds does he have?" we run into problems. First, vocabulary: Farmer, chicken, duck, bird... if a student doesn't know these words, the question becomes more difficult, and not because of any deficiency in mathematical skills. Add to that the problem that comes with not realizing that "chickens" and "ducks" are both subsets of the larger category "birds" (if a student has heard of chickens but not ducks, they might legitimately answer "4"), and the various conjugations of the irregular verb "to have" (you know that "has" and "have" mean the same thing in this sentence... does an English learner know that?) and a student might get the answer wrong for several reasons that have nothing to do with their level of math proficiency, and math proficiency is what this test is supposed to be assessing.
So what do we do about it? More accurate results could come by letting students use a translating dictionary for the non-English portions of the test. This would of course have to be tied to an increase in the time allowed, since looking up 3 or 4 words per questions will most likely more than double the time necessary to finish the test. Even without the dictionary, English learners need more time to read and comprehend English texts, so the time extension or even giving unlimited time would go a long way toward redressing this inequity. Unfortunately, both of these suggestions make-up the "axis of evil" for standardized test makers: the argument is that the point of the test being "standardized" is that scores can be compared fairly because every student takes the exact same test under the exact same conditions. Any variation in conditions destroys standardization by this view. I would argue that for all subjects other than English, the opposite is true, and a fair and equitable measuring stick for students proficiency in subjects other than English cannot be attained until we sacrifice these sacred cows on the altar of equal opportunity.

(Resources: 1 2)

-"Medium-Head Boy!.... You see, he doesn't know!"

How Standardized Testing is Killing American Education: Reason #8

Third in a series.

8) "Scaled scores" don't tell you anything about student learning: Standardized tests scores are given as "scaled scores." This means that your score is not based directly on how many questions you got right: a student who answered 25% of the questions correctly would not receive a score half that of a student who answered 50% of the questions correctly. Rather, the scores tell you how many other students that took the test scored worse than you did. A student who scores in the 35th percentile did not necessarily get 35% of the questions correct. What happened is that 35% of the students who took the same test got less questions correct than that student did. It's possible that they got 35% of the questions correct, but it's just as possible that they got 5% of the questions correct, or 75%, or even 90%. A scaled score doesn't tell us anything about the number of questions answered correctly.
Likewise, improvement on a scaled score doesn't necessarily indicate improvement in learning. A student could answer 45% of the questions correctly one year and 55% the next year. Their scaled score could improve, or drop, or stay the same, depending on whether other students improved similarly or not. Ideally, we want all students to improve, don't we? Well, if that happens at the same rate, our scaled scores will not change at all, and will give no indication that the outcome we most desire is actually taking place!
Scaled scores are deceptive on several counts. First of all, it is not uncommon for someone to think that someone with a scaled score under 50% has mastered less than 50% of the material. That is not true. Someone with a scaled score of 50% scored higher than 50% of the students who took the same test. In other words, this is a totally average student. Right in the middle. Typical of American students in general. This students actual score could tell us a lot about the state of American education: if an average student has an actual score of 20%, we would be disappointed; likewise, an "average" actual score of 85% would be very encouraging. Unfortunately, the only score we're ever exposed to is the scaled score, which doesn't tell us a lot about what (or whether) students are actually learning.

(Resources: 1 2)

-"USA Today has come out with a new survey: Apparently three out of four people make up 75 percent of the population."

How Standardized Testing is Killing American Education: Reason #9

Second in a series.

9) Norming is biased: "Norming" refers to comparing one students' results against all other students to determine how they compare to the population at large. Most of the time, however, the "population at large" scores are compared to is actually a smaller sample of the entire population which is judged to be a representative sample of the entire population. This smaller sample is given the test early, and those results are used to set up a virtual spread of scores.
So, you have two problems: how do you assure that your sample is truly representative of the larger population? You can select for race, number of years in the country, socio-economic status, parents' education, region of the country, gender, age, and a host of other variables that may or may not have some bearing on results, but no matter how big your sample is, you're always going to have sampling error. Choosing a representative sample is also really hard and expensive, so instead, samples tend to be less representative in favor of choosing students from the same geographical area, often close to the location of the test-makers offices. For the SAT, that meant that upper-middle class, predominantly white students were the sample that the test was normed against for years. Remember a few years back when they "rescaled" the scores and people complained that they were lowering the bar by making grading "easier"? What actually happened was that a more representative sample was used and the college board realized that their sample had been skewing the Norm high for years. The new scores are more accurate because they're based on a more representative sample.
The second big problem is just regular old sampling error. You can't get away from it. When you compound the sampling error inherent in choosing test questions with the sampling error from the group used to set the Norm, the reliability of the test results becomes shakier and shakier.
Several years ago, as Reformed Math made Integrated courses more popular, California debuted Integrated Math Standardized tests as options for schools. For several years, the results were impossible to norm: that is to say, results did not fit a normal distribution as you would expect from an unbiased test. Results had to be fiddled with and forced artificially into a normal curve. You'd think that this would reveal a flaw in the testing (even more than the normal level of error which is considerable) and states and districts might hold back on making major decisions based on these scores. No such luck. Bureaucracy reigns supreme, and the wheels of progress have too much inertia to stop turning, even if it means innocent students are crushed underneath.

(Resources: 1 2)

- "He uses statistics as a drunken man uses lampposts—for support rather than for illumination."

The Annual Standardized Testing Rant: First in a series!

Welcome back to my favorite topic: how standardized tests are killing American education. I've tackled this topic before, so this year I'm going to go for a series of the main reasons I detest standardized testing so much in the form of a "top ten" list." Here we go (drum roll, please!):

10) Sampling error makes it impossible to get accurate results: "Sampling Error" refers to the inherent error that exists when you choose a small sample of all possible items to evaluate mastery of the entire set. For standardized tests, there are millions of possible questions that could be asked to assess students' mastery of the standards that students are supposed to learn in a given year. To create a usable test, a small number of those possible questions must be chosen. The assumption is that the questions are chosen carefully enough so that they are representative of all possible questions. In other words, if a student answers 70% of the sample questions correctly, the assumption is that they would have answered 70% of all possible questions correctly.
"Sampling error" is a mathematical term that refers to the probability that the sample score is close (usually 90% or 95% accuracy is checked for) to the actual score the student would have received if tested on all questions. You see this number when political polls results are reported, it's called the "margin of error." So if candidate A is poled at 40% and candidate B is polled at 45% but the margin of error is + or - 7%, you would say that they are in a statistical tie. The margin of sampling error is greater than the difference between the results, meaning that the poll doesn't really indicate a clear advantage for either candidate.
For the California standards tests, students scores are grouped into "quintiles," where a student in the bottom 20% is in quintile 1, students in the next 20% (21% to 40%) are in quintile 2, etc. A student who is in the 3rd percentile is in quintile 1 and receives a score of 200. A student in the 19th percentile is also in the 1st quintile and also receives a score of 200. A student in the 22nd percentile would be in quintile 2 and receives a score of 400. Quintile 3 gets 600, 4 gets 800 and 5 gets 1000. The problem is, if you look at the average number of questions correct of a student in quintile 3 and the average number of questions correct of a student in quintile 4, the difference is less than the margin of error due to sampling error! Students could go up or down 1 quintile just by choosing different questions to include in the test, without any additional learning or skills on the students' part.
It seems immoral to me to attach such high stakes to tests that suffer from this tragic flaw from the outset. I think that we can use these tests as long as we acknowledge their limited ability to give us accurate data. When we make major funding decisions as if these results are objective fact and not broadly fallible approximations, we are playing Russian Roulette with our kids' education and future. Our kids deserve better than that.

(Resources: 1 2)

-"Definition of Statistics: The science of producing unreliable facts from reliable figures."

Thursday, January 04, 2007

New and Improved Blogging!

Hey, all,

I've added a feature to my blog that may or may not be of interest to you. On the left side of the page, there's now a list of "labels" (below the Blog Archive which is sorted by date). This will let you read the posts you really want to without having to wade through all that other stuff. Why read 22 posts on politics looking for that one on Harry Potter? Now you don't have to! You're welcome. Let me know if you can think of any more labels that would make it easier to find your favorite posts on my blog (and before you even start, I've already rejected "digital facial hair enhancement" so forget it! You'll just have to use the "Photoshop" label like everyone else!)

- "Gentlemen, we can rebuild him..."

100 hours of Pelosi

It's no more than we should expect, but it's still entertaining.

It should be no surprise that new Speaker Pelosi announced that there would be a new bipartisan tenor to the House under her leadership. After all, it's what her party complained about all through their last 12 years in the minority, and especially the last 6 years with 2 branches of the federal goverment in Republican hands.

It should also not be surprising that she has unveiled a plan for the "first 100 hours" of the 110th congress (that's about 2 weeks, I think) that pretty much shuts the Republican minority out and pushes through several key pieces of Democrat-backed legislation that languished under Republican control.

And of course, we should not be surprised that the Republicans in congress are complaining about Pelosi's decision to rush legislation that has been crafted by Democrats to votes without allowing them sufficient time to amend or debate.

Come on people, it's politics.

Of course Pelosi is going to come in promising to change everything that her opponents had been doing wrong, and to treat them better than her party had been treated while in the minority.

And of course as soon as she has the authority to actually make decisions, she's going to succumb to the temptation to push through as much broadly supported, non-controversial (or at least not-very-controversial) legislation as she can, especially when it's stuff that Democrats across the country were promising as part of taking control of Congress and it presents the appearance that things are really changing.

And of course she promises to let the Republicans play as soon as this really important stuff is settled. 100 hours, that's it, honest.

And of course the Republicans pull out the exact same complaints and arguments that the Democrats had been leveling at them for the last 6 (or, depending on how you look at it, 12) years that they're being sidelined and that legislation important to them is getting rushed through or put into drawers in committee meeting rooms. Because this is politics and everything has a half-life no longer than the daily news cycle, they are able to make these arguments without the slightest acknowledgement of any irony or awkwardness, brashly declaring "You're not playing fair!" without even a blush acknowledging the unsaid follow-up "You're doing it the same way we did for years!" or even a sheepish "I guess after the way we led, we should have expected this."

The memory of the political news cycle is notoriously short, and that's working out great for all parties involved. Pelosi can push forward in a sharply partisan fashion while promising that it's just for 100 hours, knowing full well that her prior promises of bipartisan partnership are no longer worth reporting on, and that 125 legislative hours from now nobody will bother reporting on whether or not she kept a promise she made two weeks previously. Promises are political fool's gold. Everyone reports on the promises and they get touted as accomplishments, but follow-up is poor and not "exciting" news. The politicians know it and they play it for all it's worth.

The same can be said for the Republicans in congress who feel complete freedom to blast Pelosi with the same scornful words that were justly leveled at them in years past, without shame or fear of being called "flip-floppers" (unless they run for President, I guess). The press corps is like an institutionalized version of Tom Hanks' character "Mr. Short Term Memory" from Saturday Night Live. If it happened more than a couple of days ago, it's not news. Their job is to report on what's happening now, not remind us of what happened before. The lack of accountability gives politicians license to make grandiose promises and count on the news reporting only on the promises and not coming back to follow-up.

So take the promises with a grain of salt. Check on the Congress in 200 hours or so for signs of bipartisanship in the agenda. Until then, it's politics as usual.

- "I want answers now, or I want them eventually!"

Friday, December 15, 2006

Belated Christmas Card (from Alec Baldwin)

Michelle and I were talking recently about what we would put on a Christmas "newsletter" if we were to write one, and I was reminded of this hilarious sketch from Saturday Night Live for Christmas of 1995. Alec Baldwin took the opportunity of being on live TV a week before Christmas to deliver a live, video Christmas card to everyone he forgot to mail one to. Here's the transcript:

During the Christmas season, I received many Christmas cards. Unfortunately, because I was so busy, I didn't have time to send out any myself. So, if I could, I'd like to use this as my Christmas card to everyone who was kind enough to send one to me.[ puts Santa hat on his head and begins]

"Dear Friend, or Relative, or Business Associate. Merry Christmas, or Hanukkah, or Kwanzaa, or Solstice, or Voodoo Day.

Boy, what a year it's been - me, with my acting, and, you doing whatever it is that you do. Thanks for the Christmas card, it was very beautiful, or humorous. I enjoyed the photo of your kid, or kids. Boy, he, or she, or they are really getting big. Have you seen our mutual friend, if we have one? Can you believe what he or she is up to? Boy, some people!

Has your son, or daughter, or sister, or brother, or husband, or wife still have that drug problem? All you can do is trust in God, or, if you prefer, voodoo.

How is Granny, or Nana, or Mimi, or Yaya? So so? Hey, how about the professional sports team that we both root for? They should fire, or rehire that manager of theirs. He's a character!

Well, gotta go. By the way, sorry about throwing up on your carpet that time, or times. I hope you receive many presents from Santa, or Hanukkah Guy, or the Voodoo Man.. and that the coming year is as good as, or better than, or nothing like the last year.

Love, Alec."

Friday, October 06, 2006

Pro Life?

I've just been thinking about this for a while... I thought it might be worth writing down my thoughts.

"Pro Life" is a politically charged slogan that has been aligned with a single issue in the last 30 years: abortion. It has been a polarizing issue, and one of a few issues that spring to mind when people answer the poll question "how important are 'moral' issues to you in deciding who (or what) to vote for?". The wording seems to make it obvious on which side of the debate Christians (or any person of conscience) should lie: who would say "yes" to the question "Are you 'Pro Death'?".

So, if you had asked me a year ago if I was "Pro Life," I would have said "yes," and I would have meant exactly what you thought I meant. I would have meant that I am opposed to abortion in the same way that I am opposed to murder: I believe that just because a person is still living inside of its mother's womb doesn't mean that its mother gets to decide whether or not it should die.

In the past year, however, I've heard a few things that have challenged that definition as being too narrow. First of all, I read Jim Wallis's book God's Politics over the summer. He says that Christian's need to be "Consistently Pro-Life" and not limit our passion for survival to the unborn. I also heard a powerful sermon from my pastor in which she talked about "Pro-Life" needing to inform our opinions not only of whether or not children or born, but the quality of life they experience after that event.

So, am I "Pro Life"? Yes. But it doesn't just mean I'm opposed to abortion being a legal option in this country.

Because I am "Pro Life," I think that we have a responsibility to defend the innocent Fur, Zaghawa and Massaleit people of Darfur from genocide at least as vigorously as we are defending the people of Iraq from a violent insurgency.

Because I am "Pro Life," I think that the worst act one person can commit against another person is taking their life, and we as a society must guard our own humanity by relegating those guilty of such crimes to a life of imprisonment without parole instead of making ourselves just like them by murdering them in righteous indignation.

Because I am "Pro Life," I think that we have a responsibility to provide millions of poor and dying widows and orphans in Africa with the food, water and medication they need to survive the ravages of extreme poverty and AIDS... a responsibility that should usurp our responsibility to the billion dollar portfolios of American pharmaceutical companies.

Because I am "Pro Life," I think that we have a responsibility to children and mothers in poor neighborhoods to insist that they be able to walk to a grocery store with as much selection and prices as low as the supermakets in suburban neighborhoods, and they should be able to walk there without exhausting themselves or passing a dozen liquor stores on the way.

Because I am "Pro Life," I think that we have a responsibility to make choices when we shop and when we vote that allow both farmers in Central and South America and peasant children in Southeast Asia to receive a living wage for the back-breaking labor that they do from dawn to dusk, seven days a week, from childhood until they are too old to stand... a responsibility that trumps our desire to save a few bucks, or wear the same shoes as Michael Jordan, or have our favorite coffee every morning.

Because I am "Pro Life," I think that we have a responsibility to a child born (not aborted) into a culture and cycle of poverty to adequately fund and reform public education so that their zip code doesn't determine whether they end up in college, jail, or Iraq.

Because I am "Pro Life," I think that we have a responsibility to exhaust every diplomatic strategy possible (and maybe even compromise on our own preferences for the sake of peace) before launching pre-emptive strikes against countries that might possibly be able to attack us at some indeterminate point in the future assuming our intelligence is not faulty.

Because I am "Pro Life," I think that we should be upset when the CEO and board of directors of an American company smugly award themselves multi-million dollar bonuses for the money they've made for their company... money they've made by "cutting costs" with waves of layoffs that take away the livelihoods of American workers and the pensions of retired workers, and "outsourcing" by creating a factory just across the Rio Grande where they pay pennies on the dollar to people so desperate for food that they'll work for slave wages.

And yes, because I am "Pro Life," I think that we should work to save the lives of as many unborn children as we can. If we are called as Christians to stand up for the defenseless and weak against the strong and the powerful, who is more defenseless than a child still dependent on its mother's body for every necessity of life? I don't believe, however, that making it against the law will accomplish this goal by itself. If abortion is murder, and murder is illegal, than reversing Roe v. Wade should make abortion as rare as murder is... and murder in our country is not nearly rare enough. I dare say that there may be a few "Pro Life" people out there that would be satisfied if abortion was still prevalent, but confined to the same neighborhoods in which murder is so prevalent in our country today, but I am not one of them.

The irony is that this is one point on which leading Republicans and Democrats agree: both parties want there to be less abortions performed than are being performed currently. The problem is that one side is committed to making it illegal as the only path that they're willing to pursue, and the other side is committed to keeping it legal no matter what. The debate has been trapped in this one question which our experience has shown will not in and of itself put an end to abortion... it will just give us a new label ("criminal") to put on the doctors and mothers who continue to pursue this option.

Will the number of abortions go down if it is made illegal? Maybe. Probably. I hope so. And if even a handful of lives are saved, then I think this is a path worth pursuing. But it shouldn't be the only path we pursue. If the war on drugs has taught us nothing else, it's that simply making something illegal doesn't make the problem go away... it may actually end up creating horrible violence both home and abroad as well as defining a steady source of income for organized crime, cartels, gangs, and street thugs. I'm not saying that these exact outcomes will be the results of making abortion illegal again... the point is that I don't know (and neither does anyone else) exactly what other issues might arise as a result of addressing abortion only by making those who seek it into criminals.

So what should we do? I think that if we truly believe that the unborn are people, then we should continue to work to make abortion illegal, since we define it as murder. Just because people still get killed in drive-by shootings doesn't mean that it should be legal. But it should mean that our work is not done once it is illegal. Let's also work to make this a world and a nation in which less people want to have abortions: where a poor mother knows that her child will be able to eat enough healthy food and have alternatives to gangs and be able to get into and afford a four-year university. Where children born in poor countries are more likely to die of old age than malnutrition or AIDS before they're old enough to go to school. Where a woman doesn't have to give up her career (and a vital source of income for a growing family) when she has a child because childcare is affordable (and even provided by her employer because of government subsidies and tax breaks), and health insurance plans cover children without charging exorbitant premiums or co-payments. And we need to use our resources to create places and communities where those women who are desperate and without resources can find hope and a family that they lost when they found out they were pregnant (check this out to see an example of what I'm talking about).

We can be Pro Life without being Pro-Death-Penalty, Pro-Big-Business, Pro-Military-Imperialism, or Anti-Capital-Gains/Inheritance-Taxes-On-The-Super-Rich. I would actually argue that we must be. Let's not feel defeated if the number of abortions actually comes down before it's illegal because of bi-partisan efforts to make "Pro-Life" apply to post-natal life as well as pre-natal life. Let's rejoice for those few lives saved and work and pray for more and more each day.

- "Ye know, the church says wearin' one o' them's a sin, darlin'." "So's this. Darlin'."

Monday, May 15, 2006

You know who I like? Corky.

So, Karl Rove blames the President's low approval ratings on the war in Iraq. Okay, Karl... what's your point? If we aren't thrilled about the war, who should we disapprove of?
Maybe Karl thought we should disapprove of the soldiers. It's their fault; they're not fighting hard enough. They should take a page from the suicide bombers play book and make sure that if they're going to die that they take out a dozen insurgents with them. Or Iraqi civilians. Whatever. Just keep that body count climbing... for freedom!
It seems to me that in a goverment where the President also holds the title of commander in chief, and in which he asked Congress to give up their authority to commit military forces to him, it's only fair for the President to be the recipient of at least some of the disapproval that comes up when the poorly-planned, ill-advised, under-funded, unpopular, unilateral war is not going well.
At any rate, it seems that Rove feels it's too bad that people are focusing on this whole "war" thing, because Bush is really a swell guy. Or rather, he's "likeable."
That's right, Rove actually seemed to be giving more weight to Bush's "likability" rating than his approval rating. Rove is less concerned about whether people approve of the job the President is doing than he is with whether or not they "like" him.
I'm a little concerned when one of the President's top advisors (or whatever he is now) is very concerned that people find the Presidnet to be "likeable." I mean, likeability is an important trait for the main character of a sitcom or family drama. Especially if you give the main character Down's syndrome and make them incredibly naive, and then week after week have them duped into commiting crimes or mean pranks by people that they think are their friends ("But why would he lie? I don't understand..." "But he said it would help people, I don't understand..." "But he said he was my friend, I don't understand..."). Who's more likeable than Corky? Maybe Scooby-Doo. Or Mr. Bean. I like him a lot.
Problem is, I would be digging myself a bomb shelter in the backyard if any of these characters were actually President ("But he told me the button was for ice cream, not missiles! Why would he lie? I don't understand...") . It would be nice if I liked someone who was qualified to be President, but if it comes down to it, I'll vote for someone I don't particularly care to socialize with if I approve of the job they'll do or have done as President.
Maybe that's the Democrats' problem. I mean, how likeable were Gore or Kerry? This bodes poorly for the future: Hilary? Not really likeable. Scary, yes. Likeable? Uh... is Corky a Democrat?

- "You like me! You really like me!"

Friday, May 05, 2006

IVBCF Alum...

Hey, everyone,

On the off chance that you are a graduate of UCLA's InterVarsity Christian Fellowship Chapter, you are invited to the BCF Alumni Conference! For details, follow this link to another link on my brother's blog which will hopefully send you to the organizers and not an endless series of links.

- No quote, sorry.