Is the SAT Obsolete?

Interesting story in Inside Higher Ed about the movement to make SAT and ACT scores optional at competitive-entry colleges:

Colleges that have conducted in-depth analyses of the value of standardized tests have frequently ended up questioning the tests' use. For example, the University of California recently studied whether SAT subject tests helped admissions decisions and found - generally - that they do not. Hamilton College, prior to abandoning an SAT requirement in 2006, conducted a five-year experiment being SAT-optional. During that time, the 40 percent of students who didn't submit SAT scores performed slightly better at Hamilton - a highly competitive liberal arts college - than did those who did submit scores. And in a finding consistent with studies at other colleges, Hamilton found that when it went test-optional, it received more applications from students at the top of their high school classes and many more applications from minority students.

More here.

reason on SAT's history here. Virgnia Postrel defends the test here.

Comments

[robc]

They stopped publishing it about the time I started there, but Georgia Tech used to have a public formula for predicting GPA based on high school GPA and SAT score. I think while it might have worked, it had such huge error bars that it became silly to publish. IIRC (its been over 20 years), a 4.0 GPA and 1600 SAT predicted a 3.25 GPA. 🙂

Its quite possible that was right, on average.

Anyway, the point being that SAT scores seemed to have a strong correlation at GT. I know it did in my case, much more so that my HS GPA (I had a much higher GPA at Tech than high school). At a liberal arts school, I can see how it might not correlate as well, but in a engineering/math/science school I would expect it to predict fairly well.

[Mr. Nice Guy]

As long as no one thinks SAT scores are some perfect indicator of any individual's possible performance but they in general are A useful and valid predictor of such then I'm not sure what all the hype is about. It's an indicator. No indicator is supposed to be perfect (we all know some people are more religious than others and there are many ways a measurer could "indicate" that fact, such as how many times a day they pray, how many times a week they go to church, etc., but each of those, while having some validity, is also not perfect, i.e. because a person can be very religious and not go to church, etc).

The great thing about the SAT is that it allowed many people to bypass class disadvantages by demonstrating that they could score high on it. I think it's sad that liberals today hate on it.

I can't resist suspecting that TAO did poorly on his and will find my support of the SAT to be intellectual snobbery (I scored really well on it TAO, enough to get a free ride :)). C'mon TAO, whether it be graduate education or the SAT, don't hate the playa, hate the GAME!

[Mr. Nice Guy]

"the point being that SAT scores seemed to have a strong correlation at GT. I know it did in my case, much more so that my HS GPA"

That's the idea I've always got from people who worked in setting admissions policy, because the standards for grades are so inconsistent at the various high schools (i.e. grade inflation).

"At a liberal arts school, I can see how it might not correlate as well, but in a engineering/math/science school I would expect it to predict fairly well."

I bet you're on to something there robc, there is a qualitative difference to those kinds of schools.

On a less related note, I would hate to see the SAT scores of the kind of person who would click on these ads to my right showing Palin with a shotgun and a moose.

[VM]

Hamilton is still a place of rank twaddlenockery. Was when i went there. still is.

[Warren]

When I was a high-school senior (way back in 82) I did a report on the SAT. I upset my classmates by defending it. If the goal is to admit the students that will achieve the highest GPA, the SAT was the best single predictor of that. Admissions committees without benefit of SAT scores could not do better (though they could achieve other goals such as racial diversity).

This post is interesting in that it seems to contradict my conclusion. I suspect we're dealing with outliers. Prestigious liberal art schools. My guess is that the students that didn't submit scores to these schools would have scored highly anyway.

[Warren]

Mr Nice Guy,
Who's TAO?

[The Angry Optimist]

I can't resist suspecting that TAO did poorly on his and will find my support of the SAT to be intellectual snobbery

Have I told you today that you are an asshole?

How did is no one's business and anyone who goes around bragging about it is *surprise* a grade-A asshole.

How about some manners and modesty there, MNG? Keep in mind you did spin those brilliant scores into a low-rent PhD, so I wouldn't get all nostalgic for the old days.

[VM]

intellectual snobbery?

tee hee.

so, MNG, are you heading to Clinton to let 'em know that they're wrong? 🙂

(srsly - who cares, MNG about someone's SATs? after graduation from high school they become irrelevant)

Hay Dru - you a fan of your current town's college football team?
[ducks runs off]

🙂

[The Angry Optimist]

I think there is some value in the admissions tests, FWIW. For one, the tests will help smart-but-bored students who are understimulated by high school. It should also help the smart-kid-stuck-a-bad-school demographic.

[Warren]

sorry I asked

[VM]

LOL, warren!

[The Angry Optimist]

Hay Dru - you a fan of your current town's college football team?
[ducks runs off]

Hrmph. I'm just the glad the undergrads weren't back or they would have burned the city down.

I do have to say that the shine is off of Tressel now. Most people aren't all that optimistic about the Wisconsin and Penn State games. Add that to the national championships debacles and maybe Columbus can gin up enough outrage to actually fire some of the mob-cronies Tressel brought with him from Youngstown.

srsly - who cares, MNG about someone's SATs? after graduation from high school they become irrelevant

For reals. I cannot fathom why a grown adult would still be bragging about a test he took when he was 16. Ridiculous.

[VM]

Tressel sure did have some shine for a while!

If they do get rid of his cronies, Traficant can re-form his garage band!

just heard someone call into sports radio about the Bears, "back when I played ball" was how he started. just as asinine as bragging about SATs.

[Episiarch]

As someone who scored high on the SATs, I wish they had been weighted more, myself. I would much rather have gone to Yale than Johns Hopkins.

[Sausage Swingin\' Libber]

The SAT, in my case, while not totally meaningless, was off the mark in many ways. My score: 1100. High school GPA: 2.88. Above average, but certainly not stellar. I graduated college with a 3.82 . . . and discovered along the way that I had an IQ in the top 1% of the general population. My SAT score and high school GPA certainly did not predict my collegiate success, nor did it accurately reveal my underlying intellectual capabilities.
Used on its own, it's highly unreliable - in combination with a myriad of other factors in can be useful.

[galthran]

The great thing about the SAT is that it allowed many people to bypass class disadvantages by demonstrating that they could score high on it.

Amen to that. I was both poor and lazy in high school (2.3 graduating gpa) but since I was good at taking standardized tests I got a full ride with room and board to the local university.

[Mad Max]

The SAT is, for most colleges, one factor in a "holistic" admissions process. Now some want to get rid of it altogether, because studies show that a pure holistic system is a much better predictor of college success.

Sorry, I just laughed so hard I almost suffocated.

It's one thing for a professor to rank the students in her class via a holistic process - the prof is a subject matter expert who's had the chance to examine lots of evidence of student performance, and she's assessing the students only in the subject where she herself is an expert. That's not the same as an admissions officer making a global and largely subjective judgment about how to rank applicants on the basis of their entire lives to date - including nonacademic lives.

Where are the studies showing the predictive value of the holistic system?

[FatDrunkAndStupid]

I don't know that it necessarily has to be the SAT, but obviously standardized tests perform an important function. Without them, schools have no objective criteria by which to judge applicants.

[Abdul]

The problem with our SAT experiences and the discussion of modern college admissions is that the SAT has changed drastically in the last ten years. It used to be well correlated with IQ, but the ETS has redesigned the test to break the correlation (hence, adding an essay section).

It seems there's a good market niche for a psychometrician who could design a test that would indicate college success. Perhaps a test that allows takers to demonstrate how well they can parrot their professors' preferred theories and interests.

[robc]

Without them, schools have no objective criteria by which to judge applicants.

This is why I find the "new" SAT writing section so funny. Universities are ignoring it because it is so subjective. The only universities that would probably be interested in that section are those that are getting rid of using the SAT altogether.

[stuartl]

Watching my daughter going through the process, it appeared that the schools further from us counted SAT scores higher because they did not know the school/school system she came from.

A "C" in calculus but a 5 on the AP, vs. an A and a 3 is useful information for the admissions process.

[SugarFree]

Replace the SAT with an IQ test. Done.

Funny story: Kentucky has always been a fan of the ACT for some reason. And all the universities in the state publicized their GPA/ACT admittance formula. My junior year, the star quarterback in my high school received a combined ACT of 8. This is slightly lower than the 10 that randomized guessing would produce. He took it three weeks later and scored a 22, the absolute minimum to get in with his team-admittance minimum 2.0 GPA.

Now, I'm not professing shock at academic cheating for athletics, but the sheer audacity it took for the local paper to feature him on the front page touting his "miraculous improvement" was amazing.

[libertarian democrat]

As a med school applicant, currently, I am being asked by many schools my SAT scores in combination with the MCAT (premed test). I think it's because it's still seen as one of the best predictors of academic potential, but I am not sure.

As someone who scored high on the SATs, I can tell you that I still would have considered attending a school that didn't require them (if I liked it for other reasons) because it might mean they have a more "personal" administration, which is a great thing. The school I went to I think required them, but the administration was great, and that was part of what sold me on it (they let me transfer in without being accepted, in the first few weeks of freshman classes. Very nice, helpful people).

[libertarian democrat]

Also the verbal and math scores on the SAT and GRE are still pretty highly correlated with IQ scores. That doesn't mean they should measure IQ, as the validity of a college performance test may not be that closely related to IQ (at the above average and up level).

[Mo]

I think the cottage industry around raising standardized test scores have decreased the predictive quality of the test scores. Being able to pay private tutors that teach the test and SAT test taking techniques raise scores without making the students more prepared for college. It also gives students from well-off families a greater leg up over those that don't. This may be the reason why what worked 10-20 years ago is less effective today.

[PC]

Doing some work in an admissions office during college one can see a big need for the SAT. There are countless people from rubber stamp schools that pump out high GPAs but horrendously low SATs, that should raise serious red flags. The SAT, while focusing on math and sciences really tests basic reasoning skills. The SAT is also a good benchmark to show just how deficient American Schools have become.

[Episiarch]

Replace the SAT with an IQ test. Done.

But that would discriminate against dumb people! And retards!

[SugarFree]

[dabs my eyes dry]

WSTOTR?!?

[libertarian democrat]

I've never taken one of those courses or had a tutor, but I think a majority of what they teach are test taking strategies. This helps students perform better on college tests (more validity there, of some sort) and to a lesser extent is useful in real life situations, because they are basic reasoning skills that many people lack.

[robc]

SugarFree,

I got power back yesterday. 5 hours short of a full week.

Anyway, on topic, I took the ACT too, since I grew up in KY.

I think the SAT measures intelligence while the ACT measures knowledge. That's probably simplistic, but the 2 extra sections of the ACT are more about what you have learned.

I dont know though, you never hear the ACT discussed. It seems the set of people who took both are mostly KY high school students who wanted to go out of state to school. What other states are primarily ACT?

[libertarian democrat]

However, I was having a meeting the other day with a clinical psychologist and statistician who works on figuring out how much time people with learning disabilities should get. For some, I think it makes some sense, in that they may do very well in college overall, despite being bad in time sensitive arenas, but that is not the case for most of them.

He said that the vast majority of people he sees aren't in either of those categories. Most are rich or spoiled children whose parents won't give up on the idea that there must be something wrong if they aren't doing well. Luckily, their requests are summarily rejected.

[SugarFree]

got power back yesterday. 5 hours short of a full week.

Son of a bitch fuck. The only time our power was off for a week was the 2002 ice storm. What the hell was going on? Did all the line crews leave for the Gulf Coast and just got back?

[robc]

SugarFree,

Did all the line crews leave for the Gulf Coast and just got back?

Some of that, they got back 1-2 days in. The last few days, there have been crews in town from all over the place, trying to get schools up by today. There were about 40 schools still without power on Friday. I think Im back becuase of an elementary school nearby.

I heard saturday afternoon (in a bar, so YMMV) that they had to order transformers from out of town, they ran out.

Everyone isnt back on yet, but I think it is close. 280k homes were without power originally, they finally got below 100k on Friday, I think. It seems everyone I knew that was still out got power back Sat or Sun.

[Mo]

lib dem,

I don't know what college you went to, but I took very few multiple choice tests. The ones I took still gave partial credit for showing your work and eliminated the "obvious" wrong answers, so that you had to actually figure out the problems. Standardized test taking strategies have limited value outside of taking standardized tests.

robc,

MY gf said that she took the ACT in high school. She's from Iowa. Everyone that I know that took the ACT is from the Midwest. Note that there are SAT subject tests which address the knowledge thing.

[SugarFree]

Thank goodness they rushed to get the schools back open. I'm sure those parents were getting really sick of babysitting their own kids. Imagine the horror of spending time with your own children. [shudder]

(Can you tell I really hated school?)

[Susan White]

I dont know about caolleges but I do know that thanks to the "No Child Left Behind" act, that High School SATs have been so "dumbed down" that any 6 year old can pass them. Its all for the sake of Federal funding! Imagine that.

Susan
http://www.FireMe.To/udi

[The Angry Optimist]

What other states are primarily ACT?

Here's a Wiki Map that seems to make sense. I'm in Ohio and we were almost all ACT.

[libertarian democrat]

Mo,

Like I said, I don't really know exactly what they teach. But, when answering a short answer sort of question (which was most common in my classes), pairing down the possibilities (in your head) and recognizing any misleading text is important. I am not sure, however, if there is misleading or extraneous info in the SATs. I took the GREs and MCATs more recently, and both of those have them. The GRE is by the same people as the SAT, so I thought they were similar, but that might be a difference.

[Seitz]

I do have to say that the shine is off of Tressel now.

Man, I hope this doesn't mean the end of Tressel's blog (though he hasn't updated in a year).

The great thing about the SAT is that it allowed many people to bypass class disadvantages by demonstrating that they could score high on it.

Really? I thought the great thing about the SAT was that I could coast through high schools, do well on a test on one Saturday, and get into a pretty good undergraduate university (note: this strategy also works fairly well for the LSAT).

[Seitz]

Also, the excerpt in the post refers to the SAT subject test, which in my day were called the Achievement Tests. They have my blessing if they want to drop those. I took the German subject test, and did OK, but by percentile, my score sucked. Why? Because most people who take German in high school aren't dumb enough to take the German subject exam. It's primarily taken by native speakers, which kind of defeats the purpose.

[Seitz]

Oops, my bad on the Tressel Link.

[Abdul]

Mo,

In response to your comment about the rise in SAT prep courses, I read in a Charles Murray column that no independent study has found a serious rise in scores due to test prep courses and tutoring.

[R C Dean]

Where are the studies showing the predictive value of the holistic system?

Exactly. Sure, the SAT is imperfect. So what's the alternative? Where's the data showing the alternative is better?

I don't know that it necessarily has to be the SAT, but obviously standardized tests perform an important function. Without them, schools have no objective criteria by which to judge applicants.

Pedantic quibble: I would say standardized tests provide a uniform criteria by which to judge applicants. And that is no small thing.

[bubba]

Hamilton became a refuge for people with high GPAs who sucked at standardized tests.

That's great if you're the only college doing it, but it's not something that is likely to generalize.

[Seitz]

I read in a Charles Murray column that no independent study has found a serious rise in scores due to test prep courses and tutoring.

I took the LSAT after using books to study, never took a class. I later taught a bit for the Princeton Review. I think the value in test prep classes, outside of what you can get from a book, are the practice tests in a test environment. I don't think it would have mattered for me, but for some students, I think they do better having been through at least a process simulating the real thing. I gave myself practice tests, but there were always distractions.

In my view, the problem with prep classes is that you're paying a lot of money for teachers that may very well be not very good. The PR required teachers to have achieved a certain score. So that's great, you have people who did well on the test teaching the material. The problem is, a lot of the stuff either came naturally to me, or was fairly easy to learn. But I found it really hard to teach the material to people who just didn't get it. We could spend two hours on one of the games and they just couldn't grasp it, which was as much my fault as theirs. But that's a lot of money to pay for someone (like me) who isn't good at teaching the material.

[jayjayhawker]

The GRE should be next

[SugarFree]

Very little prep possible for an IQ test. Sounds like my idea would wipe out a parasitical industry. Win-win.

[libertarian democrat]

Very little prep possible for an IQ test. Sounds like my idea would wipe out a parasitical industry. Win-win.

Actually, you can prep for it, if you know what sort of test it is. Practice can help at alot of common subtests.

[The Angry Optimist]

we can tie these phenomena back to government interference in the education market. But for government subsidization of education, in the forms of loans and grants, there would be no educational inflation (that is, your HS Diploma is almost worthless nowadays and a BA/BS is becoming increasingly worthless) because the government is sending people to college who do not have any business being there.

[SugarFree]

Well, we can keep it secret, right? SAT and ACT do that already. In fact, couldn't most if not all of the test be generated at random while taking it on the computer? I know that was how the GRE fucked me over in the vocabulary portion...

[libertarian democrat]

I loved the GRE. The increasing feeling of punishment as you do better and better.

But I mean, some standard things in IQ tests (and while I question their validity, they seem to have predictive validity) include things like digit span memory, digit symbol reproduction, and the like, can be practiced and improved upon, if only a little. I am sure if it was as important as the SAT, alot of people would do anything they could for even modest increases.

I am not an expert in psychometrics, I just think they are fun, but I am pretty sure most of the other standard subtests could be improved with practice on the general principles.

[jtuf]

The SAT is greatly over rated and I'm all for scrapping it. Testing general intellegence is impossible, but I would like to see colleges use a skills test. A test of math skills or language tests in multiple languages would provide an atlernative way for high school students to shine even if they are stuck in crumbling schools.

[DJP]

I'm surprised nobody has mentioned the fundamental reason that universities are going through the mental gymnastics of acting like the SAT doesn't matter: non-Asian-minority (NAM) SAT performance.

The fact of the matter is that NAMs tend to perform about 1 standard deviation below non-NAMs. This fact largely persists even when correcting for everything else (family income, GPA, high school demographics).

Regardless of the usefulness of the SAT, the more weight that is given to it, the harder it is to fill NAM affirmative action quotas.

The reality, of course, is that the SAT has a stronger correlation with college GPA than any other practical metric. There is extensive real research on this, as opposed to the convulated cases you will find some PC college admissions offices trying to argue.

[TallDave]

a highly competitive liberal arts college

Permit me a howl of derisive laughter at this phrase: bwahahahahahahahaha!

40 percent of students who didn't submit SAT scores performed slightly better at Hamilton -

So we've proved liberal arts colleges require no intellectual skills -- something the job market has been telling us for years.

[TallDave]

I mean, really, wake me up when they find this applies to the technical schools. You know, the places that teach things that are actually useful.

non-Asian-minority (NAM) SAT performance.

Once again, this is either racist against Asians (by punishing them) or everyone else (saying all the other minorities are dumber than Asians).

We're never going to achieve a colorblind meritocracy if every racial discrepancy is given a handout.

[sheesh]

talldave shows yet again what a moron he is.

[vinny gates]

I support the standardized tests simply because I was one of those bored-by-high-school (thanks, lowest common denominator public education!) kids who got into a decent school (well, University of Louisville) with an awful GPA and a good ACT score. Otherwise I'd be stuck at some bumfuck eastern KY community college.

[Mr. Nice Guy]

"there would be no educational inflation (that is, your HS Diploma is almost worthless nowadays and a BA/BS is becoming increasingly worthless) because the government is sending people to college who do not have any business being there."

TAO you intellectual snob! BA/BS's are held by some of the smartest people in the world, like Sarah Palin. How dare you demean the educational achievement of folks with them as "increasingly worthless!" You are a rank elitist and asshole!

Most of the posters here are probably in the group that "the government is sending to college who do not have any business being there" (either through subsidized loans, state supported schools, etc). The "old" system meant: you gots the money and the brains you go, you no gots the money but gots the brains, you no go, which is hardly a valuable merit system. State supported education has allowed for all kinds of folks who had much to contribute but could not get the scarce scholarships available to prove themselves in college and go on to contribute quite a bit to society (like Milton Friedman, Rutgers undergaduate).

Of course subsidization plays a part, but there are actually market forces to drive grade inflation: more kids pass, the higher the retention rates, the more tuition the schools get. The more kids pass the more likely they are to send their kids to that school (have a favorable opinion of the school). And there are for educational inflation as well: creating the need for new products is what good organizations do, and by bringing education to more people (like bringing tvs to more people) it of course leads to offering the cream of the crop that grad degree (like the plasma tv).