THE CHRONICLE OF HIGHER EDUCATION

March 9, 2001

Two Cheers for an End to the SAT

By Alfie Kohn

One imagines the folks at the College Board blushing deeply when, a few years back, they announced that the “A” in SAT no longer stood for “Aptitude.” Scarlet, after all, would be an appropriate color to turn while, in effect, conceding that the test wasn’t — and, let’s face it, never had been — a measure of intellectual aptitude. For a brief period, the examination was rechristened the Scholastic Assessment Test, a name presumably generated by the Department of Redundancy Department. Today, literally — and perhaps figuratively — SAT doesn’t stand for anything at all.

It wasn’t the significance of the shift in the SAT’s name that recently produced an epiphany for Richard C. Atkinson, president of the University of California. Rather, the tipping point in deciding to urge the elimination of the SAT as a requirement for admission came last year during a visit to the upscale private school his grandchildren attend. There, he watched as 12-year-olds were drilled on verbal analogies, part of an extended training that, he said in announcing his proposal, “was not aimed at developing the students’ reading and writing abilities but rather their test-taking skills.” More broadly, he argued, “America’s overemphasis on the SAT is compromising our educational system.”

Of course, it must be pointed out that U.C., assuming its policy-making bodies accept their president’s advice, would not be the first institution to drop the SAT. Hundreds of colleges and universities, including Bates, Bowdoin, Connecticut, and Mount Holyoke Colleges, no longer require the SAT or ACT. A survey by FairTest, a Cambridge, Mass.-based advocacy group, reported that such colleges are generally well-satisfied that “applicant pools and enrolled classes have become more diverse without any loss in academic quality.”

On balance, this latest and most significant challenge to the reign of the SAT is very welcome news indeed. There is a possible downside as well, but we should begin by recognizing that even before colleges began hopping off the SAT bandwagon, the assumption that they needed something like the test to help them decide whom to admit was difficult to defend, if only because of a powerful counterexample to our north: No such test is used in Canada. But the more one learns about the SAT in particular, the more one wonders what took Atkinson so long, and what is taking many of his counterparts even longer. Consider:

* The SAT is a measure of resources more than of reasoning. Year after year, the College Board’s own statistics depict a virtually linear correlation between SAT scores and family income. Each rise in earnings (measured in $10,000 increments) brings a commensurate rise in scores. Other research, meanwhile, has found that more than half the difference among students’ scores can be explained purely on the basis of parents’ level of education.

* Aggregate scores don’t reflect educational achievement. SAT results are still sometimes used to compare one state with another or one year with another. Unfortunately, not only is the test voluntary, but participation rates vary enormously by state and district. The researchers Brian Powell and Lala Carr Steelman, writing in a 1996 issue of the Harvard Educational Review, reported that those rates account for a whopping 85 percent of the variance in scores; when fewer students take the test, a state’s results end up looking much better. Similarly, even if it is true that average national scores have declined over the decades (once we factor in the statistical readjustment that took place in 1996), that is mostly because more students, relatively speaking, are now taking the test.

* Individual scores don’t reflect a student’s intellectual depth. The verbal section of the SAT is basically just a vocabulary test. It is not a measure of aptitude or of subject-area competency. So what does it measure, other than the size of students’ houses?

An interesting 1995 study with students at East Carolina University classified them as taking a “surface” approach to their assignments (meaning they memorized facts and did as little as possible); a “deep” approach (informed by a genuine desire to understand and a penchant for connecting current lessons with previous knowledge); or an “achieving” approach (where performance, particularly as compared with that of others, mattered more than learning). SAT scores turned out to be significantly correlated with both the surface and achieving approaches, but not at all with the deep approach. (That finding has been replicated with the results of other standardized tests taken by younger students, lending support to the criticism that such examinations tend to measure what matters least.)

* SAT’s don’t predict the future. A considerable amount of research, including but not limited to a summary of more than 600 studies published by the College Board in 1984, has found that only about 12 to 16 percent of the variance in freshman grades could be explained by SAT scores, suggesting that they are not particularly useful even with respect to that limited variable — and virtually worthless at predicting how students will fare after their freshman year (and whether they will graduate).

* SAT’s don’t contribute to diversity. Far from offering talented minority students a way to prove their worth, the overall effect of the SAT has been to ratify entrenched patterns of discrimination. Maria Blanco, a regional counsel with the Mexican American Legal Defense and Educational Fund, remarked recently that the SAT “has turned into a barrier to students of color” because it “keeps out very qualified kids who have overcome obstacles but don’t test very well.” Colleges looking to put together a racially and ethnically diverse student body are, therefore, already likely to minimize the significance of standardized-test scores.

Unhappily, though, some people committed to affirmative action — and even more who are opposed to it — have treated the SAT as a marker for merit and then argued about whether it is legitimate to set scores aside. Should a desire for equity sometimes override the desire for excellence? But that question is utterly misconceived. SAT’s, like other standardized tests, do not further the cause of equity or excellence. Such tests privilege the privileged and reflect a skill at taking tests. Few people — other than those who profit handsomely from its administration — will mourn the SAT when it finally breathes its last.

*And now the bad news: Unless we are very careful, a long-overdue move to jettison SAT scores may simply ratchet up the significance accorded to other admissions criteria that are little better and possibly even worse. Atkinson suggested that, at least in the short run, colleges might switch to the SAT 2, better known as achievement tests. While that may be a step forward in some respects, it may have the effect of creating a standardized, exam-based high-school curriculum that could squeeze out other kinds of teaching. That is already beginning to happen as states impose their own exit tests: Teachers feel compelled to cover vast amounts of content, often superficially, rather than letting students discover ideas.

The more ominous threat, though, is that, as the SAT fades, it will be replaced by high-school grades. There is a widespread assumption that less emphasis on scores as an admissions criterion has to mean more emphasis on grades, as though nature has decreed an inverse relationship between the two. But for grades to be given more emphasis would be terribly unfortunate. On the most obvious level, grades are unreliable indicators of student achievement. A “B” from one teacher or school doesn’t equate to a “B” from somewhere else; in fact, some studies have shown that a given assignment may even receive two different grades from a single teacher who reads it at two different times. Most people know that is true; tests like the SAT are more dangerous because they are falsely assumed to be objective.

What is far more disturbing about even the current emphasis on grades, let alone the prospect of enhancing their significance, is the damage they do when students are led to compulsively groom their transcripts.

Researchers have found three consistent effects of focusing attention on traditional grades. First, interest in the learning itself tends to decline. Many studies have shown that the more people are rewarded for doing something, the more they tend to lose interest in whatever they had to do to get the reward. While it’s not impossible for a student to be concerned about getting high marks and also to enjoy playing with ideas, the practical reality is that there is a negative correlation between a grade orientation and a learning orientation.

Second, focusing on grades tends to reduce the quality of students’ thinking. One series of studies by the researcher Ruth Butler found that graded students were significantly less creative than those who received only qualitative feedback. The more the task required creative thinking, in fact, the worse the performance of students who knew they were going to receive a grade. In another experiment by two University of Rochester researchers, reported in 1987, students who were told they would be graded on how well they learned a social-studies lesson had more trouble understanding the main point of the assigned text than did students who were told that no grades would be involved. Even on a measure of rote recall, the graded group remembered fewer facts a week later.

Finally, concern about grades often reduces a student’s preference for challenging tasks. Those who cut corners — who choose short books, undemanding projects, and “gut” courses — are not being lazy so much as rational; they are responding to the imperative to bring up their grade-point averages.

If it’s worrisome that SAT coaching sessions take time away from meaningful intellectual pursuits, then it’s worse that an admissions policy that causes students to become obsessed with grades could undermine the intellectual value of virtually everything they do in high school. Indeed, it can create intellectual dispositions that persist in and beyond college. From that perspective, complaints about “grade inflation” are a spectacular exercise in missing the point. The problem isn’t that too many students are getting A’s; the problem is that too many students are getting the idea that the whole point of school is to get A’s.

The only thing worse than placing added emphasis on the G.P.A. is placing added emphasis on relative G.P.A. Some state systems now want to guarantee acceptance to all students in a top percentage of their class. Here, the emphasis is not merely on performance (as opposed to learning), but on victory. A considerable body of data demonstrates that creating competition among students is decidedly detrimental with respect to achievement and motivation to learn. The urgent question should not be whether high-school class rank is correlated with college grades, but whether secondary schools can maintain (or create) a focus on intellectual exploration when their students are forced to view their classmates as obstacles to their own success.

*Where does all this leave us? Those willing to ask the truly radical questions about college admissions might consider an observation offered 30 years ago during a public lecture at the Educational Testing Service by the psychologist David McClelland. Rather than asking what criteria best predict success in higher education, he asked whether colleges should even be looking for the most-qualified students. “One would think that the purpose of education is precisely to improve the performance of those who are not doing very well,” he mused. “If the colleges were interested in proving that they could educate people, high-scoring students might be poor bets because they would be less likely to show improvement in performance.”

Many of us will find that challenge too unsettling, preferring that we continue to admit those students who will probably be easiest to educate. But even if we are looking for the “best” students, we ought to see G.P.A. numbers and SAT scores as a matched set of flawed criteria. Grades-and-tests, at best, will predict future grades-and-tests. Although some would dispute that, there is good evidence that grades don’t predict later-life success, in occupational or intellectual terms. In the 1980’s, a review of 35 studies, published in the American Educational Research Journal, concluded that academic indicators (grades and tests) from college — never mind high school — accounted for less than 3 percent of the variance in eventual occupational performance as judged by income, job-effectiveness ratings, and job satisfaction. Moreover, those indicators had no predictive power whatsoever for M.D.’s and Ph.D.’s.

When Mount Holyoke College, after a lengthy study by faculty members, announced last year that it would stop requiring students to submit SAT scores, the president, Joanne Creighton, did not limit her criticism to that test. “There has been a kind of reductionism in higher education, reducing students and institutions to numbers,” she said. Similarly, Atkinson said that he had recommended “that all campuses move away from admission processes that use narrowly defined quantitative formulas and instead adopt procedures that look at applicants in a comprehensive, holistic way.”

Doing so will not be an easy sell, if only because it is faster and therefore cheaper for universities that hear from tens of thousands of applicants to continue reducing each one to a numerical formula, rather than to weigh each as an individual. A move from SAT to G.P.A. — or SAT 1 to SAT 2 — will merely fine-tune the formula. That would be a pity, because the attention given Atkinson’s proposal has provided us with an opportunity to confront larger and more lasting issues.