{"id":9189,"date":"2014-03-06T09:20:46","date_gmt":"2014-03-06T14:20:46","guid":{"rendered":"http:\/\/scienceblogs.com\/principles\/?p=9189"},"modified":"2014-03-06T09:20:46","modified_gmt":"2014-03-06T14:20:46","slug":"the-new-sat-reading-gaming-and-jargon","status":"publish","type":"post","link":"http:\/\/chadorzel.com\/principles\/2014\/03\/06\/the-new-sat-reading-gaming-and-jargon\/","title":{"rendered":"The New SAT, Reading, Gaming, and Jargon"},"content":{"rendered":"

Given the academic circles I run in, it’s not surprising that one of the most repeated stories crossing my social media feeds yesterday had to do with the changes to the SAT<\/a>. Starting in 2015, the essay section will no longer be mandatory, and they’re going to reconfigure the reading and math sections to emphasize different categories of questions.<\/p>\n

My slightly cynical take on this is that changes seem to be driven more by marketing than education– stories about this all mention that the changes make the new SAT more like the ACT, which has been gaining in popularity in recent years. Which cuts into the income of ETS, so they’re re-arranging things to keep the cash flow coming. There’s a lot of buzzwordplay about this– “critical thinking” and so on– but most of it sounds like the sort of stuff you come up with after the fact, to justify something you’ve already decided to do for more concrete reasons. But then, I might be a little biased, because the SAT’s essay section was responsible for my first taste of Internet fame, back when Dave Munger and I did the Blogger SAT Challenge<\/a> in 2006.<\/p>\n

One interesting (to me, anyway, and it’s my blog…) spin-off of this was a Twitter exchange with a former student, who tweeted<\/a> “Memorizing big words is not a useful skill.” I don’t disagree with that, but it illustrates the big problem with the SAT’s, and massive standardized tests in general. And also maybe a bit of a shift in the way we approach language.<\/p>\n

What I mean by that is that I think the vocabulary section was never intended to be a memorization test, but a proxy test for wide reading. I say this mostly because while I did very well on the Verbal portion of the SAT back in the day (780, and this was in 1988 before they started “recentering” things), I never spent time memorizing anything in preparation for the test. I didn’t really need to, because I spent a lot of time reading. A lot<\/em>. Mostly genre fiction by that point, but really, just about anything with words in a row.<\/p>\n

And the thing is, if you do that, you pick up a bunch of vocabulary more or less incidentally, and also the ability to figure out approximate meanings from context and other clues. Which is more or less what the SAT vocabulary section was testing– “Here are some uncommon words, can you figure out what they mean?”<\/p>\n

I think the original idea of the SAT vocabulary questions was to pick up those students who did a lot of reading, on the theory that doing a lot of reading (and thus acquiring a large vocabulary) was a good proxy for “scholastic aptitude.” The problem, of course, is that this can be gamed. If you don’t do a lot of reading, you can still do well on the test provided you’re willing to devote a bunch of time to memorizing the definitions of obscure words. Which is dumb and pointless, but probably takes less time than getting a large vocabulary the “right” way. Which makes it an attractive option for the indolent children of the idle rich, contributing to the birth of the modern test-prep industry.<\/p>\n

So, to the extent that, over time, this has become more of a memorization test than a “reading widely” test, it’s probably good to dump the vocabulary section. But the whole business remains in a sort of Red Queen race condition– whatever they replace the vocabulary questions with will quickly be gamed in the same way that the vocabulary section was. There’s just too much money around for it not to– students will flounder with the new test for a year or two, but five years from now, high-schoolers will be doing something just as academically pointless as memorizing lists of big word, and ETS will start looking at new changes.<\/p>\n

The fundamental problem, of course, is one of scale. The SAT is taken by millions of students at a time, and needs to be graded quickly and consistently. Which requires machine-gradable multiple-choice questions or the functional equivalent thereof– even the essays get looked at for only a couple of minutes apiece, because that’s the only way to power through that many tests.<\/p>\n

This constraint opens the door for gaming the tests, and is probably part of why the scores are only weakly correlated with future college performance. Students can do very well on the SAT via a lot of strategies that exploit weaknesses introduced by the scale, but when they get to college and their work gets more individual scrutiny, the weaknesses hidden by the bubble-sheet tests get exposed.<\/p>\n

So, what’s the “shift in the way we approach language” bit? Another thing crossing my social media feeds was a post on Facebook from a scientist who had written an article that was (in the author’s opinion) excessively edited, asking whether to argue over the changes. Responses were sort of split, with some people saying that it couldn’t hurt, while others said it was futile. A few of the latter camp suggested that the editing was probably necessary to remove jargon terms.<\/p>\n

One of the words cited as possible jargon was “herbivory,” as in “behaving like a herbivore.” Which made me sort of scratch my head, because that doesn’t seem like an especially challenging word, particularly for an educated audience. And this was close on the heels of my “What Is Color?”<\/a> post, which got a bunch of comments about my use of “disturbance” and “spectrometer.” Which, again, I was a little surprised by– “disturbance” doesn’t seem like that hard a word (I wouldn’t hesitate to use it with SteelyKid, and she’s half the age of the target audience), and “spectrometer” is defined in the video.<\/p>\n

And this seems like something where the expectations have changed a lot. In the process of researching the book-in-progress, I’ve read a lot of stuff about the history of science. Most of this is of recent vintage, but some of the stuff I read was basically popular science writing from the 1940’s and 50’s. And it’s striking how much more the writers of that time expect from their audience, in an SAT vocabulary sort of sense– they throw out moderately obscure words, and expect readers to roll with them. A lot of the stuff I was reading seemed like it would be flagged for excessive jargon by the folks who regularly opine on such things in the science-communication social media I follow.<\/p>\n

There’s a sense, of course, in which this is a good thing–the target audience these days is much broader and more diverse in a socioeconomic kind of sense. A lot of those authors also expect their readers to get allusions to Classical mythology, which probably wasn’t elementary-school stuff in the 40’s, either. I don’t think it’s a bad thing to move beyond aiming pop-science writing at folks who have already been to Oxford.<\/p>\n

But at the same time, the collision of these two strands of my social-media universe makes me wonder if we haven’t capitulated to the SAT-prep vision of the world. That is, we’ve internalized the idea that acquisition of vocabulary is something that happens through pointless memorization of lists of words, rather than as an organic part of the reading process. We seem to be much less accepting of writing that expects something of the reader, instead driving everything toward the Up Goer Five<\/a> common denominator. And that, I think, is not a positive development, from either a science literacy or a college prep standpoint.<\/p>\n","protected":false},"excerpt":{"rendered":"

Given the academic circles I run in, it’s not surprising that one of the most repeated stories crossing my social media feeds yesterday had to do with the changes to the SAT. Starting in 2015, the essay section will no longer be mandatory, and they’re going to reconfigure the reading and math sections to emphasize… Continue reading The New SAT, Reading, Gaming, and Jargon<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,18,13,80,130,339,7,37,11,52,131],"tags":[],"class_list":["post-9189","post","type-post","status-publish","format-standard","hentry","category-academia","category-books","category-education","category-history_of_science","category-journalism","category-outreach","category-physics","category-pop_culture","category-science","category-science_books","category-science_writing","entry"],"_links":{"self":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/9189","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/comments?post=9189"}],"version-history":[{"count":0,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/9189\/revisions"}],"wp:attachment":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/media?parent=9189"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/categories?post=9189"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/tags?post=9189"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}