{"id":2823,"date":"2008-08-12T10:40:23","date_gmt":"2008-08-12T10:40:23","guid":{"rendered":"http:\/\/scienceblogs.com\/principles\/2008\/08\/12\/peer-review-does-not-define-sc\/"},"modified":"2008-08-12T10:40:23","modified_gmt":"2008-08-12T10:40:23","slug":"peer-review-does-not-define-sc","status":"publish","type":"post","link":"http:\/\/chadorzel.com\/principles\/2008\/08\/12\/peer-review-does-not-define-sc\/","title":{"rendered":"Peer Review Does Not Define Science"},"content":{"rendered":"

In the comments to yesterday’s post about science in popular media, ZapperZ responds with a comment that illustrates the problem<\/a>:<\/p>\n

\n

I am not saying that the media shouldn’t report ABOUT science, as accurately as they can. I am saying that DOING science isn’t done in popular media. Science isn’t done that way, especially when “research” is done haphazardly with little regards for proper scientific methodology. The popular media simply does not know how to do that.<\/p>\n

Now, one can argue that they should. But till they actually get to that stage, science has only been properly done in various respected scientific journals, where work are peer-reviewed so that all relevant information and methodology are addressed and the work is presented in a clear, accurate manner. I sincerely doubt that the NY Times piece would pass peer-review.<\/p>\n<\/blockquote>\n

I think this is not only wrong, but actually counter-productive. Peer review is an important part of the modern scientific establishment, but peer review is not the core of science. Holding peer review as the only standard for what counts as doing science is a step toward making a scientific guild system, which is something we absolutely do not want.<\/p>\n

<\/p>\n

Saying that only peer-reviewed articles (or peer-reviewable articles) count as science only reinforces the already pervasive notion that science is something beyond the reach of “normal” people. In essence, it’s saying that only scientists can do science, and that science is the exclusive province of geeks and nerds.<\/p>\n

That attitude is, I think, actively harmful to our society. It’s part of why we have a hard time getting students to study math and science, and finding people to teach<\/strong> math and science. We shouldn’t be restricting science to refereed journals, we should be trying to spread it as widely as possible.<\/p>\n

Peer review and refereed journals are a good check on science, but they do not define the essence of science. Science is, at its core, a matter of attitude and procedure. The essence of science is looking at the world and saying “Huh. I wonder why that happens?” And then taking a systematic approach to figuring it out.<\/p>\n

To some extent, this is the Mythbusters<\/cite> question– does that show count as real science, or not? I’m with zombie Feynman<\/a>: “‘Ideas are tested by experiment.’ That is the core of science. Everything else is bookkeeping.”<\/p>\n

The New York Times<\/cite> piece on radioactive countertops was not scientific by any definition, but there’s no good reason why it couldn’t’ve been scientific– had they done some simple tests, and reported the data, rather than just stringing together multiple anecdotes with some alarmist filler, that would’ve been a great thing. And that’s what we should be demanding of the popular media– don’t just repeat claims and anecdotes, do some tests. Confirm facts with observations.<\/p>\n

What we need is for more people to understand and appreciate the scientific approach to the world. Demanding that the popular media stop trying to “do science” until they can pass peer review is not going to help with that– if anything, it’s likely to make matters worse. Getting the New York Times<\/cite> to do some experiments, even at the Mythbusters<\/cite> level, before writing their stories just might make a positive difference.<\/p>\n

(Also, as I said in the comments, ZapperZ is not being terribly consistent here. While the New York Times<\/cite> article wouldn’t pass any sort of peer review standard, the Health Physics Society report held up as an alternative<\/a> is pretty shoddy, as well– it’s kind of vague about the methods, doesn’t report critical data (How many granite samples did they measure? What was the uncertainty in their average?), and uses a chain of shaky assumptions. It’s closer to science than the Times<\/cite> piece, but I wouldn’t give it a passing grade in a freshman physics lab, let alone call it worthy of peer review.)<\/p>\n","protected":false},"excerpt":{"rendered":"

In the comments to yesterday’s post about science in popular media, ZapperZ responds with a comment that illustrates the problem: I am not saying that the media shouldn’t report ABOUT science, as accurately as they can. I am saying that DOING science isn’t done in popular media. Science isn’t done that way, especially when “research”… Continue reading Peer Review Does Not Define Science<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"1","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11],"tags":[],"class_list":["post-2823","post","type-post","status-publish","format-standard","hentry","category-science","entry"],"_links":{"self":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/2823","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/comments?post=2823"}],"version-history":[{"count":0,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/posts\/2823\/revisions"}],"wp:attachment":[{"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/media?parent=2823"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/categories?post=2823"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/chadorzel.com\/principles\/wp-json\/wp\/v2\/tags?post=2823"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}