Reason Magazine
- reason.com
- reason.tv
- reason.org
- Donate
- Site Search:
Reason on Facebook
Reason on Twitter
Reason on YouTube
Reason RSSGet Reason E-mail Updates!
Manage your Reason e-mail list subscriptionsadvertisements
So Does This Mean Science Is Bullshit?
Via Plastic comes this CNN report of an incoherent, computer-generated paper being accepted for presentation at an academic conference.
[A trio of] MIT graduate students questioned the standards of some academic conferences, so they wrote a computer program to generate research papers complete with "context-free grammar," charts and diagrams.
The trio submitted two of the randomly assembled papers to the World Multi-Conference on Systemics, Cybernetics and Informatics (WMSCI), scheduled to be held July 10-13 in Orlando, Florida.
To their surprise, one of the papers--"Rooter: A Methodology for the Typical Unification of Access Points and Redundancy"--was accepted for presentation.
The prank recalled a 1996 hoax in which New York University physicist Alan Sokal succeeded in getting an entire paper with a mix of truths, falsehoods, non sequiturs and otherwise meaningless mumbo-jumbo published in the quarterly journal Social Text, published by Duke University Press.
Whole thing here.
The MIT pranksters' site is here.
Stuff about the Sokal hoax here.
Matt|4.18.05 @ 11:26AM|#
I realize the bulk of the embarrassment here is supposed to be
felt by the people reviewing the submissions, but I think the
people (humans!) whose papers were not accepted for presentation
ought to be most thoroughly humiliated.
What does it say about your ability to A) have an interesting
thought, and B) express it, when a panel of reviewers finds your
paper less interesting than some computer-generated jibber-jabber?
That's gotta hurt.
|4.18.05 @ 11:36AM|#
Matt,
Yeah, good point, except, you have to wonder what the actual
standards for acceptance are, given that they obviously are not
qualitative. Probably had to do with some sort of technicality
"minimum 3 graphs!" That's not to say that it's not humiliation to
get beat out by random-generated jibba-jabba, but, it's quite
obvious that one shouldn't feel TOO bad about getting turned down
by these idiots.
|4.18.05 @ 11:36AM|#
"You have seen that kind of people who will never let on that
they don't know the meaning of a new big word. The more ignorant
they are, the more pitifully certain they are to pretend you
haven't shot over their heads."
By Mark Twain - From: A Connecticut Yankee in King Arthur's
Court
|4.18.05 @ 11:41AM|#
"when a panel of reviewers finds your paper less interesting
than some computer-generated jibber-jabber? That's gotta
hurt"
No one reveiwed the computer-generated paper. So that does not say
anything about the quality of the reviews or the other papers
submitted to the conference. It only says that the conference
standard are low.
|4.18.05 @ 11:42AM|#
You know - this is really funny and amusing and I'm sure the
pranksters did this not to cause any harm or whatever...
But this is just the type of shit that anit-science creationists
use against everything from stem cell research to evolutionary
theory.
Jeff|4.18.05 @ 11:59AM|#
We must remember that this was an academic conference, NOT a
science conference. Big difference. Science conferences tend to be
specialized to a specific discipline and host lots of researchers
trying to find holes in the others' theories. This is a required
function of the scientific process.
An acaedemic conference, on the other hand, is a waste of time by
which several tenured university professors, textbook writers and
ciriculum overlords can justify the ungodly and artificially
inflated sums of cash they pocket. These conferences, like the
universities that generate them, need not generate anything
actually useful to fulfill their purpose.
|4.18.05 @ 12:02PM|#
Ironically, the meta-approach used by the researchers (creating a computer program to generate passably semantically-connected text to pass muster in terms of human readability) fits within the field of cybernetics. I would not be surprised to find out that a "true paper" based on this episode (with today's report as a data point) is accepted for presentation at next year's WMSCI.
Warren|4.18.05 @ 12:04PM|#
Will,
All the more reason to set the house in order, and smoke out the
pompous and pretentious.
Taking on a science conference is fine as far as it goes. What this
country really needs is an army of pranksters attacking the major
media with fake news stories. A steady stream of CNN gotchas would
go a long way toward illuminating journalism.
|4.18.05 @ 12:11PM|#
Actually it looks like it was a science conference. My experience of science conferences is that for the most part abstracts for presentation are not reviewed. It is important to differentiate conference abstracts, which usually refer to work in progress and peer reviewed journal articles which represent a finished product. This doesn't suprise me, friends of mine have put up some pretty silly posters at conferences, and I'm glad they did, science is better when there is some playfulness in the process. The output in papers, however needs to be well thought out, which more often than not, it is. The difference between this and the Sokal case is that Sokal got a paper published in a reviewed journal and these guys got some silliness published in an un-reviewed conference proceedings at best. I remember one such presentation on "MRI-Mechanical Resonance Imaging" which explained a method for hitting people on the head with a hammer as a low cost alternative to "real" MRI or magnetic resonance imaging.
|4.18.05 @ 12:12PM|#
Warren,
You mean fake stories like
100 Shiite hostages being held by Sunnis in Iraq?
|4.18.05 @ 12:14PM|#
"So does this mean that science is bullshit?"
A lot of what passes for it can be. Science research has just as
much propensity to be inflated selfserving bullshit as any other
field, especially when someone's willing to fund it. This has been
long recognized. Look at Gulliver's trip to the floating island of
Laputa ("the whore").
Me, I want to submit my learned scientific paper about distilling
moonbeams from cucumbers....
|4.18.05 @ 12:18PM|#
An abstract for any conference isn't a big deal.
They're just looking for formatting, buzz words, and then it's off
to the rest of the stack to check off. I've never actually heard of
an abstract being rejected.
If it were selected for a talk or for *acutal* publication
following peer-review...then I'd be impressed. :)
|4.18.05 @ 12:27PM|#
Conference standards can be notoriously low, but there's a
method to that madness:
1) Conferences are there for people to introduce ideas and talk
about them. Some will be found wanting, some meritorious, but the
point is to get people together and talk and get feedback. Indeed,
some professional societies actually have a by-law stating that any
member can give a talk at the conference. The American Physical
Society has a special "crackpot session" where the obvious frauds
are allowed to give their presentations. (I've never attended that
session, so I can't comment.)
2) OK, you're probably thinking that these conferences are
therefore a bunch of nonsense. But keep in mind that conference
presentations don't usually carry the same prestige as
peer-reviewed articles (unless it's an invited talk or a conference
with a reputation for stringent standards), so the host
organization isn't conferring any real seal of approval on the
talks that it accepts.
Basically, professional scientists don't go to conferences to
uncritically accept every presentation that they see. They go there
in part to mingle with people that they know, in part to examine
recent work by groups that they are familiar with, and in part to
see and evaluate what else might be out there. The key word is
"evaluate". Most evaluation is done on-site, and the reviewers
usually limit themselves to weeding out the most obvious garbage.
Since abstracts are short and highly technical, it's not always
obvious what's garbage, and so reviewers err on the side of letting
stuff in. They figure that if something is nonsense the audience
will ignore it, or perhaps even attack the presenter during the
Q&A.
This reliance on the distributed intelligence of the audience
shouldn't be at all disturbing to people who enjoy blogs.
|4.18.05 @ 12:31PM|#
Warren,
While we are quite away from a steady stream of fake news used to
discredit the press, some people have already started...
http://www.zug.com/pranks/media/
My favorite is the WTF-TV (great name) report on how Kenny Rogers
gets involved in a brawl during a book signing.
|4.18.05 @ 12:39PM|#
Similar theories about what it takes to get a paper accepted
have been tested much more often than has been published. About 10
years ago, a colleague and I submitted identical papers to a
metaphysics conference at Harvard. One was submitted using a
white-male-sounding name; the other under a female, latino-sounding
name. The paper with the latino female name tacked on was accepted;
the other was not.
There was no mention that the papers were identical -- most likely
because they were reviewed - if at all - by separate reviewers.
That or they just flagged all minorty or female sounding names for
acceptance (which was our theory at the time).
|4.18.05 @ 12:39PM|#
That's a hoot, I especially enjoy the graphs. Latency
(time-delay) is measured in Celcius(temperature), a signal-to-noise
ratio(unit-less)is measured in nanometers (distance).
Other gems: ...that we can do whole lot to adjust.... (
How about "a bunch of stuff" ). And Reality aside, we would
like deploy a methodology for how Rooter would behave in
theory.
|4.18.05 @ 12:50PM|#
Saying this as someone who was a graduate student in computer science: this conference is complete bullshit. Everyone knows this conference is complete bullshit. They spam people asking for contributions and everyone hates them. Most computer science conferences have real papers with real contributions. This really shouldn't be taken as a general comment on the academic standards of the field (which, in my opinion, are extremely high)
|4.18.05 @ 1:02PM|#
This seems a weak prank, even by the standards of ego-inflated MIT students. Remember when a bunch of them went to Vegas and tried to count cards, then get rich off writing a book? That was great. No one else had ever tried something that audacious. Except the thousands of people who have been trying to do it since the 60s. But *these* guy were from MIT! (oh my!)
Warren|4.18.05 @ 1:11PM|#
phocion,
No. That falls more under the "lying to the guys in carge of the
guns and bullets so they will shoot at folks you don't like"
heading.
Duncan,
YES. That's it exactly, we need alot more of that.
|4.18.05 @ 1:25PM|#
Remember when a bunch of them went to Vegas and tried to
count cards, then get rich off writing a book?
I have long heard urban legends of such things. I even know
somebody who claims to have worked with some consortium that grew
out of that MIT effort. She supposedly got trained by this
consortium and would go to Vegas now and then, and supposedly there
is a team of grad students at my school who did lots of computer
simulations to support the effort.
But she has never come back to Santa Barbara on Monday flush with
cash. Since this technique only boosts your odds to something like
51%, there are lots of downs as well as ups. Supposedly the way it
works is that there are these mysterious investors behind it, and
they assume the risk of the downs and take the winnings from the
ups, and only pay a small commission to the card counters.
But there's this elaborate system and a network of investors and
mathematicians running the entire thing! And armies of grad
students working it for them! And it works!
Yeah, right.
|4.18.05 @ 2:33PM|#
Hey Thoreau, try using Google before making an ass out of
yourself:
http://wired-vig.wired.com/wired/archive/10.09/vegas.html
http://www.amazon.com/exec/obidos/ASIN/0743249992/reasonmagazineA/
|4.18.05 @ 2:36PM|#
Don't know if it's legit or not, but here's the link to the Wired article about the MIT guys in Vegas: http://wired-vig.wired.com/wired/archive/10.09/vegas.html
|4.18.05 @ 3:13PM|#
I stand corrected.
I still doubt that the person I know was involved in it,
however.
|4.18.05 @ 3:18PM|#
Actually, in the World Series of Poker a year ago, one of "the guys" from MIT was coaching another poker player. Supposedly. I don't know how you would substantiate any such claims.
|4.18.05 @ 3:27PM|#
I've seen a 60 min documentary on A&E or Discovery on the
MIT card counting cabal. My recollection is that they made a good
deal of money, but then various factors, including casino's
refusing admittance to members they had ID'd, investors squabbling
with players over who got how much of the proceeds, and 'who is in
charge' infighting eventually ground it to a halt. It was a fun
little ride though for the first while, and certainly was
successful according to the documentary.
"These conferences, like the universities that generate them, need
not generate anything actually useful to fulfill their purpose."
Well said, Jeff.
|4.18.05 @ 3:27PM|#
abc: Hey Thoreau, try using Google before making an ass out
of yourself:
Sir! Even if thoreau were to make an ass of himself, he's still
twice the non-ass that most of the rest of us strive to be!
You know, I did have a foggy recollection of reading the WIRED
article about the MIT card-counting ring and almost mentioned it
earlier, but my memory was so vague (I couldn't remember whether
I'd read it in WIRED, or Science, or Discover, or somewhere else)
it would have just sounded even more like an urban legend.
|4.18.05 @ 3:29PM|#
.... which is why I didn't mention it earlier. (Forgot to include the point of my previous post.)
|4.18.05 @ 3:46PM|#
No problem, abc.
I still maintain that 99% of the people you meet who claim to have
been involved are probably full of it. Especially if they claim
recent involvement but never show up flush with cash.
I read the Wired article that you linked to. It sounds like any
card counting scheme can be beaten by frequent shuffling and larger
decks, and that the casions are catching on. I find it difficult to
believe that something like the MIT scheme could replicated again,
at least not with card counting and blackjack. No doubt somebody
somewhere in the future will find another way to beat the odds on
some game, but it will be a new trick. If somebody comes around and
claims to be involved in an old scheme that the casinos are wise
to, but never seems to have much money, be very, very
suspicious.
|4.18.05 @ 4:29PM|#
Thoreau: I watched a documentary about this team a few months
ago. My memory is a bit fuzzy, but they did make a lot of money for
a while. The team was organized as a limited liability partnership,
and had investors putting up the money. It was organized by a
professional card counter (and MIT alum), using MIT students
because they were intelligent and had no criminal records. Then
they started losing lots of money when their players started to
have conflicts with the management (the players claimed these two
events were unrelated, though the management disagreed.) Blackjack
is a beatable game*, though it's difficult and will get you kicked
out of casinos. One of the big tellls is betting patterns with a
high degree of volatility: a long string of minimum bets followed
by an enormous bet when the count is good. This is why they were a
team--one group would sit at tables betting the minimum, while
another would be signaled to come to the table and bet large
amounts when the time was right.
* Beatable in the sense that the expected value of returns can be
greater than the initial investment. Interestingly, since there is
a positive probability of loss, theoretically if you play long
enough you'll hit a long enough string of losses that you lose
everything. Of course the probability of this happening in actual
play may be remote.
|4.18.05 @ 4:33PM|#
I'm not sure the odds were even all the way up to 51%, with
counting. My impression of the MIT blackjack antics was that they
tried to do to much at once (with too much money), and got sloppy.
I think there's just no way to take the emotion out of play. You
have to be a great actor, a great counter, and sensible with money
all at once. Now, they have electronic databases of just counters
(nevermind the outright cheats), on top of the millions of cameras.
You thought Vegas was ruthless with the mob in charge, now it's run
by real capitalists, heh.
As a side note, I discovered a book a few years ago about some
alleged escapades involving a roulette computer hidden in a shoe.
It's called The Eudemonic Pie, and the story seems to have been
overlooked by the latest gambling craze. Could be a complete load,
but it looked interesting. If only I could find it amongst these
boxes...
|4.18.05 @ 4:35PM|#
Lucas-
That point about a long string of losses is an interesting one. I
was recently talking to a physicist who works on the diffusion of
light in biological tissue. Much of the mathematics used to
describe such light also has applications in the theory of
gambling. He was remarking that a chain of consecutive losses can
reduce your expected payoff lower than the standard theory would
predict. He was talking about gambling, because he likes to gamble,
but similar ideas apply to diffusion of light in tissue.
I always find it interesting when a scientist's other pursuits leak
over into his or her work. For me, my moonlighting as an instructor
at a photography school has given me some insights that have
improved my work. I'm still waiting to find a way to apply my work
on the mathematics of voting to my physics, however.
|4.18.05 @ 4:57PM|#
I'm still waiting to find a way to apply my work on the
mathematics of voting to my physics.
Puleeze thoreau. If you're physicist, I'm a priest. Physicists have
known, quantified, and explained, for over a century, that there is
a strong stocastic component in nature.
Physics
|4.18.05 @ 5:09PM|#
Puleeze thoreau. If you're physicist, I'm a priest.
Physicists have known, quantified, and explained, for over a
century, that there is a strong stocastic component in
nature.
Father Richard-
Um, where did I deny that there's a strong stochastic component in
nature?
Maybe you mean that I acted as though my colleague's insight on
diffusion was startling. No, not really startling. But many
textbooks and research articles ignore the possibility of random
walks that terminate early. And, to be fair, in many situations
those early terminations don't matter. But there are situations
where they do, and my colleague and I might collaborate on studies
of such systems, and I thought it was a neat insight.
Or, maybe you don't think that mathematical models of voting could
have any connection to nature. Well, my work on the mathematics of
voting has to do with (admittedly deterministic) algorithms that
take each voter's preference order and then deduce a winner. This
is for voting systems like Instant Runoff, Condorcet, or Borda,
where voters rank all candidates rather than just voting for a
single person. I started pondering a question about the nature of
such algorithms in regard to strategy, and have been working on it
as a hobby. I'm very close to proving a theorem about the
properties of such algorithms, and the proof involves the way that
shapes fit together in higher dimensions. I'm sure if I think about
it enough I can find an application for that insight in my physics.
Even though as an optics person I work on objects in regular old 3D
space (no 20+ dimensions for me, or whatever the string theorists
are currently using), sometimes the phase space of a system, or the
parameter space of a model or algorithm, will have higher
dimensions.
Anyway, Father Richard, hope I didn't confuse you too much with the
jargon.
|4.18.05 @ 6:11PM|#
Or, maybe you don't think that mathematical models of voting
could have any connection to nature.
Mathematics is wonderful. But math is math, not science. Sometimes
the mathematics precedes the science that uses it, like the Lorentz
transformations. But math is pure logic and does not require
validation of its premises, it only requires logic.
Science requires validation through repeatable experiments. As a
scientist, the burden of proof is on you. That is what make you a
scientist, not a mathematitician.
|4.18.05 @ 6:16PM|#
So, because I contemplated the possibility that a piece of
mathematics might have uses in science you concluded that I'm not a
scientist?
And, believe it or not, we physicists are actually aware of the
difference between physics and pure mathematics. No, really! We
aren't always up on the latest insights from the philosophy of
science, but we know that there's a difference between physics and
mathematics. Some of us even tell jokes about mathematicians!
Jokes! Can you believe it?
|4.18.05 @ 6:26PM|#
Sorry, I just get touchy when somebody tries to lecture me on
what real science is all about, or the difference between
science and math or between science and engineering, or religion,
or whatever. I once had a creationist start quizzing me on the
scientific method. I had a comparative literature major try to
explain to me that science is really just a cultural
construct.
And once upon a time I was actually involved in a situation where a
physicist, an engineer, and a mathematician tried to fix a vacuum
cleaner.
|4.18.05 @ 6:43PM|#
thoreau,
I encourage you to use whatever mathematical formulations are
useful to you in your work as a physicist and disregard, or set
aside, those that are not.
|4.18.05 @ 6:51PM|#
So, Richard, now that you're a priest...
Bless me father, for I have sinned. It has been 3 weeks since my
last confession.
|4.18.05 @ 9:12PM|#
Continuing with that special about the MIT card counting ring,
lately The History Channel has been playing shows about beating
Vegas. It features various cheaters and the various things they do
to cheat at the games of chance. I saw one where one guy played
roulette and would change the bet (by removing chips) after the
ball had fallen, and when he couldn't get away with that, he
started changing the bet on his losses, instead of his wins. That
way, when he would win, he would have $5010 on the table, but if he
lost, he would only have $15 on the table when the chips were
picked up.
I thought to myself, "the casino should be asking, 'what kind of
person would have a string of $15 bets that lost, only to have the
one that won be $5010?'" He said that he was able to get away with
it, but if someone had betting patterns like that, he will probably
be asked to leave, and never come back.
|4.18.05 @ 9:23PM|#
When a similar prank was pulled on a social studies conference,
the right wing press went apeshit shouting that this obviously
proved the worthlessness of modern social studies.
I'm sure that National Review article about the worthlessness of
hard science higher education is going to appear any time now.
|4.18.05 @ 11:20PM|#
I'm sure that National Review article about the
worthlessness of hard science higher education is going to appear
any time now.
Um, joe, it probably won't appear in NR, but I'm sure that some
right-wing publication will hammer away at this. Cuz, you know,
those scientists believe in things like stem cells, evolution, and
global warming.
(And yes, I know, some people here don't believe in global warming,
and yes, I know, you have perfectly good reasons for it, yadda
yadda. My only point is that conservatives have plenty of reasons
to dislike scientists.)
|4.18.05 @ 11:52PM|#
you so-called "scientist"!
Just ask Father Richard: I'm a mathematician, not a scientist
;)
M. Simon|4.19.05 @ 12:21AM|#
thoreau said:
And once upon a time I was actually involved in a situation
where a physicist, an engineer, and a mathematician tried to fix a
vacuum cleaner.
Jeez, that sucks. Or maybe not
|4.19.05 @ 1:08AM|#
Jeez, that sucks. Or maybe not
By the time we were done with that vacuum cleaner it had gone from
suck to blow.
Thank-you, I'm here all week! Don't forget to tip your server!
|4.19.05 @ 9:32AM|#
"By the time we were done with that vacuum cleaner it had gone
from suck to blow."
What did you do with the spare parts?
|4.19.05 @ 2:22PM|#
"I still maintain that 99% of the people you meet who claim to
have been involved are probably full of it. Especially if they
claim recent involvement but never show up flush with cash."
thoreau, you can't just go around being all skeptical like that!
Jeez! After all of the state high school wrestling champions,
former special operations/Medal of Honor awardees, and people with
PhD's in obscure subjects that I've met I can't imagine how you
could be so skeptical and untrusting!
Seriously, it's amazing how many people claim to be something that
they're not and will swear to it all the way down the line. Some of
those folks even get jobs based on faked credentials.
http://www.reason.com/0501/fe.ps.cut.shtml
http://www.pownetwork.org/phonies/phonies9.htm
Compared to those claims, pretending to be part of the MIT crew
just pales.
|4.19.05 @ 9:57PM|#
thoreau,
Late in getting back on this thread. I want apologize for the snide
remark I make about your qualifications as a physicist. I turned
some skepticism (or ignorance) I had about a part of one your posts
into personal attack. My foolishness.
Best Regards
Site comments/questions:
Media Inquiries and Reprint Permissions:
(310) 367-6109
Editorial & Production Offices:
3415 S. Sepulveda Blvd.
Suite 400
Los Angeles, CA 90034
(310) 391-2245