So Does This Mean Science Is Bullshit?

Via Plastic comes this CNN report of an incoherent, computer-generated paper being accepted for presentation at an academic conference.

[A trio of] MIT graduate students questioned the standards of some academic conferences, so they wrote a computer program to generate research papers complete with "context-free grammar," charts and diagrams.

The trio submitted two of the randomly assembled papers to the World Multi-Conference on Systemics, Cybernetics and Informatics (WMSCI), scheduled to be held July 10-13 in Orlando, Florida.

To their surprise, one of the papers--"Rooter: A Methodology for the Typical Unification of Access Points and Redundancy"--was accepted for presentation.

The prank recalled a 1996 hoax in which New York University physicist Alan Sokal succeeded in getting an entire paper with a mix of truths, falsehoods, non sequiturs and otherwise meaningless mumbo-jumbo published in the quarterly journal Social Text, published by Duke University Press.

Whole thing here.

The MIT pranksters' site is here.

Stuff about the Sokal hoax here.

Comments

[Matt]

I realize the bulk of the embarrassment here is supposed to be felt by the people reviewing the submissions, but I think the people (humans!) whose papers were not accepted for presentation ought to be most thoroughly humiliated.

What does it say about your ability to A) have an interesting thought, and B) express it, when a panel of reviewers finds your paper less interesting than some computer-generated jibber-jabber? That's gotta hurt.

[Evan Williams]

Matt,

Yeah, good point, except, you have to wonder what the actual standards for acceptance are, given that they obviously are not qualitative. Probably had to do with some sort of technicality "minimum 3 graphs!" That's not to say that it's not humiliation to get beat out by random-generated jibba-jabba, but, it's quite obvious that one shouldn't feel TOO bad about getting turned down by these idiots.

[Gene]

"You have seen that kind of people who will never let on that they don't know the meaning of a new big word. The more ignorant they are, the more pitifully certain they are to pretend you haven't shot over their heads."
By Mark Twain - From: A Connecticut Yankee in King Arthur's Court

[a]

"when a panel of reviewers finds your paper less interesting than some computer-generated jibber-jabber? That's gotta hurt"

No one reveiwed the computer-generated paper. So that does not say anything about the quality of the reviews or the other papers submitted to the conference. It only says that the conference standard are low.

[Will]

You know - this is really funny and amusing and I'm sure the pranksters did this not to cause any harm or whatever...

But this is just the type of shit that anit-science creationists use against everything from stem cell research to evolutionary theory.

[T, Ph.D.]

Ain't got time for no jibber-jabber!

[Jeff]

We must remember that this was an academic conference, NOT a science conference. Big difference. Science conferences tend to be specialized to a specific discipline and host lots of researchers trying to find holes in the others' theories. This is a required function of the scientific process.
An acaedemic conference, on the other hand, is a waste of time by which several tenured university professors, textbook writers and ciriculum overlords can justify the ungodly and artificially inflated sums of cash they pocket. These conferences, like the universities that generate them, need not generate anything actually useful to fulfill their purpose.

[keith]

Ironically, the meta-approach used by the researchers (creating a computer program to generate passably semantically-connected text to pass muster in terms of human readability) fits within the field of cybernetics. I would not be surprised to find out that a "true paper" based on this episode (with today's report as a data point) is accepted for presentation at next year's WMSCI.

[Warren]

Will,
All the more reason to set the house in order, and smoke out the pompous and pretentious.

Taking on a science conference is fine as far as it goes. What this country really needs is an army of pranksters attacking the major media with fake news stories. A steady stream of CNN gotchas would go a long way toward illuminating journalism.

[rghunter]

Actually it looks like it was a science conference. My experience of science conferences is that for the most part abstracts for presentation are not reviewed. It is important to differentiate conference abstracts, which usually refer to work in progress and peer reviewed journal articles which represent a finished product. This doesn't suprise me, friends of mine have put up some pretty silly posters at conferences, and I'm glad they did, science is better when there is some playfulness in the process. The output in papers, however needs to be well thought out, which more often than not, it is. The difference between this and the Sokal case is that Sokal got a paper published in a reviewed journal and these guys got some silliness published in an un-reviewed conference proceedings at best. I remember one such presentation on "MRI-Mechanical Resonance Imaging" which explained a method for hitting people on the head with a hammer as a low cost alternative to "real" MRI or magnetic resonance imaging.

[phocion]

Warren,

You mean fake stories like 100 Shiite hostages being held by Sunnis in Iraq?

[just another lurker]

"So does this mean that science is bullshit?"

A lot of what passes for it can be. Science research has just as much propensity to be inflated selfserving bullshit as any other field, especially when someone's willing to fund it. This has been long recognized. Look at Gulliver's trip to the floating island of Laputa ("the whore").

Me, I want to submit my learned scientific paper about distilling moonbeams from cucumbers....

[Doc]

An abstract for any conference isn't a big deal.

They're just looking for formatting, buzz words, and then it's off to the rest of the stack to check off. I've never actually heard of an abstract being rejected.

If it were selected for a talk or for *acutal* publication following peer-review...then I'd be impressed. 🙂

[thoreau]

Conference standards can be notoriously low, but there's a method to that madness:

1) Conferences are there for people to introduce ideas and talk about them. Some will be found wanting, some meritorious, but the point is to get people together and talk and get feedback. Indeed, some professional societies actually have a by-law stating that any member can give a talk at the conference. The American Physical Society has a special "crackpot session" where the obvious frauds are allowed to give their presentations. (I've never attended that session, so I can't comment.)

2) OK, you're probably thinking that these conferences are therefore a bunch of nonsense. But keep in mind that conference presentations don't usually carry the same prestige as peer-reviewed articles (unless it's an invited talk or a conference with a reputation for stringent standards), so the host organization isn't conferring any real seal of approval on the talks that it accepts.

Basically, professional scientists don't go to conferences to uncritically accept every presentation that they see. They go there in part to mingle with people that they know, in part to examine recent work by groups that they are familiar with, and in part to see and evaluate what else might be out there. The key word is "evaluate". Most evaluation is done on-site, and the reviewers usually limit themselves to weeding out the most obvious garbage. Since abstracts are short and highly technical, it's not always obvious what's garbage, and so reviewers err on the side of letting stuff in. They figure that if something is nonsense the audience will ignore it, or perhaps even attack the presenter during the Q&A.

This reliance on the distributed intelligence of the audience shouldn't be at all disturbing to people who enjoy blogs.

[Duncan]

Warren,

While we are quite away from a steady stream of fake news used to discredit the press, some people have already started...

http://www.zug.com/pranks/media/

My favorite is the WTF-TV (great name) report on how Kenny Rogers gets involved in a brawl during a book signing.

[independent worm]

Similar theories about what it takes to get a paper accepted have been tested much more often than has been published. About 10 years ago, a colleague and I submitted identical papers to a metaphysics conference at Harvard. One was submitted using a white-male-sounding name; the other under a female, latino-sounding name. The paper with the latino female name tacked on was accepted; the other was not.

There was no mention that the papers were identical -- most likely because they were reviewed - if at all - by separate reviewers. That or they just flagged all minorty or female sounding names for acceptance (which was our theory at the time).

[Drooling Richard]

That's a hoot, I especially enjoy the graphs. Latency (time-delay) is measured in Celcius(temperature), a signal-to-noise ratio(unit-less)is measured in nanometers (distance).

Other gems: ...that we can do whole lot to adjust.... ( How about "a bunch of stuff" ). And Reality aside, we would like deploy a methodology for how Rooter would behave in theory.

[Adam]

Saying this as someone who was a graduate student in computer science: this conference is complete bullshit. Everyone knows this conference is complete bullshit. They spam people asking for contributions and everyone hates them. Most computer science conferences have real papers with real contributions. This really shouldn't be taken as a general comment on the academic standards of the field (which, in my opinion, are extremely high)

[Geophile]

This seems a weak prank, even by the standards of ego-inflated MIT students. Remember when a bunch of them went to Vegas and tried to count cards, then get rich off writing a book? That was great. No one else had ever tried something that audacious. Except the thousands of people who have been trying to do it since the 60s. But *these* guy were from MIT! (oh my!)

[Warren]

phocion,
No. That falls more under the "lying to the guys in carge of the guns and bullets so they will shoot at folks you don't like" heading.

Duncan,
YES. That's it exactly, we need alot more of that.

[thoreau]

Remember when a bunch of them went to Vegas and tried to count cards, then get rich off writing a book?

I have long heard urban legends of such things. I even know somebody who claims to have worked with some consortium that grew out of that MIT effort. She supposedly got trained by this consortium and would go to Vegas now and then, and supposedly there is a team of grad students at my school who did lots of computer simulations to support the effort.

But she has never come back to Santa Barbara on Monday flush with cash. Since this technique only boosts your odds to something like 51%, there are lots of downs as well as ups. Supposedly the way it works is that there are these mysterious investors behind it, and they assume the risk of the downs and take the winnings from the ups, and only pay a small commission to the card counters.

But there's this elaborate system and a network of investors and mathematicians running the entire thing! And armies of grad students working it for them! And it works!

Yeah, right.

[abc]

Hey Thoreau, try using Google before making an ass out of yourself:

http://wired-vig.wired.com/wired/archive/10.09/vegas.html

http://www.amazon.com/exec/obidos/ASIN/0743249992/reasonmagazinea-20/

[Lowdog]

Don't know if it's legit or not, but here's the link to the Wired article about the MIT guys in Vegas: http://wired-vig.wired.com/wired/archive/10.09/vegas.html

[thoreau]

I stand corrected.

I still doubt that the person I know was involved in it, however.

[Lowdog]

Actually, in the World Series of Poker a year ago, one of "the guys" from MIT was coaching another poker player. Supposedly. I don't know how you would substantiate any such claims.

[Poustman]

I've seen a 60 min documentary on A&E or Discovery on the MIT card counting cabal. My recollection is that they made a good deal of money, but then various factors, including casino's refusing admittance to members they had ID'd, investors squabbling with players over who got how much of the proceeds, and 'who is in charge' infighting eventually ground it to a halt. It was a fun little ride though for the first while, and certainly was successful according to the documentary.

"These conferences, like the universities that generate them, need not generate anything actually useful to fulfill their purpose." Well said, Jeff.

[Stevo Darkly]

abc: Hey Thoreau, try using Google before making an ass out of yourself:

Sir! Even if thoreau were to make an ass of himself, he's still twice the non-ass that most of the rest of us strive to be!

You know, I did have a foggy recollection of reading the WIRED article about the MIT card-counting ring and almost mentioned it earlier, but my memory was so vague (I couldn't remember whether I'd read it in WIRED, or Science, or Discover, or somewhere else) it would have just sounded even more like an urban legend.

[Stevo Darkly]

.... which is why I didn't mention it earlier. (Forgot to include the point of my previous post.)

[abc]

hey, thoreau - sorry about the "ass" comment. I haven't slept in two days.

[thoreau]

No problem, abc.

I still maintain that 99% of the people you meet who claim to have been involved are probably full of it. Especially if they claim recent involvement but never show up flush with cash.

I read the Wired article that you linked to. It sounds like any card counting scheme can be beaten by frequent shuffling and larger decks, and that the casions are catching on. I find it difficult to believe that something like the MIT scheme could replicated again, at least not with card counting and blackjack. No doubt somebody somewhere in the future will find another way to beat the odds on some game, but it will be a new trick. If somebody comes around and claims to be involved in an old scheme that the casinos are wise to, but never seems to have much money, be very, very suspicious.

[Lucas Wiman]

Thoreau: I watched a documentary about this team a few months ago. My memory is a bit fuzzy, but they did make a lot of money for a while. The team was organized as a limited liability partnership, and had investors putting up the money. It was organized by a professional card counter (and MIT alum), using MIT students because they were intelligent and had no criminal records. Then they started losing lots of money when their players started to have conflicts with the management (the players claimed these two events were unrelated, though the management disagreed.) Blackjack is a beatable game*, though it's difficult and will get you kicked out of casinos. One of the big tellls is betting patterns with a high degree of volatility: a long string of minimum bets followed by an enormous bet when the count is good. This is why they were a team--one group would sit at tables betting the minimum, while another would be signaled to come to the table and bet large amounts when the time was right.

* Beatable in the sense that the expected value of returns can be greater than the initial investment. Interestingly, since there is a positive probability of loss, theoretically if you play long enough you'll hit a long enough string of losses that you lose everything. Of course the probability of this happening in actual play may be remote.

[Geophile]

I'm not sure the odds were even all the way up to 51%, with counting. My impression of the MIT blackjack antics was that they tried to do to much at once (with too much money), and got sloppy. I think there's just no way to take the emotion out of play. You have to be a great actor, a great counter, and sensible with money all at once. Now, they have electronic databases of just counters (nevermind the outright cheats), on top of the millions of cameras. You thought Vegas was ruthless with the mob in charge, now it's run by real capitalists, heh.

As a side note, I discovered a book a few years ago about some alleged escapades involving a roulette computer hidden in a shoe. It's called The Eudemonic Pie, and the story seems to have been overlooked by the latest gambling craze. Could be a complete load, but it looked interesting. If only I could find it amongst these boxes...

[thoreau]

Lucas-

That point about a long string of losses is an interesting one. I was recently talking to a physicist who works on the diffusion of light in biological tissue. Much of the mathematics used to describe such light also has applications in the theory of gambling. He was remarking that a chain of consecutive losses can reduce your expected payoff lower than the standard theory would predict. He was talking about gambling, because he likes to gamble, but similar ideas apply to diffusion of light in tissue.

I always find it interesting when a scientist's other pursuits leak over into his or her work. For me, my moonlighting as an instructor at a photography school has given me some insights that have improved my work. I'm still waiting to find a way to apply my work on the mathematics of voting to my physics, however.

[Drooling Richard]

I'm still waiting to find a way to apply my work on the mathematics of voting to my physics.

Puleeze thoreau. If you're physicist, I'm a priest. Physicists have known, quantified, and explained, for over a century, that there is a strong stocastic component in nature.

Physics

[thoreau]

Puleeze thoreau. If you're physicist, I'm a priest. Physicists have known, quantified, and explained, for over a century, that there is a strong stocastic component in nature.

Father Richard-

Um, where did I deny that there's a strong stochastic component in nature?

Maybe you mean that I acted as though my colleague's insight on diffusion was startling. No, not really startling. But many textbooks and research articles ignore the possibility of random walks that terminate early. And, to be fair, in many situations those early terminations don't matter. But there are situations where they do, and my colleague and I might collaborate on studies of such systems, and I thought it was a neat insight.

Or, maybe you don't think that mathematical models of voting could have any connection to nature. Well, my work on the mathematics of voting has to do with (admittedly deterministic) algorithms that take each voter's preference order and then deduce a winner. This is for voting systems like Instant Runoff, Condorcet, or Borda, where voters rank all candidates rather than just voting for a single person. I started pondering a question about the nature of such algorithms in regard to strategy, and have been working on it as a hobby. I'm very close to proving a theorem about the properties of such algorithms, and the proof involves the way that shapes fit together in higher dimensions. I'm sure if I think about it enough I can find an application for that insight in my physics. Even though as an optics person I work on objects in regular old 3D space (no 20+ dimensions for me, or whatever the string theorists are currently using), sometimes the phase space of a system, or the parameter space of a model or algorithm, will have higher dimensions.

Anyway, Father Richard, hope I didn't confuse you too much with the jargon.

[Drooling Richard]

Or, maybe you don't think that mathematical models of voting could have any connection to nature.

Mathematics is wonderful. But math is math, not science. Sometimes the mathematics precedes the science that uses it, like the Lorentz transformations. But math is pure logic and does not require validation of its premises, it only requires logic.

Science requires validation through repeatable experiments. As a scientist, the burden of proof is on you. That is what make you a scientist, not a mathematitician.

[thoreau]

So, because I contemplated the possibility that a piece of mathematics might have uses in science you concluded that I'm not a scientist?

And, believe it or not, we physicists are actually aware of the difference between physics and pure mathematics. No, really! We aren't always up on the latest insights from the philosophy of science, but we know that there's a difference between physics and mathematics. Some of us even tell jokes about mathematicians! Jokes! Can you believe it?

[thoreau]

Sorry, I just get touchy when somebody tries to lecture me on what real science is all about, or the difference between science and math or between science and engineering, or religion, or whatever. I once had a creationist start quizzing me on the scientific method. I had a comparative literature major try to explain to me that science is really just a cultural construct.

And once upon a time I was actually involved in a situation where a physicist, an engineer, and a mathematician tried to fix a vacuum cleaner.

[Drooling Richard]

thoreau,

I encourage you to use whatever mathematical formulations are useful to you in your work as a physicist and disregard, or set aside, those that are not.

[Poustman]

So, Richard, now that you're a priest...

Bless me father, for I have sinned. It has been 3 weeks since my last confession.

[Stevo Darkly]

Any connection between Richard and The Father Drooling Mysteries?

[Shawn Smith (in Las Vegas)]

Continuing with that special about the MIT card counting ring, lately The History Channel has been playing shows about beating Vegas. It features various cheaters and the various things they do to cheat at the games of chance. I saw one where one guy played roulette and would change the bet (by removing chips) after the ball had fallen, and when he couldn't get away with that, he started changing the bet on his losses, instead of his wins. That way, when he would win, he would have $5010 on the table, but if he lost, he would only have $15 on the table when the chips were picked up.

I thought to myself, "the casino should be asking, 'what kind of person would have a string of $15 bets that lost, only to have the one that won be $5010?'" He said that he was able to get away with it, but if someone had betting patterns like that, he will probably be asked to leave, and never come back.

[joe]

When a similar prank was pulled on a social studies conference, the right wing press went apeshit shouting that this obviously proved the worthlessness of modern social studies.

I'm sure that National Review article about the worthlessness of hard science higher education is going to appear any time now.

[thoreau]

I'm sure that National Review article about the worthlessness of hard science higher education is going to appear any time now.

Um, joe, it probably won't appear in NR, but I'm sure that some right-wing publication will hammer away at this. Cuz, you know, those scientists believe in things like stem cells, evolution, and global warming.

(And yes, I know, some people here don't believe in global warming, and yes, I know, you have perfectly good reasons for it, yadda yadda. My only point is that conservatives have plenty of reasons to dislike scientists.)

[Stevo Darkly]

They ain't no such thing as "stem cells," you so-called "scientist"!

[thoreau]

you so-called "scientist"!

Just ask Father Richard: I'm a mathematician, not a scientist 😉

[M. Simon]

thoreau said:

And once upon a time I was actually involved in a situation where a physicist, an engineer, and a mathematician tried to fix a vacuum cleaner.

Jeez, that sucks. Or maybe not

[thoreau]

Jeez, that sucks. Or maybe not

By the time we were done with that vacuum cleaner it had gone from suck to blow.

Thank-you, I'm here all week! Don't forget to tip your server!

[saw-whet]

"By the time we were done with that vacuum cleaner it had gone from suck to blow."

What did you do with the spare parts?

[saw-whet]

Oops, I meant extra parts. Aw heck, this thread is closed.

[rob]

"I still maintain that 99% of the people you meet who claim to have been involved are probably full of it. Especially if they claim recent involvement but never show up flush with cash."

thoreau, you can't just go around being all skeptical like that! Jeez! After all of the state high school wrestling champions, former special operations/Medal of Honor awardees, and people with PhD's in obscure subjects that I've met I can't imagine how you could be so skeptical and untrusting!

Seriously, it's amazing how many people claim to be something that they're not and will swear to it all the way down the line. Some of those folks even get jobs based on faked credentials.

http://www.reason.com/0501/fe.ps.cut.shtml
http://www.pownetwork.org/phonies/phonies9.htm

Compared to those claims, pretending to be part of the MIT crew just pales.

[thoreau--Navy Seal Officer, Fi]

rob-

Why are you so skeptical?

[Drooling Richard]

thoreau,

Late in getting back on this thread. I want apologize for the snide remark I make about your qualifications as a physicist. I turned some skepticism (or ignorance) I had about a part of one your posts into personal attack. My foolishness.

Best Regards

[thoreau]

No problem, Richard.