Ever since the Science paper came out earlier this week, the world of academia has been a-flutter with excitement and controlled outrage. In the midst of all this, I decided to do a series of posts examining the situation at hand. This is the third and concluding portion of the series and prior to this:

• Predatory Open Access: Part 1 – A Sting Op and Indictment of the OA Model: I talked about the paper that has set us all aflutter and how it has pretty much trashed the Indian open access system.
• Predatory Open Access: Part 2 – Peer Review in OA and Ethics of “Sting Op Research”: I talk about the issue of faltering peer review and look briefly into the ethics of sting operation in medical research.

In this concluding part, I try to highlight the fact that Science effort was, by far, not the first such endeavor in scamming journals into publishing papers of dicey quality (just as a matter of joke or sting operations) and I look back at some other similar episodes.

Penis Captivus and Sir William Osler:

In the Philadelphia Medical News of December 13, 1884, a certain Dr. Egerton Yorick Davies reported a case report (via a letter to the editor) of a new disease called Penis Captivus. He reported (text via Life in the Fast Lane blog):

At bedtime, when going to the back kitchen to see if the house was shut up, a noise in the coachman’s room attracted his attention, and, going in, he discovered to his horror that the man was in bed with one of the maids. She screamed, he struggled, and they rolled out of bed together and made frantic efforts to get apart, but without success. He was a big, burly man, over six feet, and she was a small woman, weighing not more than ninety pounds. She was moaning and screaming, and seemed in great agony, so that after several fruitless attempts to get them apart, he sent for me. When I arrived I found the man standing up and supporting the woman in his arms, and it was quite evident that his penis was tightly locked in her vagina, and any attempt to dislodge it was accompanied by much pain on the part of both. It was, indeed, a case “De cohesione in coitu.” I applied water, and then ice, but ineffectually, and at last sent for chloroform, a few whiffs of which sent the woman to sleep, relaxed the spasm, and released the captive penis, which was swollen, livid, and in a state of semi-erection, which did not go down for several hours, and for days the organ was extremely sore. The woman recovered rapidly and seemed none the worse.

In the days to ensue, there would be vehement debates and doubts whether the case report was true or fabricated and whether there truly was such a case. People questioned at first, “Penis Captivus: Did it occur?” (BMJ, 1979). Some sounded more sanguine “Penis Captivus: It occurred” (BMJ, 1980):

Legend has it that Egerton Yorick Davies was none other than Sir William Osler himself. Harvey Cushing, in his seminal work, The Life of Sir William Osler, says Egerton Y. Davis was Osler’s “fanciful half, who first and last got him into a good deal of trouble.” Legend goes that Theophilus Parvin, who was on the editorial board of the Medical News along with Sir William Osler, was extremely enthusiastic about vaginismus and related conditions and in the issue preceding Egerton Davies’ case report, he had written an extensive editorial about vaginismus, which earned Osler’s scorn. He had a wager with friends that he could get a case report published in the journal on a similar, more exaggerated condition; and he ended up authoring one of the first medical spoofs and which ended up being the most cited paper on the topic.

There have been many that followed in his footsteps, but few would have been more brash and outrageous than Alan Sokal, the New York Physics professor, who has been immortalized by his Sokal Hoax/Affair.

Sokal Affair:

In 1996, the “humanities” journal Social Text, a post-modernist publication out of Duke University, there was a special theme issue on “Science Wars” in which post-modernist scholars from the social sciences and humanities attacked the scientific method of study. Alan Sokal, in response to such allegations, wrote up a spoof paper, titled, with a flourish, as: Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity. In that paper, he claimed that quantum physics is a linguistic and social construct and actually critiques the scientific objectivity from the post-modernist viewpoint. Admittedly, the journal, Science Text, at that time, did not practice peer review, but its editorial board comprised of post-modernist scholars of international acclaim and fame. That the article was a parody and not written seriously, was revealed by Sokal in a subsequent publication in Lingua Franca, entitled, “A Physicist Experiments with Cultural Studies”. In this blistering article, he parodied the loosey-goosey approach of the social studies and humanities experts towards scientific rigor and objectivity. In this paper, Sokal opens, lambasting the post-modernists, saying:

For some years I’ve been troubled by an apparent decline in the standards of intellectual rigor in certain precincts of the American academic humanities. But I’m a mere physicist: if I find myself unable to make head or tail of jouissance and différance, perhaps that just reflects my own inadequacy.

So, to test the prevailing intellectual standards, I decided to try a modest (though admittedly uncontrolled) experiment: Would a leading North American journal of cultural studies — whose editorial collective includes such luminaries as Fredric Jameson and Andrew Ross — publish an article liberally salted with nonsense if (a) it sounded good and (b) it flattered the editors’ ideological preconceptions?

The answer, unfortunately, is yes. Interested readers can find my article, “Transgressing the Boundaries: Toward a Transformative Hermeneutics of Quantum Gravity,” in the Spring/Summer 1996 issue of Social Text. It appears in a special number of the magazine devoted to the “Science Wars.”

What’s going on here? Could the editors reallynot have realized that my article was written as a parody?

He continues in this vein for a while and alternately derides or criticizes the approach taken by the publishers of his parody paper for their lack of objectivity and personal biases. The publication of the fact that it was a parody was immediately followed by a storm of debates and questions. There were people, on either side of the divide, asking whether it was ethical on behalf of Sokal to have authored a hoax paper and what were the ethics of the practice. It was largely viewed to be an intellectual con, considering the fact that journals and academic publishers always assumed that there was academic and scientific integrity behind the construction of a paper.

One of the most famous names to have come down heavily on Sokal and associates was Jacques Derrida. Himself the target of much ridicule from the media for the scam, he hit back calling Sokal’s stunt as “sad” (triste). The scholarly world was cleaved into two as Derrida launched the serious allegation of intellectual bad faith against Sokal and his associates.

However, this event clearly set the tone for the fact that scholars were thinking about what could go through the editorial sieves and what got caught. Subsequent to this hoax paper being published and the massive media attention and academic gossip it generated, Social Text instituted a conventional peer review system; and slowly but surely, peer review began to get entrenched as the “industry standard” of the academic publishing world.

The Rosenham Experiment:

Ironically, this was another of the spoof experiments which was published in Science. David Rosenham, a prominent psychologist, conducted this study in order to question the validity of the psychiatric diagnoses. He published the findings in 1973 in Science under the brilliant, tongue-in-cheek title: On being sane in insane places. He conducted the study in two phases. In the first phase, 8 participants, 5 men and 3 women, feigned auditory hallucinations in order to get admitted to 12 different psychiatric institutions spread all over the USA. Once admitted, they claimed to have been relieved of all symptoms; however, they were still kept admitted with diagnoses of different variants of Schizophrenia. The average stay was of 19 days and at release, all but one were signed out with diagnosis of “Schizophrenia (in remission)”.

The second phase happened when hospital administrators at one facility got enraged at Rosenham’s con and challenged him to send pseudo-patients that their doctors would then weed out. In the following week, of the 193 new patients that reported to the facility, 41 were identified as potential pseudo-patients, with 19 of them being identified by at least one psychiatrist and one other staff member as a pseudo-patient. The fact, however, was that Rosenham had not sent a single pseudo-patient to the facility in the said time period. In his sombre tone, Rosenham concludes his paper, saying:

 It is clear that we cannot distinguish the sane from the insane in psychiatric hospitals.  The hospital itself imposes a special environment in which the meaning of behavior can easily be misunderstood.  The consequences to patients hospitalized in such an environment – the powerlessness, depersonalization, segregation, mortification, and self-labeling – seem undoubtedly counter-therapeutic.

I do not,  even now, understand this problem well enough to perceive solutions.  But  two matters seem to have some promise.  The first concerns the proliferation of community mental health facilities, of crisis intervention centers, of the human potential movement, and of behavior therapies that, for all of their own problems, tend to avoid psychiatric labels, to focus on specific problems and behaviors, and to retain the individual in a relatively non-pejorative environment.  Clearly, to the extent that we refrain from sending the distressed to insane places, our impressions of them are less likely to be distorted.  (The risk of distorted perceptions, it seems to me, is always present, since we are much more sensitive to an individual’s behaviors and verbalizations than we are to the subtle contextual stimuli than often promote them.  At issue here is a matter of magnitude.  And, as I have shown, the magnitude of distortion is exceedingly high in the extreme context that is a psychiatric hospital.)

The second matter that might prove promising speaks to the need to increase the sensitivity of mental health workers and researchers to the Catch 22 position of psychiatric patients.  Simply reading materials in this area will be of help to some such workers and researchers.  For others, directly experiencing the impact of psychiatric hospitalization will be of enormous use.  Clearly, further research into the social psychology of such total institutions will both facilitate treatment and deepen understanding.

I and the other pseudopatients in the psychiatric setting had distinctly negative reactions.  We do not pretend to describe the subjective experiences of true patients.  Theirs may be different from ours, particularly with the passage of time and the necessary process of adaptation to one’s environment.  But we can and do speak to the relatively more objective indices of treatment within the hospital.  It could be a mistake, and a very unfortunate one, to consider that what happened to us derived from malice or stupidity on the part of the staff.  Quite the contrary, our overwhelming impression of them was of people who really cared, who were committed and who were uncommonly intelligent.  Where they failed, as they sometimes did painfully, it would be more accurate to attribute those failures to the environment in which they, too, found themselves than to personal callousness.  Their perceptions and behaviors were controlled by the situation, rather than being motivated by a malicious disposition.  In a more benign environment, one that was less attached to global diagnosis, their behaviors and judgments might have been more benign and effective.

The paper has since remained a landmark in the criticism of scientific diagnostic method of psychiatric illnesses as we are, eve today, grappling with the debate to define what is normal and what is not.

Hunting the Hunters: Trapping Predatory Open Access Journals:

In what can be described to be the closest to the con pulled off by John Bohannon on a series of journals, Phil Davis, a Graduate Student at Cornell pulled the wool over the eyes of Bentham Publications, who spammed him with a series of emails soliciting either papers or asking him to sit on the editorial board of journals he had absolutely no expertise for. In order to check the veracity of the claims made by the organizers, he submitted a gobbledygook paper using the SCIGen software. The SCIGen website introduces itself as:

SCIgen is a program that generates random Computer Science research papers, including graphs, figures, and citations. It uses a hand-written context-free grammar to form all elements of the papers. Our aim here is to maximize amusement, rather than coherence.

Davis generated a paper titled: Deconstructing Access Points and submitted it to a Bentham published journal, The Open Information Science Journal (TOISCIJ). Davis added a couple of fictitious, but believably common names. In a blog post on the Scholarly Kitchen, Davis describes how he set about setting his institutional affiliation:

The manuscript was given two co-authors, David Phillips and Andrew Kent.  Any similarity to real or fictitious, living or dead academics is purely coincidental, as was their institutional affiliation: The Center for Research in Applied Phrenology based in Ithaca, New York.  If the acronym didn’t reveal the farce right away, phrenology is the pseudoscience of reading personality traits from the lumps on one’s head.

The article was accepted after four months and they were asked to submit the 800$ author processing charges. The journal wrote back, saying:

Davis subsequently retracted the paper and I am sure, is still wondering how the paper got accepted. If anyone even read the paper, they would have noticed the complete balderdash that it actually was. The methods section went:

In this section, we discuss existing research into red-black trees, vacuum tubes, and courseware [10]. On a similar note, recent work by Takahashi suggests a methodology for providing robust modalities, but does not offer an implementation [9]. Clearly, if throughput is a concern, our methodology has a clear advantage. A recent unpublished undergraduate dissertation [22] proposed a similar idea for kernels [1, 9, 16, 17]. Continuing with this rationale, the choice of IPv4 in [12] differs from ours in that we simulate only appropriate configurations in our method [1]. Unfortunately, the complexity of their method grows logarithmically as heterogeneous models grows. We had our method in mind before Butler Lampson published the recent little- known work on amphibious models. Obviously, despite substantial work in this area, our approach is evidently the application of choice among security experts.

The whole paper goes on in this thread, weaving nonsense, grammatically correct, context free language, inserting random technological jargon and sounding and looking all professional on the surface, but being essentially, vacuous beneath all the brouhaha. Even I, a highly un-tech-related person can feel that something is amiss in the whole business!

Davis subsequently submitted another paper to a similar conference, the WMSCI 2005, where they went after 165 people came up with donations so that they could go and showcase their bogus paper!

You can catch their whole show here: LINK to videos.

Closing Comments:

Like in every other field of work, in academic publishing there are charlatans. The onus should be on us to keep our heads up and know which way to look. There can be a lot of debate about the open access model and whether or not the fiduciary considerations place a binding bias towards publishing, no matter what comes up, but at the end of the day, pay-walling of scholarly resources is an even more heinous crime. It robs more people of the right to know, to read, to learn, to share their work.

One way of keeping things clean is a de facto post-publication peer review system, which is not a big deal in today’s world of online access, twitter and blogging. We need to push to bring out the whole journal article, including datasets and other “raw, unprocessed material” as accessible, web-based supplementaries, so that anyone who doubts the truth or accuracy of execution of the research project can get their hands dirty with the data and see for themselves how things stack up.

The academia from the developing world is especially at risk to fall prey to such charlatans and there should be spread of awareness about such charlatans once they are spotted.

It seems that even in the academic world, the old adage of “survival of the fittest” is coming in to play!