A Skeptic Over Coffee: Young Blood Part Duh

Does this cloudy liquid hold the secret to vitality in your first 100 years and beyond? I can’t say for sure that it doesn’t. What I can say is that I would happily sell it to you for $8,000.

Next time someone tries to charge you a premium to intravenously imbibe someone else’s blood plasma, you have my permission to tell them no thanks. Unless there’s a chance that it is fake, then it might be worth doing.

Californian company Ambrosia LLC has been making the rounds in publications like the New Scientist hype-machine to promote claims that their plasma transfusions show efficacy at treating symptomatic biomarkers of aging. Set up primarily to exploit rich people by exploiting younger, poorer people on the off chance that the Precious Bodily Fluids of the latter will invigorate the former, the small biotech firm performed a tiny study of over-35s receiving blood plasma transfusions from younger people. It’s listed on clinicaltrials.gov and everything.

First of all, to determine the efficacy of a treatment it’s important that both the doctors and the patients are blinded to whether they are administering/being administered the active therapeutic. That goes all the way up the line from the responsible physician to the phlebotomist to the statistician analyzing the data. But to blind patients and researchers the study must include a control group receiving a placebo treatment, which in this case there was not. So it’s got that going for it.

To be fair, this isn’t actually bad science. For that to be true, it would have to be actual science. Not only does a study like this require a control to account for any placebo effect*, but the changes reported for the various biomarkers may be well within common fluctuations.

Finally, remember that if you assess 20 biomarkers with the common confidence cutoff of p=0.05, chances are one of the twenty will show a statistical difference from baseline. That is the definition of a p-value at that level: a 1 in 20 chance of a difference being down to random chance. Quartz reports the Ambrosia study looked at about 100 different biomarkers and mentions positive changes in 3 of them. I don’t know if they performed statistical tests at a cutoff level of 0.05, but if so you should expect on average 5 of 100 biomarkers in a screen to show a statistical difference. This isn’t the first case of questionable statistics selling fountain of youth concepts.

All of this is not to say that the experiments disprove the positive effects of shooting up teenage PBFs. It also generated zero conclusive evidence against the presence of a large population of English teapots in erratic orbits around Saturn.

You could conclude by saying “more scientific investigation is warranted” but that would imply the work so far was science.

* The placebo effect can even apply to as seemingly objective a treatment as surgery. Take this 2013 study that found no statistical difference in the outcomes of patients with knee problems treated with either arthroscopic surgery or a surgeon pretending to perform the surgery.

I

Is the future of scientific publishing in-house open access?

WeirdFuture
Photo from flickr user Tom Marxchivist, 1952 cover by Basil Wolverton, used under CC attribution license.

Those of you that frequent theScinder know that I am pretty passionate about how science is disseminated, and you have probably noticed that, like their brethren in newsprint and magazine before them, the big-name publishers don’t know exactly how to react to a changing future, and despite what traditional publishers would have you believe, they are not immune to publishing tripe.

Nature may be butting heads with Duke University over requesting waivers for the open access policy in place there. Apparently the waiver request isn’t even necessarily based on the practical implementation of Duke’s open access policy (Nature allows articles to be made freely available in their final version 6 months after publication), but it does raise the question: how much hassle will universities and their faculty put up with before they take matters into their own hands? As MIT’s Samuel Gershman points out, modern publishing doesn’t cost all that much. Even the fairly exorbitant fees charged to authors by the “gold standard” of open access publishers may be a transient relic of the conventional (turning archaic?) publishing business model. This provides incentive for predatory publishing (as discussed in this article at The Scientist and the basis for the Bohannon article published in Science last October) But if peer review and editing is largely volunteer labour, performed as an essential component of the role of a researcher and with the bill largely footed as a public expenditure, why keep paying enormous subscription fees for traditional publishing? If the trend catches on, as it almost certainly will, leading institutions will continue to adopt open access policies and libraries will see less and less reason to keep paying for outdated subscriptions.

Relevant links:

Scholarly Publishing: Where is Plan B?

California univerisity system consider boycotting Nature Publishing Group

Samuel Gershman’s ideal publishing model, the Journal of Machine Learning Research

Ender’s Game gets it wrong

You loved Ender’s Game, you read it young and considered the story and lessons within to be an important part of your development as a young adult. The loveable, relateable protagonist Ender is a misunderstood genius, destined to become an essential player in the survival of the human race. He was picked on and made fun of, constantly challenged by the pecking order. And all of this torment, from both authority and peers, arose because of his unique abilities as a human. Eventually he is vindicated by singlehandedly saving humanity from a perceived alien menace. Probably, you didn’t just like Ender, you wanted to be Ender, and thought you were two peas in a pod, gifted geniuses before your time. This is also why we like Peter Parker so much, and commonly described criteria for delusions in the Diagnostic and Statistical Manual of. Mental Disorders

But I’m not here to berate you for your literary preferences. I too loved Ender’s Game, identifying with the gifted and talented cast as a reflection of myself and my own cadre (for some reason, everyone considered themselves an Ender, never a Crazy Tom, Dink Meeker, Petra, or even a Bean). The audience as the protoganist is a common trait of tremendously succesful story franchises. Exemplia gratis: Harry Potter, Star Wars, The Matrix, etc., all involve a relatable character plucked from obscurity to “save the world,” blank masks ripe to be painted with our own faces.

A much more drastic feat of projection accompanies our personal escapisms in the story of Ender. The major plot driver of the story is a human projection of our centralized “insect” hierarchy on the Formics. In case you haven’t read or watched yet, the key to defeating the alien menace is to stomp the queen, shutting down the society from the top down. [Spoiler alert one sentence ago.] Once that happens, all the workers stop dead in their tracks like marionettes off strings. Fans of E.O. Wilson’s works and the concept of the superorganism will recognize that this isn’t quite how control plays out in a hive. Ants surely don’t consider themselves to be toiling in some Hymenopteran gulag. Instead, each individual is living out their dream job, which just so happens to be whatever form of labour is in short supply (though for bumblebees there is quite a bit of bullying involved in persuading workers to stay the line). Think Brave New World not 1984. In most cases, the queen is actively involved in negative feedback to repress rival queens among her offspring. This means that the destruction of the queen typically doesn’t destroy the colony’s ability to survive, for ants and many bees a new queen will be along shortly. And the executive control of the queen is almost non-existent. For many cases, it may be more accurate to think of the queen as toiling for the sake of the workers rather than the other way around. From a pure superorganism perspective, the queen is functionally the genitalia, where the workers are the organs, digits and nervous system of a body which just happens to consist of physically separate parts.

Ultimately in Ender’s Game the humans aren’t fighting the Other, they are fighting against their own reliance on central control as mirrored by the Formics. The Formics serve as a model of central authority across interstellar distances, and the vital trait that enables their society is faster than light telepathy. Not something real-world humans are likely to develop anytime soon. Ender empowers his lieutenants with tactical autonomy to overcome communication and planning lags from a central authority.

On balance, us humans are a bit like the Formics in our reliance on the human pseudo-superorganism and it’s technological appendages. Our society is increasingly ever-more reliant on science and technology, but the risk of technological and scientific literacy of most individuals falling woefully behind society as a whole is very real. If the societal infrastructure of the technical elite were destabilized or removed (stomping the queen), how well would society as a whole get along? How many of us have basic skills in coding or electronics (that’s a biased rhetorical: my undergraduate training was in electrical engineering). This is a real concern that doesn’t require the use of a D.R. device [doomsday weapon from the book] to be realized. If the role of science, technology, engineering, and mathematics in education, policy, and culture doesn’t maintain pace with our reliance on those fields for survival, we will face a very real crisis. If the worst scenarios predicted by planetary climatologists are realized, this crisis has already gained an awful lot of momentum. Confronted with problems progressing faster than our ability to deal with them, humans functionally become the unguided Formics of Ender’s Game. What a disheartening answer to Fermi’s Paradox*.

Note: there is plenty of discussion of author Orson Scott Card’s public bigotry available elsewhere on the internet. I deliberately avoid that entirely here.

*Fermi’s paradox: The universe is really big, with plenty of potential for life to develop and follow similar paths as on Earth. Why aren’t we constantly bombarded with wayward or directed radio communications from these civilizations? See Drake equation

Open Access Death Knell. Or Is It?

Is_this_tomorrowrev
I told you publication was a fiat currency

Last week, Science published a critical investigation into the state of peer review in open access journals. John Bohannon, the author,  generated a faux paper describing a set of fictional experiments testing the effects of secondary metabolites from lichen on cultured cancer cells. These papers were sent to a selection of open access journals sampled from the Directory of Open Access Journals (DOAJ) and Larry Beall’s infamous list. The lichen species, secondary metabolite, and cancer cell line were varied randomly to generate iterations of the paper with slight differences, but the shoddy results were the same in each. In a few of the iterations I examined, the structures didn’t even match the secondary metabolite described. The researchers didn’t exist, their names random combinations from a database of first and last names with random middle initials, and the institutions they worked at were fictional. A “dose-dependent” growth inhibition effect in ethanol buffer (with no EtOH in controls) spans five orders of magnitude, and shows growth rates all with overlapping confidence intervals at 95%.

Of 304 papers submitted to various open access journals, 157 were accepted, many of them without any real review taking place. 98 were rejected, and 49 were still up in the air at press time. The article seems to make a solid case against the relatively nascent open access model of publishing, and that is certainly the tone represented by the article and associated press coverage. However, if I assume that the average reader of Science is scientifically literate, then I would expect that most readers will remain unconvinced that open access is dead and dangerous.

In Who’s Afraid of Peer Review Bohannon combines language from both scientific and journalistic writing styles, taking advantage of the credibility implied by describing sample-selection and procedural decisions in a semi-scientific manner, as well as the journalist’s ability to make general claims with a strength that would be questionable in a scientific article.

Examples:

And the acceptances and rejections of the paper provide the first global snapshot of peer review across the open-access scientific enterprise.

137 of the journals chosen for this investigation were pulled from a black list maintained by Jeffrey Beall at the University of Colorado Boulder. In places (such as the general description of results) the overlap between Beall’s list and the journals selected from the DOAJ is not clear. In the original sample, 16 of these journals are in both the DOAJ and Beall’s list, but it is difficult to tell if they made it into the final analysis because 49 of the 304 journals selected for submission were thrown out for “appearing derelict” or failing to complete the article review by press time.

For the publishers on his [Beall’s] list that completed the review process, 82% accepted the paper. Of course that also means that almost one in five on his list did the right thing—at least with my submission. A bigger surprise is that for DOAJ publishers that completed the review process, 45% accepted the bogus paper.

This is somewhat misleading, as it implies that the 45% and 82% results are exclusive of each other. I could not tell just from reading the paper what proportion of the 16 journals found in both Beall’s list and the DOAJ made it to the final analysis. Furthermore, I know this is misleading based on how Jeffrey Beall, who is quite close to the subject, interpreted it: “Unfortunately, for journals on DOAJ but not on my list, the study found that 45% of them accepted the bogus paper, a poor indicator for scholarly open-access publishing overall.”

Acceptance was the norm, not the exception.

157/304 journals (51.64%) accepted the paper. While this is a majority, I would hardly qualify acceptance as a typical result when the split is so nearly even, especially when 137 of the 304 journals had already been blacklisted. Discrediting open access in general based on the results reported is not a fair conclusion.

Overall, the article just misses making a strong critical statement about the state of scientific publication, instead focusing only on problems with predatory publishing in open access. By ignoring traditional journals, we are left without a comparison to inform what may be quite necessary reform in scientific publishing. Bohannon’s article is likely to be seen and read by a large number of people in both science and scientific publishing. Editors can be expected to be on alert for the sort of fake paper used by Bohannon and Science, making any comparison to traditional publishing models just about impossible for now. Finally, the overall effect is to damn innovation in publishing, particularly open access models, and it is not surprising that the sting article was published by the “king-of-the-hill” of traditional scientific journals. It is possible that the backlash against open access and publishing innovation in general will actually impede necessary progress in scientific publishing.

As long as an academic career is judged blindly on marketing metrics such as publication frequency and researchers continue to accept excessive publication fees, there will remain an incentive for grey market “paper-mills” to gather up unpublishable papers for profit. Overall, the open access model has thus far incorporated too much from traditional publishing and not enough from the open source movement.

Science magazine warns you that open access is too open, I say that open access is not too open enough.

text in block quotes is from Who’s Afraid of Peer Review by John Bohannon, Science, Oct. 4 2013

Original version of image here

EDIT: link to John Bohannon’s article

Referencing “I Fucking Love Science” in Your Article Title is a Great Way to Attract Web Traffic

Negging your audience is also important, apparently.

Gregor_Mendel_MonkplusText

John Skylar had an interesting article last week, picked up by Mashables shortly after it went a bit viral. He points out that the beautiful images posted on I Fucking Love Science (hereafter referred to as IFLS) are not all that great of a representation of what science is, and fawning over them does not equate a “love of science,” (though I would argue that stock photography of people pipetting is also “not science”). After I point out that Skylar’s seemingly hostile position on IFLS was actually just an aggressive technique for internet-writers to drive traffic, I want to quickly defend IFLS for what it actually provides. IFLS is what science students, profs and professionals would refer to as their outreach work: it drives interest and awareness among the general populace, many who would never be exposed to it otherwise. For those working in scientific fields, it provides a window into other fields as well. Usually the image is accompanied by a short explanatory text and a link to the original article, or at least the popular press take on it. I would expect that the nominal goal for outreach, not just IFLS, is to increase scientific literacy in the general population. But once again there is a pervasive conflation of terms, and it isn’t confusing data with people and money, it is conflating results with a method. I would claim that in actuality, Mythbusters is potentially better for improving scientific literacy than IFLS, and zombie Richard Feynman would back me up.

I studied science and engineering at the undergraduate and graduate level for about five years, doing a little research along the way to boot. In all of my coursework, I can’t recall taking a single class that actually taught science. The coursework that probably came the closest was in statistics. In lieu of teaching a few courses a year on the intricacies of the scientific method, elegant experiments, etc., courses in a science department almost invariably teach the history and current consensus of a field, by and large treating this information as static facts. In short, they focus on the results and tend to ignore where these results come from. The contents of the science coursework taught at a typical university is not science, it is trivia.

Science is a method for figuring out if an idea we have about the world is false, nothing more. You have heard this before if you ever competed in the science fair as a kid. Science is a comparison of guesses and givens, where guesses are the results we expect if some idea we have about the world might be true. Givens are the data, gathered by observations, measurement, and sometimes assumption. Where the guesses and the givens don’t agree, the original idea is WRONG, simple as that. The best science is based on the ideas that are most readily falsifiable, not necessarily the most complicated. A vague theory is hard to disprove, but it still can be very, very useless.

Ultimately, scientific literacy is not knowing how many flarks jive a phoouon with a 90 rad spin, but the ability to be confronted with a claim and confer upon it a vote of confidence or no confidence in that idea’s reality. The best way to love science is to use it to inform your view of the world, regardless of your profession. Next time another human tries to sell you on an idea, ask to see their p-value, and don’t trust averages without error bars. If you start with an unsparing application of science, the survival rates of nonsense will plummet. Maybe that effect would even trickle up to the higher echelons of U.S. government, and eventually they might enact reasonable policies with an outlook beyond four years at a time, including emphasising a strong and stable investment in academic research.


Hat tip to Neal Stephenson’s novel Anathem for influencing my interpretation of “givens.”

Feynman has the best description of science that I have yet found.

Image from http://commons.wikimedia.org/wiki/File:Gregor_Mendel_Monk.jpg originally from the book “Mendel’s Principles of Heredity: A Defence.” Scans of book at http://www.archive.org/details/mendelsprinciple1902bate