Rubbish in, Garbage Out?

Extraordinary claims require extraordinary press releases?

You have probably read a headline in the past few weeks stating that NASA has verified that an infamous, seemingly reactionless propulsion drive does in fact produce force. You also might not have read the technical report that spurred the media frenzy (relative to the amount of press coverage normally allocated to space propulsion research, anyway), instead relying on the media reports and their own contracted expert opinion. The twist is that it seems to be the case that no one else- excepting perhaps the participants of the conference it was presented at– has read it either, and this includes myself and likely the authors of almost any other material you find commenting on it. The reason is that the associated entry in the NASA Technical Reports Server only consists of an abstract.

The current upswing of interest and associated speculation on the matter of this strange drive is eerily reminiscent of other recent \begin{sarcasm}groundbreaking discoveries\end{sarcasm}: FTL neutrinos measured by the OPERA experiment and the Arsenic Life bacterium from Mono Lake, California. Both were later refuted, some important people at OPERA ended up resigning, and the Arsenic Life paper continues to boost the impact factors of the authors and publisher as Science Magazine refuses to retract it. (current citations according to Google Scholar number more than 300).

I would venture that the manner of disclosing the OPERA findings was done more responsibly than the Arsenic Life paper. Although both research teams made use of press releases to gain a broad audience for their findings (note this down in your lab notebook as “do not do” if you are a researcher), the OPERA findings were at the pre-publication stage and disclosed as an invitation to greater scrutiny of their instrumentation, while the arsenic life strategy was much less reserved. From the OPERA press release:

The OPERA measurement is at odds with well-established laws of nature, though science frequently progresses by overthrowing the established paradigms. For this reason, many searches have been made for deviations from Einstein’s theory of relativity, so far not finding any such evidence. The strong constraints arising from these observations makes an interpretation of the OPERA measurement in terms of modification of Einstein’s theory unlikely, and give further strong reason to seek new independent measurements.

Notice the description of the search for exceptions to Einstein’s relativity as ” . . . so far not finding any evidence. . .” That despite the data they are reporting doing exactly that if anomalous instrumentation could be ruled out. This was a plea for help, not a claim of triumph.

On the contrary, the press seminar associated with the release of Felisa Wolfe-Simon et al.’s A bacterium that can grow by using arsenic instead of phosphorus issued no such caveats with their claims. Likewise it was readily apparent in the methods sections of their paper that the Arsenic Life team made no strong efforts to refute their own data (the principal aim of experimentation), and the review process at Science should probably have been made more rigorous than standard practice. It is perhaps repeated too often without consideration, but I’ll mention the late, great Carl Sagan’s assertion that “extraordinary claims require extraordinary evidence.” The OPERA team kept this in mind, while the Arsenic Life paper showed a strong preference to sweep under the carpet any due diligence in considering alternative explanations. Ultimately, the OPERA results were explained as an instrumentation error and the Arsenic Life discovery has been refuted in several independent follow-up experiments (i.e. [1][2]).

Is propellant-less propulsion on par with Arsenic Life or FTL neutrinos in terms of communicating findings? In this case I would lean toward the latter: more of a search for instrumentation error than a claim of the discovery of Totally New Physics. The title of the tech report “Anomalous Thrust Production from an RF Test Device Measured on a Low-Thrust Torsion Pendulum” denotes the minimum requisite dose of skepticism.

Background reading below, but by far the best take on the subject is xkcd number 1404. The alt-text: “I don’t understand the things you do, and you may therefore represent an interaction with the quantum vacuum virtual plasma.”

23/08/2014 several typos corrected
[UPDATE Full version of tech report: http://rghost.net/57230791%5D via comments from http://ow.ly/ADJqb .
http://www.wired.co.uk/news/archive/2014-07/31/nasa-validates-impossible-space-drive .
http://www.wired.co.uk/news/archive/2014-08/07/10-qs-about-nasa-impossible-drive .
http://www.wired.com/2014/08/why-nasas-physics-defying-space-engine-is-probably-bogus/?mbid=social_twitter .
http://en.wikipedia.org/wiki/Quantum_vacuum_plasma_thruster .
http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20140006052.pdf .
http://www.wired.co.uk/news/archive/2013-02/06/emdrive-and-cold-fusion .
http://www.aiaa.org/EventDetail.aspx?id=18582 .

Much ado about sitting

A few years ago, athletic shoe companies began to cash in on a study or two suggesting that running in shoes was dangerous, guaranteed to ruin your joints and your life, make you less attractive and confident, etc. (at least, that’s how it was translated to press coverage). The only viable answer, vested marketing implied, was to buy a new pair of shoes with less shoe in them.

Despite the obvious irony, consumers flocked in droves to purchase sweet new kicks and rectify their embarrassing running habits. Much like any other fitness craze, popular active-lifestyle magazines ran articles about the trend spinning a small amount of scientific research into definitive conclusions right next to advertisements for the shoes themselves. Fast forward to 2014 wherein the makers of arguably the most notorious shoes in the minimalist sector, the Vibram Five Fingers line, have moved to settle a lawsuit alleging the claimed health benefits of the shoes were not based on evidence. The market frenzy for minimalist footwear appears to have sharply abated. There are even blatant examples of market backlash in the introduction of what could be described as “marshmallow shoes,” such as the Hakko, with even more padding than runners were used to before the barefoot revolution.

An eerily similar phenomenon has appeared, i.e. market capitalisation on nascent scientific evidence, in the latest demon threatening our health: sitting. At the bottom of the orogenic marketplace for accessories designed to get workers a bit less semi-recumbent in the workplace. This market was virtually non-existent only a few years ago, yet now is substantial enough to have spawned an entire genre of internet article.

There is even a new term gaining traction for the condition: “sitting disease.” I sure hope it’s not catching. For now at least the term seems to remain quarantined in quotation marks most places it is used.

Many of the underlying articles in science journals are what is euphemistically referred to as survey science. Long generational time, lack of uniform cultivation standards and potential ethical considerations make Homo sapiens a rather poor model organism. Even if survey data were considered reliable (a dubious assumption), this only reveals associations. Even accelerometer studies, like those at the Mayo Clinic, only measure activity for a few weeks. The results can’t tell you that sitting alone causes obesity. An equally fair hypothesis would be that obesity increase the likelihood to stay sitting, but that’s just called inertia.

Although the studies and their press coverage motivate a burgeoning marketplace for NEAT accessories they don’t actually tell us much in the way of new information. A sedentary lifestyle is unhealthy. Attempts to increase the amount of low-intensity activity throughout the day, such as using a walking desk, are likely to motivate appetite. Without considering diet (and downplaying the importance of exercise), a standing desk, sitting ball, or occasional walking meeting is not likely to have tremendous health benefits when taken alone. And despite the rhetoric, maintaining a smoking habit to break up your sit-time with walks to the outdoors is probably not an equivalent trade-off. Presenting health management in such an unbalanced, single-variable way seems more motivated by trendiness for some, revenue for others, and both for the press. It is not that sitting is actually good for you, it’s just myopic to focus solely on that one health factor. As part of a a sedentary lifestyle gestalt, yes, it does play a role in promoting ill-health. Then again, if you think about it, you probably already knew that before it was cool.


Avoid sensationalist science journalism, consider the sources:
Ford, E.S., and Caspersen, C.J. (2012). Sedentary behaviour and cardiovascular disease: a review of prospective studies. Int J Epidemiol 41, 1338–1353.
Hamilton, M.T., Hamilton, D.G., and Zderic, T.W. (2007). Role of low energy expenditure and sitting in obesity, metabolic syndrome, type 2 diabetes, and cardiovascular disease. Diabetes 56, 2655–2667.
Katzmarzyk, P.T., Church, T.S., Craig, C.L., and Bouchard, C. (2009). Sitting time and mortality from all causes, cardiovascular disease, and cancer. Med Sci Sports Exerc 41, 998–1005.
Rosenkranz, R.R., Duncan, M.J., Rosenkranz, S.K., and Kolt, G.S. (2013). Active lifestyles related to excellent self-rated health and quality of life: cross sectional findings from 194,545 participants in The 45 and Up Study. BMC Public Health 13, 1071.
Rovniak, L.S., Denlinger, L., Duveneck, E., Sciamanna, C.N., Kong, L., Freivalds, A., and Ray, C.A. (2014). Feasibility of using a compact elliptical device to increase energy expenditure during sedentary activities. Journal of Science and Medicine in Sport 17, 376–380.
Schmid, D., and Leitzmann, M.F. (2014). Television Viewing and Time Spent Sedentary in Relation to Cancer Risk: A Meta-analysis. JNCI J Natl Cancer Inst 106, dju098.
Young, D.R., Reynolds, K., Sidell, M., Brar, S., Ghai, N.R., Sternfeld, B., Jacobsen, S.J., Slezak, J.M., Caan, B., and Quinn, V.P. (2014). Effects of Physical Activity and Sedentary Time on the Risk of Heart Failure. Circ Heart Fail 7, 21–27.