In a recent article first-authored by William Laurance researchers report that, rather unremarkably, publishing more papers before receiving a PhD predicts that an individual will have a more successful career in research, measured solely by publication frequency. They also considered first language, gender precociousness of first article, and university prestige. If publication frequency before attaining the PhD is the best predictor of career publication frequency, just how good is it? They report an r2 value of about 0.14 for the best model incorporating pre-PhD publications, with models lacking this predictor faring much worse.
If I have a model that only explains 14% of the deviance of the data, well, I think it is time to find a new model. When they included the first three years immediately following attaining a PhD, the r2 value jumped to 0.29 for publications alone, and slightly better when the model includes one or more of the other predictors. Better, but still pretty pathetic. If you are hiring people with a 29% rate of picking the right candidate based on some metric of success chances are you won’t be in charge of hiring for long. The paper only looked at the first ten years immediately following the PhD degree, so including the first three years is a bit like predicting rain when you are already wet. Why were the models so miserable? The range of publication frequency over the first ten years was pretty wide, from 0 to 87 papers published. On top of that, their sample consisted only of individuals who had managed to land a university faculty job. That’s right, one or more of these scientists landed a tenure-track position with zero publications. Jealous?
The sample selection is a pretty major flaw of the paper, in my opinion. The scientists surveyed were all on one rung or another of the assistant/associate/full professor ladder, which is to say that everyone they considered were extremely high acheivers among the total population of people holding biology PhDs. The rate of biology PhDs attaining faculty positions six years post-degree has dropped from 55% in 1973, to 15% in 2006 . Since their data only represented successful academics, their models had no chance of predicting which individuals would drop out of research altogether as opposed to going on to become a principal investigator. Predicting whether an individual is able and willing to continue in science research would be a lot more telling than whether they published 2 versus 10 articles per year their first decade out of grad school.
Using publication frequency as the sole measure of success is certainly rife with limitations (though they do mention a close correlative agreement with h-index). What about quality? What about real, meaningful, contributions to the field? What about retractions? I would be much more interested in a model that could predict whether a researcher would have to withdraw an article during their career than how many articles they might generate. Hopefully with a bit better r2 than 0.14, though.
Publication is often referred to as the “currency” of academia. Well I’d like to posit that this currency is purely fiat. If inflation continues as it has been doing , the rate of fraudulent papers can only increase . In my estimation, 300 papers with 3 retractions is worth a lot less than a “measly” 30 papers total. The commonplace occurrence of papers that must be withdrawn (not to mention fraudulent papers never outed, frivolous claims and tenuous conclusions) has broader implications beyond an individual’s career or a journal’s bottom line. When bad science becomes the new normal, public trust deteriorates, and anti-science sentiments thrive.
The authors of the paper did have what I would consider a good take-home: faced with two applicants, one with a PhD from a prestigious university and the other from a lesser-known institution, pick the one with the better publication record. I would go one further and encourage hiring decisions to be informed by actually reading the papers. And vet the sources in these papers’ references. It’s not too hard, and if your job description includes hiring new talent, it’s your job. ‘A’s hire ‘A’s, and ‘B’s hire ‘C’s. Don’t be a ‘B,’ Science (with a capital ‘S’) depends on it.
Laurance et al Predicting Publication Success for Biologists Bioscience Oct. 2013
via conservation bytes