Evaluating scientific evidence

In an era where PR rules the news and superlatives rule science, how can a reader really know what’s what?

Critical analysis skills are a key survival skill, but facts-on-demand has taken over in many modern educational structures. And despite the best intentions, the ‘openness’ of the internet has simply confused things. Opinions on scientific issues regularly rub shoulders with evidence and sometimes it can be hard to tell which is which (for scientists and non-scientists alike).

And what is ‘scientific evidence’ anyway? I wrote about this a few years ago, but it’s much more complex than I had room to explain.

I recently stumbled across this great series on how to evaluate scientific publications, from the German peer-reviewed medical magazine Deustches Ärtzeblatt. The papers are useful for teaching, for critical news audiences, and for practicing scientists. All articles are open access, translated from German. The series started in 2009 – I haven’t found a contents list or an apparent end-date for the series, so I will keep this updated as they get published.

The series is aimed at medical scientists interpreting clinical trials, so not all of them may be relevant to sciences that do stats a bit differently from the linear standard (e.g. ecology). But they are great starting point for anyone wanting to understand basic biostatistics, and I guarantee even the most seasoned scientist will find something useful in one of the papers. Most of them at least have an absolute gem of a quote buried in the text somewhere.

One should avoid conclusions that are supported neither by one’s own data nor by the findings of others. It is wrong to refer to an exploratory data analysis as a proof.

(Also see my previous post on causal language)

Part 1: Critical Appraisal of Scientific Articles

Part 2: Study Design in Medical Research

Part 3: Types of Study in Medical Research

Part 4: Confidence Interval or P-value?

Part 5: Requirements and Assessment of Laboratory Tests

Part 6: Systematic Literature Reviews and Meta-Analyses

Part 7: Descriptive Statistics. The Specification of Statistical Measures and Their Presentation in Tables and Graphs.

Part 8: Avoiding Bias in Observational Studies

Part 9: Interpreting results in 2 x 2 Tables

Part 10: Judging a Plethora of p-Values. How to Contend With the Problem of Multiple Testing

Part 11: Data Analysis of Epidemiological Studies

Part 12: Choosing Statistical Tests