scientific evidence

All impact metrics are wrong, but (with more data) some are useful.

Manu Saunders6 Comments

A couple of years ago I wrote about some of the limitations of relying on Altmetrics as an indicator of a paper’s impact, because it doesn’t pick up all online mentions.

Yes, impact metrics are flawed; experts have been pointing this out for years. And I’m not singling out Altmetrics here, there are a few different impact metrics used by different journals for the same goal, e.g. PlumX, Dimensions, CrossRef Event Data.

Despite their flaws, we’re all still using them to demonstrate how our work is reaching global audiences. I used them recently in a promotion application and a major grant application.

But I’m now questioning whether I will keep using them, because they are deeply flawed and are consistently misused and misinterpreted. They are literally a measure of quantity without any context: the number of shares or mentions, but no indication of how and why they are being shared.

This is problematic for a few reasons. Continue reading →

Bees and meat consumption: a spurious correlation

Manu SaundersLeave a comment

I’ve seen and heard a few claims circulating that removing meat from your diet is essential to ‘save the bees’. This is misleading and draws a long bow between lots of random correlations to promote a particular agenda.

Sure, intensive meat production contributes to some big environmental problems and there are plenty of reasons to reduce your meat consumption. But there is absolutely no scientific evidence to support claims that eating meat is bad for bees.

Meat comes in many forms. From wild game to highly-processed ‘meat products’, from large-scale intensive feedlots to diversified low-intensity grazing systems, from locally-produced to high food miles. It is generally impossible to make blanket statements about all meat. Continue reading →

Times have changed: dealing with dodgy science in the internet age

Manu Saunders13 Comments

Dodgy science, dodgy scientists and dodgy humans are not a new thing. And dodgy scientific papers have been published since the dawn of scientific publishing. In 1667 an article on ‘snakestones’, a pseudoscience medical cure with absolutely no basis in truth, appeared in one of the first issues of the oldest known scientific journal, Philosophical Transactions of the Royal Society (now Phil Trans A, one of the most prestigious modern scientific journals).

Since then, disreputable papers have made regular appearances in reputable journals. And there are different scales of disreputable. The paper claiming that octopi originated from outer space was clearly far-fetched, while the scholars who recently argued there was a ‘moral panic’ over free-ranging cats simply highlighted how interdisciplinary research is often challenged by opposing methodological approaches (note: I agree with most ecologists that free-ranging cats are not good for wild animals, including insects). Continue reading →

Insectageddon is a great story. But what are the facts?

Manu Saunders63 Comments

Hype is an ineffective communication strategy, especially when based on limited facts. There are many elements to effective communication – simply raising awareness about a problem is not enough if audiences don’t engage with the facts and participate in developing solutions.

The latest instalment in the Insect Armageddon saga is out. I wasn’t going to write about it. After my previous posts, I didn’t want to sound like a stuck record. But I’ve had a few media requests, some from journalists who found my original blogs. Most journalists I spoke to have been great, and really understand the importance of getting the facts straight. But a few seemed confused when they realised I wasn’t agreeing with the apocalyptic narrative – ‘other scientists are confirming this, so why aren’t you?’

This latest review paper has limitations, just like the German and Puerto Rican studies that received similar hype over the last few years. This doesn’t make any of them ‘bad’ studies, because every single research paper has limitations. No single study can answer everything neatly. Science takes time. Continue reading →

Data, not just for scientists

Manu Saunders7 Comments

Early last year I wrote a post on ecology and mathematics that was inspired by an online discussion happening at the time. Although comprehensive advanced maths skills are not essential to being an influential or inspiring ecologist, a good level of mathematical knowledge and understanding of statistical analysis is definitely necessary to create honest science and communicate the importance of your work to others.

But it’s not just ecologists who need mathematical common sense. Anyone who deals with, or is interested in science needs to understand the ambiguity of an average, or the difference between a regression and a correlation. In fact, anyone who cares about the society they live in should be aware how deeply statistics and data now influence the way we live – policies and decisions on anything from what product choices you find in retail stores to how much tax you pay are all based on data.

Why does this matter to us? Well, if those data are a bit dodgy, or haven’t been analysed and presented appropriately, problems arise. And when these kinds of data misrepresentations are used to fuel public opinion or inform government policy, there can be serious impacts on communities, individuals and ecosystems. Continue reading →