How do you review a conceptual paper?

Manu Saunders

Scientific disciplines grow from new concepts, ideas, theories and expert opinions, not just data. But conceptual papers are the hardest kind of scientific paper to publish.

Too many researchers…seem to think that any non-empirical paper is simply an essay and devoid of deeper scholarship. Nothing could be further from the truth. More than once I have received comments from reviewers claiming a paper is nothing more than an essay, implying essays are little more than opinions. But aren’t all papers “opinions” in one form or another?

~ Rudy Hirschheim (2008)

By definition, a conceptual paper doesn’t present original data…but it must present an original concept. It synthesises knowledge from previous work on a particular topic, and presents it in a new context to provide a springboard for new research that will fill knowledge gaps.

Conceptual papers shouldn’t follow the status quo; they need to show how moving beyond the current norm will enhance knowledge.

Most researchers have experienced the struggle to get a conceptual paper published. I’ve co-authored two conceptual papers and I’m currently battling to get a third into existence. All have been more difficult to get through peer review than any of the data papers I’ve published.

“Pollinators, pests, and predators: Recognizing ecological trade-offs in agroecosystems”:
6 journals; 12 months before acceptance.

“Limitations of the ecosystem services versus disservices dichotomy”: 1 journal, 2 reject-resubmit revisions; 12 months before acceptance

Current manuscript: 5 journals, 2nd reject-resubmit from the last journal.

So why is it so hard to contribute a new idea to Ecology? A few thoughts from my experience as author and reviewer, please leave your own in the comments.

Journals don’t educate reviewers about differences in article styles

Most ecology journals have one or more categories for authors to submit conceptual papers for review. These are called various things, like Forum articles, Viewpoints, Perspectives, Opinions etc. Aside from brief (often vague) descriptions of style and format on the journal’s website, there are no criteria provided for reviewers to judge these categories. As a reviewer, I’ve never received advice on how to review a conceptual paper for these journals. And I’ve never been encouraged to consult the journal’s website for more information on the category I’ve been asked to review. This matters, because conceptual papers have very different expectations compared to data papers, the gold standard of scientific publications.

Editors don’t agree on journal scope, author guidelines and expectations

As an author, some of the rejections I’ve received have been from editors rejecting us without review because our paper “didn’t meet the criteria of a Review paper”. Even though we’d submitted our paper under one of the journal’s specific conceptual categories (see above), not under the Review category. Other times, we submitted to a journal using the journal’s proposal system for conceptual papers. The editor that read the proposal liked our idea and invited us to submit the full paper. A different editor then handled our full submission and rejected it without review. So why do journals have so many different categories for submission, if editors aren’t trained to recognise these categories?

Do journals prefer new ideas from senior/established researchers?

I have no data on this, so don’t quote me. Why do so many high-impact journals only publish “invited” conceptual papers?

Scientists aren’t trained to write (and therefore review) conceptual papers

Concepts are embedded in context and based on expert opinion. Yet the current paradigm in science training and practice is facts/evidence-based. As scientists, we’re trained not to have opinions and to control context out of experiments. For peer review, we’re trained to identify standards of study design, statistical analyses, presentation and formatting etc. But there are very few standards for judging someone else’s ideas. It’s easy to subconsciously revert to subjective opinion and critique conceptual papers based on personal views/experience on the topic.

So here are some thoughts on reviewing conceptual papers:

  • Conceptual papers DON’T NEED TO PRODUCE DATA. “But meta-analysis!…” One of the most common criticisms from reviewers that I’ve experienced is that we haven’t produced data to back up our ‘new idea’. Conceptual papers aren’t about data, they’re about context.
  • Focus on how the argument is built, not the argument itself. As a reviewer, you don’t need to agree with the authors’ opinion/new idea, but you do need to be convinced that their argument is a valid contribution to the literature. As an author, you need to convince everyone, including people beyond your peers, that your idea is valid within its context. Many Humanities disciplines teach these skills, e.g. the ‘old-fashioned’ and misunderstood Rhetoric.
  • If a journal doesn’t tell you whether the conceptual paper you’re reviewing is a resubmission, ask the editor. Before pointing out flaws, new reviewers should know what elements the authors have added onto their original text in response to previous review comments. I think this is more important for conceptual papers than data papers, because critiques of conceptual papers are more likely to be subjective than objective.

I can’t find any published guidelines for reviewing a conceptual paper in Ecology. But here are some from other disciplines:

Gilson LL & Goldberg CB (2015) Editors’ comment: So, what is a conceptual paper? Group & Organization Management, 40: 127-130.

Hirshcheim R (2008) Some Guidelines for the Critical Reviewing of Conceptual Papers. Journal of the Association for Information Systems, 9: 432-441.

Whetten DA (1989) What constitutes a theoretical contribution? The Academy of Management Review, 14: 490-495.

© Manu Saunders 2018

 

18 thoughts on “How do you review a conceptual paper?”

  1. Marco Mello June 21, 2018 / 10:26 PM

    Very nice post! We really need to talk about this. Yes, there are many kinds of ecological papers nowadays and nobody (authors, reviewers or editors) receives training in how to write or handle them. You refer to “data papers”, but maybe this term has a different meaning now. It refers to papers that present data sets with metadata. “Research papers” seems to be a more adequate term to refer to empirical studies. If we think about the broad category “theoretical synthesis”, which encompasses theoretical papers, its elements vary from opinions, essays, reviews, meta-analyses, and discursive models to mathematical models. Maybe our journals should give specific guidelines for each kind of paper in the current highly diverse writing ecosystem.

    Liked by 1 person

    • Manu Saunders June 21, 2018 / 10:38 PM

      Thanks Marco! I use ‘data papers’ to refer to all research/empirical papers. Agree that there are so many different perceptions of paper categories, especially across disciplines & cultures!

      Liked by 1 person

  2. Bartosz Bartkowski June 22, 2018 / 1:24 AM

    Thanks for this interesting post on a very important topic. Having so far published mainly conceptual papers myself (they are a bit more “acceptable” in environmental/ecological economics, I guess), I can only agree with your recommendations. I particularly think the second one is important: that conceptual work should not be judged (as it often is, alas) by whether the reviewer agrees, but mainly based on whether it offers a novel perspective and how the argument is developed. So it’s about novelty, logic and consistency, not data.

    Liked by 1 person

  3. cinnabarreflections June 22, 2018 / 7:09 AM

    Excellent post! I think that one reason conceptual papers and new ideas get a rough ride may be that they potentially threaten the foundation upon which scientists in that discipline may have based their own work, and perhaps their career. In that sense, science can be quite conservative, and not as objective as it should be. I would welcome your thoughts on this.

    Liked by 2 people

    • Manu Saunders June 22, 2018 / 9:09 AM

      Thanks! Wasn’t it Max Planck who made the observation ‘science advances one death at a time’?…I think fields dominated by particular researchers/research groups can be harder to break into by younger researchers with new ideas

      Liked by 1 person

    • Bartosz Bartkowski June 22, 2018 / 3:10 PM

      This sounds a lot like the Kuhnian/Lakatosian variant of the theory of science… Conceptual papers implicitly or explicitly question parts of the paradigm (Kuhn) or hard core (Lakatos) of a discipline.

      Liked by 1 person

  5. Luke O'Loughlin July 1, 2018 / 9:36 AM

    Good post Manu! You should write it up as a conceptual essay 😉 My own experience trying to get a concept paper published was 5 journals, two with multiple rounds of peer-review, and 18 months before acceptance. You could summarize almost all rounds of reviews we received as “R1 = I don’t agree with this idea, Reject”, “R2 = It’s fine, Accept”, but because R1 was always way more vocal than R2 (read: wrote far more content in their review), we just kept getting rejected. The points you’ve raised here are exacting what we kept getting frustrated about through the whole process. It sucks as well because often a concept paper will be the most fun to write (but clearly not the most fun to get published)….

    Liked by 1 person

    • Manu Saunders July 2, 2018 / 3:20 PM

      Thanks Luke! Yes, they are often the most rewarding to write, which adds to the frustration. And perhaps editors are more likely to err on side of reject with contradictory reviews, compared to same situation with data papers? Just have to keep trying!

      Like

  6. Pingback: Moving on from the insect apocalypse to evidence-based conservation – Ecology is not a dirty word
  7. Pingback: What is a literature review? – Ecology is not a dirty word
  8. Pingback: No evidence of seasonal blog post views in the southern hemisphere: response to Leather (2020) and Ollerton (2020) – Ecology is not a dirty word
  9. Pingback: How not to do peer review – Ecology is not a dirty word
  10. Pingback: Last year – Ecology is not a dirty word
  11. Pingback: Let’s talk about standards for scholarly opinion articles – Ecology is not a dirty word
  12. Pingback: New paper: we don’t know how extreme fire impacts Australian invertebrates – Ecology is not a dirty word
  13. Kaz Ohashi (@KazOhashi_Lab) January 3, 2022 / 11:29 AM

    This is a great post! I believe that the major progress in science is achieved through the development of conceptual frameworks, not data themselves. Data are also important, of course, but only in terms of fostering concepts or confirming their correctness.

    I totally agree with you that many people underestimate the importance of conceptual papers. This might be related to the fact that a lot of people aren’t trained to develop hypotheses before collecting data (some even hate hypothesis-driven studies). This will also be the main reason why people tend to focus on meta-analyses rather than conceptualization of new ideas.

    “When are hypotheses useful in ecology and evolution?”
    doi.org/10.1002/ece3.7365

    My tentative strategy to cope with this prevailing attitude would be submitting to journals that specialize in review articles. But as you pointed out, we need a more fundamental improvement in our thinking.

    The following editorial was a great help to me in writing a synthetic review paper for Biological Reviews. I thought it might also help us review a conceptual paper.

    “The anatomy of an excellent review paper”
    doi.org/10.1111/1365-2435.13207

    Like

  14. Wasif Hussain December 21, 2022 / 3:02 AM

    This was very helpful to a person who hasn’t written a professional paper and now writing a conceptual paper.

    Scientific Method and it’s limitations are also nicely pointed out here. I am trained as a Sculptor and then specialized as designer. So what these journal authors might be rejecting as “mere subjective” in our domain even that can be tested, refined and improved. Subjectivity can also be supported by logic, reasoning, in fact in fine arts or design domains, we develop ideas with the help of critics, authors, and news paper style articles. Even without reaping the benefits of scientific method we have been building concepts and contexts, producing results, and contributing to social gains.

    My context is that I am proposing a new word (word+terminology+lexicon) in Urdu Language in Pakistan, instead of using the english word design to signify that particular domain and what it caters to.

    Your statement “Conceptual papers aren’t about data, they’re about context.” hits home.

    Like

Comments are closed.