Guidelines for effective manuscript evaluation

Thank you for agreeing to review a manuscript for Psychotherapy.

Although the editor and associate editors make final decisions about manuscripts, I view the successful culmination of the journal review process as collaborative between the editorial consultants and action editor for a given manuscript. As an editorial consultant, you provide essential input to the scholarly peer-review process, and your written evaluation is a crucial part of that input.

In an effort to aid in the review process, I suggest the following five criteria:

• Respect
• Balance
• Education
• Specificity
• Thoroughness

Each of these is discussed below.

I suspect all of us have received reviews in our careers that have been unnecessarily caustic, demeaning or even outright hostile. When this occurs I think it ultimately impugns the reputation of the journal and editorial board from which it originates.

So first and foremost, I believe reviews should be written in a respectful and, dare I say, even a kind manner.

Since Psychotherapy will likely be rejecting most of the work that is submitted, there is no need to add insult to injury. I do not mean to suggest that reviewers refrain from honestly conveying what they see as limitations of the manuscript and this may be honestly stated.

Although I would suggest making comments from the first person, such as "I believe", "In my opinion", "as I see it", "my question is", "It seems to me" etc. I also recognize that in some cases material that is submitted can be problematic, to the point of creating exasperation in the reviewer. In such cases I would ask that you frame your comments in the review as if you are sitting at a table with the manuscript's author, sharing a cup of coffee together. And only write in your review what you would say to the author face-to-face in that setting.

I believe one of the best routes to demonstrating respect in the review process is commenting on both the strengths and limitations of a manuscript.

There are no perfect research studies, case presentations or theoretical formulations, all will have strengths and limitations. When writing a review, it is very easy to focus on problems or limitations. These are important, but be sure to recognize strengths in the manuscript.

Appreciate that something useful is likely to have occurred during the hundreds, perhaps thousands, of hours that were spent conducting the research and preparing the manuscript. At the same time, be clear about problems and limitations you see in the hypotheses, design, analyses, or writing. But again, try to use positively toned, constructive comments regarding limitations.

I expect all of us have received reviews where we learned useful information or facts regarding the area of study, and subsequently our work was improved and our thinking expanded. When this occurred, the review process has been educational in nature. I would like to encourage this educational model for the reviews provided at Psychotherapy.

That is, to disseminate knowledge and consequently to stimulate authors to improve their communication and methodology from a more consultative, helpful, and supportive position than that of a gatekeeper model. This is not to imply that reviews should be uncritical. They naturally must be critical. It does imply, however, that it is not enough simply to accept or reject a paper and communicate that to the authors.

The process of submitting a manuscript and receiving an editorial evaluation should be a positive educational experience for authors. Thus, in addition to helping editors make decisions, manuscript reviews should be knowledgeable critiques aimed at helping authors improve their scientific functioning (e.g., writing, thinking, manuscript preparation, methodology).

In sum, we want authors to feel like they have gained something from the review process at Psychotherapy.

In order for authors to feel like they have gained useful knowledge from the review process it is vital that specific feedback be given with regard to how authors might strengthen their work. That is, recommendations must be concrete and specific enough that the author can make operational sense of what is being said, what the criticisms are, and what suggestions are being made.

Sweeping criticisms (e.g., "this paper has little to offer") without specific examples of what is impeding the paper's contributions are unhelpful, confusing and frustrating to both the Action Editor and author alike. It is helpful, however, to give specific illustrations of problems and suggestions.

Related to this issue, it is most helpful when your review lists points in a sequentially numbered format (i.e. 1), 2), 3), etc.) because the Action Editor can then easily reference specific comments when communicating with the author.

An effective review for empirical submissions should explicitly address

• how clearly the purposes are presented and then connected to what follows;
• the soundness of the methodology in light of the topic and purposes of the study as well as issues of reliability, internal and external validity; and
• the cogency of the conclusions (e.g., the connection of the conclusions to the data and the purposes).

In addition, given that one of the primary aims of the journal is to provide research that has clinical utility for applied practice, reviewers should expect that authors of empirical papers report effect sizes (i.e. d, g or r) and use more straightforward clinical significance methodology for psychotherapy outcome findings (i.e. unchanged, reliable change, movement into functional distribution, clinically significant change, and deterioration).

An effective review for conceptual submissions should explicitly address

• the theoretical clarity, coherence, and cogency of these arguments;
• what are the similarities and differences of these ideas in relation to alternative theoretical positions; and
• the soundness and viability of these ideas in relation to alternative theoretical conceptualizations.

Further, I would ask reviewers of conceptual pieces to try and put aside parochial views regarding different theoretical orientations and instead embrace the opportunity to engage with authors of various perspectives. That is, what can you offer the authors from your own theoretical perspective and approach that might expand and enliven the thinking on this issue?

In addition, as most approaches to therapy share similar constructs of interest but not the terms or labels for them, such reviews offer the opportunity to dialogue across different theoretical "languages." When this occurs it may help all of us to communicate more effectively with one another.

A thorough review will also attend to such factors as the importance of the piece and the quality of the writing. Lucid writing is, of course, a primary concern and needs to be addressed in the review.

The length of a review is only one indicator of its thoroughness. At the same time, it may be helpful for reviewers to know the Editor's view of the minimum length of a sound (qua thorough) review. Experience suggests that it is most difficult to perform a thorough review in less than about a single-spaced page. On the other hand, reviews that are much more than two pages may reflect over-editing.

Please do not indicate whether you believe the paper should be accepted or rejected in the text of your review. This information will be indicated on the Manuscript Evaluation sheet (PDF, 97KB)opens in new window.

Also please remember that this manuscript is a confidential document and you should not distribute, discuss, cite or refer to the work before it is published.

Finally, if reviewing a manuscript poses a conflict of interest that may make it difficult to offer a balanced and fair judgment, please notify the Action Editor and return it.