How to Be A Good Reviewer

Amanda Cote (Michigan State University) provided the following excellent advice about how to proceed with reviews. ICA Game Studies wholeheartedly endorses her recommendations!

ICA Game Studies Division: Reviewing Tips

Update your ICA profile and paper submission system keywords! This year, reviews will be assigned automatically based on profile/paper keyword matches.

  • Sign in to your ICA profile → Manage profile → Edit bio → Choose relevant keywords
  • Also choose relevant keywords in the paper submission system when you submit your papers.
  • Try to balance a wide selection of keywords (to aid in matching) with your areas of expertise (to allow you to review effectively).
    • Move beyond extremely broad terms like “games”  (we all share that interest!)
    • Choose your broad methodological background (qualitative research or quantitative research) and at least one specific methodology (e.g. in-depth interviews, survey research, experimental research)
  • Don’t panic if you get a paper that’s not a great match; follow the suggestions below to be an effective reviewer regardless!

How to be a Good Reviewer

  1. Remember that reviews are meant to HELP. It’s easy to feel like being a reviewer means you must be super critical all the time. That’s not necessarily the best way to support and encourage good work! Communicate both what you like about a paper and where you think the authors still need work. Give specific, actionable suggestions for improvement if you feel the work needs help. Try to write the kind of review you would like to receive.
  2. Keep the review criteria in mind. Is this submission relevant to game studies (as a whole, not just your specific area)? Is the argument interesting? Does the research include a useful theoretical or empirical contribution to existing conversations? In short, would you or other game studies scholars like to hear more about this research? These are the biggest criteria to evaluate first. Once you answer these big questions, then you can provide feedback on smaller concerns like writing quality.
  3. Evaluate the submission on its own terms. Not all of us are focused on triple-A gaming titles, nor are we all native English speakers, and that’s great! Our broadness is part of what makes our division awesome. However, to keep us international and vibrant, we need to set aside our preconceptions of what a game study should look like, and keep our minds open to new ideas. Try to find the meat (or vegetarian alternative) of the piece. Does the study use a game? Can you understand the main message of the paper, even if the English isn’t perfect? Then use that comments to the editor section to voice your concerns, and focus on the core of the paper in the author comments section. And remember—you’re giving feedback on the study that exists, not what the authors could have done in a hypothetical ideal study.
  4. Don’t be afraid to admit your own weaknesses. The diversity of approaches in our field mean you will sometimes receive work that’s not exactly in your wheelhouse. If there’s a part you don’t understand, that’s ok! You can say that and defer to the other reviewer on that point – that’s why there’s more than one person reviewing each piece.
  5. Focus on what you DO know. Not everyone is going to be an expert at everything, and we’re a very small division with a very diverse set of topics, not to mention oodles of methods. However, we all have PhDs or are on our way – we know how to structure arguments and literature reviews, and we know if something will make sense to a general audience or not. If that’s where you need to focus your feedback, go for it!
  6. Organize your review. For writing reviews, everyone has a different style. But you should try to make sure that readers can follow your argument clearly. It can be helpful to start your review with a (short) summary of what the paper aims at, what the authors did, and the key findings. In a next step, you can dive deeper into these questions and explain where the authors have done a good job and where you see weaknesses in their arguments / methods / conclusions / writing. Feel free to use bullet points and / or flowing text to give your feedback.
  7. Think big picture. It’s tempting to point out every single thing that you like or dislike in a paper. But at a certain point, too much feedback, especially on small issues, becomes overwhelming (and can feel cruel; see #1). Try to identify the biggest strengths and weaknesses in the paper. What do you really want the authors to fix to make this the best possible paper? What do you think they must keep? Give detailed feedback on the 2-3 biggest pros and cons. Then, if you really need to, briefly mention other areas that could be improved. But make sure it’s clear where the authors should focus most of their revision energy.
  8. Remember you’re not alone! There are many great resources out there to help you write a useful, helpful review (whether it’s positive, critical, or both). We’ve included some favorites at the end of this document, but new resources emerge every day. Also feel free to ask colleagues or mentors how they approach the reviewing process. Because we all take a slightly different approach, exploring other peoples’ strategies can help you develop your own.
  9. Always remember that reviewing is a learning opportunity too. Want to find ways to improve your own papers? Want to know what other people’s papers look like pre-publication so that you don’t feel like an imposter when you’re working on your own? Want to learn about common errors to look out for in your writing? Reviewing is an amazing chance to learn about all of those things! The more you review, the better you get at seeing your own stuff like a reviewer will before you submit it, and who doesn’t like shorter revise & resubmits?
  10. You can do it! Any perspective on a manuscript can be meaningful feedback and we highly appreciate your thorough comments on a piece. We’re all here because we care about understanding games more deeply; your feedback is a big part of that!

Resources on How to be a Helpful Reviewer