Systematic literature reviews

Are you getting distracted from your main aim during a literature review? Don’t worry, this is a common challenge, particularly for more extensive projects like a thesis.

What strategies can enhance the effectiveness of your searches while avoiding irrelevant information?

It is worthwhile to search a range of research databases, not just in what are depicted as top tier journals. Sometimes innovative research may not receive public recognition early.

Experiment with keywords a little bit first and reflect on what might give you the most relevant articles for your themes and concepts. Try incorporating Boolean operators (AND, OR, NOT).

What criteria can you use to identify valuable research?

We have all spent time reviewing articles and book chapters that often delve into similar subject matter. Is there a more efficient approach?

Here is an example. Brian Belcher, Katherine Rasmussen, Matthew Kemshaw, and Deborah Zornes (2016) focused on a systematic literature review about the evaluation of transdisciplinary research quality. Their research question involved a number of embedded terms: What are appropriate principles, criteria, and indicators for defining and assessing research quality in transdisciplinary research? They first checked the title and abstract only. They also defined the search parameters by expanding on “quality” in their topic coverage:

Must refer to research quality* definitions and criteria

(*societal relevance, effectiveness, impact, or related aspects of relevance will be considered aspects of quality)

They also specified that the article must involve at least one of these:

  • Present a framework for research assessment
  • Offer an outline of knowledge and optimal methodologies in appraising research, along with suggestions for quality standards or evaluation processes
  • Assess existing quality criteria and propose alternative or supplementary benchmarks
  • Carry out empirical examinations of research undertakings that furnish recommendations for evaluating quality

This search involved a broad multidisciplinary approach. The researchers specified that the discussion must be relevant to the environment, natural resources management, sustainability, livelihoods, or related areas of human–environmental interactions. They also stated that an explicit reference to these subject areas was not compulsory. (You might need a narrower set of parameters as you may not be exploring a multidisciplinary topic. A specified period for publications is a standard search parameter.)

For the articles that met these criteria, they then evaluated both the abstract and the complete article. There are many guides available on how to evaluate the trustworthiness of research, with details about what should be included in quantitative or qualitative research. When some key details are absent, the clarity and even the transparency of the research process is called into question. This applies to qualitative research such as that based on the phenomenological approach just as much as lab-based research supported by modelling and statistical inferences. I have edited most kinds in various fields.

How can you comprehensively document information?

You could organise a summary with author, title and year, research methodologies, data sources and outcomes using a table or spreadsheet. This can help track research streams, the evolution of our understanding of a topic or the development of new processes and inventions. It is probably best to devise your own simple template. As well as taking more comprehensive notes on each relevant article, it saves you a lot of time later when you systematically document your findings and references in an organised way like a table or spreadsheet at the same time. This makes comparisons easier to see and arrange. It also leads you to think about further options for inclusion in the literature review. Some references in the articles can be a good start, but don’t feel you have to read everything.

When do you start writing?

For a thesis, once you have formed your research questions, it can be helpful to start writing the actual literature review, introduction and methodology chapters early. Why? This gives you a focus for your further search of the literature. It also breaks up the intense reading by developing this into writing, which can help to keep you on track.

Save, save, save

Finally, a word about backing up. As well as your normal storage place for files, it is a good idea to consistently create backups in two other secure locations. This could be any combination of an external hard drive, a Dropbox, another cloud-based application and your dedicated computer, if you have one. Also, keep a log of what is stored and where so that you can check that everything is saved in three places. When you rewrite and add to a draft, give it a new version number. You might be surprised how often people do not do this. If they decide they want to move something from a draft of one chapter into another chapter, it might have disappeared unless they have kept and numbered all versions of their drafts.

Through these simple steps, maintaining your focus and keeping track of your literature review and writing can be much easier. Above all, when you need a break, take one!