Study Selection
Publisert
This is important to identify the studies that are relevant to the topic of interest and identified question.
About this phase
|
What happens before this step |
Completed search and a list over identified references that meet the inclusion criteria (included studies) and if sampling included in the synthesis (studies included in the synthesis) |
|
Description |
Evaluate the results of the literature search against the inclusion and exclusion criteria. This may be a cyclical process going back and forth between title and abstract, full text and inclusion. |
|
Why is this step important? |
To identify the studies that are relevant to the topic of interest and identified question. In your protocol a decision was made about taking a comprehensive or theoretical approach to sampling. Talk through this again so that the whole team is familiar with the screening and study selection approach. |
|
Responsible |
Project leader and at least one other team member |
|
Tasks |
Pilot your inclusion criteria together during a data driven screening exercise (screen a random allocation of 300 studies for both title and abstract and full text) to determine the baseline inclusion rate (guidance coming shortly). Continue screening using a ranking algorithm and other machine learning (ML) functions if applicable. For help in considering which ML functions can be applied to the review and which screening and/or analysis software the team is going to use. See the Machine Learning Team Sharepoint Room or contact the ML team for more information. (Access limited to NIPH authors) |
|
Tools |
Criteria for inclusion and exclusion and a screening program: Eppi Reviewer or RAYYAN |
| Deliverable |
List over included studies |
Study selection and sampling is one of the steps that is most different from systematic reviews of interventions. The aim of a QES is not to identify all relevant studies published on a topic (comprehensive study selection) but to identify a broad range of relevant studies that provide different perspectives, contexts, populations etc. of your topic of interest.
The study selection process in a QES is rarely linear. Most often screening (title and abstract, full text), assessing methodological limitations and descriptive data extraction happen simultaneously in a cyclical way. It is also possible to sample from the studies that meet the inclusion criteria if the search identifies a large number of relevant studies. For guidance on sampling please see Ames 2019, The EPOC Guidance on sampling studies in a QES and the webinar “Selecting studies and assessing methodological limitations.”
Another issue to consider when thinking through study selection is language. Qualitative studies are difficult and time consuming to translate. Think through if you want to pragmatically limit the review to languages spoken by review team members or include studies in any language. Translation programs like Deep L translate use natural language processing to translate whole PDFs. They attempt to translate meaning, but this is not always successful. They are a good option for seeing if a study meets your inclusion criteria but may not be good enough to provide translation of extracted data for analysis.
Relevant resources:
Cochrane webinar “Selecting studies and assessing methodological limitations.”
Ames HMR, Glenton C, Lewin S. “Purposive sampling in a qualitative evidence synthesis: A worked example from a synthesis on parental perceptions of vaccination communication.” BMC Medical Research Methodology (2019) 19:26
Suri H. Purposeful sampling in qualitative research synthesis. Qual Res J. 2011;11(2):63–75.
- EPOC QES Template
- WEIRD (Ways of Evaluating Important and Relevant Data) tool
- EPOC Qualitative Evidence Syntheses guidance on when to sample and how to develop a purposive sampling frame
Noyes J, Booth A, Cargo M, Flemming K, Harden A, Harris J, Garside R, Hannes K, Pantoja T, Thomas J. Qualitative evidence. Cochrane Handbook for Systematic Reviews of Interventions. 2019 Sep 23:525-45.