Evidence Synthesis & Systematic Reviews

We've broken down the steps of evidence synthesis into four very general areas: 

• Background and Prep Work
• Gather Information
• Synthesize Information
• Write Up Report

A systematic review, one of the common forms of evidence synthesis, will take approximately 12 to 18 months to complete. 

Remember: these steps are at a conceptual level, and are highly generalized. Be sure to seek appropriate guidance and mentorship when conducting your evidence synthesis. 

Links to more complete steps for conducting evidence synthesis can be found under the What Software and Guidance Exists to Help Me Conduct an Evidence Synthesis tab. 

Your first step is to prepare to conduct an evidence synthesis. 

You will need to approach this from a couple of different angles -- the human component and the protocol component. 

The human component of evidence synthesis: 

Evidence synthesis can only be accomplished with a team. You'll need at least three people on your team. 

Your team might be bigger than three, depending on the goals of your work. For example, a systematic review and meta-analysis might benefit from a statistician as a member of your team. 

The protocol component of evidence synthesis: 

Ultimately, you should develop a protocol for your evidence synthesis.

Protocols do many important things: 

• Help you create a solid question with an achievable scope
• Ensure everyone on your team is on the same page
• Smooth out points of disagreement before getting deep into the project
• Give you guidance and milestones
• Improve reproducibility and reduce bias
• Improve the likelihood your work will be published

Many journals, including PLoS journals, BMJ and BMJ Open, Systematic Reviews, BioMed Central, and others, strongly encourage pre-registration of the review process. 

PRISMA-P offers guidance on how to write a robust protocol. Be sure to read the Explanation and Elaboration paper along with reviewing the checklist. 

You should also search to see if similar projects area already underway. OSF Registries and PROSPERO are two common places to look for registered protocols. This helps ensure your evidence synthesis is novel, and reduces duplication of work.

The data you  analyze for your evidence synthesis is made up of the articles you find. That's why the search component is so important -- and so time consuming. It can take a few weeks to a few months to create a systematic search.

Searches should be exhaustive.  A systematic search for an evidence synthesis project must capture all articles relevant to your question. Poorly constructed searches yield poor results, missing relevant articles. A good search will use a combination of thesaurus terms, free text terms, synonyms, variations, Boolean operators, follow appropriate search syntax, and more.  

Librarians are expert searchers and have a deep expertise across databases. They document searches and follow best practices for constructing a search. Documentation improves the reproducibility and reliability of your evidence synthesis, and is a critical component of your write-up.

After you have collected your data (composed of all the articles you locate from your searches), you need to synthesize your findings. 

Generally, your next steps entail: 

• Selecting articles for inclusion in your analysis. You and your team members will screen all the articles based on the rules you set up in your protocol. This step begins with a title/abstract screening to remove clearly unrelated studies, then moves to screening full-text studies.
• Extracting the data from the articles. Data extraction is done by hand and by filling in a pre-prepared form with important points.
• Evaluating for Risk of Bias and quality. You will assess the studies you selected to include in your study for any biases in the studies and their quality. Depending on your discipline and your study, different types of evaluation frameworks and checklists already exist.  

At a minimum, at least two people will screen each article. When there is a disagreement on whether or not to include an article, the third person will be the tie-breaker. Extracting and evaluating should be done by members of the team who are trained on the criteria.  

Evidence Synthesis also follows a standard reporting mechanism.

The Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) is one such mechanism, and is endorsed by numerous journals. Some journals require you to follow the PRISMA standard in order for your manuscript to be accepted. For example, PLoS Medicine and Systematic Reviews require submissions follow the PRISMA Statement and include a completed checklist and flow diagram for publication. 

Following PRISMA will help you clearly present your findings and methodology. 

Ready to start tackling an evidence synthesis project? 

Check out how a librarian can partner, or fill out our application form!