Evidence Synthesis: Part 1
This blog post is the first in a series exploring Evidence Synthesis. We’re going to start by looking at two types of evidence synthesis: literature reviews and systemic reviews. To help me with this topic I looked at a number of research guides from other institutions, e.g., Cornell University Libraries.
The Key Differences Between a Literature Review and a Systematic Review
Literature Review |
Systemic Review |
|
Purpose |
To provide a comprehensive overview and synthesis of existing research literature on a specific topic. It may identify trends, gaps, contradictions, and areas for future research. | Seeks to answer a specific research question by rigorously identifying, appraising, and synthesizing all relevant studies on the topic using predetermined criteria and methods. It aims to minimize bias and provide a reliable summary of the current evidence. |
Scope |
The scope can be broader and more flexible, allowing for the inclusion of various types of studies, such as empirical research, theoretical papers, and conceptual frameworks. | A systematic review has a narrower scope, focusing on empirical research studies that meet predefined criteria for inclusion. It typically excludes non-peer-reviewed literature, opinion pieces, and anecdotal evidence to ensure the reliability of findings. |
Search Strategy |
May be less structured, with searches conducted across multiple databases, journals, and other sources using keywords and subject headings relevant to the topic. | Highly structured and systematic, following predefined protocols to identify all relevant studies. It includes comprehensive searches of multiple databases, gray literature sources, and reference lists of relevant articles to minimize publication bias. |
Inclusion Criteria |
Inclusion may be less stringent, allowing for studies based on relevance to the topic and the author’s judgment. | Inclusion criteria are clearly defined and applied consistently to all studies. Typically includes factors such as study design, population characteristics, intervention/exposure, outcome measures, and language of publication. |
Data Extraction & Synthesis |
May involve summarizing key findings, themes, and arguments from individual studies in a narrative format. | Data extraction and synthesis is conducted systematically and transparently using predefined methods. Data from included studies are quantitatively synthesized (if possible) using statistical techniques such as meta-analysis to calculate summary effect sizes. |
Quality Assessment |
While quality assessment of individual studies may be mentioned in a literature review, it is typically less rigorous and systematic compared to a systematic review. | Quality assessment of included studies is a critical component, involving the evaluation of study design, risk of bias, methodological rigor, and internal validity. Only studies judged to be of high quality are included in the final analysis. |
Overall, while both literature reviews and systematic reviews involve reviewing existing research literature, systematic reviews adhere to more rigorous and transparent methods to minimize bias and provide robust evidence to inform decision-making in education and other fields. If you are interested in learning about other evidence synthesis this decision tree created by Cornell Libraries (Robinson, n.d.) is a nice visual introduction.
ChatGPT
Along with exploring evidence synthesis I am also interested in generative A.I. I want to be transparent about how I used A.I. to create the table above. I fed this prompt into ChatGPT:
“List the differences between a literature review and a systemic review for a graduate student of education“
I wanted to see what it would produce. I reformatted the list into a table so that it would be easier to compare and contrast these two reviews much like the one created by Cornell University Libraries (Kibbee, 2024). I think ChatGPT did a pretty good job. I did have to do quite a bit of editing, and make sure that what was created matched what I already knew. There are things ChatGPT left out, for example time frames, and how many people are needed for a systemic review, but we can revisit that in a later post.
References
Kibbee, M. (2024, April 10). Libguides: A guide to evidence synthesis: Cornell University Library Evidence Synthesis Service. Cornell University Library. https://guides.library.cornell.edu/evidence-synthesis/intro