Systematic Reviews: Questions, Methods and Usage

Hansen, Henrik and Trifkovic, Neda (2013): Systematic Reviews: Questions, Methods and Usage.

Abstract

While their application in the context of development aid is quite new, systematic reviews have been used and debated in other fields for various decades. Although evidence gathering and synthesis in the context of development aid inarguably faces challenges distinct from other fields, it is expected that experiences about systematic review methodology from other contexts can provide useful inputs to the field of development aid evaluation.

Systematic reviews operate in a complex multidisciplinary environment, which requires acknowledging the influence of institutions and social interaction. The Evaluation Study suggests that the scarcity of comparable evidence about the effects of development interventions necessitates that authors of systematic reviews change their strategies when assessing the strength of evidence or synthesizing data. The focus on asking the ‘right’ questions in international development reviews is important precisely because no review process is immune to bias. The study emphasizes that since systematic reviews in international development may be vulnerable to a range of biases. Systematic reviews should not aim, at all cost, at pursuing the classical approach suitable for traditional, ‘easy-to-measure’ situations.

• Item Type: MPRA Paper
• Language: English
• Keywords: Systetmatic Reviews; International Development
• Subjects: C - Mathematical and Quantitative Methods > C0 - General; O - Economic Development, Innovation, Technological Change, and Growth > O2 - Development Planning and Policy > O20 - General

References

3ie. (2012a). Quality Assurance Services. 3ie: International Initiative for Impact Evaluation.

3ie. (2012b). 3ie Impact Evaluation Glossary. New Delhi: International Initiative for Impact Evaluation.

3ie. (2013). Systematic Reviews. 3ie: International Initiative for Impact Evaluation.

Ashrafian, H., Darzi, A. & Athanasiou, T. (2011). Evidence Synthesis: Evolving Methodologies to Optimise Patient Care and Enhance Policy Decisions, in: Athanasiou, T. and Darzi, A. (Eds.), Evidence Synthesis In Healthcare: A Practical Handbook for Clinicians, (pp. 1–46). London: Springer.

Atkins, D., Eccles, M., Flottorp, S., Guyatt, G. H., Henry, D., Hill, S., et al. (2004). Systems for Grading the Quality of Evidence and the Strength of Recommendations I: Critical Appraisal of Existing Approaches The GRADE Working Group. BMC Health Services Research, 4(1), 38.

Baird, S. Garfein, R., McIntosh, C., Özler, B., (2012). Effect of a cash transfer programme for schooling on prevalence of HIV and herpes simplex type 2 in Malawi: a cluster randomised trial. The Lancet, 379(9823), pp.1320–1329.

Barnett-Page, E. & Thomas, J. (2009). Methods for the Synthesis of Qualitative Research: a Critical Review. BMC Medical Research Methodology, 9(1), 59.

Bastian, H., Glasziou, P. & Chalmers, I. (2010). Seventy-Five Trials and Eleven Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med, 7(9).

Bates, S., Clapton, J. & Coren, E. (2007). Systematic Maps to Support the Evidence Base in Social Care. Evidence & Policy: A Journal of Research, Debate & Practice, 3(4), 539–551.

Berelson, B. (1952). Content Analysis in Communication Research. Glencoe, IL: Free Press.

Birge, R. T. (1932). The Calculation of Errors by the Method of Least Squares. Physical Review, 40(2), 207–227.

BMJ. (2013). Practice. The British Medical Journal.

Boeije, H. R., Van Wesel, F. & Alisic, E. (2011). Making a Difference: Towards a Method for Weighing the Evidence in a Qualitative Synthesis. Journal of Evaluation in Clinical Practice, 17(4), 657–663.

Booth, A. (2001). Cochrane or Cock-eyed? How Should We Conduct Systematic Reviews of Qualitative Research? In Qualitative Evidence-based Practice Conference, Taking a Critical Stance. May 14-16 2001. Coventry: Coventry University.

Braun, V. & Clarke, V. (2006). Using Thematic Analysis in Psychology. Qualitative Research in Psychology, 3(2), 77–101.

Briner, R. B. & Denyer, D. (2012). Systematic Review and Evidence Synthesis as a Practice and Scholarship Tool, in: Rousseau, D. M. (Ed.), The Oxford Handbook of Evidence-Based Management. Oxford: Oxford University Press.

Bronson, D. E. & Davis, T. S. (2011). Finding and Evaluating Evidence:Systematic Reviews and Evidence-Based Practice: Systematic Reviews and Evidence-Based Practice. Oxford: Oxford University Press.

Broome, M. E. (1993). Integrative Literature Reviews for the Development of Concepts, in: Rodgers, B. L. and Knafl, K. A. (Eds.), Concept development in nursing: foundations, techniques, and applications. Philadelphia, PA: Saunders Co.

Brunt, L. (2001). The Advent of the Sample Survey in the Social Sciences. Journal of the Royal Statistical Society. Series D (The Statistician), 50(2), 179–189.

Campbell Collaboration. (2008). User Involvement in the Systematic Review Process. Oslo: The Campbell Collaboration.

Campbell Collaboration. (2012). What Is a Systematic Review? The Campbell Colalboration.

Campbell Collaboration. (2013). Background: The Campbell Collaboration. The Campbell Collaboration.

Campbell, R., Pound, P., Pope, C., Britten, N., Pill, R., Morgan, M., et al. (2003). Evaluating Meta-ethnography: a Synthesis of Qualitative Research on Lay Experiences of Diabetes and Diabetes Care. Social Science & Medicine, 56(4), 671–684.

CEBM. (2013). Levels of Evidence. Oxford Centre for Evidence-based Medicine.

Chalmers, I, Hetherington, J., Newdick, M., Mutch, L., Grant, A., Enkin, M., et al. (1986). The Oxford Database of Perinatal Trials: Developing a Register of Published Reports of Controlled Trials. Controlled clinical trials, 7(4), 306–324.

Chalmers, I, Hedges, L. V. & Cooper, H. (2002). A Brief History of Research Synthesis. Evaluation & the health professions, 25(1), 12–37.

Chambers, R. (1998). Clinical Effectiveness Made Easy: First Thoughts on Clinical Governance. Oxford: Radcliffe Med. Press.

Clapton, J., Rutter, D. & Sharif, N. (2009). SCIE Systematic Mapping Guidance. Social Care Institute for Excellence.

Cochrane, A. L. (1972). Effectiveness and Efficiency: Random Reflections on Health Services. London: Nuffield Provincial Hospitals Trust.

Cochrane, A. L. (1979). 1931-1971: A Critical Review, with Particular Reference to the Medical Profession, in: Teeling-Smith, G. and Wells, N. (Eds.), Medicines for the year 2000, (pp. 1–11). London: Office of Health Economics.

Cochrane Collaboration. (2012a). Cochrane Reviews. The Cochrane Collaboration.

Cochrane Collaboration. (2012b). History. The Cochrane Collaboration.

Cochrane Collaboration. (2013a). About The Cochrane Library. The Cochrane Library.

Cochrane Collaboration. (2013b). Glossary. The Cochrane Collaboration.

Cook, D J, Sackett, D. L. & Spitzer, W. O. (1995). Methodologic Guidelines for Systematic Reviews of Randomized Control Trials in Health Care from the Potsdam Consultation on Meta-Analysis. Journal of clinical epidemiology, 48(1), 167–171.

Cooper, H. M. & Hedges, L. V. (1994). The Handbook of Research Synthesis. New York: Russell Sage Foundation.

Counsell, C. (1997). Formulating Questions and Locating Primary Studies for Inclusion in Systematic Reviews. Annals of internal medicine, 127(5), 380–387.

CRD. (2009). Systematic Reviews. CRD’s Guidance for Undertaking Reviews in Health Care. York:Centre for Reviews and Dissemination, University of York.

Davies, P. (2000). The Relevance of Systematic Reviews to Educational Policy and Practice. Oxford Review of Education, 26(3-4), 365–378.

DFID. (2012). Systematic Reviews in International Development : An Initiative to Strengthen Evidence-Informed Policy Making. Department for International Development.

DFID. (2013). Systematic Reviews. Department for International Development.

Dixon, E., Hameed, M., Sutherland, F., Cook, Deborah J. & Doig, C. (2005). Evaluating Meta-analyses in the General Surgical Literature. Annals of Surgery, 241(3), 450–459.

Dixon-Woods, M., Cavers, D., Agarwal, S., Annandale, E., Arthur, A., Harvey, J., et al. (2006). Conducting a Critical Interpretive Synthesis of the Literature on Access to Healthcare by Vulnerable Groups. BMC Medical Research Methodology, 6(1), 35.

Downs, S. H. & Black, N. (1998). The Feasibility of Creating a Checklist for the Assessment of the Methodological Quality Both of Randomised and Nonrandomised Studies of Health Care Interventions. Journal of epidemiology and community health, 52(6), 377–384.

Duflo, E., Glennerster, R. & Kremer, M. (2008). Using Randomization in Development Economics Research: A Toolkit, in Schultz, T. P. and Strauss, J. (Eds.) Handbook of Development Economics, (pp. 3895-3962). Amsterdam: Elsevier.

Dunn, P. M. (1997). James Lind (1716-94) of Edinburgh and the Treatment of Scurvy. Archives of Disease in Childhood - Fetal and Neonatal Edition, 76(1), F64–F65.

Duvendack, M., Hombrados, J. G., Palmer-Jones, R. & Waddington, H. (2012). Assessing “what Works” in International Development: Meta-analysis for Sophisticated Dummies. Journal of Development Effectiveness, 4(3), 456-471.

Edwards, A. G., Russell, I. T. & Stott, N. C. (1998). Signal Versus Noise in the Evidence Base for Medicine: An Alternative to Hierarchies of Evidence? Family practice, 15(4), 319–322.

Egger, M. & Smith, G. D. (1998). Meta-analysis Bias in Location and Selection of Studies. British Medical Journal, 316(7124), 61–66.

EPPI-Centre. (2009a). What Is a Systematic Review? EPPI-Centre.

EPPI-Centre. (2009b). User Involvement. EPPI-Centre.

EPPI-Centre. (2009c). International Development Review Group. EPPI-Centre.

EPPI-Centre. (2009d). History of Systematic Reviews. EPPI-Centre.

Evans, D. & Pearson, A. (2001). Systematic Reviews: Gatekeepers of Nursing Knowledge. Journal of Clinical Nursing, 10(5), 593–599.

Finfgeld, D. L. (2003). Metasynthesis: The State of the Art—So Far. Qualitative Health Research, 13(7), 893–904.

Fisher, R. (1925). Statistical Methods for Research Workers. Edinburgh: Oliver & Boyd.

Freeman, L. (1977). A Set of Measures of Centrality Based on Betweenness. Sociometry, 40(1), 35–41.

Garg, A. X., Hackam, D. & Tonelli, M. (2008). Systematic Review and Meta-analysis: When One Study Is Just Not Enough. Clinical Journal of the American Society of Nephrology, 3(1), 253–260.

Glass, G. V. (1976). Primary, Secondary, and Meta-Analysis of Research. Educational Researcher, 5(10), 3–8.

Goldschmidt, P. G. (1986). Information Synthesis: a Practical Guide. Health Services Research, 21(2 Pt 1), 215–237.

Gough, D., Thomas, J. & Oliver, S. (2012). Clarifying Differences Between Review Designs and Methods. Systematic reviews, 1(1), 28.

Greenhalgh, Trisha, Robert, G., Macfarlane, F., Bate, P., Kyriakidou, O. & Peacock, R. (2005). Storylines of Research in Diffusion of Innovation: a Meta-narrative Approach to Systematic Review. Social science & medicine (1982), 61(2), 417–430.

Grégoire, G., Derderian, F. & Le Lorier, J. (1995). Selecting the Language of the Publications Included in a Meta-analysis: Is There a Tower of Babel Bias? Journal of clinical epidemiology, 48(1), 159–163.

Guyatt, G. (1991). Evidence-based Medicine. ACP J Club, A16–114.

Hampton, J. R. (1998). The End of Medical History? Journal of the Royal College of Physicians of London, 32(4), 366-375.

Harden, A. (2008). Methods for the Thematic Synthesis of Qualitative Research in Systematic Reviews. BMC Medical Research Methodology, 8(1), 45.

Heaton, J. (2004). Reworking Qualitative Data. SAGE.

Heyvaert, M., Maes, B. & Onghena, P. (2013). Mixed Methods Research Synthesis: Definition, Framework, and Potential. Quality & Quantity, 47(2), 659–676.

Higgins, J. P. & Green, S. (Eds.). (2011). Cochrane Handbook for Systematic Reviews of Interventions. The Cochrane Collaboration.

Holsti, O. R. (1969). Content Analysis for the Social Sciences and Humanities. Reading, MA: Addison-Wesley Pub. Co.

Jadad, A. R. (2000). Systematic Reviews and Meta-analyses on Treatment of Asthma: Critical Evaluation. British Medical Journal, 320(7234), 537–540.

Jadad, A. R., Cook, D J, Jones, A., Klassen, T P, Tugwell, P, Moher, M., et al. (1998). Methodology and Reports of Systematic Reviews and Meta-analyses: a Comparison of Cochrane Reviews with Articles Published in Paper-based Journals. JAMA: the Journal of the American Medical Association, 280(3), 278–280.

Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J., Gavaghan, D. J., et al. (1996). Assessing the Quality of Reports of Randomized Clinical Trials: Is Blinding Necessary? Controlled clinical trials, 17(1), 1–12.

Jones, K. (2004). Mission Drift in Qualitative Research, or Moving Toward a Systematic Review of Qualitative Studies, Moving Back to a More Systematic Narrative Review. Qualitative Report, 9(1), 95–112.

Jorgensen, A. W., Hilden, J. & Gotzsche, P. C. (2006). Cochrane Reviews Compared with Industry Supported Meta-analyses and Other Meta-analyses of the Same Drugs: Systematic Review. BMJ, 333(7572), 782–0.

Kavanagh, J., Campbell, F., Harden, A. & Thomas, J. (2011). Mixed Methods Synthesis: A Worked Example, in: Hannes, K. and Lockwood, C. (Eds.), Synthesizing Qualitative Research: Choosing the Right Approach, (pp. 113–136). Chichester, UK: John Wiley & Sons, Ltd. http://onlinelibrary.wiley.com/doi/10.1002/9781119959847.ch6/summary

Kelly, K. D., Travers, A., Dorgan, M., Slater, L. & Rowe, B. H. (2001). Evaluating the Quality of Systematic Reviews in the Emergency Medicine Literature. Annals of Emergency Medicine, 38(5), 518–526.

Kitchenham, B. & Charters, S. (2007). Guidelines for Performing Systematic Literature Reviews in Software Engineering. Keele University and Durham University Joint Report.

KMIN. (2012). Systematic Reviews of Measurement Instruments. Knowledgecenter measurement instruments.

Krippendorff, K. (2003). Content Analysis: An Introduction to Its Methodology. London: SAGE Publications Ltd.

Kung, J., Chiappelli, F., Cajulis, O. O., Avezova, R., Kossan, G., Chew, L., et al. (2010). From Systematic Reviews to Clinical Recommendations for Evidence-Based Health Care: Validation of Revised Assessment of Multiple Systematic Reviews (RAMSTAR) for Grading of Clinical Relevance. The Open Dentistry Journal, 4, 84–91.

Lau, J, Ioannidis, J. P. & Schmid, C. H. (1998). Summing up Evidence: One Answer Is Not Always Enough. Lancet, 351(9096), 123–127.

Lavis, J. N. (2009). How Can We Support the Use of Systematic Reviews in Policymaking? PLoS Med, 6(11), e1000141.

Lind, J. (1753). A Treatise of the Scurvy. In Three Parts. Containing an Inquiry into the Nature, Causes and Cure, of That Disease. Together with a Critical and Chronological View of What Has Been Published on the Subject. Edinburgh: Printed by Sands, Murray and Cochran for A Kincaid and A Donaldson.

Mallett, S. & Clarke, M. (2002). The Typical Cochrane Review. How Many Trials? How Many Participants? International Journal of Technology Assessment in Health Care, 18(04), 820–823.

Marmot, M., Adelstein, A. & Bulusu, L. (1984). Lessons from the Study of Immigrant Mortality. Lancet, 1(8392), 1455–1457.

Mathison, S. (2005). Encyclopedia of Evaluation. Thousand Oaks, CA: SAGE Publications, Inc.

Moher, D., Pham, B., Klassen, Terry P, Schulz, K. F., Berlin, J. A., Jadad, A. R., et al. (2000). What Contributions Do Languages Other Than English Make on the Results of Meta-analyses? Journal of Clinical Epidemiology, 53(9), 964–972.

Moher, D., Tetzlaff, J., Tricco, A. C., Sampson, M. & Altman, D. G. (2007). Epidemiology and Reporting Characteristics of Systematic Reviews. PLoS Medicine, 4(3), e78.

Mulrow, C. D. (1994). Rationale for Systematic Reviews. BMJ (Clinical research ed.), 309(6954), 597–599.

NHS. (2011). Glossary. National Institute for Health and Clinical Excellence. Retrieved February 26, 2013, from http://www.nice.org.uk/website/glossary/glossary.jsp?alpha=D

Noblit, G. W. & Hare, R. D. (1988). Meta-Ethnography: Synthesizing Qualitative Studies. Newbury Park, CA: SAGE Publications, Inc.

Oakley, A. (2002). Social Science and Evidence-based Everything: The Case of Education. Educational Review, 54(3), 277–286.

Oxman, A. D. & Guyatt, G. H. (1991). Validation of an Index of the Quality of Review Articles. Journal of clinical epidemiology, 44(11), 1271–1278.

Oxman, A. D. & Guyatt, G. H. (1993). The Science of Reviewing Researcha. Annals of the New York Academy of Sciences, 703(1), 125–134.

Paterson, B. L., Canam, C., Jillings, C. & Thorne, S. E. (2001). Meta-Study of Qualitative Health Research: A Practical Guide to Meta-Analysis and Meta-Synthesis. Thousand Oaks, CA: SAGE Publications Ltd.

Pawson, R., Greenhalgh, T., Harvey, G. & Walshe, K. (2004). Realist Synthesis - an Introduction. Manchester: ESRC Research Methods Programme.

Pawson, R., Greenhalgh, Trisha, Harvey, Gill & Walshe, Kieran. (2005). Realist Review-a New Method of Systematic Review Designed for Complex Policy Interventions. Journal of health services research & policy, 10 Suppl 1, 21–34.

Peters, C. C. (1933). Summary of the Penn State Experiments on the Influence of Instruction in Character Education. Journal of Educational Sociology, 7(4), 269–272.

Petticrew, M. & Roberts, H. (2008). Systematic Reviews in the Social Sciences: A Practical Guide. Oxford: Blackwell Publishing.

Pieper, D., Buechter, R., Jerinic, P. & Eikermann, M. (2012). Overviews of Reviews Often Have Limited Rigor: a Systematic Review. Journal of Clinical Epidemiology, 65(12), 1267–1273.

Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., et al. (2006). Guidance on the Conduct of Narrative Synthesis in Systematic Reviews. Lancaster: Lancaster University.

PRISMA Statement. (2009). The PRISMA Statemant. PRISMA - Transparent Reporting of Systematic Reviews and Meta-Analyses.

Rees, R. & Oliver, S. (2012). Stakeholder Perspectives and Participation in Reviews, in: Gough, D., Oliver, S., and Thomas, J. (Eds.), An Introduction to Systematic Reviews, (pp. 17–34). London: SAGE Publications Ltd.

Robinson, J. P. & Shaver, P. R. (1973). Measures of Social Psychological Attitudes. Revised. Ann Arbor, MI: University of Michigan, pp. 750.

Rosoff, J. I. (1975). Is Support of Abortion Political Suicide? Family Planning Perspectives, 7(1), 13.

Russell, R., Chung, M., Balk, E. M., Atkinson, S., Giovannucci, E. L., Ip, S., et al. (2009). Issues and Challenges in Conducting Systematic Reviews to Support Development of Nutrient Reference Values: Workshop Summary: Nutrition Research Series, Vol. 2. Rockville (MD): Agency for Healthcare Research and Quality (US).

Sackett, D., Rosenberg, W. M., Gray, J. A., Haynes, R. B. & Richardson, W. S. (1996). Evidence Based Medicine: What It Is and What It Isn’t. BMJ (Clinical research ed.), 312(7023), 71–72.

Sandelowski, M. & Barroso, J. (2002). Finding the Findings in Qualitative Studies. Journal of Nursing Scholarship, 34(3), 213–219.

Schlosser, D. (2007). Appraising the Quality of Systematic Reviews. Focus Technical Brief, 17, 1–8.

Schmidt, F. L., Hunter, J. E., Pearlman, K., Hirsh, H. R., Sackett, P. R., Schmitt, N., et al. (1985). Forty Questions About Validity Generalization and Meta-Analysis. Personnel Psychology, 38(4), 697–798.

Schnore, L. F. & Fagin, H. (1967). Urban Research and Policy Planning. Beverly Hills: Sage Publications.

Schreiber, R., Crooks, D. & Stern, P. N. (1997). Qualitative Meta-analysis, in: Morse, J. M. (Ed.), Completing a Qualitative Project: Details and Dialogue, (pp. 311–326). Thousand Oaks, CA: Sage Publications, Inc.

Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., et al. (2007). Development of AMSTAR: a Measurement Tool to Assess the Methodological Quality of Systematic Reviews. BMC Medical Research Methodology, 7(1), 10.

Shen, L. (1981). The Taiwan issue in Peking’s foreign relations in the 1970s: a systematic review. Chinese Yearbook of International Law and Affairs, 1, 74–96.

Skinner, C. J., Holt, D. & Smith, T. M. F. (1989). Analysis of Complex Surveys. Chichester: Wiley.

Smith, M. L. & Glass, G. V. (1977). Meta-analysis of Psychotherapy Outcome Studies. The American Psychologist, 32(9), 752–760.

Smith, M. L. & Glass, G. V. (1980). Meta-Analysis of Research on Class Size and Its Relationship to Attitudes and Instruction. American Educational Research Journal, 17(4), 419.

Smith, M. L., Glass, G. V. & Miller, T. I. (1980). The benefits of psychotherapy. Baltimore: Johns Hopkins University Press.

Snilstveit, B. (2012). Systematic Reviews: From “bare Bones” Reviews to Policy Relevance. Journal of Development Effectiveness, 4(3), 388–408.

Snilstveit, B., Oliver, S. & Vojtkova, M. (2012). Narrative Approaches to Systematic Review and Synthesis of Evidence for International Development Policy and Practice. Journal of Development Effectiveness, 4(3), 409–429.

Stewart, R., Van Rooyen, C. & De Wet, T. (2012). Purity or Pragmatism? Reflecting on the Use of Systematic Review Methodology in Development. Journal of Development Effectiveness, 4(3), 430–444.

Stigler, S. M. (1986). The History of Statistics: The Measurement of Uncertainty Before 1900. Cambridge, MA: Harvard University Press.

Stigler, S. M. (1989). Francis Galton’s Account of the Invention of Correlation. Statistical Science, 4(2), 73–79.

Stone, P. J., Dunphy, D. C., Smith, M. S. & Ogilvie, D. M. (1968). The General Inquirer: a Computer Approach to Content Analysis. User’s Manual. Cambridge, MA: MIT Press.

Tashakkori, A. & Teddlie, C. (1998). Mixed Methodology: Combining Qualitative and Quantitative Approaches. Thousand Oaks, CA: Sage Publications, Inc.

Thomas, J. & Harden, A. (2008). Methods for the Thematic Synthesis of Qualitative Research in Systematic Reviews. BMC Medical Research Methodology, 8(1), 45.

Thompson, S. G. & Higgins, J. P. (2002). How Should Meta-regression Analyses Be Undertaken and Interpreted? Statistics in medicine, 21(11), 1559–1573.

Vuturo, G. J., McCormick, W. C. & Krischer, J. P. (1980). Potential and Actual Drug Product Selection Rates in Florida. Contemporary pharmacy practice, 3(3), 163–170.

Waddington, H., White, H., Snilstveit, B., Hombrados, J. G., Vojtkova, M., Davies, P., et al. (2012). How to Do a Good Systematic Review of Effects in International Development: a Tool Kit. Journal of Development Effectiveness, 4(3), 359–387.

Weber, R. P. (1990). Basic Content Analysis. London: SAGE Publications Ltd.

Weed, D. L. (2013). The Quality of Nutrition and Cancer Reviews: a Systematic Assessment. Critical reviews in food science and nutrition, 53(3), 276–286.

Weinstein, M. C., Read, J. L., MacKay, D. N., Kresel, J. J., Ashley, H., Halvorsen, K. T., et al. (1986). Cost-effective Choice of Antimicrobial Therapy for Serious Infections. Journal of general internal medicine, 1(6), 351–363.

White, H. & Waddington, H. (2012). Why Do We Care About Evidence Synthesis? An Introduction to the Special Issue on Systematic Reviews. Journal of Development Effectiveness, 4(3), 351–358.

WHO. (2013). SURE Guides for Preparing and Using Evidence-Based Policy Briefs. World Health Organization.

Winkelstein, W., Jr. (1998). The First Use of Meta-analysis? American journal of epidemiology, 147(8), 717.

Yates, F. & Cochran, W. G. (1938). The Analysis of Groups of Experiments. The Journal of Agricultural Science, 28(04), 556–580.