Two Criteria for Good Measurements in Research: Validity and Reliability

Mohajan, Haradhan (2017): Two Criteria for Good Measurements in Research: Validity and Reliability. Published in: Annals of Spiru Haret University , Vol. 17, No. 4 (24 December 2017): pp. 56-82.

Abstract

Reliability and validity are the two most important and fundamental features in the evaluation of any measurement instrument or tool for a good research. The purpose of this research is to discuss the validity and reliability of measurement instruments that are used in research. Validity concerns what an instrument measures, and how well it does so. Reliability concerns the faith that one can have in the data obtained from the use of an instrument, that is, the degree to which any measuring tool controls for random error. An attempt has been taken here to review the reliability and validity, and threat to them in some details.

• Item Type: MPRA Paper
• Language: English
• Keywords: Validity and reliability, errors in research, threats in research.
• Subjects: B - History of Economic Thought, Methodology, and Heterodox Approaches > B4 - Economic Methodology; C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General

References

Abowitz, D. A., & Toole, T. M. (2010). Mixed Method Research: Fundamental Issues of Design, Validity, and Reliability in Construction Research. Journal of Construction Engineering and Management, 136(1), 108-116.

Allchin, D. (2001). Error Types, Perspectives on Science, 9(1), 38-58.

Allen, M. J., & Yen, W. M. (1979). Introduction to Measurement Theory. Monterey, CA: Brooks/Cole Publishing Company.

Altheide, D. L., & Johnson, J. M. (1994). Criteria for Assessing Interpretive Validity in Qualitative Research. In N. K. Denzin & Y. S. Lincoln (Eds.). Handbook of Qualitative Research, pp. 485-499. Thousand Oaks, CA: SAGE.

Babbie, E. R. (2010). The Practice of Social Research. Belmont, CA: Wadsworth.

Bajpai, S. R., & Bajpai, R. C. (2014). Goodness of Measurement: Reliability and Validity. International Journal of Medical Science and Public Health, 3(2), 112-115.

Bland, M, J., & Altman, D. (1986). Statistical Methods for Assessing Agreement between Two Methods of Clinical Measurement. The Lancet, 327(8476), 307-310.

Blumberg, B., Cooper, D. R., & Schindler, P. S. (2005). Business Research Methods. Berkshire: McGrawHill Education.

Burns, G. N., Morris, M. B., Periard, D. A., LaHuis, D., Flannery, N. M., Carretta, T. R., & Roebke, M. (2017). Criterion-Related Validity of a Big Five General Factor of Personality from the TIPI to the IPIP. International Journal of Selection and Assessment, 25, 213–222.

Campbell, D. T. (1959). Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix. Psychological Bulletin, 56(2), 81–105.

Campos, C.M.C., da Silva Oliveira, D., Feitoza, A. H. P., & Cattuzzo, M. T. (2017). Reliability and Content Validity of the Organized Physical Activity Questionnaire for Adolescents, Educational Research, 8(2), 21-26.

Chakrabartty, S. N. (2013). Best Split-Half and Maximum Reliability. IOSR Journal of Research & Method in Education, 3(1), 1-8.

Cook, D. A., & Beckman, T. J. (2006). Current Concepts in Validity and Reliability for Psychometric Instruments: Theory and Application. The American Journal of Medicine, 119, 166.e7-166.e16.

Corbett, N., Sibbald, R., Stockton, P., & Wilson, A. (2015). Gross Error Detection: Maximising the Use of Data with UBA on Global Producer III (Part 2). 33rd International North Sea Flow Measurement Workshop 20th – 23rd October 2015.

Cortina, J. M. (1993). What is Coefficient Alpha? An Examination of Theory and Applications. Journal of Applied Psychology, 78(1), 98–104.

Creswell, J. W. (2005). Educational Research: Planning, Conducting and Evaluating Quantitative and Qualitative Research (2nd Ed.). Pearson Merrill Prentice Hall.

Creswell, R. (2014). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. USA: SAGE Publications.

Crocker, L., & Algina, J. (1986). Introduction to Classical and Modern Test Theory. Philadelphia: Harcourt Brace Jovanovich College Publishers.

Cronbach, L. J. (1951). Coefficient Alpha and the Internal Structure of Tests. Psychometrika, 16, 297–334.

Cronbach, L. J., & Meehl, P. E. (1955). Construct Validity in Psychological Tests, Psychological Bulletin, 52, 281-302.

de Almeida, S. C. C. (2016). Validity and Reliability of the 2nd European Portuguese Version of the “Consensus Auditory-Perceptual Evaluation of Voice” (II EP CAPE-V). Master Thesis. Health Science School of Polytechnic Institute of Setúbal, Portugal.

Denga, D. I. (1987). Educational Measurement, Continuous Assessment and Psychological Testing. Calabar Rapid Educational Publishers Ltd.

Devillis, R. E. (2006). Scale Development: Theory and Application. Applied Social Science Research Method Series. Vol. 26 Newbury Park: SAGE Publishers Inc.

Downing, S. M. (2004). Reliability: On the Reproducibility of Assessment Data. Med Education, 38, 1006-1012.

Feldt, L. S., & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.). Educational Measurement (3rd Ed.). New York: American Council on Education and Macmillan.

Fink, A. (Ed.) (1995). How to Measure Survey Reliability and Validity. Thousand Oaks, CA: SAGE.

Fink, A., & Kosecoff, J. (1985). How to Conduct Surveys? Newbury Park, CA: SAGE Publications.

Flake, J. K., Pek, J., & Hehman, E. (2017). Construct Validation in Social and Personality Research: Current Practice and Recommendations. Social Psychological and Personality Science, 8(4), 1-9.

Forza, C. (2002). Survey Research in Operations Management: A Process-based Perspective. International Journal of Operations and Production Management, 22 (2), 152-194.

Ganesh, T. (2009). Reliability and Validity Issues in Research. Integration and Dissemination Research Bulletin, 4, 35-40.

Gluch, P. (2000). Costs of Environmental Errors (CEE): A Managerial Environmental Accounting Tool or a Symptom of Managerial Frustration? Greener Management International, 31, 87-100.

Graziano, A. M., & Raulin, M. L. (2006). Research Methods: A Process of Inquiry (6th Ed.). Boston, MA: Allyn & Bacon.

Gwet, K. L. (2008). Computing Inter-Rater Reliability and its Variance in the Presence of High Agreement. British Journal of Mathematical and Statistical Psychology, 61, 29-48.

Hallgren, K. A. (2012). Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23-34.

Hashim, N. H., Murphy, J., & O’Connor, P. (2007). Take Me Back: Validating the Wayback Machine as a Measure of Website Evolution. In M. Sigala, L. Mich and J. Murphy (Eds.). Information & Communication Technologies in Tourism, pp. 435-446, Wien: Springer-Verlag.

Haynes, M. C., Ryan, N., Saleh, M., Winkel, A. F., & Ades, V. (2017). Contraceptive Knowledge Assessment: Validity and Reliability of a Novel Contraceptive Research Tool. Contraception, 95, 190–197.

Heale, R., & Twycross, A. (2015). Validity and Reliability in Quantitative Studies. Evid Based Nurs, 18(4), 66-67.

Howell, D.C. (1995). Fundamental Statistics for the Behavioral Sciences (3rd Ed.). Duxbury Press, Belmont, California.

Huck, S. W. (2007). Reading Statistics and Research (5th Ed.). New York, NY: Allyn & Bacon.

Ihantola, E. -M., & Kihn, L. -A. (2011). Threats to Validity and Reliability in Mixed Methods Accounting Research. Qualitative Research in Accounting and Management, 8(1), 39-58.

Johnson, R. E., Kording, K. P., Hargrove, L. J., & Sensinger, J. W. (2017). Adaptation to Random and Systematic Errors: Comparison of Amputee and Non-Amputee Control Interfaces with Varying Levels of Process Noise. PLoS ONE, 12(3): e0170473.

Kane, M. T. (2013). Validating the Interpretations and Uses of Test Scores. Journal of Educational Measurement, 50, 1–73.

Keller, A. (2000). Electronic Journals: A Delphi Survey. INSPEL, 34(3-4), 187-193.

Kerlinger, H. (1964). Foundations of Behavioral Research. Holt, Rinehart and Winston, Inc., New York.

Keyton, J., King, T., Mabachi, N. M., Manning, J., Leonard, L. L., & Schill, D. (2004). Content Analysis Procedure Book. Lawrence, KS: University of Kansas.

Kimberlin, C. L., & Winterstein, A. G. (2008). Validity and Reliability of Measurement Instruments Used in Research. American Journal of Health-System Pharmacists, 65(1), 2276- 2284.

Last, J. (Ed.) (2001). International Epidemiological Association, A Dictionary of Epidemiology (4th Ed.). New York: Oxford University Press.

Leedy, P. D., & Ormrod, J. E. (2004). Practical Research, (8th Ed.). Upper Saddle River, N.J: Prentice Hall.

Legesse, B. (2014). Research Methods in Agribusiness and Value Chains. School of Agricultural Economics and Agribusiness, Haramaya University.

Lillis, A. (2006), Reliability and Validity in Field Study Research. In Z. Hoque (Ed.), Methodological Issues in Accounting Research: Theories and Methods, pp. 461-475. Piramus, London.

Madan, C. R., & Kensinger, E. A. (2017). Test–Retest Reliability of Brain Morphology Estimates. Brain Informatics, 4, 107–121.

Malhotra, N. K. (2004). Marketing Research: An Applied Orientation (4th Ed.). New Jersey: Pearson Education, Inc.

Marascuilo, L. A., & Levin, J. R. (1970). Appropriate Post Hoc Comparisons for Interaction and Nested Hypotheses in Analysis of Variance Designs: The Elimination of Type-IV Errors, American Educational Research Journal, 7(3), 397-421.

Messick, S. (1989). Validity. In R. L. Linn (Ed.). Educational Measurement (3rd Ed.). New York: American Council on Education and Macmillan.

Messick, S. (1995). Standards of Validity and the Validity of Standards in Performance Assessment. Educational Measurement: Issues and Practice, 14(4), 5-8.

Mitroff, I. I., & Silvers, A. (2009). Dirty Rotten Strategies: How We Trick Ourselves and Others into Solving the Wrong Problems Precisely. Stanford Business Press.

Moana-Filho, E. J., Alonso, A. A., Kapos, F. P., Leon-Salazar, V., Gurand, S. H., Hodges, J. S., & Nixdorf, D. R. (2017). Multifactorial Assessment of Measurement Errors Affecting Intraoral Quantitative Sensory Testing Reliability. Scandinavian Journal of Pain, 16(6), 93-98.

Morris, E., & Burkett, K. (2011). Mixed Methodologies: A New Research Paradigm or Enhanced Quantitative Paradigm, Online Journal of Cultural Competence in Nursing and Healthcare, 1(1): 27‒36.

Murphy, K. R., & Davidshofer, C. O. (2005). Psychological Testing: Principles and Applications (6th Ed.). Upper Saddle River, N.J.: Pearson/Prentice Hall.

Neyman, J., & Pearson, E. S. (1928). On the Use and Interpretation of Certain Test Criteria for Purposes of Statistical Inference: Part I. Biometrika, 20A(1/2), 175-240.

Noble, S., Spann, M. N., Tokoglu, F., Shen, X., Constable, R. T., & Scheinost, D. (2017). Multifactorial Assessment of Measurement Errors Affecting Intraoral Quantitative Sensory Testing Reliability. Cerebral Cortex, 27(11), 5415–5429.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric Theory (3rd Ed.). Mcgraw-Hill: New York.

Nwana, O. C. (2007). Textbook on Educational Measurement and Evaluation. Owerri: Bomaway Publishers.

Okoro, O. M. (2002). Measurement and Evaluation in Education. Obosi: Pacific Publisher Ltd.

Oliver, V. (2010). 301 Smart Answers to Tough Business Etiquette Questions. Skyhorse Publishing: New York, USA.

Onwuegbuzie, A. J. (2003). Expanding the Framework of Internal and External Validity in Quantitative Research. Research in the Schools, 10(1), 71-90.

Pallant, J. (2011). A Step by Step Guide to Data Analysis Using the SPSS Program: Survival Manual, (4th Ed.). McGraw-Hill, Berkshire.

Pett, M., Lackey, N., & Sullivan, J. (2003). Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research. Thousand Oaks, CA: SAGE Publications.

Reichenbacher, M., & Einax, J. W. (2011). Challenges in Analytical Quality Assurance. Springer-Verlag Berlin Heidelberg.

Robson, C. (2011). Real World Research: A Resource for Users of Social Research Methods in Applied Settings, (2nd Ed.). Sussex, A. John Wiley and Sons Ltd.

Russell, B. (1971). A Liberal Decalogue. In The Autobiography of Bertrand Russell. 3. 1944–1967, pp. 60–101. London: George Allen & Unwin.

Ryan, B., Scapens, R. W., & Theobald, M. (2002). Research Method & Methodology in Finance & Accounting (2nd Ed.). Thomson, London.

Saunders, M., Lewis, P., & Thornhill, A. (2009). Research Methods for Business Students, (5th Ed.). Harlow, Pearson Education.

Shekharan, U., & Bougie, R. (2010). Research Methods for Business: A Skill Building Approach (5th Ed.). New Delhi: John Wiley.

Sim, J., & Wright, C. (2005). The Kappa Statistic in Reliability Studies: Use, Interpretation and Sample Size Requirements. Physical Therapy, 85(3), 257–268.

Singh, A. S. (2014). Conducting Case Study Research in Non-Profit Organisations. Qualitative Market Research: An International Journal, 17, 77–84.

Sperry, L. (2004). Assessment of Couples and Families: Contemporary and Cutting Edge Strategies (1st Ed.). New York, NY: Routledge.

Straub, D. W. (1989). Validating Instruments in MIS Research. MIS Quarterly, 13(2), 147-169.

Swamy, P. A. V. B., Hall, S. G., Tavlas, G. S., & von zur Muehlen, P. (2017). On the Interpretation of Instrumental Variables in the Presence of Specification Errors: A Reply, 5(32), 1-3.

Tashakkori, A., & Teddlie, C. (1998). Mixed Methodology: Combining Qualitative and Quantitative Approaches. Thousand Oaks, CA: SAGE.

Tavakol, M., & Dennick, R. (2011). Making Sense of Cronbach’s Alpha. International journal of Medical Education, 2, 53-55.

Taylor, J. R. (1999). An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements. University Science Books.

Thatcher, R. (2010). Validity and Reliability of Quantitative Electroencephalography. Journal of Neurotherapy, 14, 122-152.

Thompson, B. (2003). Understanding Reliability and Coefficient Alpha, Really. In B. Thompson (Ed.). Score Reliability: Contemporary Thinking on Reliability Issues. Thousand Oaks, CA: SAGE.

Traub, R. E., & Rowley, G. L. (1991). An NCME Instructional Module on Understanding Reliability. Educational Measurement: Issues and Practice, 10(1), 37-45.

Turner, S.P. (1979). The Concept of Face Validity. Quality and Quantity, 13(1), 85–90.

Twycross, A., & Shields, L. (2004). Validity and Reliability-What’s it All About? Part 2: Reliability in Quantitative Studies. Paediatric Nursing, 16 (10), 36.

Waltz, C., Strickland, O., & Lenz, E. (2004). Measurement in Nursing and Health Research. New York: Springer Publishing.

Willis, J. (2007). Foundations of Qualitative Research: Interpretive and Critical Approaches. SAGE Publications.

Wilson, J. (2010). Essentials of Business Research: A Guide to Doing Your Research Project. SAGE Publications.

Yarnold, P. R. (2014). How to Assess the Inter-Method (Parallel-Forms) Reliability of Ratings Made on Ordinal Scales: Emergency Severity Index (Version 3) and Canadian Triage Acuity Scale. Optimal Data Analysis, 3(4), 50-54.

Yoshida, S., Matsushima, M., Wakabayashi, H., Mutai, R., Murayama, S., Hayashi, T., Ichikawa, H., Nakano, Y., Watanabe, T., & Fujinuma, Y. (2017). Validity and Reliability of the Patient Centred Assessment Method for Patient Complexity and Relationship with Hospital Length of Stay: a Prospective Cohort Study. BMJ Open, 7 (e016175), 1-8.

Zohrabi, M. (2013). Mixed Method Research: Instruments, Validity, Reliability and Reporting Findings. Theory and Practice in Language Studies, 3(2), 254-262.