Guidelines for data fusion with international large scale assessments: Insights from the TALIS-PISA link

Gil-Izquierdo, María and Cordero, José Manuel (2017): Guidelines for data fusion with international large scale assessments: Insights from the TALIS-PISA link.

Abstract

The educational effectiveness research has experienced a substantial improvement in the last decades thanks to the refinement of large-scale international assessments. Those surveys provide researchers and policy makers with comparative micro data that can be exploited in cross-national studies in order to evaluate educational policies or determinants of educational achievement. This paper focuses on the potential uses and misuses that can be made with the so-called TALIS-PISA link created by the OECD. This is a recently developed instrument that allows for connecting data about teacher characteristics and practices collected in TALIS with students´ academic performance measured in PISA. However, the statistical and technical aspects regarding this link between both surveys are far from straightforward. In this paper we explore the main problematic issues of the data fusion process and provide some guidelines for researchers interested in performing empirical analyses using the resulting dataset.

• Item Type: MPRA Paper
• Language: English
• Keywords: Education, Teachers, International datasets, Large-scale assessments, PISA.
• Subjects: H - Public Economics > H5 - National Government Expenditures and Related Policies > H52 - Government Expenditures and Education; I - Health, Education, and Welfare > I2 - Education and Research Institutions > I21 - Analysis of Education

References

Austin, B., Adesope, O. O., French, B. F., Gotch, C., Bélanger, J. and Kubacka, K. (2015). Examining school context and its influence on teachers. Linking TALIS 2013 with PISA 2012 student data. OECD Education Working Papers, No. 115, OECD Publishing, Paris.

Boonen, T., Van Damme, J. and Onghena, P. (2014). Teacher effects on student achievement in first grade: which aspects matter most?, School Effectiveness and School Improvement, 25(1), 126-152.

Caro, D. H., Lenkeit, J. and Kyriakides, L. (2016). Teaching strategies and differential effectiveness across learning contexts: Evidence from PISA 2012, Studies in Educational Evaluation, 49, 30-41.

Chapman, C., Armstrong, P., Harris, A., Muijs, D., Reynolds, D. and Sammons, P. (eds.). (2012). School effectiveness and improvement research, policy and practice: Challenging the orthodoxy? Abingdon, Oxon: Routledge.

Cordero, J.M., Cristóbal, V. and Santín, D. (2017). Causal Inference on Education Policies: A Survey of Empirical Studies Using PISA, TIMSS and PIRLS, Journal of Economic Surveys, forthcoming doi: 10.1111/joes.12217.

Creemers, B. P. M. (1994). The effective classroom. London: Cassell.

Creemers, B. P. M. and Kyriakides, L. (2008). The dynamics of educational effectiveness: A contribution to policy, practice and theory in contemporary schools. London: Routledge.

D'Orazio, M., Di Zio, M. and Scanu, M. (2006). Statistical matching: Theory and practice. John Wiley & Sons.

Drent, M., Meelissen, M. R. and van der Kleij, F. M. (2013). The contribution of TIMSS to the link between school and classroom factors and student achievement. Journal of Curriculum Studies, 45(2), 198-224.

Echazarra, A., Salinas, D., Méndez, I., Denis, V. and Rech, G. (2016). How teachers teach and students learn: Successful strategies for school, OECD Education Working Papers, No. 130, Paris: OECD Publishing.

Ehrenberg, R. G. and Brewer, D. J. (1994). Do school and teacher characteristics matter? Evidence from high school and beyond, Economics of Education Review, 13, 1–17.

Eveleigh, F. and Freeman, C. (2012). An exploratory analysis of the TALIS and PISA link data: an investigation of the possible relationships. Contribution for the European Conference on Educational Research (ECER) (2012) Conference.

Greenwald, R., Hedges, L. V. and Laine, R. D. (1996). The effect of school resources on student achievement, Review of Educational Research, 66(3), 361-396.

Hanushek, E. A. (2011). The economic value of higher teacher quality. Economics of Education Review, 30(3), 466-479.

Hanushek, E. A., Rivkin, S. G. and Taylor, L. L. (1996). Aggregation and the Estimated Effects of School Resources. The Review of Economics and Statistics, 78(4), 611-627.

Hattie, J. A. C. (2009). Visible learning. A synthesis of over 800 meta-analyses relating to achievement. Oxon: Routledge.

House, J. D. (2009). Elementary-school mathematics instruction and achievement of fourth-grade students in Japan: Findings from the TIMSS 2007 assessment. Education, 130(2), 301.

Hox, J. (2002). Multilevel Analysis. Techniques and Applications, Mahwah: Lawrence Erlbaum Associates.

Kaplan, D. and McCarty, A. T. (2013). Data fusion with international large scale assessments: a case study using the OECD PISA and TALIS surveys. Large-scale Assessments in Education, 1(1), 1-26.

Le Donné, N., Fraser, P. and Bousquet, G. (2016). Teaching Strategies for Instructional Quality: Insights from the TALIS-PISA Link Data, OECD Education Working Papers, No. 148, Paris: OECD Publishing.

Mullis, I. V., Martin, M. O., Foy, P. and Arora, A. (2012). TIMSS 2011 international results in mathematics. International Association for the Evaluation of Educational Achievement. Herengracht 487, Amsterdam, 1017 BT, The Netherlands.

Nilsen, T. and Gustafsson, J. E. (2016). Teacher Quality, Instructional Quality and Student Outcomes. Springer International Pu.

Palardy, G. J. and Rumberger, R. W. (2008). Teacher effectiveness in first grade: The importance of background qualifications, attitudes, and instructional practices for student learning, Educational Evaluation and Policy Analysis, 30(2), 111-140.

Nye, B., Konstantopoulos, S. and Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26, 237–257.

O’Dwyer, L. M., Wang, Y., & Shields, K. A. (2015). Teaching for conceptual understanding: A cross-national comparison of the relationship between teachers’ instructional practices and student achievement in mathematics. Large-scale Assessments in Education, 3(1), 1-30. doi: 10.1186/s40536-014-0011-6

OECD (2009). PISA Data Analysis Manual, SPSS Second Edition, OECD Publishing, Paris.

OECD (2014a). TALIS 2013 Results An International Perspective on Teaching and Learning, OECD, Paris.

OECD (2014b). TALIS 2013 User Guide to the International Dataset, OECD, Paris.

OECD (2014c). PISA 2012 Technical Report, OECD Publishing, Paris.

Orlich, D. C., Harder, R. J., Callahan, R. C., Trevisan, M. S. and Brown, A. H. (2013). Teaching Strategies: A Guide to Effective Instruction, 10th edition, Wadsworth, Cengage Learning, Boston, MA.

Rässler, S. (2012). Statistical matching: A frequentist theory, practical applications, and alternative Bayesian approaches (Vol. 168). Springer Science & Business Media.

Raudenbush, S. W. and Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods, Sage Publications.

Reynolds, D., Sammons, P., De Fraine, B., Van Damme, J., Townsend, T., Teddlie, C. and Stringfield, S. (2014). Educational effectiveness research (EER): a state-of-the-art review. School Effectiveness and School Improvement, 25(2), 197-230.

Rivkin, S.G., Hanushek, E.A. and Kain, J.F. (2005). Teachers, schools, and academic achievement, Econometrica 73, no. 2: 417–58.

Rockoff, J.E. (2004) The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review 94, no. 2: 247–52.

Rowan, B., Correnti, R. and Miller, R. J. (2002). What large-scale, survey research tells us about teacher effects on student achievement: Insights from the prospects study of elementary schools. Teacher College Record, 104, 1525–1567.

Sans-Martín, A., Guardia, J. and Triadó, X.M. (2016). Educational leadership in Europe:A transcultural approach, Revista de Educación, 371, 83-106.

Schwerdt, G. and Wuppermann, A. C. (2011). Is traditional teaching really all that bad? A within-student between-subject approach. Economics of Education Review, 30(2), 365-379.

Todd, P. E. and Wolpin, K. I. (2003). On the specification and estimation of the production function for cognitive achievement, The Economic Journal, 113(485).

Van Klaveren, C. (2011). Lecturing style teaching and student performance, Economics of Education Review, 30(4), 729-739.

Wayne, A. J. and Youngs, P. (2003). Teacher characteristics and student achievement gains: A review, Review of Educational Research, 73, 89–122.

Wu, M. (2005). The role of plausible values in large-scale surveys. Studies in Educational Evaluation, 31(2-3), 114-128.

Zuzovsky, R. (2013). What works where? The relationship between instructional variables and schools' mean scores in mathematics and science in low-, medium-, and high-achieving countries. Large-scale Assessments in Education, 1(1), 2.