Using "Cheat Sheets" to Distinguish Ability from Knowledge: Evidence from a Randomized Control Trial in Chile

Díez-Amigo, Sandro (2014): Using "Cheat Sheets" to Distinguish Ability from Knowledge: Evidence from a Randomized Control Trial in Chile.

Abstract

According to the existing evidence some higher education admission tests may be screening out students who, despite a relative lack of specific knowledge, possess as much intellectual ability as their peers. If this is the case, students from disadvantaged socioeconomic backgrounds are likely to be disproportionately affected, since they generally receive a primary and secondary education of worse quality than their better-off peers, often resulting in significant knowledge gaps. Also, although in some cases these formative shortcomings might be too large to be feasibly addressed at the time of enrollment in higher education, it is plausible to think that in some cases they may perhaps be relatively easy to remedy. In view of all this, in this paper I present a diagnostics experiment, aimed at helping to better understand this issue. In particular, I custom-designed a multiple-choice test, intended to measure an individual's mathematical ability, while minimizing the reliance on previously acquired knowledge. Also, I put together a two page "cheat sheet", which outlined all the necessary concepts to successfully complete the exam, without providing any explicit answers. This test was subsequently used to evaluate the candidates applying for admission into a special access program at one of the leading Chilean universities. A staged randomized control trial was used to measure the difference in academic performance (i.e. number of correctly answered questions) across the three parts of the exam between students who received a "cheat sheet" after the first or second parts of the test, respectively. As expected, "cheat sheets" improved the average performance of candidates on the exam, but their impact varied considerably across individuals. Most importantly, "cheat sheets" proved significantly more beneficial (in terms of improved test performance) to those students who were more likely to have had a secondary education of lower quality. This result has important implications for educational policies in Chile and elsewhere, suggesting that a transition to ability-focused admission tests would facilitate the access to higher education for talented students from disadvantaged backgrounds.

• Item Type: MPRA Paper
• Language: English
• Keywords: Equality of Opportunity; Higher Education; Education Policy; Policy Evaluation; Economic Development
• Subjects: A - General Economics and Teaching > A1 - General Economics > A12 - Relation of Economics to Other Disciplines; I - Health, Education, and Welfare > I2 - Education and Research Institutions; J - Labor and Demographic Economics > J1 - Demographic Economics > J15 - Economics of Minorities, Races, Indigenous Peoples, and Immigrants ; Non-labor Discrimination; O - Economic Development, Innovation, Technological Change, and Growth > O1 - Economic Development > O15 - Human Resources ; Human Development ; Income Distribution ; Migration

References

Banerjee, A., Duflo, E., Gallego, F., and Kast, F. (2012). "Removing Higher Education Barriers to Entry: Test Training and Savings Promotion". Mimeo.

Bloom, B.S. (Editor), Engelhart, M.D., Furst, E.J., Hill, W.H., Krathwohl, D.R. (1956). Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook 1: Cognitive Domain. David McKay, New York.

Bransford, J.D., Brown, A.L., Cocking, R.R. (Editors) (1999). How People Learn. National Academy Press, Washington D.C..

Comisión de Financiamiento Estudiantil para la Educación Superior (2012). "Análisis y Recomendaciones para el Financiamiento del Sistema Estudiantil". Ministerio de Educación, Gobierno de Chile.

DEMRE (2011). "Proceso de Admisión 2011". Universidad de Chile.

DEMRE (2012). "Proceso de Admisión 2012". Universidad de Chile.

DEMRE (2013). "Proceso de Admisión 2013". Universidad de Chile.

Dickert-Conlin, S., and Rubenstein, R. (Editors) (2007). Economic Inequality and Higher Education. Access, Persistence, and Success. Russell Sage Foundation, New York.

Díez-Amigo, S. (2014). "Improving the Access to Higher Education for the Poor: Lessons from the Design and Implementation of an Experimental Special Admission Program in Chile". Mimeo.

Dinkelman, T., and Martinez, C. (2011). "Investing in Schooling in Chile: The Role of Information About Financial Aid for Higher Education". CEPR Discussion Paper DP8375.

Heckman, J.J. (1995). "Lessons from the Bell Curve". Journal of Political Economy, 103(5):1091-1120.

Hoxby, C., and Turner, S. (2012). "Expanding College Opportunities for High-Achieving, Low Income Students". SIEPR Working Paper 12-014.

Krathwohl, D.R. (2002). "A Revision of Bloom's Taxonomy: An Overview". Theory into Practice, 41(4):212-218.

Loofbourow, L. (2013). No to Profit. Boston Review, May 2013.

OECD (2011). "Education at a Glance 2011: OECD Indicators". OECD Publishing.

Pellegrino, J.W., Chudowsky, N., and Glaser, R. (Editors) (2001). Knowing What Students Know. National Academy Press, Washington D.C..

Sánchez, I. (2011). "Los Desafíos de la Educación Superior en Chile". Centro de Políticas Públicas UC: Temas de la Agenda Pública, 6(47).

Santelices, M.V., Ugarte, J.J., Flotts, P., Radovic, D., and Kyllonen, P. (2011). "Measurement of New Attributes for Chile's Admissions System to Higher Education". ETS Research Report 11-18.

Tapia Rojas, O.A., Olivares, J., and Ormazábal Díaz-Muñoz, M.R. (Editors) (1996). Manual de Preparación Matemáticas: Prueba de Aptitud Académica. Ediciones Universidad Católica de Chile, Santiago.

Williamson, C., and Sánchez, J.M. (2009). "Financiamiento Universitario: Principios Básicos para el Diseño de una Politíca Pública en Chile". Centro de Políticas Públicas UC: Temas de la Agenda Pública, 4(34).