When Excellence is not Excellent: The Impact of the Excellence Initiative on the Relative Productivity of German Universities

Cantner, Uwe and Grashof, Nils and Grebel, Thomas and Zhang, Xijie (2023): When Excellence is not Excellent: The Impact of the Excellence Initiative on the Relative Productivity of German Universities.

Abstract

Since the Bologna Process, universities have experienced a tremendous increase in competitive pressure. The Excellence Initiative is a path-breaking initiative to the historically-based egalitarian higher education system in Germany to boost universities' competitiveness and the competition among universities. Despite this systematic change, it remains rather unclear to what extent the Excellence Initiative influences the performance. In this paper, we investigate the effect of the Excellence Initiative on universities relative productivity in teaching and research and hence on the divergence or convergence of universities in these performance dimensions. Based on a unique dataset that combines publication and detailed university-level data, we apply a two-step approach by calculating relative productivity with a non-parametric procedure in the first step and using a difference-in-difference approach for estimating treatment effects of the Excellence Initiative in the second step. Overall, we note only a few moments when universities funded by the Excellence Initiative seem to excel and make progress compared to non-funded universities. In research, only some of the Excellence-funded universities, particularly the winners of the Graduate Schools and Clusters of Excellence funding line, manage to improve significantly, even if only for a few years. In teaching, we found no significant average treatment effect of the Excellence Initiative, but a slightly significant time-specific decline in relative teaching productivity two years after funding.

• Item Type: MPRA Paper
• Language: English
• Keywords: Excellence Initiative, Universities, Relative Productivity, DEA, Difference-in-difference, Funding
• Subjects: C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C22 - Time-Series Models ; Dynamic Quantile Regressions ; Dynamic Treatment Effect Models ; Diffusion Processes; I - Health, Education, and Welfare > I2 - Education and Research Institutions > I20 - General; I - Health, Education, and Welfare > I2 - Education and Research Institutions > I23 - Higher Education ; Research Institutions; I - Health, Education, and Welfare > I2 - Education and Research Institutions > I25 - Education and Economic Development; I - Health, Education, and Welfare > I2 - Education and Research Institutions > I28 - Government Policy

References

Abadie, A. and Imbens, G. W. (2016). Matching on the estimated propensity score. Econometrica, 84(2):781–807.

Abbott, M. and Doucouliagos, C. (2003). The efficiency of australian universities: a data envelopment analysis. Economics of Education review, 22(1):89–97.

Agasisti, T. (2009). Market forces and competition in university systems: theoretical reflections and empirical evidence from italy. International Review of applied economics, 23(4):463–483.

Agasisti, T. and Johnes, G. (2009). Beyond frontiers: comparing the efficiency of higher education decision-making units across more than one country. Education economics, 17(1):59–79.

Aghion, P., Dewatripont, M., Hoxby, C., Mas-Colell, A., and Sapir, A. (2010). The governance and performance of universities: evidence from europe and the us. Economic policy, 25(61):7–59.

Almus, M. and Czarnitzki, D. (2003). The effects of public r&d subsidies on firms’ innovation activities: the case of eastern germany. Journal of Business & Economic Statistics, 21(2):226–236.

Ardiyono, S. K. (2022). Covid-19 pandemic, firms’ responses, and unemployment in the asean-5. Economic Analysis and Policy, 76:337–372.

Arora, A., David, P. A., and Gambardella, A. (2000). Reputation and competence in publicly funded science: estimating the effects on research group productivity. In The Economics and Econometrics of Innovation, pages 141–176. Springer.

Arora, A. and Gambardella, A. (2005). The impact of nsf support for basic research in economics. Annales d’Economie et de Statistique, pages 91–117.

Ash, M. G. (2006). Bachelor of what, master of whom? the humboldt myth and historical transformations of higher education in german-speaking europe and the us 1. European Journal of Education, 41(2):245–267.

Auranen, O. and Nieminen, M. (2010). University research funding and publication performance—an international comparison. Research policy, 39(6):822–834.

Autio, E. and Rannikko, H. (2016). Retaining winners: Can policy boost high-growth entrepreneurship? Research policy, 45(1):42–55.

Banker, R. D., Charnes, A., and Cooper, W. W. (1984). Some models for estimating technical and scale inefficiencies in data envelopment analysis. Management science, 30(9):1078–1092.

Barra, C. and Zotti, R. (2016). Measuring efficiency in higher education: an empirical study using a bootstrapped data envelopment analysis. International Advances in Economic Research, 22(1):11–33.

Başkaya, S., & Klumpp, M. (2014). International data envelopment analysis in higher education: how do institutional factors influence university efficiency. Journal of Business and Economics, 5(11), 2085-2090.

Björk, B.-C. and Solomon, D. (2013). The publishing delay in scholarly peer-reviewed journals. Journal of informetrics, 7(4):914–923.

Bolli, T., Olivares, M., Bonaccorsi, A., Daraio, C., Aracil, A. G., and Lepori, B. (2016). The differential effects of competitive funding on the production frontier and the efficiency of universities. Economics of Education Review, 52:91–104.

Bolli, T. and Somogyi, F. (2011). Do competitively acquired funds induce universities to increase productivity? Research Policy, 40(1):136–147.

Bonaccorsi, A., Simar, D. C., et al. (2006). Advanced indicators of productivity of universities. An application of robust nonparametric methods to italian data, scientometrics, vol. 66, no. 2, 389-410.

Brinkman, P. T. and Leslie, L. L. (1986). Economies of scale in higher education: Sixty years of research. The review of higher education, 10(1):1–28.

Buenstorf, G. and Koenig, J. (2020). Interrelated funding streams in a multi-funder university system: Evidence from the german exzellenzinitiative. Research Policy, 49(3):103924.

Butler, L. (2003). Explaining australia’s increased share of isi publications—the effects of a funding formula based on publication counts. Research policy, 32(1):143–155.

Caliendo, M. and Kopeinig, S. (2008). Some practical guidance for the implementation of propensity score matching. Journal of economic surveys, 22(1):31–72.

Callaway, B. (2022). Difference-in-differences for policy evaluation. arXiv preprint arXiv:2203.15646.

Callaway, B. and Sant’Anna, P. H. (2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225(2):200–230.

Cantner, U., Krüger, J., and Hanusch, H. (2007). Produktivitäts-und Effizienzanalyse: der nichtparametrische Ansatz. Springer-Verlag.

Carayol, N., Lanoe, M., et al. (2018). The impact and design of project-based funding in science: Lessons from the anr experience. Technical report.

Carayol, N. and Matt, M. (2006). Individual and collective determinants of academic scientists’ productivity. Information Economics and policy, 18(1):55–72.

Caves, D. W., Christensen, L. R., and Diewert, W. E. (1982). The economic theory of index numbers and the measurement of input, output, and productivity. Econometrica: Journal of the Econometric Society, pages 1393–1414.

Cherchye, L. and Abeele, P. V. (2005). On research efficiency: A micro-analysis of dutch university research in economics and business management. Research policy, 34(4):495–516.

Conard, M. J. and Conard, M. A. (2000). An analysis of academic reputation as perceived by consumers of higher education. Journal of Marketing for Higher Education, 9(4):69–80.

Corsini, A. (2022). Scientific networks and idex funding. Doctoral dissertation.

Dasgupta, P. and David, P. A. (1994). The new economics of science. Research Policy, 23(5):487–521.

De Chaisemartin, C. and d’Haultfoeuille, X. (2020). Two-way fixed effects estimators with heterogeneous treatment effects. American Economic Review, 110(9):2964–96.

De Fraja, G. and Iossa, E. (2002). Competition among universities and the emergence of the elite institution. Bulletin of economic Research, 54(3):275–293.

De Groot, H., McMahon, W. W., and Volkwein, J. F. (1991). The cost structure of american research universities. The Review of economics and statistics, pages 424–431.

Del Rey, E. (2001). Teaching versus research: a model of state university competition. Journal of Urban Economics, 49(2):356–373.

Di Matteo, D. (2022). Does high-speed rail matter for tourism? evidence from italy. Research in Transportation Business & Management, page 100881.

Epple, D., Romano, R., and Sieg, H. (2006). Admission, tuition, and financial aid policies in the market for higher education. Econometrica, 74(4):885–928.

Fandel, G. (2007). On the performance of universities in north rhine-westphalia, germany: Government’s redistribution of funds judged using dea efficiency measures. European Journal of Operational Research, 176(1):521–533.

Fischer, M., Kampkötter, P., et al. (2017). Effects of german universities’ excellence initiative on ability sorting of students and perceptions of educational quality. Journal of Institutional and Theoretical Economics, 173(4):662.

Franck, E. P. and Schönfelder, B. (2000). On the role of competition in higher education—uses and abuses of the economic metaphor. Schmalenbach Business Review, 52(3):214–237.

Frietsch, R., Schubert, T., and Rothengatter, O. (2017). An analysis of the excellence initiative and its effects on the funded universities. Technical report, Studien zum deutschen Innovationssystem.

Frølich, N., Coate, K., Mignot-Gerard, S., and Knill, C. (2010). Einheit von forschung und lehre: implications for state funding of universities. Higher Education Policy, 23(2):195–211.

Gautier, A. and Wauthy, X. (2007). Teaching versus research: A multi-tasking approach to multidepartment universities. European Economic Review, 51(2):273–295.

Gawellek, B. and Sunder, M. (2016). The german excellence initiative and efficiency change among universities, 2001-2011. Technical report, Working Paper.

Geuna, A. and Martin, B. R. (2003). University research evaluation and funding: An international comparison. Minerva, 41(4):277–304.

Gnewuch, M. and Wohlrabe, K. (2018). Super-efficiency of education institutions: an application to economics departments. Education Economics, 26(6):610–623.

Goodman-Bacon, A. (2021). Difference-in-differences with variation in treatment timing. Journal of Econometrics, 225(2):254–277.

Gralka, S. (2018). Persistent inefficiency in the higher education sector: evidence from germany. Education Economics, 26(4):373–392.

Gralka, S., Wohlrabe, K., and Bornmann, L. (2019). How to measure research efficiency in higher education? research grants vs. publication output. Journal of Higher Education Policy and Management, 41(3):322–341.

Grashof, N. and Kopka, A. (2022). Artificial intelligence and radical innovation: an opportunity for all companies? Small Business Economics, pages 1–27.

Gu, J. (2012). Spatial recruiting competition in chinese higher education system. Higher Education, 63(2):165–185.

Guerzoni, M. and Raiteri, E. (2015). Demand-side vs. supply-side technology policies: Hidden treatment and new empirical evidence on the policy mix. Research Policy, 44(3):726–747.

Gush, J., Jaffe, A., Larsen, V., and Laws, A. (2018). The effect of public funding on research output: the new zealand marsden fund. New Zealand Economic Papers, 52(2):227–248.

Hachmeister, C.-D., Harde, M. E., and Langer, M. F. (2007). Einflussfaktoren der studienentscheidung. Eine empirische Studie von CHE und EINSTIEG. Arbeitspapier, 95.

Heckman, J. (1997). Instrumental variables: A study of implicit behavioral assumptions used in making program evaluations. Journal of human resources, pages 441–462.

Heckman, J. J., Ichimura, H., and Todd, P. E. (1997). Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme. The review of economic studies, 64(4):605–654.

Horstschräer, J. (2012). University rankings in action? the importance of rankings and an excellence competition for university choice of high-ability students. Economics of Education Review, 31(6):1162–1176.

Huber, M. (2021). Causal analysis: Impact evaluation and causal machine learning with applications in r.

Imboden, D., Lütjen-Drecoll, E., Bargmann, S., Casper, G., Gächter, S., Kratky, C., Nahrstedt, K., Bech Nosch, M., Pauss, F., and Scheidegger, D. (2016). Internationale expertenkommission zur evaluation der exzellenzinitiative–endbericht. Internationale Expertenkommission Exzellenzinitiative.

Jacob, B. A. and Lefgren, L. (2011). The impact of research grant funding on scientific productivity. Journal of public economics, 95(9-10):1168–1177.

Johnes, G. and Johnes, J. (1993). Measuring the research performance of uk economics departments: an application of data envelopment analysis. Oxford economic papers, pages 332–347.

Kaiser, U. and Kuhn, J. M. (2012). Long-run effects of public–private research joint ventures: The case of the danish innovation consortia support scheme. Research Policy, 41(5):913–927.

Kempkes, G. and Pohl, C. (2008). Do institutions matter for university cost efficiency? evidence from germany. CESifo economic studies, 54(2):177–203.

Kettl, D. F. (2000). The global public management revolution: A report on the transformation of governance. Brookings Institution Press.

Krücken, G. (2021). Multiple competitions in higher education: a conceptual approach. Innovation, 23(2):163–181.

Krücken, G., Blümel, A., and Kloke, K. (2009). Towards organizational actorhood of universities: Occupational and organizational change within german university administrations.

Krücken, G. and Meier, F. (2006). Turning the university into an organizational actor. Globalization and organization: World society and organizational change, pages 241–257.

Laudel, G. (2006). The art of getting funded: how scientists adapt to their funding conditions. Science and Public Policy, 33(7):489–504.

Lehmann, E. E., Meoli, M., Paleari, S., and Stockinger, S. A. (2018). Approaching effects of the economic crisis on university efficiency: A comparative study of germany and italy. Eurasian Business Review, 8(1):37–54.

Leusin, M. E. (2022). The development of al in multinational enterprises-effects upon technological trajectories and innovation performance.

Leuven, E. and Sianesi, B. (2003). Psmatch2: Stata module to perform full mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing.

Liu, J. S., Lu, L. Y., Lu, W.-M., and Lin, B. J. (2013). A survey of dea applications. Omega, 41(5):893–902.

Menter, M., Lehmann, E. E., and Klarl, T. (2018). In search of excellence: A case study of the first excellence initiative of germany. Journal of Business Economics, 88(9):1105–1132.

Mergele, L. and Winkelmayer, F. (2021). The relevance of the german excellence initiative for inequality in university funding. Higher Education Policy, pages 1–19.

Merton, R. K. (1973). The sociology of science: Theoretical and empirical investigations. University of Chicago press.

Möller, T., Schmidt, M., and Hornbostel, S. (2016). Assessing the effects of the german excellence initiative with bibliometric methods. Scientometrics, 109(3):2217–2239.

Morgan, J. (2016). Germany: an alternative route to excellence. Times Higher Education. https://www. timeshighereducation. com/features/germany-an-alternative-route-to-excellence.

Musselin, C. (2018). New forms of competition in higher education. Socio-Economic Review, 16(3):657–683.

Obermeit, K. (2012). Students’ choice of universities in germany: structure, factors and information sources used. Journal of Marketing for higher education, 22(2):206–230.

Olivares, M. and Wetzel, H. (2014). Competing in the higher education market: empirical evidence for economies of scale and scope in german higher education institutions. CESifo Economic Studies, 60(4):653–680.

Pollitt, C. (1990). Managerialism and the public services: The Anglo-American experience. Blackwell.

Quiroga-Martínez, F., Fernández-Vázquez, E., and Alberto, C. L. (2018). Efficiency in public higher education on argentina 2004–2013: Institutional decisions and university-specific effects. Latin American Economic Review, 27(1):1–18.

Rios-Avila, F., Sant’Anna, P., and Callaway, B. (2022). Csdid: Stata module for the estimation of difference-in-difference models with multiple time periods.

Robst, J. (2001). Cost efficiency in public higher education institutions. The Journal of higher education, 72(6):730–750.

Rosenbaum, P. R. and Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41–55.

Rosenbaum, P. R. and Rubin, D. B. (1985). Discussion of “on state education statistics”: A difficulty with regression analyses of regional test score averages. Journal of Educational Statistics, 10(4):326– 333.

Rubin, D. B. (2001). Using propensity scores to help design observational studies: application to the tobacco litigation. Health Services and Outcomes Research Methodology, 2(3):169–188.

Sant’Anna, P. H. and Zhao, J. (2020). Doubly robust difference-in-differences estimators. Journal of Econometrics, 219(1):101–122.

Scandura, A. (2016). University–industry collaboration and firms’ r&d effort. Research Policy, 45(9):1907–1922.

Shleifer, A. (1985). A theory of yardstick competition. The RAND journal of Economics, pages 319–327.

Smith, S., Ward, V., and House, A. (2011). ‘impact’in the proposals for the uk’s research excellence framework: Shifting the boundaries of academic autonomy. Research Policy, 40(10):1369–1379.

Strehl, F., Reisinger, S., and Kalatschan, M. (2007). Funding systems and their effects on higher education systems.

Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical science: a review journal of the Institute of Mathematical Statistics, 25(1):1.

Van der Meulen, B. (1998). Science policies as principal–agent games: Institutionalization and path dependency in the relation between government and science. Research policy, 27(4):397–414.

Vanino, E., Roper, S., and Becker, B. (2019). Knowledge to money: Assessing the business performance effects of publicly-funded r&d grants. Research policy, 48(7):1714–1737.

Wiener, M., Maresch, D., and Breitenecker, R. J. (2020). The shift towards entrepreneurial universities and the relevance of third-party funding of business and economics units in austria: A research note. Review of Managerial Science, 14(2):345–363.

Willich, J., Buck, D., Heine, C., and Sommer, D. (2011). Studienanfänger im wintersemester 2009/10: Wege zum studium, studien-und hochschulwahl, situation bei studienbeginn [first-year students in the winter semester 2009/10: Avenues to study, choice of field of study and university, situation when starting to study]. Hannover: HIS Hochschul-Informations-System GmbH.

Winston, G. C. (1999). Subsidies, hierarchy and peers: The awkward economics of higher education. Journal of economic perspectives, 13(1):13–36.

Wohlrabe, K., Bornmann, L., Gralka, S., and de Moya Anegon, F. (2019). Wie effizient forschen universitäten in deutschland, deren zukunftskonzepte im rahmen der exzellenzinitiative ausgezeichnet wurden? ein empirischer vergleich von input-und output-daten. Zeitschrift für Evaluation, 18(1):9–27.

Zhao, Z. (2008). Sensitivity of propensity score methods to the specifications. Economics Letters, 98(3):309–319.