Gaming in a benchmarking environment. A non-parametric analysis of benchmarking in the water sector

De Witte, Kristof and Marques, Rui (2009): Gaming in a benchmarking environment. A non-parametric analysis of benchmarking in the water sector.

Abstract

This paper discusses the use of benchmarking in general and its application to the drinking water sector. It systematizes the various classifications on performance measurement, discusses some of the pitfalls of benchmark studies and provides some examples of benchmarking in the water sector. After presenting in detail the institutional framework of the water sector of the Belgian region of Flanders (without benchmarking experiences), Wallonia (recently started a public benchmark) and the Netherlands (introduced already in 1997 a public benchmark), we non-parametrically measure the productivity gains by the use of a dynamic Malmquist index. The three regions, each at a different stage of the benchmarking circle, exhibit different performance trends. The ‘carrot’ and the ‘stick’ of benchmarking seem to offer an effective incentive to trigger performance. In addition, the Malmquist decompositions provide some evidence on the ‘gaming’ of the stakeholders by the water utilities.

• Item Type: MPRA Paper
• Language: English
• Keywords: Benchmarking; gaming; Malmquist decomposition; regulation; water sector
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C14 - Semiparametric and Nonparametric Methods: General; L - Industrial Organization > L9 - Industry Studies: Transportation and Utilities > L95 - Gas Utilities ; Pipelines ; Water Utilities

References

Alegre, H. (2007). Water and wastewater infrastructure asset management. Research Program, LNEC, Lisbon (in Portuguese).

Alegre, H., J. Baptista, E. Cabrera, J. Cubillo, P. Duarte, W. Hirner, W. Merkel and R. Parena (2006). Performance indicators for water supply services. Second Edition, London: IWA Publishing.

Aquawal (2006). Etude relative à l’élaboration d’indicateurs de performance des services de distribution de l’eau. Rapport Final. p. 96.

Aubin, D. and F. Varane (2007). Country report Belgium. In Water and Liberalisation, European Water Scenarios. Editors Finger, M., J. Allouche and P. Luis-Manso, IWA publishing.

Belgaqua (2007). Overzicht van de beheersstructuren in België. Belgische federatie voor de watersector, p. 32.

Camp, R. (1989). Benchmarking, the search for industry best practices that lead to superior performance. Portland: American Society for Quality Control (ASQC) Quality Press.

Caves, D.W., L.R. Christensen and W.E. Diewert (1982). The economic theory of index numbers and the measurement of input, output, and productivity. Econometrica 50(6), 1393–1414.

Cazals, C., J.P. Florens and L. Simar (2002). Nonparametric frontier estimation: a robust approach, Journal of econometrics 106(1), 1-25.

Charnes, A., W.W. Cooper and E. Rhodes (1978). Measuring the efficiency of decision making units. European Journal of Operational Research 2(6), 429–444.

Daraio C. and L. Simar (2007). Advanced robust and nonparametric methods in efficiency analysis. Series: Studies in Productivity and Efficiency, Springer.

De Witte, K. and E. Dijkgraaf (2007). Mean and bold? On separating merger economies from structural efficiency gains in the water sector. Tinbergen Institute Discussion Paper 92/03. Accepted for publication in Journal of the Operational Research Society.

De Witte, K. and R. Marques (2007a). Detecting influential observations in frontier models, a robust approach. Mimeo.

De Witte, K. and R. Marques (2007b). Designing incentives to local public utilities, an international comparison to the drinking water sector. CES Discussion Paper Series DPS 07.32.

De Witte, K. and D. Saal (2008). The regulator’s fault? On the effects of regulatory changes on profits, productivity and prices in the Dutch drinking water sector. CES Discussion Paper Series DPS 08.28.

Dijkgraaf, E., R. de Jonge, v.d. Mortel, A. Nentjes, M. Varkevisser and D. Wiersma (1997). Mogelijkheden tot marktwerking in de Nederlandse watersector. Ministry of Economic Affairs, The Hague.

Dijkgraaf, E., S.A. van der Geest and M. Varkevisser (2007). The efficiency gains of benchmarking Dutch water companies. Working Paper Erasmus University Rotterdam.

Farrell, M. (1957). The measurement of productive efficiency. Journal of the Royal Statistical Society, Series B 120(3), 253-281.

Freeman, N., B. Liner and L. Blankenship (1998). Benchmarking. The first step toward improving operating performance. Water Environment & Technology, 10(7), 45-48.

Fried, H., C.A.K. Lovell and S. Schmidt (2008). The measurement of productive efficiency and productivity growth. Oxford University Press.

Jamasb, T., P. Nillesen and M. Pollitt (2003). Gaming the regulator: a survey. The Electricity Journal 16(10), 68-80.

Jamasb, T., P. Nillesen and M. Pollitt (2004). Strategic behaviour under regulatory benchmarking. Energy Economics 26(5), 825-843.

Kingdom, B., J. Knapp, P. LaChange, and M. Olstein (1996). Performance benchmarking for water utilities. Denver: AWWARF and AWWA.

Kingdom, B. (1998). Use of performance indicators and performance benchmarking in the North American water industry—findings from studies recently completed for AWWA and WEF research foundations. Aqua, 47(6), 266–296.

Lafferty, A. and L. William (2005). Benchmarking performance indicators for water and wastewater utilities. AWWARF, Denver.

Marques, R. (2005). Regulation of public services. Sílabo Press, Lisbon.

Marques, R. (2006). A yardstick competition model for Portuguese water and sewerage services regulation. Utilities Policy 14(3), 175–184.

Marques, R. and K. De Witte (2008). Towards a benchmarking paradigm in the European public water and sewerage services. Forthcoming Public Money & Management.

Parena, R., E. Smeets¬ and I. Troquet (2002). Process benchmarking in the water industry. London: IWA Publishing.

SERV (2007). Informatiedossier. De Watersector: analyse en uitdagingen. Sociaal Ecomische Raad van Vlaanderen, p. 141.

Van Dijk, P., M. Schouten, K. Swami and M. Kooij (2007). Country report on the Netherlands. In Water and Liberalisation, European Water Scenarios. Editors Finger, M., J. Allouche and P. Luis-Manso, IWA publishing.

Vewin (2007a). Reflections on performance 2006, Association of Dutch Water Companies, Rijswijk.

Vewin (2007b). Internationale benchmark krijgt vervolg. Waterspiegel 10(1), 19-21.