Quality management of methodology and process development for official statistics

Zeelenberg, Kees and Ypma, Winfried and Struijs, Peter (2018): Quality management of methodology and process development for official statistics.

Abstract

Methodology and process development are cornerstones of official statistics and belong to the major factors that contribute to their quality. This is reflected in the European Statistics Code of Practice, which mentions that "Sound methodology underpins quality statistics" and “Appropriate statistical procedures, implemented from data collection to data validation, underpin quality statistics”. However, methodology and processes are hard to explain to the average user, and so we must find other ways to ensure trust in statistical methodology and processes, and to convince users of the quality of official statistical methodology and processes. Important elements of such an approach are independence of the methodological and development units, transparency of methods and process designs, peer reviews, internal reviews and evaluations, and internal quality management.

In this paper we look in particular at quality management. We show how quality management of methodological and development units may be based on the Code of Practice. We show how the various elements work together, and how the whole of these elements may lead to certification of the units, for example by EFQM or ISO. In 2017 the department for process development and methodology at Statistics Netherlands was certified according to ISO 9001. As an example, we discuss the various steps that have been taken to achieve this certification. In particular we focus on:

the quality procedures for internal and external reports, recommendations and briefs;

the quality assurance of statistical development projects in which methodologists and business analysts participate;

the quality assurance of methodological courses taught to statisticians;

the internal management of the department.

• Item Type: MPRA Paper
• Language: English
• Keywords: quality management, official statistics, statistical methods, process development, certification, ISO 9001
• Subjects: C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology ; Computer Programs > C81 - Methodology for Collecting, Estimating, and Organizing Microeconomic Data ; Data Access; C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology ; Computer Programs > C82 - Methodology for Collecting, Estimating, and Organizing Macroeconomic Data ; Data Access

References

ABS, 2009, ABS Data Quality Framework. http://www.abs.gov.au/websitedbs/D3310114.nsf/home/Quality:+The+ABS+Data+Quality+Framework

Braaksma B. and K. Zeelenberg, 2015, “Re-make/Re-model:” should big data change the modelling paradigm in official statistics? Statistical Journal of the IAOS 31(2), 1-10. https://doi.org/10.3233/SJI-150892

CBS (Statistics Netherlands), 2009, Checklist Quality of Statistical Outputs.http://ec.europa.eu/eurostat/documents/64157/4374310/43_CHECKLIST-QUALITY-STATISTICAL-OUTPUT-CBS-2009.pdf/2000653a-e8bc-4979-8e13-bb455b5ddc50

CBS (Statistics Netherlands), 2014, Quality Guidelines. https://www.cbs.nl/en-gb/background/2014/12/quality-guidelines-2014

CBS (Statistics Netherlands), 2017, Quality Declaration and Compliance with the Code of Practice. https://www.cbs.nl/en-gb/about-us/organisation/quality

Census Bureau, 2013, U.S. Census Bureau Statistical Quality Standards. https://www.census.gov/about/policies/quality/standards.html

ESS, 2015a, Quality Assurance Framework of the European Statistical System, 2nd edition. http://ec.europa.eu/eurostat/web/quality

ESS, 2015b, Peer Review Report on Compliance with the Code of Practice and the Coordination Role of the National Statistical Institute: Netherlands. http://ec.europa.eu/eurostat/web/quality/peer-reviews

ESS, 2017, European Statistics Code of Practice, 3rd edition. http://ec.europa.eu/eurostat/web/quality

ISO, 2015, Quality management systems – Requirements (ISO 9001:2015). https://www.iso.org/standard/62085.html Kuurstra, D. & Zeelenberg K., 2016, Statistical quality by design: certification, rules and culture. Paper presented at the European Conference on Quality in Official Statistics (Q2016), Madrid, 31 May-3 June 2016.

Smekens M., and Zeelenberg, K., 2015, Lean Six Sigma at Statistics Netherlands, Statistical Journal of the IAOS 31(4), 583-586. https://doi.org/10.3233/SJI-150930

Statistics Canada, 2017, Statistics Canada’s Quality Assurance Framework, 3rd edition. http://www.statcan.gc.ca/pub/12-586-x/4058339-eng.htm

Struijs, P., Braaksma, B., & Daas, P. J. H., 2014, Official statistics and big data. Big Data & Society, 1(1), pp. 1-6. https://doi.org/10.1177/2053951714538417

UN, 2012, Generic National Quality Assurance Framework. https://unstats.un.org/unsd/dnss/qualitynqaf/nqaf.aspx

Zeelenberg, K., 2015, Product and service innovation at Statistics Netherlands. Paper for the Seminar on Modernization of Statistical Production and Services and Managing for Efficiency at the 63rd plenary session of the Conference of European Statisticians, Geneva, 16 June 2015. http://www.unece.org/index.php?id=38920#jfmulticontent_c48917-3

Zeelenberg K., 2016, Methodological Challenges in Official Statistics. Proceedings of the 2016 International Methodology Symposium, Statistics Canada, Ottawa (Ontario, Canada). https://www.statcan.gc.ca/eng/conferences/symposium2016/program/