Microdata Adjustment by the Minimum Information Loss Principle

Merz, Joachim (1994): Microdata Adjustment by the Minimum Information Loss Principle.

Abstract

Microdata have become increasingly important for economic and social analyses. One striking problem with almost any practical analysis of microdata, microdata as a singular cross or longitudinal sample or within (static) microsimulation, is to achieve representative results. In this study a consistent solution of the microdata adjustment problem - that is to achieve representative results by re-weighting microdata to fit aggregate control data - is presented based on the Minimum Information Loss (MIL) principle. Based on information theory this principle satisfies the desired positivity constraint on the weighting factors to be computed. For the consistent solution which simultaneously adjusts hierarchical microdata (e.g. household and personal information), a fast numerical solution by a specific modified Newton-Raphson (MN) procedure with a global exponential approximation is proposed. Practical experiences for large microdata sets in a pension reform analysis with e.g. more than 60.000 households and 240 restrictions simultaneously to be achieved within the Sfb 3 microsimulation model show that this MN procedure was able to rather largely reduce the computional expenses by 75%. The available efficient PC-computer program ADJUST is also succesfully applied in a described microsimulation analyses of the recent 1990 German tax reform investigating the impacts on market and non-market labour supply within the formal and informal economy, and in a recent firm microsimulation analysion explaining factors of successful firms in the German engineering industry.

• Item Type: MPRA Paper
• Language: English
• Keywords: Microdata Adjustment, Microanalyses, Microsimulation, Minimum Information Loss, Modified Newton- Raphson Algorithm, PC program package ADJUST
• Subjects: C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology ; Computer Programs > C81 - Methodology for Collecting, Estimating, and Organizing Microeconomic Data ; Data Access; C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology ; Computer Programs > C80 - General

References

Beebout, H. (1977), "Microsimulation as a Policy Tool: The MATH Model", Policy Analysis Series No. 14, Mathematica Policy Research Inc., Washington, D.C. Beebout, H. (1986), "Evaluating Reagan administration social program changes: Two applications of MATH", in: G.H. Orcutt, J. Merz and H. Quinke, eds., Microanalytic simulation models to support social and financial policy, North Holland, Amsterdam, 83-97. Bishop, Y.M. and S.E. Fienberg (1969), "Incomplete Two-dimensional Contingency Tables", Biometrics, 25, 383-400. Bishop, Y.M. and S.E. Fienberg (1975), Discrete Multivariate Analysis: Theory and Practice, Cambridge, Mass.: MIT-Press. Bridges, B. jr. and M.P. Johnston (1976), "Estimation of social security taxes on the March current population survey", Studies of Income Distribution, No.4 (Social Security Administration, Office of Research and Statistics, US Department of Health, Education and Welfare, Washington, D.C. Byron, R.P, (1978), "The Estimation of Large Social Account Matrices", Journal of the Royal Statistical Society, Series A, 141, Part 3, 359-367 Devine, J. and R. Wertheimer (1981), "Aging Techniques Used by the Major Microsimulation Models", Working paper 1453-01, The Urban Institute Washington, D.C. Dworschak, F. and J. Merz (1982), "Die Ausgangsdatenbasis des Sfb 3-Mikrosimulationsmodells - Verknüpfung von Mikrodaten für die Simulation alternativer Rentenreformmodelle", Sfb 3- Arbeitspapier Nr. 78, Frankfurt/Mannheim. Finke, R. and H. Theil (1984), "An Extended Version of Minimum Information Estimation of Allocation Models", Economics Letters, 15, 229-233. Frank, M. and P. Wolfe (1956), "An Algorithm for Quadratic Programming", Naval Research Logistics Quarterly, 3, 95-100. Galler, H.P. (1977), "Die Ermittlung eines konsistenten Hochrechnungsrahmens für das IMDAF-69", SPESArbeitspapier Nr. 77, Frankfurt/Mannheim. Galler, H.P. and G. Wagner (1986), "The Microsimulation Model of the Sfb 3 for the Analysis of Economic and Social Policies", in: G.H. Orcutt, J. Merz and H. Quinke, eds., Microanalytic simulation models to support social and financial policy, North Holland: Amsterdam, 227-247. Goldfeld, S.M., Quandt, R. and H.F. Trotter (1966), "Maximization by Quadratic-Hill-Climbing", Econometrica, 34, 3, 541-551. Greenberger, M., Crenson, M.A. and B.C. Crissey (1976), Models in the Policy Process, New York. Hain, W. and Helberger, Chr. (1986), "Longitudinal Microsimulation of Life Income", in: G.H. Orcutt, J. Merz and H. Quinke, eds., Microanalytic simulation models to support social and financial policy, North Holland, Amsterdam, 251-270. Haveman, R. and K. Hollenbeck (eds.) (1980), Microeconomic Simulation Models for Public Policy Analysis , New York, Academic Press. Hildreth, C. (1957), "A Quadratic Programming Procedure", Naval Research Logistics Quarterly, 14, 79-85. Hollenbeck, K. (1976), "An Algorithm for Adjusting n-dimensional Tabular Data to Conform to General Linear Constraints", Proc. Am. Stat. Ass., 402-405. Houthakker, H.S. (1960), "The Capacity Method of Quadratic Programming", Econometrica, 28, 62-87. Ireland, C.T. and S. Kullback (1968), "Contingency Tables with Given Marginals", Biometrika, 55, 179-188. Jaynes, E.T. (1957), "Information Theory and Statistical Mechanics I", Physic Review, Vol. 106, 620-630. Krupp, H.-J., Galler, H.P., Grohmann, H., Hauser, R. and G. Wagner (eds.) (1981), Alternativen der Rentenreform '84, Frankfurt/New York. Merz, J. (1980), "Prognosegüte und Spektraleigenschaften ökonomischer Modelle", in: S. Stöppler (ed.), Dynamische ökonomische Systeme - Analyse und Steuerung, 2nd edition, Gabler, Wiesbaden, 31- 66. Merz, J. (1983a), "Die konsistente Hochrechnung von Mikrodaten nach dem Prinzip des minimalen Informationsverlustes", Allgemeines Statistisches Archiv, 76, 4, 342-366. Merz, J. (1983b), "The Adjustment of Microdata Using the Kalman Filtering Procedure and Optimal Control Theory", Sfb 3-Working Paper No. 122, Frankfurt/Mannheim. Merz, J. (1985), "Ein modifiziertes Newton-Verfahren zur Lösung des Hochrechnungsproblems nach dem Prinzip des minimalen Informationsverlustes", Computing, 35, 51-61. Merz, J. (1986), "Structural Adjustment in Static and Dynamic Microsimulation Models", in: G.H. Orcutt, J. Merz and H. Quinke, eds., Microanalytic simulation models to support social and financial policy, North Holland: Amsterdam, 423-446. Merz, J. (1989), Markt- und nichtmarktmäßige Aktivitäten privater Haushalte - Theoretischer Ansatz, repräsentative Mikrodaten, mikroökonometrische Analyse und Mikrosimulation wirtschafts- und sozialpolitischer Maßnahmen für die Bundesrepublik Deutschland, Habilitation (Frankfurt). Merz, J. (1990), "The 1990 German tax reform - Microsimulation of time allocation effects in the formal and informal economy", in: J.K. Brunner and H.G. Petersen, eds., Simulation models in tax and transfer policy, Campus: Frankfurt/M., New York. Merz, J. (1991a), "Microsimulation - A survey of principles, developments and applications", International Journal of Forecasting, 7, 77-104. Merz, J. (1991b), "Wirkungen der Steuerreform 1990 auf Nebenerwerb und Schwarzarbeit", in: D. Döring and P.B. Spahn (eds.), Steuerreform als gesellschaftliche Aufgabe der neunziger Jahre, Duncker und Humblot, Berlin, 283-303. Merz, J. (1993a), "Microsimulation as an instrument to evaluate economic and social programmes", Proceedings of the 49th Session of the International Statistical Institute (Firence, Italy, September 3) also available as FFB-Discussion Paper No. 5, Forschungsinstitut Freie Berufe (FFB), Department of Economics and Social Sciences, University of Lüneburg, Lüneburg, Germany. Merz, J. (1993b), "ADJUST - A program package for the adjustment of microdata by the minimum information loss principle: Program Manual", FFB-Documentation No. 1e, Forschungsinstitut Freie Berufe (FFB), Department of Economics and Social Sciences, University of Lüneburg, Lüneburg, Germany. Merz, J. (1993c), "Market and non-market labour supply and recent German tax reform impacts - Behavioural response in a combined dynamic and static microsimulation model", Proceedings of the Special International Association for Research and Wealth Conference on Microsimulation and Public Policy, University of Canberra, Canberra, Australia, December 6-10. Merz, J. (1994a), "Statisches Sonderforschungsbereich 3 - Mikrosimulationsmodell: Mainframe und PCVersion", R. Hauser, N. Ott and G. Wagner, eds., Deutsche Forschungsgemeinschaft: Mikroanalytische Grundlagen der Gesellschaftspolitik - Erhebungsverfahren, Analysemethoden und Mikrosimulation, Akademie Verlag: Berlin. Merz, J. (1994b), "Microsimulation - A survey of methods and applications for analyzing economic and social policy", FFB-Discussion Paper No. 9, Forschungsinstitut Freie Berufe (FFB), Department of Economics and Social Sciences, University of Lüneburg, Lüneburg, Germany. Merz, J. and K.G. Wolff (1993a), "Die Sfb 3 - Nebenerwerbstätigkeitsumfrage und ihre Erweiterungen als repräsentative Mikrodatenbasis zur Analyse markt- und nichtmarktmäßiger Aktivitäten privater Haushalte", in: R. Hauser, N. Ott and G. Wagner (eds.), Mikroanalytische Grundlagen der Gesellschaftspolitik - Problemem und ausgewählte Lösungsansätze, Akademie Verlag, Weinheim. Merz, J. and K.G. Wolff (1993b), "The shadow economy: Illicit work and household production - A microanalysis of West Germany, The Review of Income and Wealth, Vol. 39, No. 2, 177-194. Merz, J. und P. Buxmann (1990), "MICSIM: A PC-Microsimulationmodel for research and teaching realized with C and the relational data base system ORACLE", Sfb 3-Arbeitspapier Nr. 316, Sonderforschungsbereich 3, Mikroanalytische Grundlagen der Gesellschaftspolitik, Frankfurt/M., Mannheim. Mosteller, F. (1968), "Association and Estimation in Contingency Tables", Journal of the American Statistical Association, 63, 1-28. Oh, H.L. and F. Scheuren (1980), "Multivariate Raking Ratio Estimation in the 1973 Exact Match Study", Studies from Interagency Data Linkages Report No. 11, U.S. Department of Health, Education and Welfare, Washington, D.C. Orcutt, G.H. (1957), "A New Type of Socio-Economic Systems", The Review of Economics and Statistics, 58, 773-797. Orcutt, G.H., Caldwell, S. and R. Wertheimer II. (1976), Policy Exploration Through Microanalytic Simulation, Washington, D.C. Orcutt, G.H., Quinke, H. and J,. Merz (eds.) (1986), Microanalytic simulation models to support social and financial policy, North Holland. Amsterdam. Ortega, J.M. and W.C. Rheinboldt (1970), "Iterative Solution of Non-linear Equations in Several Va riables", Computer Science and Applied Mathematics, New York/London. Rabinowitz, P. (ed.) (1970), Numerical Methods for Nonlinear Algebraic Equations, London/New York/Paris. Shore, J.E. and R.W. Johnson (1980), "Axiomatic Derivation of the Principle of Maximum Entropy and the Principle of Minimum Cross-Entropy", IEEE Trancsactions on Information Theory, Vol. IT-26, 28- 37. Spahn, P.B., H.P. Galler, H. Kaiser, TH. Kassella and J. Merz (1992), Mikrosimulation in der Steuerpolitik, Heidelberg. Statistisches Bundesamt (1960), Stichproben in der amtlichen Statistik, Stuttgart/Mainz. Stone, R. (1976), "The Development of Economic Data Systems", Pyatt, G. et al. (eds.), Social Accounting for Development Planning with Special Reference to Sri Lanka, Cambridge. Stöppler, S. (ed.) (1980), Dynamische ökonomische Systeme - Analyse und Steuerung, 2nd edition, Gabler, Wiesbaden. Sulvetta, M.B. (1976) "An Analyst's Guide to TRIM - the Transfer Income Model' , Urban Institute Paper 99603, The Urban Institute, Washington, D.C. Theil, H. (1967), Economics and Information Theory, North Holland, Amsterdam. Theil, H. (1972), Statistical Decomposition Analysis, North Holland, Amsterdam. Theil, H., R. Finke and L.R. Flood (1984), "Minimum Information Estimation of Allocation Models", Economics Letters, 15, 251-256. Wauschkuhn, U. (1982), Anpassung von Stichproben und n-dimensionalen Tabellen an Randbedingungen, München/Wien. Webb, R. and G. Chow (1978), "TRIM user's guide", Urban Institute Working Paper, The Urban Institute, Washington D.C. Webb, R.L., Michel, R.C. and A.B. Bergsman (1990), "The historical development of the Transfer Income Model (TRIM2)", in: G.H. Lewis and R.C. Michel, eds., 1990, Microsimulation techniques for tax and transfer analysis, Urban Institute Press, Washington D.C. Widmaier, U., Niggemann, H. and J. Merz (1994), "What makes the difference between unsuccessful and successful firms in the German mechanical engineering industry? A microsimulation approach using data from the NIFA-panel", Paper presented at the XIIIth World Congress of Sociology, 18- 23 July 1994, Bielefeld, Germany Wolfson M. (1990), "Income tax/transfer integration - Policy implications and analytical challenges", J.K. Brunner and H.G. Petersen (eds), Simulation models in tax and transfer policy, Campus, Frankfurt/M., New York. Wyscarver, R.A. (1980) "The Treasury Individual Income Tax Simulation Model", Office of the Secretary of the Treasury, Office of Tax Analysis, Washington D.C.