Latent grouped structures in panel data: a review

Pionati, Alessandro (2025): Latent grouped structures in panel data: a review.

Abstract

Latent group structures in panel data models are a new and powerful approach to deal with unobserved heterogeneity in a parsimonious way. This review, with a special focus on grouped structure in unobservable traits, first analyzes the limits and opportunities of Bonhomme and Manresa (2015a)’s Grouped Fixed Effects (GFE) estimator, also discussing the literature it contributed to create. A rich selection of models enhancing clustered heterogeneity at a slope level, starting from Su et al. (2016a), is then presented. A short section investigates how the applied literature has employed in practice the GFE. Finally, the GFE of Bonhomme et al. (2022) is presented in detail together with its limits and advantages.

• Item Type: MPRA Paper
• Language: English
• Keywords: Grouped Fixed Effects, Fixed Effects, Discrete Heterogeneity
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C10 - General; C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C23 - Panel Data Models ; Spatio-temporal Models

References

Abowd, J. M., Kramarz, F., and Margolis, D. N. (1999). High wage workers and high wage firms. Econometrica, 67(2):251-333.

Acemoglu, D., Johnson, S., Robinson, J. A., and Yared, P. (2008). Income and democracy. American economic review, 98(3):808-842.

Andersen, E. B. (1970). Asymptotic properties of conditional maximum-likelihood estimators. Journal of the Royal Statistical Society, Series B, 32:283-301.

Ando, T. and Bai, J. (2016). Panel data models with grouped factor structure under unknown group membership. Journal of Applied Econometrics, 31(1):163-191.

Ando, T. and Bai, J. (2023). Large-scale generalized linear models for longitudinal data with grouped patterns of unobserved heterogeneity. Journal of Business &Economic Statistics, 41(3):983-994.

Andrews, M. J., Gill, L., Schank, T., and Upward, R. (2008). High wage workers and low wage firms: negative assortative matching or limited mobility bias? Journal of the Royal Statistical Society Series A: Statistics in Society, 171(3):673-697.

Arellano, M. and Hahn, J. (2007). Understanding bias in nonlinear panel models: Some recent developments. Econometric Society Monographs, 43:381.

Bai, J. (2009). Panel data models with interactive fixed effects. Econometrica, 77(4):1229-1279.

Bai, J. and Ng, S. (2002). Determining the number of factors in approximate factor models. Econometrica, 70(1):191-221.

Bartolucci, F., Bellio, R., Salvan, A., and Sartori, N. (2016). Modified profile likelihood for fixed-effects panel data models. Econometric Reviews, 35(7):1271-1289.

Bartolucci, F. and Nigro, V. (2010). A dynamic model for binary panel data with unobserved heterogeneity admitting a root-n-consistent conditional estimator. Econometrica, 78(2):719-733.

Bartolucci, F. and Nigro, V. (2012). Pseudo conditional maximum likelihood estimation of the dynamic logit model for binary panel data. Journal of Econometrics, 170(1):102-116.

Bartolucci, F., Pigini, C., and Valentini, F. (2024). Mcmc conditional maximum likelihood for the two-way fixed-effects logit. Econometric Reviews, pages 1-26.

Bester, C. A. and Hansen, C. (2009). A penalty function approach to bias reduction in nonlinear panel models with fixed effects. Journal of Business & Economic Statistics, 27(2):131-148.

Bester, C. A. and Hansen, C. B. (2016). Grouped effects estimators in fixed effects models. Journal of Econometrics, 190(1):197-208

Bonhomme, S., Holzheu, K., Lamadon, T., Manresa, E., Mogstad, M., and Setzler, B. (2023).How much should we trust estimates of firm effects and worker sorting? Journal of Labor Economics, 41(2):291-322.

Bonhomme, S., Lamadon, T., and Manresa, E. (2019). A distributional framework for matched employer employee data. Econometrica, 87(3):699-739.

Bonhomme, S., Lamadon, T., and Manresa, E. (2022). Discretizing unobserved heterogeneity. Econometrica, 90(2):625-643.

Bonhomme, S. and Manresa, E. (2015a). Grouped patterns of heterogeneity in panel data. Econometrica, 83(3):1147-1184.

Bonhomme, S. and Manresa, E. (2015b). Supplement to "grouped patterns of heterogeneity in panel data". Econometrica, 83(3).

Bryant, P. and Williamson, J. A. (1978). Asymptotic behaviour of classification maximum likelihood estimates. Biometrika, 65(2):273-281.

Carro, J. M. (2007). Estimating dynamic panel data discrete choice models with fixed effects. Journal of Econometrics, 140(2):503-528.

Chamberlain, G. (1980). Analysis of covariance with qualitative data. The Review of Economic Studies, 47:225-238.

Charbonneau, K. B. (2017). Multiple fixed effects in binary response panel data models. The Econometrics Journal, 20(3):S1-S13.

Chen, M., Fernandez-Val, I., and Weidner, M. (2021). Nonlinear factor models for network and panel data. Journal of Econometrics, 220(2):296-324.

Chetverikov, D. and Manresa, E. (2022). Spectral and post-spectral estimators for grouped panel data models. arXiv preprint arXiv:2212.13324.

Dhaene, G. and Jochmans, K. (2015). Split-panel jackknife estimation of fixed-effect models. The Review of Economic Studies, 82(3):991-1030.

Dzemski, A. and Okui, R. (2021). Convergence rate of estimators of clustered panel models with misclassification. Economics Letters, 203:109844.

Fernandez-Val, I. and Weidner, M. (2016). Individual and time effects in nonlinear panel models with large n, t. Journal of Econometrics, 192(1):291-312.

Freeman, H. and Weidner, M. (2023). Linear panel regressions with two-way unobserved heterogeneity. Journal of Econometrics, 237(1):105498.

Grunewald, N., Klasen, S., Martinez-Zarzoso, f. I., and Muris, C. (2017). The trade-off between income inequality and carbon dioxide emissions. Ecological Economics, 142:249-256.

Gu, J. and Volgushev, S. (2019). Panel data quantile regression with grouped fixed effects. Journal of Econometrics, 213(1):68-91.

Hahn, J. and Kuersteiner, G. (2011). Bias reduction for dynamic nonlinear panel models with fixed effects. Econometric Theory, 27(6):1152-1191.

Hahn, J. and Moon, H. R. (2010). Panel data models with finite number of multiple equilibria. Econometric Theory, 26(3):863-881.

Hahn, J. and Newey, W. (2004). Jackknife and analytical bias reduction for nonlinear panel models. Econometrica, 72:1295-1319.

Hsiao, C. (2018). Panel models with interactive effects. Journal of Econometrics, 206(2):645-673.

Hsiao, C. (2022). Analysis of panel data. Number 64. Cambridge university press.

Janys, L. and Siflinger, B. (2024). Mental health and abortions among young women: Time-varying unobserved heterogeneity, health behaviors, and risky decisions. Journal of Econometrics, 238(1):105580.

Johar, M., Johnston, D. W., Shields, M. A., Siminski, P., and Stavrunova, O. (2022). The economic impacts of direct natural disaster exposure. Journal of Economic Behavior & Organization, 196:26-39.

Kapetanios, G., Serlenga, L., and Shin, Y. (2023). Testing for correlation between the regressors and factor loadings in heterogeneous panels with interactive effects. Empirical Economics, 1-49.

Kline, P., Saggio, R., and Solvsten, M. (2020). Leave-out estimation of variance components. Econometrica, 88(5):1859-1898.

Lancaster, T. (2000). The incidental parameter problem since 1948. Journal of econometrics, 95(2):391-413.

Lentz, R., Piyapromdee, S., Robin, J.-M., et al. (2018). On worker and firm heterogeneity in wages and employment mobility: Evidence from danish register data. PIER Discussion Papers, 91.

Lewis, D. J., Melcangi, D., Pilossoph, L., and Toner-Rodgers, A. (2023). Approximating grouped fixed effects estimation via fuzzy clustering regression. Journal of Applied Econometrics, 38(7):1077-1084.

Li, H., Lindsay, B. G., and Waterman, R. P. (2003). Efficiency of projected score methods in rectangular array asymptotics. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 65(1):191-208.

Lin, C.C. and Ng, S. (2012). Estimation of panel data models with parameter heterogeneity when group membership is unknown. Journal of Econometric Methods, 1(1):42-55.

Liu, R., Shang, Z., Zhang, Y., and Zhou, Q. (2020). Identification and estimation in panel models with overspecified number of groups. Journal of Econometrics, 215(2):574-590.

Loyo, J. A. and Boot, T. (2024). Grouped heterogeneity in linear panel data models with heterogeneous error variances. Journal of Business & Economic Statistics, pages 1-13.

Lu, X. and Su, L. (2017). Determining the number of groups in latent panel structures with an application to income and democracy. Quantitative Economics, 8(3):729-760.

Lumsdaine, R. L., Okui, R., and Wang, W. (2023). Estimation of panel group structure models with structural breaks in group memberships and coefficients. Journal of Econometrics, 233(1):45-65.

MacLahlan, G. and Peel, D. (2000). Finite mixture models. Wiley Series in Probability and Statistics. John Wiley & Sons, Inc.

Miao, K., Su, L., andWang, W. (2020). Panel threshold regressions with latent group structures. Journal of Econometrics, 214(2):451-481.

Moon, H. R. and Weidner, M. (2015). Linear regression for panel with unknown number of factors as interactive fixed effects. Econometrica, 83(4):1543-1579.

Moon, H. R. and Weidner, M. (2017). Dynamic linear panel regression models with interactive fixed effects. Econometric Theory, 33(1):158-195.

Moon, H. R. and Weidner, M. (2023). Nuclear norm regularized estimation of panel regression models.

Mugnier, M. (2022). A simple and computationally trivial estimator for grouped fixed effects models. arXiv preprint arXiv:2203.08879.

Mugnier, M. (2023). A simple and computationally trivial estimator for grouped fixed effects models. arXiv preprint arXiv: 2203.08879.

Neyman, J. and Scott, E. L. (1948). Consistent estimates based on partially consistent observations. Econometrica, pages 1-32.

Oberlander, L., Disdier, A.-C., and Etilé, F. (2017). Globalisation and national trends in nutrition and health: A grouped fixed-effects approach to intercountry heterogeneity. Health economics, 26(9):1146-1161.

Okui, R. and Wang, W. (2021). Heterogeneous structural breaks in panel data models. Journal of Econometrics, 220(2):447-473.

Pesaran, M. H. (2006). Estimation and inference in large heterogeneous panels with a multifactor error structure. Econometrica, 74(4):967-1012.

Pesaran, M. H. and Yamagata, T. (2008). Testing slope homogeneity in large panels. Journal of econometrics, 142(1):50-93.

Pollard, D. (1981). Strong consistency of k-means clustering. The annals of statistics, pages 135-140.

Sarafidis, V. and Weber, N. (2015). A partially heterogeneous framework for analyzing panel data. Oxford Bulletin of Economics and Statistics, 77(2):274-296.

Su, L. and Ju, G. (2018). Identifying latent grouped patterns in panel data models with interactive fixed effects. Journal of Econometrics, 206(2):554-573.

Su, L., Shi, Z., and Phillips, P. C. B. (2016a). Identifying latent structures in panel data. Econometrica, 84(6):2215-2264.

Su, L., Shi, Z., and Phillips, P. C. B. (2016b). Supplement to "Identifying latent structures in panel data". Econometrica, 84(6):2215-2264.

Sun, Y. X. (2005). Estimation and inference in panel structure models.

Valentini, F., Pigini, C., and Bartolucci, F. (2023). Advances in maximum likelihood estimation of fixed-effects binary panel data models. In Trends and Challenges in Categorical Data Analysis: Statistical Modelling and Interpretation, pages 275-315. Springer.

Vogt, M. and Linton, O. (2017). Classification of non-parametric regression functions in longitudinal data models. Journal of the Royal Statistical Society Series B: Statistical Methodology, 79(1):5-27.

Vogt, M. and Linton, O. (2020). Multiscale clustering of nonparametric regression curves. Journal of Econometrics, 216(1):305-325.

Wang, W., Phillips, P. C., and Su, L. (2018). Homogeneity pursuit in panel data models: Theory and application. Journal of Applied Econometrics, 33(6):797-815.

Wooldridge, J. M. (2021). Two-way fixed effects, the two-way mundlak regression, and difference in differences estimators. Available at SSRN 3906345.