Counterfactual Reconciliation: Incorporating Aggregation Constraints For More Accurate Causal Effect Estimates

Cengiz, Doruk and Tekgüç, Hasan (2022): Counterfactual Reconciliation: Incorporating Aggregation Constraints For More Accurate Causal Effect Estimates.

Abstract

We extend the scope of the forecast reconciliation literature and use its tools in the context of causal inference. Researchers are interested in both the average treatment effect on the treated and treatment effect heterogeneity. We show that ex post correction of the counterfactual estimates using the aggregation constraints that stem from the hierarchical or grouped structure of the data is likely to yield more accurate estimates. Building on the geometric interpretation of forecast reconciliation, we provide additional insights into the exact factors determining the size of the accuracy improvement due to the reconciliation. We experiment with U.S. GDP and employment data. We find that the reconciled treatment effect estimates tend to be closer to the truth than the original (base) counterfactual estimates even in cases where the aggregation constraints are non-linear. Consistent with our theoretical expectations, improvement is greater when machine learning methods are used.

• Item Type: MPRA Paper
• Language: English
• Keywords: Forecast Reconciliation; Non-linear Constraints; Causal Machine Learning Methods; Counterfactual Estimation; Difference-in-Differences
• Subjects: C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods

References

Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association, 105(490), 493-505.

Acemoglu, D., Naidu, S., Restrepo, P., & Robinson, J. A. (2019). Democracy does cause growth. Journal of political economy, 127(1), 47-100.

Allegretto, S., Dube, A., Reich, M., & Zipperer, B. (2017). Credible research designs for minimum wage studies: A response to Neumark, Salas, and Wascher. ILR Review, 70(3), 559-592.

Almeida, V., Ribeiro, R., & Gama, J. (2016). Hierarchical time series forecast in electrical grids. In K.J. Kim and N. Joukov (eds.), Information Science and Applications (ICISA) 2016. Singapore: Springer, pp. 995-1005.

Angrist, Joshua D., and Jörn-Steffen Pischke. (2008). Mostly Harmless Econometrics: An Empiricist’s Companion, Princeton, NJ: Princeton University Press.

Athanasopoulos, G., Ahmed, R. A., & Hyndman, R. J. (2009). Hierarchical forecasts for Australian domestic tourism. International Journal of Forecasting, 25(1), 146-166.

Athey, S., & Imbens, G. W. (2017). The state of applied econometrics: Causality and policy evaluation. Journal of Economic Perspectives, 31(2), 3-32.

Athey, S., Bayati, M., Doudchenko, N., Imbens, G., & Khosravi, K. (2021). Matrix completion methods for causal panel data models. Journal of the American Statistical Association. https://doi.org/10.1080/01621459.2021.1891924

Bertrand, M., Duflo, E., & Mullainathan, S. (2004). How much should we trust differences-in-differences estimates? The Quarterly Journal of Economics, 119(1), 249-275.

Cunningham, S. (2021). Causal Inference. The Mixtape. New Haven: Yale University Press.

De Chaisemartin, C., & D'Haultfoeuille, X. (2022a). Difference-in-differences estimators of intertemporal treatment effects (No. w29873). National Bureau of Economic Research.

De Chaisemartin, C., & D'Haultfoeuille, X. (2022b). Two-way fixed effects and differences-in-differences with heterogeneous treatment effects: A survey (No. w29691). National Bureau of Economic Research.

Doudchenko, N., & Imbens, G. W. (2016). Balancing, regression, difference-in-differences and synthetic control methods: A synthesis (No. w22791). National Bureau of Economic Research. https://www.nber.org/system/files/working_papers/w22791/w22791.pdf

van Erven T., Cugliari J. (2015). Game-Theoretically Optimal Reconciliation of Contemporaneous Hierarchical Time Series Forecasts. In: Antoniadis A., Poggi JM., Brossat X. (eds) Modeling and Stochastic Learning for Forecasting in High Dimensions. Lecture Notes in Statistics, vol 217. Springer, Cham. https://doi.org/10.1007/978-3-319-18732-7_15

Gay, D. M. (1990). Usage summary for selected optimization routines. Computing science technical report, 153, 1-21.

Gamakumara, P. (2020). Probabilistic forecast reconciliation. Doctoral dissertation, PhD thesis. Monash University.

Gechert, S., Havranek, T., Irsova, Z., & Kolcunova, D. (2021). Measuring capital-labor substitution: The importance of method choices and publication bias. Review of Economic Dynamics. https://doi.org/10.1016/j.red.2021.05.003

Gobillon, L., & Magnac, T. (2016). Regional policy evaluation: Interactive fixed effects and synthetic controls. Review of Economics and Statistics, 98(3), 535-551.

Hastie, T., Montanari, A., Rosset, S., & Tibshirani, R. J. (2019). Surprises in high-dimensional ridgeless least squares interpolation. arXiv preprint arXiv:1903.08560. https://arxiv.org/abs/1903.08560

Hollyman, R., Petropoulos, F., & Tipping, M. E. (2021). Understanding forecast reconciliation. European Journal of Operational Research, 294(1), 149-160.

Hyndman, R. J., Ahmed, R. A., Athanasopoulos, G., & Shang, H. L. (2011). Optimal combination forecasts for hierarchical time series. Computational Statistics & Data Analysis, 55(9), 2579-2589.

Hyndman, R. J., & Athanasopoulos, G. (2018). Forecasting: principles and practice, second ed., OTexts, Melbourne, Australia. https://otexts.com/fpp2/index.html

Imbens, G. W. (2020). Potential outcome and directed acyclic graph approaches to causality: Relevance for empirical practice in economics. Journal of Economic Literature, 58(4), 1129-79.

Imbens, G. W., & Rubin, D. B. (2015). Causal inference in statistics, social, and biomedical sciences. Cambridge University Press.

LaLonde, R. J. (1986). Evaluating the econometric evaluations of training programs with experimental data. The American Economic Review, 76(4) 604-620. Lee, D. S., McCrary, J., Moreira, M. J., & Porter, J. R. (2021). Valid t-ratio Inference for IV (No. w29124). National Bureau of Economic Research.

Liu, L., Wang, Y., & Xu, Y. (2021). A Practical Guide to Counterfactual Estimators for Causal Inference with Time-Series Cross-Sectional Data. https://arxiv.org/pdf/2107.00856.pdf

Mankiw, N. G., Romer, D., & Weil, D. N. (1992). A contribution to the empirics of economic growth. The Quarterly Journal of Economics, 107(2), 407-437.

Nickell, S. (1981). Biases in dynamic models with fixed effects. Econometrica: Journal of the econometric society, 1417-1426.

Panagiotelis, A., Athanasopoulos, G., Gamakumara, P., & Hyndman, R. J. (2021). Forecast reconciliation: A geometric view with new insights on bias correction. International Journal of Forecasting, 37(1), 343-359.

Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688.

Spiliotis, E., Abolghasemi, M., Hyndman, R. J., Petropoulos, F., & Assimakopoulos, V. (2021). Hierarchical forecast reconciliation with machine learning. Applied Soft Computing. https://doi.org/10.1016/j.asoc.2021.107756

U.S. Bureau of Labor Statistics (2021a). Quarterly Census of Employment and Wages. Accessed in June 2021: https://www.bls.gov/cew/

U.S. Bureau of Labor Statistics (2021b). Quarterly Census of Employment and Wages: Handbook of Methods. https://www.bls.gov/opub/hom/cew/pdf/cew.pdf

U.S. Bureau Economic Analysis (2021a). GDP by State. Accessed in June 2021: https://www.bea.gov/data/gdp/gdp-state.

U.S. Bureau Economic Analysis (2021b). SQGDP2 series are available at: https://apps.bea.gov/iTable/iTable.cfm?reqid=70&step=1&isuri=1&acrdn=1#reqid=70&step=1&isuri=1&acrdn=1

Varadhan, R. (2015). Alabama: Constrained nonlinear optimization. R package version 2015.3-1.

Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., et al. (2020). SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17, 261–272. doi:10.1038/s41592-019-0686-2

Wickramasuriya, S. L., Athanasopoulos, G., & Hyndman, R. J. (2019). Optimal forecast reconciliation for hierarchical and grouped time series through trace minimization. Journal of the American Statistical Association, 114(526), 804-819.

Xu, Y. (2017). Generalized synthetic control method: Causal inference with interactive fixed effects models. Political Analysis 25(1): 57–76.