The Comparison of Methods for IndividualTreatment Effect Detection

Semenova, Daria and Temirkaeva, Maria (2019): The Comparison of Methods for IndividualTreatment Effect Detection. Published in: CEUR Workshop Proceedings , Vol. 2479, (26 September 2019): pp. 46-56.

Abstract

Today, treatment effect estimation at the individual level isa vital problem in many areas of science and business. For example, inmarketing, estimates of the treatment effect are used to select the mostefficient promo-mechanics; in medicine, individual treatment effects areused to determine the optimal dose of medication for each patient and soon. At the same time, the question on choosing the best method, i.e., themethod that ensures the smallest predictive error (for instance, RMSE)or the highest total (average) value of the effect, remains open. Accord-ingly, in this paper we compare the effectiveness of machine learningmethods for estimation of individual treatment effects. The comparisonis performed on the Criteo Uplift Modeling Dataset. In this paper weshow that the combination of the Logistic Regression method and theDifference Score method as well as Uplift Random Forest method pro-vide the best correctness of Individual Treatment Effect prediction onthe top 30% observations of the test dataset.

• Item Type: MPRA Paper
• Language: English
• Keywords: Individual Treatment Effect; ITE; Machine Learning; Random Forest; XGBoost; SVM·Random; Experiments; A/B testing; Uplift Random Forest
• Subjects: C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods and Methodology: General > C10 - General; M - Business Administration and Business Economics ; Marketing ; Accounting ; Personnel Economics > M3 - Marketing and Advertising > M30 - General

References

Breiman, L.: Random forests. Machine learning45(1), 5–32 (2001)

Buzmakov, A.: Machine learning for subgroup discovery under treatment effect.arXiv preprint arXiv:1902.10327 (2019)

Chen, T., Guestrin, C.: Xgboost: A scalable tree boosting system. In: Proceedingsof the 22nd acm sigkdd international conference on knowledge discovery and datamining. pp. 785–794. ACM (2016)

Diemert Eustache, Betlei Artem, Renaudin, C., Massih-Reza, A.: A large scalebenchmark for uplift modeling. In: Proceedings of the AdKDD and TargetAdWorkshop, KDD, London,United Kingdom, August, 20, 2018. ACM (2018)

Guelman, L.: Optimal personalized treatment learning models with insurance ap-plications (2015)

Guelman, L., Guill ́en, M., P ́erez-Mar ́ın, A.M.: Uplift random forests. Cyberneticsand Systems46(3-4), 230–248 (2015)

Gutierrez, P., G ́erardy, J.Y.: Causal inference and uplift modelling: A review ofthe literature. In: International Conference on Predictive Applications and APIs.pp. 1–13 (2017)

Hansotia, B., Rukstales, B.: Incremental value modeling. Journal of InteractiveMarketing16(3), 35 (2002)

Hansotia, B.J., Rukstales, B.: Direct marketing for multichannel retailers: Issues,challenges and solutions. Journal of Database Marketing & Customer StrategyManagement9(3), 259–266 (2002)

Jaroszewicz, S.S.M., Rzepakowski, P.: Ensemble methods for uplift modeling. DataMin. Knowl. Discov29(6), 1531–1559 (2015)

Jaskowski, M., Jaroszewicz, S.: Uplift modeling for clinical trial data. In: ICMLWorkshop on Clinical Data Analysis (2012)

Lo, V.S.: The true lift model: a novel data mining approach to response modelingin database marketing. ACM SIGKDD Explorations Newsletter4(2), 78–86 (2002)

Manahan, C.: A proportional hazards approach to campaign list selection. SASUser Group International (SUGI) 30 Proceedings (2005)

Rzepakowski, P., Jaroszewicz, S.: Decision trees for uplift modeling with single andmultiple treatments. Knowledge and Information Systems32(2), 303–327 (2012)

Tian, L., Alizadeh, A.A., Gentles, A.J., Tibshirani, R.: A simple method for esti-mating interactions between a treatment and a large number of covariates. Journalof the American Statistical Association109(508), 1517–1532 (2014)

Weisberg, H.I., Pontes, V.P.: Estimation of individual causal effects (April 2014),http://www.freepatentsonline.com/8688610.html

Weisberg, H.I., Pontes, V.P.: Post hoc subgroups in clinical trials: Anathema oranalytics? Clinical trials12(4), 357–364 (2015)