Statistical analysis of heaped duration data

Petoussis, Kos and Gill, Richard and Zeelenberg, Kees (1997): Statistical analysis of heaped duration data.

Abstract

This paper shows how heaping of duration data, e.g. caused by rounding due to memory effects, can be analyzed. If the data are heaped Cox's partial likelihood approach, which is often used in survival analysis, is no longer appropriate. We show how this problem can be overcome by considering the problem as a missing data problem. A variant of Cox's Proportional Hazard Model is constructed that takes heaping into account, and is estimated by maximum likelihood using the EM algorithm, with many nuisance parameters, simultaneously for all parameters. Ingredients of our method are application of the EM algorithm, Cox regression and nonparametric maximum likelihood calculation with `predicted' data in each M step. An example from practice, where jackknife is used to estimate the variances, illustrates the power of the new methodology.

• Item Type: MPRA Paper
• Language: English
• Keywords: heaping; duration data; survival analysis; Proportional Hazard Model; profile likelihood; EM algorithm
• Subjects: C - Mathematical and Quantitative Methods > C2 - Single Equation Models ; Single Variables > C23 - Panel Data Models ; Spatio-temporal Models; C - Mathematical and Quantitative Methods > C4 - Econometric and Statistical Methods: Special Topics > C41 - Duration Analysis ; Optimal Timing Strategies; J - Labor and Demographic Economics > J6 - Mobility, Unemployment, Vacancies, and Immigrant Workers > J64 - Unemployment: Models, Duration, Incidence, and Job Search

References

CBS, 1991. Socio-economic panel survey, contents, concept and organization, 1991, SDU/Publishers, the Hague.

Cox, D.R. and D.V. Hinkley, 1978. Problems and solutions in theoretical statistics (Chapman & Hall).

Cox, D.R., 1975. Partial likelihood. Biometrika 62 p. 269.

Dempster, A.P., N.M. Laird and D.B. Rubin, 1976. Maximum Likelihood from Incomplete Data via the EM algorithm. Journal of Royal Statistical Society B 19, pp.1-38.

Gorter D. and E. Hoogteijling, 1990a. Determinanten van werkloosheidsduur, 1985-1987. Internal report (Department of Statistical Methods, Statistics Netherlands, Voorburg).

Gorter D. and E. Hoogteijling, 1990b. Duur van het zoeken naar werk. Report (Department of Statistical Methods, Statistics Netherlands, Voorburg).

Heijtjan, D.F. and D.B. Rubin, 1991. Ignorability and coarse data. Annals of Statistics 19, pp.2244-53.

Heijtjan, D.F. and D.B. Rubin, 1990. Inference from coarse data via multiple imputation with application to age heaping. Journal of the American Statistics Association 85, pp.304-14.

Johansen, S., 1983. An extension of Cox's regression model. International Statistical Review, 51, pp. 165-174.

Kalbfleisch, J.G., and R.L. Prentice, 1980. The statistical analysis of failure time data, John Wiley, New York.

Louis, T.A., 1982. Finding the observed information matrix when using the EM algorithm. J. R. Statist. Soc. B 44, No. 2, pp.226-233.

Meilijson, I., 1989. A fast improvement to the EM algorithm on its own terms. Journal of the Royal Statistical Society B 51, pp.127-38.

Torelli, N. and U. Trivellato, 1993. Modelling inaccuracies in job-search duration data. Journal of Econometrics 59, pp.185-211.

Wolter, K.M., 1985. Introduction to variance estimation. Springer-Verlag, New York Inc.

Zwemmer, W., 1995. Parallel computing in statistics. Report (Department of Statistical Methods, Statistics Netherlands, Voorburg).