Forecasting with Econometric Methods: Folklore Versus Fact

Armstrong, J. Scott (1978): Forecasting with Econometric Methods: Folklore Versus Fact. Published in: Journal of Business No. 51 (1978): pp. 549-564.

Abstract

Evidence from social psychology suggests that econometricians will avoid evidence that disconfirms their beliefs. Two beliefs of econometricians were examined: (1) Econometric methods provide more accurate short-term forecasts than do other methods; and (2) more complex econometric methods yield more accurate forecasts. A survey of 21 experts in econometrics found that 95% agreed with the first statement and 72% agreed with the second. A review of the published empirical evidence yielded little support for either of the two statements in the 41 studies. The method of multiple hypotheses was suggested as a research strategy that will lead to more effective use of disconfirming evidence. Although this strategy was suggested in 1890, it has only recently been used by econometricians.

• Item Type: MPRA Paper
• Language: English
• Keywords: econometric models, forecasting
• Subjects: B - History of Economic Thought, Methodology, and Heterodox Approaches > B2 - History of Economic Thought since 1925 > B23 - Econometrics ; Quantitative and Mathematical Studies; C - Mathematical and Quantitative Methods > C0 - General > C01 - Econometrics; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C53 - Forecasting and Prediction Methods ; Simulation Methods

References

Armstrong, J. Scott. 1968. Long-range forecasting for international markets: the use of causal models. Pp. 222-27 in Robert L. King (ed.), Marketing and the New Science of Planning. Chicago: American Market Association.

Armstrong, J. Scott. 1978a. Long-Range Forecasting: From Crystal Ball to Computer. New York: Wiley-Interscience.

Armstrong, J. Scott. 1978b. Advocacy and objectivity in science. Wharton School Working Paper. Mimeographed. Philadelphia: Wharton School of Management, University of Pennsylvania.

Armstrong, J. Scott, and Grohman, Michael, C. 1972. A comparative study of methods for long-range market forecasting. Management Science 19:211-21.

Asch, S. E . 1965. Effects of group pressure upon the modification and distortion of judgments. In Harold Proshansky and Bernard Seidenberg (eds.), Basic Studies in Psychology. New York: Holt, Rinehart & Winston.

Ash, J. C. K., and Smyth, D. J. 1973. Forecasting the United Kingdom Economy. Farnborough: Saxon House.

Barber, Bernard. 1961. Resistance by scientists to scientific discovery. Science 134596-602.

Bassie, V. Lewis. 1958. Economic Forecasting. New York: McGraw-Hill.

Bassie, V. Lewis. 1972. A note on scientific method in forecasting. Pp. 1211-18 in B. G. Hickman (ed.), Econometric Models of Cyclical Behavior. Vol. 2. Studies in Income and Wealth no. 36. New York: Columbia University Press.

Brown, T. Merritt. 1970. Specification and Uses o f Econometric Models. New York: St. Martin's.

Chamberlin. T. C. (1890) 148:754-59. 1965. The method of multiple working hypotheses. Science 148: 754-759.

Chapman, Loren J., and Chapman, J. R. 1969. Illusory correlation as an obstacle to the use of valid psychodiagnostic signs. Journal of Abnormal Psychology 74:271-80.

Christ, Carl F. 1951. A test of an econometric model for the United States, 1921-47. Pp. 35-107 in Conference on Business Cycles. New York: National Bureau for Economic Research. (Discussion on pp. 107-29.)

Christ, Carl F. 1956. Aggregate econometric models. American Economic Review 46:385-408.

Christ, Carl F. 1975. Judging the performance of econometric models of the U.S. economy. International Economic Review 16 (1975): 54-74.

Claudy, John G. 1972. A comparison of five variable weighting procedures. Educational and Psychological Measurement 32:311-22.

Cooper, J. Phillip, and Nelson, Charles R. 1975. The ex ante prediction performance of the St. Louis and FRB-MIT-PENN econometric models and some results on compos ite predictors. Journal of Money, Credit and Banking 7:l-32.

Cooper, Ronald L . 1972. The predictive performance of quarterly econometric models of the United States. Pp. 813-926 in Bert G. Hickman (ed.), Econometric Models of Cyclical Behavior. Vol. 2. Studies in Income and Wealth no. 36. New York: Columbia University Press. (Discussion on pp. 926-47.)

Dawes, Robyn M., and Corrigan, Bernard. 1974. Linear models in decision making. Psychological Bulletin 81:95-106.

De Grazia, Alfred; Juergens, Ralph E.; and Steccini, Livio C. 1966. The Velikovsky Affair. Hyde Park, N.Y.: University Books.

Elliott, J. W. 1973. A direct comparison of short-run GNP forecasting models. Journal of Business 46:33-60.

Fair, Ray C. 1971. A Short-Run Forecasting Model of the United States. Lexington, Mass.: Heath.

Fischhoff, Baruch, and Beyth, Ruth. 1975. I knew it would happen: remembered prob abilities of once-future things. Organizational Behavior and Human Performance 13:l-16.

Friend, Irwin, and Taubman, P. 1964. A short-term forecasting model. Review of Eco nomics and Statistics 46:229-36.

Geller, E. Scott, and Pitz, Gordon F. 1968. Confidence and decision speed in the revision of opinion. Organizational Behavior and Human Performance 3:190-201. Forecasting with Econometric Methods 563

Granger, C. W. J., and Newbold, P. 1974. Economic forecasting: the atheist's view point. In G. A. Renton (ed.), Modelling the Economy. London: Heinemann.

Grant, Donald L., and Bray, Douglas W. 1970. Validation of employment test for telephone company installation and repair occupations. Journal of Applied Psychology 54:7-15.

Greenwald, Anthony G. 1975. Consequences of prejudice against the null hypothesis. Psychological Bulletin 82: 1-20.

Haitovsky, Yoel; Treyz, G.; and Su, V. 1974. Forecasts with Quarterly Macroeconomic Models. New York: Columbia University Press.

Ibrahim, I. B., and Otsuki, T. 1976. Forecasting GNP components using the method of Box and Jenkins. Southern Economic Journal 42:461-70.

Johnston, Roy, and McNeal, B. F. 1964. Combined MMPI and demographic data in predicting length of neuropsychiatric hospital stay. Journal of Consulting Psychology 28:64-70.

Jorgenson, Dale W.; Hunter, J.; and Nadiri, M. I. 1970. The predictive performance of econometric models of quarterly investment behavior. Econometrica 38:213-24.

Kosobud, Richard F. 1970. Forecasting accuracy and uses of an econometric model. Applied Economics 2:253-63.

Lawshe, C. H., and Schucker, R. E. 1959. The relative efficiency of four test weighting methods in multiple prediction. Educational and Psychological Measurement 19:103-14.

Leser, C. E . V. 1968. A survey of econometrics. Journal of the Royal Statistical Society 131, series A: 530-66.

Levenbach, Hans; Cleary, J. P.; and Fryk, D. A. 1974. A comparison of ARIMA and econometric models for telephone demand. Proceedings of the American Statistical Association: Business and Economics Statistics Section, pp. 448-50.

Liebling, Herman I.; Bidwell, Peter T.; and Hall, Karen E. 1976. The recent performance of anticipations surveys and econometric model projections of investment spending in the United States. Journal of Business 49:451-77.

McLaughlin, Robert L , 1973. The forecasters' batting averages. Business Economics 358-59.

McNees, Stephen K. 1974. How accurate are economic forecasts? New England,Eco nomic Review (November-December), pp. 2-19.

McNees, Stephen K. 1975. An evaluation of economic forecasts. New England Eco nomic Review (November-December), pp. 3-39.

Mahoney, Michael J. 1977. Publication prejudices: an experimental study of confirma tory bias in the peer review system. Cognitive Therapy and Research 1: 161-75.

Markland, Robert E. 1970. A comparative study of demand forecasting techniques for military helicopter spare parts. Naval Research Logistics Quarterly 17:103-19.

Narasimham, Gorti; Castellino, V. F.; and Singpunvalla, Nozer D. 1974. On the predic tive performance of the BEA quarterly econometric model and a Box-Jenkins type ARIMA model. Proceedings of the American Statistical Association: Business and Economics Section, pp. 501-4.

NATO. 1967. Forecasting on a Scientific Basis. Lisbon: NATO.

Naylor, Thomas H.; Seaks, T. G.; and Wichern, D. W. 1972. Box-Jenkins methods: an alternative to econometric models. International Statistical Review 40:123-37.

Nelson, Charles R. 1972. The prediction performance of the FRB-MIT-PENN model of the U. S. economy. American Economic Review 5:902-17.

O'Herlihy, C.; Fane, G.; Gwilliam, K. M.; and Ray, G. F. 1967. Long-term forecasts of demand for cars, selected consumer durables and energy. National Institute Economic Review 40:34-61.

Platt, John R. 1964. Strong inference. Science 146:341-53.

Pruitt, D. G. 1961. Informational requirements in making decisions. American Journal of Psychology 74:433-39.

Reiss, Albert J., Jr. 1951. The accuracy, efficiency and validity of a prediction instrument. American Journal of Sociology 56:552-61.

Ridker, Ronald G. 1963. An evaluation of the forecasting ability of the Norwegian national budgeting system. Review of Economics and Statistics 45:23-33.

Rippe, R. D., and Wilkinson, M. 1974. Forecasting accuracy of the McGraw-Hill anticipations data. Journal of the American Statistical Association 69:849-58.

Rosenthal, Robert, and Fode, K.L. 1963. The effect of experimenter bias on the performance of the albino rat. Behavioral Science 8:183-89.

Rosenthal, Robert, and Rosnow, Ralph L . 1969. Artifact in Behavioral Research. New York: Academic Press.

Schmidt, Frank L . 1971. The relative efficiency of regression and simple unit predictor weights in applied differential psychology. Educational and Psychological Measurement 31:699-714.

Scott, Richard D., and Johnson, R. W. 1967. Use of the weighted application blank in selecting unskilled employees. Journal of Applied Psychology 51:393-95.

Sims, Christopher A. 1967. Evaluating short-term macroeconomic forecasts: the Dutch performance. Review of Economics and Statistics 49:225-36.

Stuckert, R. P. 1958. A configurational approach to prediction. Sociometry 21:225-37.

Suits, Daniel B. 1962. Forecasting and analysis with an econometric model. American Economic Review 52: 104-32.

Summers, David A . , and Stewart, Thomas R. 1968. Regression models of foreign policy beliefs. Proceedings of the American Psychological Association, pp. 195-96.

Vandome, Peter. 1963. Econometric forecasting for the United Kingdom. Bulletin of the Oxford University Institute of Economics and Statistics 25:239-81.

Wason, P. C. 1960. On the failure to eliminate hypotheses in a conceptual task. Quarterly Journal of Experimental Psychology 12: 129-40.

Wason, P. C. 1968. On the failure to eliminate hypotheses-a second look. 4.165-74 in P. C. Wason and P. N. Johnson-Laird (eds.), Thinking and Reasoning. Baltimore: Penguin.

Wesman, A. G., and Bennett, G. K. 1959. Multiple regression vs. simple addition of scores in prediction of college grades. Educational and Psychological Measurement 19:243-46.

Worswick, G. D. N. 1974. Preston, R. S., The Wharton Annual and lndustry Forecasting Model. Journal of Economic Literature 12:117-18.