Do AI Occupational-Exposure Scores Measure AI? AIOE and Eloundou (2024) Largely Capture Cognitive Content; Webb (2020) Does Not

Rai, Sudhanshu (2026): Do AI Occupational-Exposure Scores Measure AI? AIOE and Eloundou (2024) Largely Capture Cognitive Content; Webb (2020) Does Not.

Abstract

A growing empirical literature uses pre-built "AI occupational exposure" scores (most prominently the AI Occupational Exposure (AIOE) index of Felten, Raj, and Seamans (2021) and the GPT-4 task-exposure measure of Eloundou et al. (2024)) as occupation-level treatments or predictors for AI's labor-market effects. We show that two of the most-cited scores, AIOE and Eloundou's GPT-4 measure, substantially re-label cognitive task content rather than capturing AI-specific exposure, a construct-validity problem that does not extend to a third, differently-built score (Webb 2020, patent-based). This note horse-races these three measures against transparent cognitive/manual task-content indices and the established Autor–Dorn Routine Task Intensity (RTI) measure. Across 773 occupations: (i) the ten-plus AI "applications" underlying AIOE collapse to a single factor (first principal component ≈ 88%); (ii) AIOE and Eloundou each correlate strongly with a cognitive-ability index (+0.85 / +0.70) and negatively with a manual-ability index (−0.91 / −0.83), correlate 0.86 with each other, but only moderately with RTI (−0.33 / −0.30): the confound is specifically cognitive ability level, not the classic routine-task polarization axis; (iii) each score's positive wage association reverses sign controlling for cognitive content but is barely affected by controlling for RTI; and (iv) the much-cited pre-ChatGPT "AI foresight" wage-divergence pattern collapses to near-zero under the cognitive control (not under the RTI control). Critically, this collapse is not universal: Webb's patent-text-overlap score is essentially uncorrelated with AIOE (r=0.03) and Eloundou (r=−0.03), only weakly related to cognitive content (r=0.13), and its modest wage associations do not reverse under cognitive control. The cognitive-content collapse is a signature of how a score is built: subjective crowd-relatedness ratings (AIOE) or LLM/human task judgments (Eloundou), not an inherent property of occupational AI-exposure measurement. Studies using relatedness- or judgment-based exposure scores should control for cognitive content and re-interpret accordingly; a patent-based measure is not shown here to have the same problem, though its own construct validity is untested.

• Item Type: MPRA Paper
• Language: English
• Keywords: AI occupational exposure, Felten-Raj-Seamans index, GPT-4 task exposure, construct validity, task content measures, routine task intensity, Autor-Dorn RTI, patent-based automation exposure
• Subjects: C - Mathematical and Quantitative Methods > C3 - Multiple or Simultaneous Equation Models ; Multiple Variables > C38 - Classification Methods ; Cluster Analysis ; Principal Components ; Factor Models; C - Mathematical and Quantitative Methods > C5 - Econometric Modeling > C52 - Model Evaluation, Validation, and Selection; C - Mathematical and Quantitative Methods > C8 - Data Collection and Data Estimation Methodology ; Computer Programs > C81 - Methodology for Collecting, Estimating, and Organizing Microeconomic Data ; Data Access; J - Labor and Demographic Economics > J2 - Demand and Supply of Labor > J24 - Human Capital ; Skills ; Occupational Choice ; Labor Productivity; O - Economic Development, Innovation, Technological Change, and Growth > O3 - Innovation ; Research and Development ; Technological Change ; Intellectual Property Rights > O33 - Technological Change: Choices and Consequences ; Diffusion Processes

References

References: Autor, D. H., & Dorn, D. (2013). The Growth of Low-Skill Service Jobs and the Polarization of the US Labor Market. American Economic Review, 103(5), 1553–1597. https://doi.org/10.1257/aer.103.5.1553

Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2024). GPTs are GPTs: Labor market impact potential of LLMs. Science, 384(6702), 1306–1308. https://doi.org/10.1126/science.adj0998

Felten, E., Raj, M., & Seamans, R. (2021). Occupational, industry, and geographic exposure to artificial intelligence: A novel dataset and its potential uses. Strategic Management Journal, 42(12), 2195–2217. https://doi.org/10.1002/smj.3286

Frank, M. R., Javadian Sabet, A., Simon, L., Bana, S. H., & Yu, R. (2026). AI-exposed jobs deteriorated before ChatGPT. arXiv:2601.02554. https://arxiv.org/abs/2601.02554

Frey, C. B., & Osborne, M. A. (2013). The Future of Employment: How Susceptible Are Jobs to Computerisation? Oxford Martin School Working Paper. https://oms-www.files.svdcdn.com/production/downloads/academic/The_Future_of_Employment.pdf

Lund, C., et al. (2026). AI Exposure Scores: what they measure, what they miss, and what comes next. arXiv:2606.23633. https://arxiv.org/abs/2606.23633

Webb, M. (2020). The Impact of Artificial Intelligence on the Labor Market. SSRN 3482150. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3482150