Accounting for research quality: Research audits and the journal rankings debate

Rowlinson, Michael and Harvey, Charles and Kelly, Aidan and Morris, Huw and Todeva, Emanuela (2015): Accounting for research quality: Research audits and the journal rankings debate. Published in: Critical Perspectives on Accounting , Vol. 26, (2015): pp. 2-22.

Abstract

The question of whether and how research quality should be measured, and the consequences of research audits such as the UK’s Research Excellence Framework (REF) – formerly the RAE – are considered in relation to the role of journal ratings such as the Association of Business Schools Academic Journal Quality Guide (the ABS Guide). Criticism of the ABS Guide has distracted attention from the results of successive RAEs, where the panel for Business and Management has been one of the most selective in its allocation of the highest grades, especially when compared with the neighbouring field of Economics. If the ABS Guide had been used to grade outputs submitted for Business and Management in the RAE 2008 then many more outputs would have received the highest grades, especially in accounting where outputs from journals such as Critical Perspectives on Accounting, which are highly rated in the ABS Guide, appear to have been downgraded by the RAE panel. The alleged bias against accounting in the ABS Guide rests on a particular interpretation of citation impact factors for journals, and a narrow definition of subject fields. Critics of the ABS Guide would be better advised to direct their attention to scrutinizing the results of the REF and considering whether it provides an adequate research ranking for UK business schools. 15% of all full time students in the UK study business and management, including accounting and finance, but only 6.7% of the full time equivalent research active staff submitted in the RAE 2008 were in business and management, or accounting and finance. Research audits are therefore forcing the separation of teaching from research in UK business schools. With an estimated ratio of 71 full time students per research active faculty member in UK business schools, it may be time to consider a more appropriate, inclusive, and economical form of ranking for research in business and management.

• Item Type: MPRA Paper
• Language: English
• Keywords: Critical Accounting journals Journal rankings Research audit United Kingdom
• Subjects: A - General Economics and Teaching > A2 - Economic Education and Teaching of Economics > A20 - General; B - History of Economic Thought, Methodology, and Heterodox Approaches > B2 - History of Economic Thought since 1925 > B26 - Financial Economics

References

AACSB. 2010. Eligibility procedures and accreditation standards for business accreditation. Tampa FL.: AACSB International - The Asscociation to Advance Collegiate Schools for Business.

ABDC. 2010. Australian business deans council journal quality guide.

Adler, N. J., & Harzing, A. W. 2009. When knowledge wins: Transcending the sense and nonsense of academic rankings. Academy of Management Learning & Education, 8(1): 72-95.

Barker, K. 2007. The uk research assessment exercise: The evolution of a national research evaluation system. Research Evaluation, 16(1): 3-12.

Battilana, J., Anteby, M., & Sengul, M. 2010. The circulation of ideas across academic communities: When locals re-import exported ideas. Organization Studies, 31(6): 695-713.

Baum, J. A. C. 2011. Free-riding on power laws: Questioning the validity of the impact factor as a measure of research quality in organization studies. Organization, 18(4): 449-466.

Beattie, V., & Goodacre, A. 2006. A new method for ranking academic journals in accounting and finance. Accounting and Business Research, 36(2): 65-91.

Bessant, J., Birley, S., Cooper, C., Dawson, S., Gennard, J., Gardiner, M., Gray, A., Jones, P., Mayer, C., McGee, J., Pidd, M., & Rowley, G. 2003. The state of the field in UK management research: Reflections of the research assessment exercise (rae) panel. British Journal of Management, 14(1): 51-68.

Burrell, G. 2011. Journal rankings do not recognise quality. Times Higher Education.

Burrows, R. 2012. Living with the h-index? Metric assemblages in the contemporary academy. Sociological Review, 60(2): 355-372.

Cooke, B. 2012. Kill the REF in complex circumstances, Vol. 2012.

Curtis, B. 2007. Some impacts of the performance-based research fund on academics. Paper presented at the Public Sociologies: Lessons and Trans-Tasman Comparisons: TASA / SAANZ 2007 Joint Conference Proceedings.

DiMaggio, P. J., & Powell, W. W. 1991. The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. In W. W. Powell, & P. J. DiMaggio (Eds.), The new institutionalism in organizational analysis: 63-82. Chicago: University of Chicago.

Dunne, S., Harney, S., & Parker, M. 2008. The responsibilities of management intellectuals: A survey. Organization, 15(2): 271-282.

Easton, G., & Easton, D. M. 2003. Marketing journals and the research assessment exercise. Journal of Marketing Management, 19(1/2): 5-24.

Espeland, W. N., & Sauder, M. 2007. Rankings and reactivity: How public measures recreate social worlds. American Journal of Sociology, 113(1): 1-40.

Geary, J., Marriott, L., & Rowlinson, M. 2004. Journal rankings in business and management and the 2001 research assessment exercise in the uk. British Journal of Management, 15(2): 95-141.

Harney, S., & Dunne, S. forthcoming. More than nothing? Business and management studies and the UK research audit. Critical Perspectives on Accounting.

Harvey, C., Kelly, A., Morris, H., & Rowlinson, M. 2010. The association of business schools academic journal quality guide version 4.

Harzing, A.-W. (Ed.). 2013. Journal quality list (forty-eight ed.). Melbourne, Australia: Harzing.com.

HEFCE. Hefce research funding. 30 Nov 2009 HEFCE. The research excellence framework: A brief guide to the proposals. 24 January

Hoepner, A. G. F., & Unerman, J. 2012. Explicit and implicit subject bias in the abs journal quality guide. Accounting Education, 21(1): 3-15.

Hussain, S. 2011. Food for thought on the abs academic journal quality guide. Accounting Education, 20(6): 545-559.

Hussain, S. 2012. Further food for thought on the ABS guide. Accounting Education, 21(1): 17-22.

Jeacle, I., & Carter, C. 2011. In Tripadvisor we trust: Rankings, calculative regimes and abstract systems. Accounting Organizations and Society, 36(4-5): 293-309.

Kelly, A., Morris, H., Rowlinson, M., & Harvey, C. (Eds.). 2009. The Association of Business Schools academic journal quality guide version 3: Association of Business Schools.

Macdonald, S., & Kam, J. 2011. The skewed few: People and papers of quality in management studies. Organization, 18(4): 467-475.

Mingers, J. 2008. Measuring the research contribution of management academics using the Hirsch-index. Kent Business School Working Paper Series, 162.

Mingers, J., & Harzing, A. W. 2007. Ranking journals in business and management: A statistical analysis of the Harzing data set. European Journal of Information Systems, 16(4): 303-316. Mingers, J., & Willmott, H. 2010. Moulding the one-dimensional academic: The performative effects of journal ranking lists. Kent Business School Working Paper Series, 239.

Mingers, J., & Willmott, H. 2012. Taylorizing business school research: On the 'one best way' performative effects of journal ranking lists. Human Relations.

Mingers, J., Watson, K., & Scaparra, P. 2009. Estimating business and management journal quality from the 2008 research assessment exercise in the UK, Kent Business School Working Paper Series.

Mingers, J., Watson, K., & Scaparra, P. 2012. Estimating business and management journal quality from the 2008 Reserach Assessment Exercise in the UK. Information Processing and Management.

Parker, M. 2005. Fucking. In C. Jones, & D. O'Doherty (Eds.), Manifestos for the business school of tomorrow. Parker, M., & Thomas, R. 2011. What is a critical journal? Organization, 18(4): 419-427.

Power, M. 1997. The audit society: Rituals of verification. Oxford: Oxford University Press. QAA. 2012. Institutional review of higher education institutions in england and northern ireland: A handbook for higher education providers, Second edition ed.

RAE2008. 2009. RAE2008 panel I UoA 36 Business and Management Studies subject overview report: Hefce. Rafols, I., Leydesdorff, L., O'Hare, A., Nightingale, P., & Stirling, A. 2012. How journal rankings can suppress interdisciplinary research: A comparison between innovation studies and business & management. Research Policy, 41(September): 1262-1282.

Rowlinson, M., Harvey, C., Kelly, A., Kestinova, D., Morris, H., & Todeva, E. 2010. Business history in the UK Research Assessment Exercise. European Business History Association (EBHA) Newsletter, June: 4-12.

Sangster, A. 2011. The ABS journal quality guide: A personal view. Accounting Education, 20(6): 575-580.

Saunders, J., & Wong, V. 2011. Manoeuvring towards research decline the RAE and the decline of Britain's international research standing. European Journal of Marketing, 45(4): 484-512.

Saunders, J., Wong, V., & Saunders, C. 2011. The research evaluation and globalization of business research. British Journal of Management, 22(3): 401-419.

Starbuck, W. H. 2005. How much better are the most-prestigious journals? The statistics of academic publication. Organization Science, 16(2): 180-200.

Taylor, J. 2011. The assessment of research quality in UK universities: Peer review or metrics? British Journal of Management, 22(2): 202-217.

Tol, R. S. J. 2010. The research output of business schools and business school scholars in ireland, Working Paper. Dublin: Economic and Social Research Institute, Dublin, Ireland.

Tourish, D. 2010. Publish or be damned, Times Higher Education.

Tourish, D. 2011. Leading questions: Journal rankings, academic freedom and performativity: What is, or should be, the future of leadership? Leadership, 7(3): 367-381.

Tourish, D. 2012. The effects of journal rankings, Association of Business Schools, Annual Research Conference: Managing Business and Management Research. British Library, London.

Warwick Business School (2012). Financial information. accesssed12 October Willmott, H. 2011. Journal list fetishism and the perversion of scholarship: Reactivity and the abs list Organization, 18(4): 429-442.

Willmott, H., Edwards, P., & Parker, M. 2011. Workshop on 'the political economy of academic publishing' Warwick Organization Theory Network (WOTNet). Warwick Business School.