NiTRO Creative Matters

Perspectives on creative arts in higher education

Business as usual

By Professor Ross Woodrow — More than ten years of ERA (Excellence of Research in Australia) data gives a clear picture of the trajectory of creative arts research in academe. 

For the period from 2006 to 2013, the total number of research outputs in field of research (FoR) 19 Creative Arts and Writing increased by 36%. This makes the 14% increase for the period 2011 to 2016 look far less like a cause for optimism, especially considering the national average discipline growth of 17%. The full-time equivalent number of staff producing research outputs in FoR19 compared to the previous ERA has shown a marginal increase of 4%. This ends the good news, such as it is.

The only two areas that matter in relation to ERA measures are quality scores (1 to 5) and grant income. The quality of the research in Creative Arts and Writing, as measured by the gold medal standard of “5” rating, is at the bottom end of the 22 broad disciplines in academe.

As with past ERA exercises the field of Creative Arts and Writing falls last of the twenty-two disciplines in the percentage of Category One grants awarded. However, the percentage of the Australian Competitive Grants research income, that totalled $5.08 billion for the ERA2018 reference period, has now dropped so close to 0% that it isn’t recorded on the published graph.

Does this prove, what some Associate Research Deans (ARDs) in the HASS sector and many PVCs Research have suspected all along, that NTRO outputs in the creative fields are rubbish research and researchers producing them have no chance of getting an ARC grant?

Far from proving any such thing, it simply demonstrates the effect of this deeply-embedded prejudice against any form of creative arts research that does not conform to the humanities and social science research models and methodologies. It suggests that, at best, university-based gatekeepers for ARC grant applications, have not facilitated and encouraged grant applications from researchers producing NTROs in the FoR 19 or, at worst, they have discouraged or blocked them.

The ARC statistical data claims that FoR 19 Creative Arts and Writing have a similar success rate to other humanities and social science disciplines. Creative Arts and Writing ranks in the mid-range of the 22 discipline clusters for ERA volume of outputs submitted and number of staff. It is therefore an absurd and misleading suggestion that it is not a cause for concern that FoR 19 is awarded almost zero percent of ARC grant income because a negligible number of applications are submitted.

What are ARDs and PVCRs doing about ramping up the number of ARC grant applications in Creative Arts and Writing? Very little, especially now, as the ERA “evidence” shows that researchers producing NTROs don’t get category one grants, and this, they believe, is because the quality of their research is stagnant and that’s a disciplinary problem. Giving extra funds to support the production of NTROs by researchers in the creative disciplines gets no institutional return. They are best left to do “business as usual” as one arts and humanities research dean recently wrote in an official university document recommending withdrawing research centre funding for the creative arts.

Concern over the difficulty in judging NTROs as research, by DASSH Associate Deans Research, came to a head at their 2017 annual meeting when they agreed to commission a pilot study based on a survey of 30 key NTRO decision makers in the 19 field “involved in scoring ERA submissions, assessing ARC applications and other forms of research assessment at the national level”. The reason for a focus on this field is noted in the background to the survey:

“While every other FoR has increased its average score in each of the ERA rounds – as universities better understand what is expected of them, and as the ERA exercise drives changes in practice – FoR19 is the only code in which the average score across the sector has decreased in each ERA round. The ADR network was concerned that this speaks to a continuing uncertainty as to how these judgments are being made in practice.” (1)

For anyone looking at the ERA data, a very selective form of tunnel vision is required to avoid noticing that one of the largest fields in this cluster gets close to zero percentage of ARC grants awarded. Here the diagnosis and prognosis for the problems obstructing the development of research culture in the creative arts fields are commingled around the lack of gold medals, leading to the misguided pre-emptive assumption that the symptomatic cause must lie with the peer-reviewers in FoR 19.

The thirty experts surveyed in August 2018 offered little consensus in rating the importance of nineteen possible criteria they might use in making quality judgements about NTROs. Most often respondents took the op-out option of “it depends” which took comments to a second, qualitative stage of analysis. Nevertheless, it was heartening to see that the majority expert opinion does not support “The presence of a strong academic research question in the research statement” as being very important. The majority also believed that it is relevant “Whether the NTRO has been critically reviewed” but not very important to judgement of its quality. In other words, a majority of expert peer-reviewers of non-discursive research outputs are resisting the pressure to move closer to existing humanities research models and methodologies.

The strong reliance on the starting point of “the research question” in social science and humanities is mostly antithetical to the outcomes of creative practices which hardly ever yield “propositional claims about states of affairs.” (2) Such approaches, by default, deny the primacy of the non-discursive forms that creative arts research takes which are often shot through with complexity, sometimes including contradictory or unresolved dilemmas, yet nonetheless in operation enhances an understanding of the underlying issue, human condition or state of affairs. It is evident from even a cursory scan of the research statements that support NTROs in the ERA that artists and other creative arts researchers rarely even mention the term “research question” and statistically are five times more likely to use the terms “metaphor” or metaphoric” when giving the background to the development of a particular output. (3)

The thirty experts found consensus on only one of the nineteen criteria in making quality judgements as being very important: “Demonstrated familiarity in the research statement with the current state of knowledge in the relevant academic disciplines.” Peer-reviewers in Creative Arts and Writing are meticulously doing their job by taking each work they review on its individual merits, measured against the research statement that demonstrates the researcher is aware of the operational, historical, contemporary and conceptual context in which their work operates.  I fear that the ARDs were attempting to refract NTROs through the lens of their own research methodology with this survey and they missed the obvious; that they might be part of the problem. For inconsistent judgement by expert peer-reviewers of NTROs is not the reason for the inertia in gaining higher ERA scores in FoR19.

It is now evident that the ERA, like the HERDC, is a quantitative system but with a far greater focus on grant income. The quantity and quality of publication outputs of any sort will not alone achieve the gold medal of a “5” or silver medal “4” ranking in the ERA.  The number of “5” ratings given at the sub-discipline level within the 22 broad research fields is correlated categorically with grant income.  For example, in:

  • FoR11 Medical and Health Sciences, where 36% of the Category One grants were awarded, 170 disciplines across the 36 institutions measured were ranked at “5”

  • FoR09 Engineering, with 9% Cat One grants, 76 disciplines ranked at 5

  • FoR17 Psychology and Cognitive Sciences  3% grant income with 20 disciplinary 5 rankings.

In FoR19 Creative Arts and Writing, only one 5 rating was given in ERA 2018 and this was achieved by an institution with the two individual researchers who rank first and second in terms of number and value of ARC grants awarded in this 19 FoR over the past ten years. As already noted, this one “5” ranking placed Creative Arts and Writing second to last place above FoR12 Built Environment and Design on the list of two-digit discipline codes for the number of “5” ratings. Four of the sub-disciplines in FoR19 were ranked at “5” and three of these were from the institution that ranked at “5” overall.

Creative arts research is very different to conventional forms of research and if the quality of the insights and understandings achieved can be captured through processes of peer review offered by the ERA there is no further justification or argument needed for awarding ARC category one grants for non-discursive research outcomes such as a novel, exhibition, film or any other creative arts work that has no identifiable instrumental value. The lack of progress in such acceptance over the past ten years suggests this will not happen with the current ERA system and ARC grant distribution protocols.

The situation is currently no different to the twenty years prior to the introduction of the ERA in 2010 when researchers producing NTROs were specifically excluded by the ARC from grant applications – back to business as usual.


1. Alan McKee “Analysis of the criteria used by key decision makers in Australia to judge the academic quality of NTROs” an unpublished report commissioned by DASSH, 2018 2

2. Tom Barone and Eliot Eisner Arts Based Research, Sage, 2012

3. In 2010 I conducted a small concordance analysis of research statements for outputs submitted in ERA 2010.’Shrunk_to_this_Little_Measure’

The indications then, as with a follow up larger concordance study of research statements submitted for ERA2015, suggested that the exegetical framing of creative-based research is specifically distinctive and does not correlate strongly with traditional science writing or discursive research practices in the humanities and social sciences.

Professor Ross Woodrow was the founding Director of the Griffith Centre for Creative Arts Research. His involvement with creative arts research and research education spans more than twenty years. He was one of the lucky few to receive an ARC small project grant for an exhibition, Bush Burial in 1996 during the brief period when the “J” category allowed artists in academe to submit NTROs for ARC grants.

More from this issue

More from this issue

By Jenny Wilson — This edition of NiTRO was prompted by responses to a survey conducted last year, which asked readers what they would like to see more about in NiTRO. It is also timely, given the recent announcement of results of the ERA exercise and research engagement and impact assessments. We have devoted two editions to an exploration of the state of play for creative arts research.
By Professor Craig Batty — It was heartening to read QUT Vice-Chancellor, Margaret Sheil, write in support of the arts and humanities in the last edition of NiTRO.
By Professor Vanessa Tomlinson and Charulatha Mani — Drawing on Draper and Harrison’s earlier reflections in NiTRO on doctoral projects at Queensland Conservatorium (QCGU), I met with Charulatha Mani, an artist-researcher who has recently submitted her PhD on intersections between early opera and Karnatik music.
By Professor Carole Gray — In relation to the progress of creative arts research within higher education institutions, Jen Webb asks the important question “Are we there yet?” In this article I would like to partially address this question by focusing on a key component of a practice-led submission for PhD - namely the inclusion and presentation of artefacts as part of the overall argument, about which there has been a long debate. Their status can be ambiguous and the concept of ‘exhibition’ is - I would argue - problematic in this context.

Successful filmmaking requires the filmmakers to be invisible. Any trace of the maker in the film is usually scorned at, particularly in commercial films, that is unless the film requires the filmmaker to be in the film.

“Are we there yet?” is a searching but also ambiguous question posed about creative practice research and the academy. In fact, yes, we are now deeply ensconced in the academic sector and its intersections with ways of governing knowledge and research. Of course, systems need to be developed and conformed to if we are to be able to ‘play the game’ … but ultimately this is also a highly differentiated and differentiating sector … segmented and divided by New Public Management discourse and practices.

I have been deeply involved in creative art and design research since the mid 1990s but have never worked in an art and design faculty. Instead, I found a home in IT and computer science where from the outset, there was a remarkable openness to having artists amongst the mix of people from different disciplines. My very first research grant for studying collaboration between artists and technologists … funded a series of artist residencies over four years.

In step with profound changes in the form and function of universities, creative arts research has been undergoing a process of transformation. While the past decade has been spent consolidating the creative arts into the evolving academy … the landscape we now face promises ongoing dramatic changes.

By Professor Jeri Kroll — There is no denying that creative arts in the university have been successful over recent decades. Yet Jen Webb still asks, in a July 2018 NiTRO piece, “Are we there yet?” - the ‘we’ being the collective staff and students of the creative and performing arts disciplines.

The UQ Drama Creative Fellowship, piloted in 2014, brings a playwright of national standing to UQ’s School of Communication and Arts each year to provide workshops, masterclasses and lectures. These activities have focussed both on the craft of playwriting and on the dramaturgy, or attributes, of the playtext. In 2019, UQ Drama took a different approach to the Fellowship.