The problem of historical blindness, some theories, and a new NTRO benchmarking project
For Ross Gibson (1956-2023)
Whilst we await yet another review of research reporting and evaluation in Australia, and a pause and possible abolition of the heavily bureaucratic ERA exercise, it is worth asking questions of first principle. Is creative practice research any more ‘excellent’ now than it was prior to ERA, and has ERA strengthened disciplines or weakened them? We were told that ERA was not only a research quality benchmarking exercise, but that it could/would provide a mechanism to improve the standard of research in Australia over time. An excellent article by my colleagues Jen Webb and Ross Gibson (who sadly passed away recently) shows that of all the FoRs, Creative Arts and Writing has shown a stagnant set of scores for the entire history of the ERA assessment exercise. Over the period 2010-18, there was no change to the score in 18 UoEs, 3 saw improved scores, and 2 saw declines. In contrast to this the Chemical Sciences, which uses citation based evaluation, had 4 UoEs showing no change, 12 showing an improved score, and 3 showing a declined score (Webb & Gibson, 2019).
By way of background, I have been working as an academic since the mid-1990s across four different institutions (WSU, UoW, QUT and MQ) and my career has been built on creative practice research and the leadership of creative practice-based disciplines and schools. In that time, I have worked as an ERA code leader and Head of School at three of these institutions and assessed ERA for all four rounds (2010, 2012, 2015, 2018). A period of time at my current employer was also spent as an invited member of a DVC-R level ‘research strategy group’ in which the broad directions of the university were discussed across the full range of disciplines and challenged me to articulate the methods and approaches of creative arts research to the most remote of discipline peers. This has given me some highly resolved perspectives. From where I sit, and if I am to be completely honest, there hasn’t been much change to the standard of research produced in that period. At the same time, there has been a slow but perceivable weakening of institutional support for creative arts disciplines across the sector. Whilst I acknowledge that some institutions have recently made moves to expand their creative arts operations, they are clearly in the minority. This slow weakening of the disciplines over a long period of time has stimulated some reflection about the conditions that have given rise to it.
Some theories
One only has to look at the number of closures and mergers over the past 10-15 years which have seen formerly discrete creative arts schools (and even faculties) disappear or be diluted within mega schools and faculties as a result of mergers. There seems to have been declining support from university leaders for creative arts disciplines nationally. My first theory is that these disciplines are not seen to respond well to standard institutional research metrics such as external grant income, and there is a necessary intensity to their teaching, which imposes limits on how efficient they can be in what seems like an endless drive to teach more students at lower cost. This has forced some institutions to de-prioritise, or even close some of the disciplines as they are not seen as good horses in the research and rankings arms races, and they are expensive in comparison to large scale theoretical/studies disciplines when it comes to teaching because they require more contact hours, smaller groups, and specialist facilities.
I propose that these issues largely stem from the extent to which creative arts research and the broader disciplines are valued by institutions and the ways various institutions have chosen to engage with rankings and research assessment systems. It has been my observation that creative arts disciplines are not always well understood within institutions and there can be difficulties experienced by those outside the disciplines in understanding the discipline norms and research practices. This leads me to my second theory: that the prospects for the disciplines can be directly correlated to the representation of leaders within the academic management network who fully understand them.
The flourishing of creative arts research depends on having well adapted and enabling mechanisms for research management so that it does not become an endless exercise of jamming square pegs into round holes, or creating a series of procedural hoops that serve no purpose and do not enable good research.
This health of the disciplines relies on senior creative arts researchers having appropriate input into research management systems as experts. At some institutions this happens. At others it doesn’t because some academic units are now so large, due to the aforementioned mergers, that research management happens at a significant distance from creative arts disciplines. Craig Batty’s (2019) article ‘University management should take creative practice research seriously… before it’s too late’ presents a clear account of what can go wrong when the research management structures are too distant from the disciplines, inappropriate or misguided directives issued, and unhelpful hoops put up for researchers by those who lack sufficient insight into the research practices of the disciplines they govern. Jen Webb and Ross Gibson advance the same point, arguing that amongst senior management in the sector “there is little… understanding of creative-practice-led research’s tendency to deepen an internalised sense of understanding before well-defined, externalised or explicated knowledge is produced” (2019).
The ‘Non-Traditional Research Output’. Are we there yet?
Perhaps the most telling clue of this inside/outside discipline problem of not being understood is the term ‘non-traditional research output’. With all due respect to the title of this outlet, this term makes me extremely uncomfortable. ‘Non-traditional’ according to whom? Are the authors aware of the history of the disciplines most engaged with this category of output?
From the perspective of someone whose primary research interest is in music and technology the idea that creative practice outputs are ‘non-traditional’ ignores the history of this discipline. If we look back into the 20th century for evidence, a number of specialist institutes for music research were established. France was a major centre for them, but they sprung up in all parts of the world. In the 1950s, electronic music composer and theorist Pierre Schaeffer established the Institut National Audiovisuel, Groupe de Recherches Musicales (INA-GRM) in Paris, with a mission to create a platform for research into new music and technology. The INA-GRM Studio was a platform for countless composer residencies and new, research driven musical works. Indeed, this studio was an early platform for the composer Iannis Xenakis who explored the intersections of mathematics and music. In addition to countless investigative musical compositions and studies, two of the most important treatises on music composition emerged from the scene around this facility, including Iannis Xenakis’ ‘Formalized music: thought and mathematics in composition’ (1963) and Pierre Schaeffer’s ‘Traité des objet musicaux’ (1966), both of which delivered field changing findings on creative process and the specific materials and techniques of music composition that were discovered through practice-based investigation.
A little later, in the early 1970s, the Institute for Research and Coordination in Acoustics/Music (IRCAM) was established in Paris – a large-scale, purpose-built research facility for electronic music, funded by the French Government and located at the Centre Pompidou. Founded by French composer Pierre Boulez, this has been operating creative practice-based research programs for over 50 years. In this period, IRCAM’s research outputs have consisted of compositions, performances, publicly distributed software, and research publications. These outputs have existed within an ecological framework where knowledge is produced through and across a range of media which are in close dialogue with one another. This symbiotic relationship between audiovisual media and text has been a marked feature of the major international conferences for computer music (ICMC, NIME and others) for decades. These events typically consist of an extensive concert program of investigative practice (with program notes) alongside a program of paper sessions. It is unremarkable within the field and one could not contemplate a successful event where both presentation modes were not present. Most importantly, the notion of traditional/non-traditional is entirely redundant.
In Australia, through the 1960s-80s university electronic music studios appeared in number as important centres for composition research – La Trobe University, the University of Adelaide in the 70s, and the Australian Centre for Arts and Technology (ACAT) at ANU in the 1980s. Indeed, as an undergraduate student I honed my skills and completed my honours project in the Sydney University Experimental Sound Studio (SUESS) during the late 1980s. At that time, SUESS worked in close collaboration with the Sydney Conservatorium of Music Electronic Music Studio through the electronic music collective ‘watt’ (of which I was a member). The primary outputs of this network of studios were musical and audio-visual compositions which were investigative in nature. Many of these works were based on emerging or experimental technologies, such as the Fairlight CMI and explored new compositional methods and processes. My composition teachers, such as Peter Sculthorpe and Martin Wesley-Smith, were not forced to present their outputs as ‘non-traditional’. They were seen as field leaders and innovators and influenced a generation of composers.
So, if we were to simply take the discipline of music, we can see there is at least a 70 year history of practice-based outputs in music, produced within government funded research institutions and universities – where the mission and outputs have been explicitly presented as research.
It does beg the question: how many decades need to elapse before these outputs are seen as ‘traditional’ by those outside the disciplines?
I would argue that the term is blind to the history of the disciplines it covers. While it is true that some of the other creative arts disciplines may not have had such explicit historical trails of practice based research, the examples from music are startling. I would advocate that we drop the term ‘NTRO’ and go for a more affirming title. The binary encompassed by the traditional/non-traditional nomenclature implies a concerning power relationship that is entirely unhelpful at best, and insulting at worst. It is hard to imagine that such a term would be proposed by anyone from within the disciplines.
In light of the above, I ask myself: why have we gone so far backwards in the past 70 years? This is surely a problem of advocacy and the way in which the disciplines connect to our national research institutions and universities. It’s a problem of leadership (in a management context) and the way in which the disciplines participate in it (or not). It also does not strike me as purely coincidental that this decline has occurred during the onslaught of rankings systems and research assessment exercises across the sector over the past decade or two. Whilst many may point to the Dawkins reforms of the late 80s as posing fundamental challenges to the creative arts in the academy, in my experience the real challenges have arisen in the way in which the disciplines have been managed in the more recent research assessment regimes. Depending on the outcome of the current review of the ARC and ERA, the challenges may well intensify. It is more important than ever that there are strong and clear representations made to policy makers.
Benchmarking NTRO processes
As a way of unpacking the baseline question of ‘what has happened?’, I have been wanting to better understand how different institutions have been handling NTROs for collection and verification purposes, beyond my personal experience of four institutions. I say this because in my experience the processes around NTROs have been variable and in the worst cases fraught due to attempts to blindly equate NTROs with traditional humanities style publication processes with (at times) little regard for disciplinary norms and the specificities of particular kinds of research practice that occur in the creative arts.
Sometimes processes are locally invented, with imagined criteria in order to ‘perform rigour’, that are out of step with how discipline peers view outputs in the field.
In my experience, it usually happens when criteria are imposed by those outside the disciplines, or by those who have limited experience with creative practice-based outputs and assessment. A frequently recurring issue seems to be the tension between the humanities and the creative arts, and their divergent methods of investigation and knowledge production (Beudel, 2022).
I would assert that the stakes around the bureaucratic processes are high, because at their worst, poor processes can demoralise practice-led researchers or impose significant additional workloads on them for no clear benefit. On the other hand, good processes can allow researchers to develop strong programs of research, increasing the quality, reach and impact of their work over time.
The first step in this process is to find out exactly what is happening across the sector. To this end, I have commenced a benchmarking project in which I am aiming to gather as much information as possible from DDCA (and wider) institutions on the handling and verification of NTROs, or more specifically ‘creative works’, across the sector. Important discipline-based reports already exist, such as ASPERA’s Screen Production Research Reporting (Batty, Glisovic, Berry, & et al., 2017), which provides a comprehensive overview of the main issues from the screen production discipline, and builds upon the work of Kerrigan, Leahy, Cohen (Kerrigan, Leahy, & Cohen, 2016) and others. That said, its concerns strongly align with those expressed here and can arguably be applied across all creative arts disciplines. The authors’ concerns are not limited to collection processes but extend to research management structures and the kinds of people who make decisions in the collection process. In short, there is clear overlap in this report with the concerns that I and many of my peers share about whether research administration and management processes are enabling for creative practice-based researchers.
Given the overlap in concerns, and building upon this initial good work from my screen colleagues, it would seem useful to conduct a further survey and benchmarking process at multi-disciplinary level to illuminate the extent to which this screen report may be a litmus test for the sector at large, and to see whether processes may have changed in the five years since its publication, following the introduction of engagement and impact assessments among other changes. The benchmarking process will report back the specifics of collection across as many institutions in the sector from which I am able to obtain responses. By way of summary, the questions explore the submission and verification process, including the material submitted, quantity of explanatory text required, the decision-making process for verification (who decides and what expertise do they have?), peer recognition requirements, whether a points scoring system is used for scale, and what kind of guidance material is provided to researchers who are submitting outputs for verification. This information is useful because it provides an evidence base and a central view of areas of alignment and divergence between institutions in their management of outputs that should be helpful for research leaders and managers.
Once this project is complete, results will be made publicly available in a report. Above all, my interest is in practical measures to support a climate where creative practice-led research can thrive within systems and structures that are properly tuned to support it and where researchers feel part of a stimulating research community. Everyone wants to be understood and valued, and for the institution to support creative practice research in the same way it might support other types of research. I think it’s time to assert creative practice methods more strongly in the conversation around research because we know there are deep and important insights gained through practice that are unavailable to other methods. My hope is that taking some practical steps like this will assist in re-positioning the disciplines more strongly in the sector into the future.
References
Batty, C. (2019). University management should take creative practice research seriously … before it’s too late. NiTRO: Non-Traditional Research Outcomes (20). Retrieved from https://nitro.edu.au/articles/2019/4/13/university-management-should-take-creative-practice-research-seriously-before-its-too-late
Batty, C., Glisovic, S., Berry, M., & et al. (2017). Screen production research reporting: an ASPERA scoping report: Australian Screen Production Education and Research Association.
Beudel, S. (2022). The “living humanities”: Creative arts and academic humanities in tension. TEXT, 26(2). doi:10.52086/001c.40217
Kerrigan, S., Leahy, G., & Cohen, H. (2016). Still a burning issue: measuring screen production research. Studies in Australasian Cinema, 10(1), 79-96. doi:10.1080/17503175.2016.1140257
Schaeffer, P. (1966). Traité des objets musicaux. Paris: Éditions du Seuil
Webb, J., & Gibson, R. (2019). Accounting for creative arts research. NiTRO: Non-Traditional Research Outcomes (21). Retrieved from https://nitro.edu.au/articles/2019/6/7/accounting-for-creative-arts-research
Xenakis, I. (1963). Musiques formelles: nouveaux principes formels de composition musicale. La Revue musicale, 253-254.
Links
INA-GRM https://inagrm.com/en
IRCAM https://www.ircam.fr/
NIME – the International Conference on New Interfaces for Musical Expression https://www.nime.org/
watt https://www.australianmusiccentre.com.au/product/catno/TP074
Julian Knowles is a professor of music, media and creative practice at Macquarie University. He has held Head of School positions for creative arts disciplines at WSU, UoW, and QUT. He is currently Chair of Discipline for Media & Communications at Macquarie University. His research focuses on the creative potentials of new technologies and his creative output spans music, performance, media art, and screen. He is currently developing a new experimental audio-visual work ‘Solar Halo’ for weather data and live performers to be premiered at Experimental Intermedia in NYC in December 2023.
IMAGE: Zeynep Akcay, a frame from KAM, animated film 2021, directed by Dr. Zeynep Akcay