In the competitive world of university research and its evaluation, driven by national priorities as well as developments in data-science and algorithmic thinking, it is imperative that creative research is assessed by criteria that are relevant to disciplinary practice, and that are agreed upon by peers from both within and outside of the university sector, underpinned by values of generosity, openness and care.
Excellence is contextual, diverse, provocative, difficult and incommensurable. While we all agree excellence is a laudable goal – who would argue that research should not be excellent? – it is also an overused ‘buzzword’ that needs careful nuance and expanded, generous thinking.
Arising from panels and seminars held jointly by the Australian Council of Deans and Directors of Creative Arts (DDCA) and the Australian Consortium of Humanities Researchers and Centres (ACHRC) in 2022, this manifesto highlights the key issues creative researchers in Australia face when navigating the territory of university research, the artistic world, and internal and national research assessment and reporting requirements.
As leaders in creative research in our institutions, we want to foster success, engagement, ambition and sensitivity to the needs of the sector. As artists, we want to focus on making and supporting creative work. And yet, despite our passion for, and experience in, advocating for creative research excellence, we struggled in writing this manifesto. To whom should a manifesto be addressed: Researchers? Audiences? Research office managers? Funding bureaucrats? In reality, it is all of these people.
We aspire to work in a world in which the creative arts are valued equally as contributors to knowledge as their non-artistic counterparts, and where practitioner-researchers are able to thrive in their scholarly careers without the constant need to justify the validity of their work.
With this manifesto we pinpoint issues we believe are salient to all these stakeholders, providing simple yet powerful principles to guide further creative and political work.
Excellence and quality start locally
How we define excellence and quality is the first and core problem in assessing excellence in creative research. What do different stakeholders determine excellence or quality in creative research to be? In this way, excellence needs to be locally understood and contextualised, diverse in its expression if commensurate in its robustness.
Quality starts locally and needs to be understood and assessed within the context in which – and for which – a work is made. Different projects, and the researchers working across them, have different aims and therefore different quality or success criteria. The outcomes that emerge from the aims and context of a project are also different from project to project, and these need to be communicated within the assessment exercise.
Research assessment should be designed in such a way that practitioner-researchers can clearly identify and signal the success criteria by which a work should be appraised.
Scale is important but it comes in different guises
In determining excellence in creative research, university-based exercises often start by seeking the creative equivalencies of those found in traditional research. This can be helpful, but it can also be a distraction from the core qualities of creative research.
The ‘scale’ of a project, for example, can be a useful equivalency when thinking about excellence in creative research. A larger or more complex project will give the researcher(s) more opportunity for examining problems from multiple angles and in greater depth over sustained periods of time. But recognising that there is diversity in what ‘scale’ might entail for creative research is vital, because creative projects scale in different ways.
Scale may come in the form of interdisciplinary connections, innovating across knowledge domains and practices. Equally, scale may involve deep complexity and innovation within a single domain, practice, set of tools or materials. Scale may also simply be evident in the scope of an output: the number of components or display contexts; the physical size or temporal duration; the size of the public it engages; or the diversity in its modes of public engagement. Compression is also a form of scale.
These notions of scale are neither commensurate nor incommensurate, and there is no hierarchy among them. But how creative researchers approach the question of scale will relate to their project aims and therefore their success criteria, and so variance in concepts of scale must be considered when appraising creative research.
Creative research takes time
Longitudinal development, provenance and impact for creative research projects are not well represented in the short-term reporting cycles that have become favoured in recent years. Yet the development time and time ‘in market’ both make vital contributions to a project’s inherent and perceived excellence.
While research reporting frequently hinges on the initial dissemination venue or outlet for a given work, the reality of creative research trajectories is that the path a work takes across exhibition venues or publication outlets, media forms and public engagements, more accurately indicates its excellence value. Building this narrative will bring far more insight into a work or project than simply recounting its first public showing.
The art world is not a sole arbiter of merit
One of the most important distinctions between creative and conventional research is that there is no direct correlation between the paths that conventional scholarly publications take to come into the world, and the pathways for creative research works. Artists in universities are often engaged in work that is not well understood either by the university or by the broader arts community, making navigating these markers of excellence complicated.
While blind peer review ensures that publication even in the most esteemed journals is a function of merit, such an equivalent does not always exist in creative outlets; for example, selection for exhibition at a major art institution, where reputation, seasonal or strategic ‘fit’, as well as the ‘who you know’ principle, might guide curatorial selection decisions.
Similarly, art grants are not necessarily awarded on the basis of research potential and their applicant pool – indeed, the main state/territory and national art grant bodies in Australia explicitly discourage applicants from identifying as university researchers and from framing their application as research funding.
Relying on art-world status to judge the merit of creative research will only ever go part of the way to uncovering its meaningfulness, value or originality.
Research metrics are not the only data
Citations are simply one amongst many possible data points and should not be prioritised above other forms of engagement. This is particularly evident with creative works, where a much more diverse ecology of data points is required to profile a work’s provenance and impact. Big-data scraping and profiling techniques common in the commercial world can be more useful in gauging the impact of creative research than reliance on academic citations.
Data collection exercises enable us to see more and see differently, and this would augment our knowledge and understanding of our research. What data collection processes are possible to better serve creative research? How can qualitative and quantitative evidence be interwoven in ways that communicate excellence and quality? Can we design data-accompanied evaluations rather than data-only evaluations? We need to create rich stories of excellent research that engages and has impact – without concomitant ranking and hierarchies – and ensure that these are understood and accepted by assessment processes.
Competition amongst institutions entrenches privilege
While ranking exercises such as ERA (Excellence in Research for Australia) may appear to have fostered in universities a focus on quality in a way that is fair and achievable regardless of location, they have also tended to stratify institutions, both internally and inter-institutionally. Rankings exercises make universities pick ‘winners’ from amongst their disciplines, and universities with fewer resources, often smaller regional universities, exclude some disciplines in favour of others. Despite good intentions, ERA processes have manufactured, stratified and entrenched inequalities between universities, and ultimately ERA results are used as a rationale for defunding.
Quality exercises, as much as they are collaboratively designed to assess disciplines or cohorts rather than individuals, tend to highlight the success of individuals, reward some institutions, and direct resources in hierarchical and unfair ways. This reactive response to research quality outcomes can lead to a resource-intensive churn of people, projects and priorities in the system, always playing catch-up.
Transparency of reporting is vital
If national research reporting is understood as an exercise designed to increase research excellence in universities, then transparency of reporting is vital. Simply put, a rising tide raises all ships; transparent research reporting holds the potential for institutions and researchers to demonstrate best practice and learn from each other in resistance to entrenched stratification.
Transparency also creates trust in the system. It is very difficult for individuals and groups to act in good faith in a non-transparent system. Transparent processes demonstrate the values and purpose that are inherent but often encoded invisibly in the system, and they reduce the negative effects of competition.
If Australia is to reach its goals of international excellence in research across all disciplines, it needs transparency, trust, openness, and a values-based approach to excellence in creative research designed to support the place of creative practice in the university.
We have to get this right
We aspire to a system of peer-reviewing research for excellence and quality that understands the value of creative practice and the important role of creative practitioners in the university environment.
We desire to work in a sector in which our creative endeavours are valued and celebrated regardless of their monetary and prestige indicators, and regardless of constantly changing internal and external goalposts.
We hope this manifesto reignites a discussion about the values and purposes that constitute the foundation for quality evaluation exercises in research, regardless of discipline, in order to support all universities, all researchers, all artforms, and all methodologies and approaches.
Associate Professor Grayson Cooke is an interdisciplinary scholar and media artist, Associate Professor in the School of Arts and Social Sciences at Southern Cross University. Grayson has presented media art and live audio-visual performance works in Australia and internationally, and he has exhibited and performed in major international festivals such as the Japan Media Arts Festival, NeMaf in Seoul, VIDEOFORMES in France, TIVA in Taipei, and the FILE Festival in Sao Paulo. As a scholar he has published widely in academic journals, and he is an associate editor for the online peer-reviewed journal Transformations. He holds an interdisciplinary PhD from Concordia University in Montreal.
Professor Craig Batty is an award-winning educator, researcher and supervisor in the areas of screenwriting, creative writing and screen production. He is also an expert in creative practice research methodologies. He is currently editor of the Journal of Screenwriting, and co-editor of Media Practice and Education. He has published over 100 books, book chapters, journal articles and creative practice research works, as well as industry articles, book reviews and interviews. He has also guest edited 10 journal special issues. Craig has also worked on a variety of screen projects as a writer, script editor and script consultant. Craig’s substantive position is Executive Dean of UniSA Creative.
Associate Professor Tully Barnett works in Creative Industries in the College of Humanities, Arts and Social Sciences at Flinders University. Her DECRA Fellowship was for a project looking at digitisation as a cultural practice. She is a Chief Investigator for the ARC Linkage project Laboratory Adelaide: The Value of Culture looking for qualitative and quantitative methodologies for measuring and reporting the intangible and non-financial benefits of cultural activities, institutions and events, with collaborators Julian Meyrick and Robert Phiddian. She serves on the advisory board of the Australasian Consortium of Humanities Research Centres (ACHRC) and the executive committee of the Australasian Assocation of Digital Humanities (aaDH).
IMAGE: Grayson Cooke Australian Cloud Atlas, The Lume Melbourne 2022, audiovisual installation, image courtesy @lmcabbott