NiTRO Creative Matters

Perspectives on creative arts in higher education

NTRO: A Model for Change

By Professor Ross Woodrow — The decision by the Australia Research Council (ARC) to achieve the long-mooted merging of the Higher Education Research Data Collection (HERDC) and the Excellence in Research for Australia (ERA) exercise by adeptly disappearing the HERDC has been welcomed by many discipline leaders, and not just those in the creative arts. With the inclusive ERA becoming the singular evaluation of research quality across Australia, there couldn’t be a better time to rethink the classification of research in universities.

The ARC’s attempt to establish an authoritative ranked journal list in 2009 in preparation for the first ERA exercise caused consternation rather than consensus. After decades of collecting and auditing journal data sets built on the gold standard of peer review, the HERDC was recognised as essentially being a quantitative exercise that rewarded volume in the allocation of funding to universities.

For the ERA 2015 exercise, the ARC published a Submission Journal List, formulated after gaining advice from peak discipline bodies and submissions from universities. The variability in the editorial rigour of publications on this unranked eligible journal list did not raise any concern because, by developing an information technology platform or a System to Evaluate the Excellence of Research (SEER), the ARC has ensured quality control is finely tuned to the needs of each discipline. ARC-appointed authorities, such as ERA reviewers and expert panel members, access a dashboard showing the pattern of journal and other publications for institutions in their area of authority and are therefore best placed to make qualitative judgement on the reputation of a journal or book publisher or exhibition venue or other mechanism of peer management.

The all-encompassing nature of the ERA research data evaluation makes the distinction between “traditional” and “non-traditional” outputs redundant.  The Non-Traditional Research Output (NTRO) designation was instigated by the ARC as an administrative necessity to identify non-HERDC eligible outputs such as creative works, and, more recently, translations and reports to external bodies. Each of these NTROs submitted for evaluation also required a research statement outlining the research background, contribution, and significance of the particular work. Consequently, for the past two iterations of the ERA at least, the evaluation of creative works and other NTROs have been more secure than many so-called “traditional” research publications since they have had to pass through three layers of peer-review. They have been scrutinised by editorial or peer-review selection processes by publishers, gallery directors, curators, and selection panels before publication. Post publication, the outputs have undergone verification and evaluation by Research Deans and Officers in each university and, finally, by external ERA reviewers. In addition to this, a number of universities, such as the University of Sydney, also appoint external peer assessors to oversee all creative research outputs collected in its research data repository. Peer judgement remains fundamental to evaluating all research, and the ARC returned to first principles and the use of the latest data management and visualisation to construct the comprehensive measures of quality for NTROs. In bringing recognition to creative arts research, the ARC has fortuitously developed a model that can become the standard procedure across all disciplines, including those currently using citation metrics, to evaluate research publications in future ERA exercises.

The ERA exercise has made manifest the ability of disciplinary authorities to evaluate research on a granulated five point scale of quality. Logically, it follows that the old HERDC two-step allocation of 5 research points for “A1” books and 1 research point for all other modes of publication or dissemination could be reconfigured to create a three-step 5/3/1 point allocation for individual research outputs.

The allocation of the highest ranking to books was always absurdly anomalous in its discipline exclusivity. Few would dispute the fact that most books that are published by reputable academic publishers represent a substantial investment of research time. However, so does a major discovery in physics or a massive longitudinal study in social science, reported in a journal article; equally, so does a major architectural, information technology or engineering innovation reported at a conference. The same applies to a significant novel, film, theatrical, or musical piece that provides insights into a culture, historical moment, or social issue, or to a major original public art work that encapsulates place and identity, or to an important exhibition that shifts perceptions of meaning and time.

Several Australian universities have developed protocols for internal recognition of a 5-point book equivalent for creative arts research and more granulated ranking beyond the binary 5/1. The universal application of a tripartite hierarchical ranking of outputs in future developments of the ERA exercise would bring the institutional ranking of quality of individual outputs closer to the ultimate five-point scaling at the discipline level.  Significantly, a “Substantial”, “Major”, and “Standard” designation, for example, could be applied to outputs regardless of their form or mode of production. By not being tied to particular publication forms, the focus would be on the quality of the research, and a particular book would not automatically rank as “Substantial”—just as a ground-breaking journal article could rank as “Substantial”.

The outmoded 5/1 binary ranking of research used in HERDC has no place in the tractable ERA methodology if it is to continue to present a finely-grained picture of research quality performance by discipline within each Australian institution.

Professor Ross Woodrow is Director of the Griffith Centre for Creative Arts Research and Deputy Director Research at the Queensland College of Art, Griffith University, Brisbane . He received his BA from the University of Queensland. His MPhil and PhD from the University of Sydney were both traditional thesis submissions although he has supervised to completion more than twenty studio-based Doctorates along with a number of thesis submissions.  He is the Executive and founding Editor of the peer-reviewed journal Studio Research and he has gained extensive experience with the ERA in preparation of submission data and as an ERA Reviewer.  As an artist and theorist his publication profile includes a wide range of traditional and creative outputs.

More from this issue

More from this issue

Independent artists are faced with a challenging and transforming landscape that requires adaptive resilience in order to thrive creatively, today and in the future. How do we, as tertiary educators, empower and enable artists to build strong and flexible, professional contemporary art practices? To address this issue, my current research draws models of praxis from artist-run initiatives (ARI) in the Visual Arts industry, specifically from my experience as director of Boxcopy Contemporary Art Space.

By Su Baker, President, Australian Council of Deans and Directors of Creative Arts — Over 2 decades the creative art academic community has grown and matured as a sector -  so have the questions of method and purpose of publically funded research, that influence the processes of evaluation. Discussions around impact and ‘end-user’ value is a live issue at the ARC and we look forward to the new thinking that will shortly emerge. The creative arts depend almost entirely on end-user experience, and the impact of these experiences aspire to have real and meaningful impact on peoples lives.
By Dr Jenny Wilson. DDCA’s Research officer Jenny Wilson caught up with Henk Borgdorff in Amsterdam in April 2016,  hot on the heels of his recent speaking tour of European and UK universities, art and music schools, to find out more about artistic research and European experiences of the politics of art and higher education.
By Professor Graeme Sullivan Visual arts has no singular function because it can be called on to do just about anything. Arts’ usefulness is because it is edgeless and homeless—art is masterful at shape shifting and form fitting
By Professor Jeri Kroll Since the Strand report (1998), scholars have been unpacking the manifold ways in which creative works can be research. Explaining the usefulness of questions to doctoral candidates not only keeps supervisors honest, but also keeps at the forefront of everyone’s mind why theory is unavoidable.
By Professor Paul Draper and Professor Scott Harrison Communities of profession, the old academy and the new academy, intimately rub up against each other and while some research may still be considered ‘more equal’ than others for now – this evolving mix can only positively impact on the rise of artistic research, its acceptance in society and its measurement by governments and universities.
By Associate Professor Cheryl Stock AM — The narrative of knowledge is almost always underpinned by the cognitive but how we know the world is often through the experiential. Whilst we have moved a long way in redefining knowledge in research terms to include the processes and outcomes of our practices (artistic, creative, professional) and importantly have privileged the artist’s voice as the expert in this recasting of what a knowledge claim might look like, some art forms prove more problematic than others in this endeavour.
By Dr Danny Butt — During the 1990s and 2000s, as readers of NiTRO know well, an intensive debate took place among art and design academics as to whether their practices and those of their graduate students could be called research, and if so what “contribution to knowledge” might be made by the creative output, as distinct from the writing that has traditionally accompanied submissions in higher degrees in creative arts.
By Professor Margaret Sheil — On my last outing in an ACUADS conference,  I was described by Flinders University’s Julian Meryick as the “artist’s ideal of a scientist… impatient with the reduction of everything down to short term utility.” So as I venture once again into the creative arts domain,  I draw on a scientific analogy. The principle of chemical equilibrium refers to a system in which the rate of consumption of inputs is the same as that at which outputs are produced so that the system is in a stable state of consumption and production.
By Associate Professor Robert Burke and Dr. Andrys Onsman — Criticism of the scientific methods of doing research has increasingly pointed out that all experimental research involves some sort of creative leap. In the performing arts such creative leaps are fundamental to artistry.
By Dr Leo Berkeley — The creative practice of filmmaking, understood as a form of academic research, has been growing in scale and significance within Australian universities for several years.  While doctorates involving the making of a film have been occurring for decades, it is only relatively recently that the academic screen production community has been seeking to more systematically establish how the production of a film can lead to the discovery of new knowledge.  
By Dr Jenny Wilson As many in creative arts grappled with the amalgamation challenges of the 90s, few were aware that the Dawkins reforms also had increased the centrality of research to university funding. This ‘blissful ignorance’ was not to last.
By Professor Brad Buckley and Associate Professor John Conomos — Recently, there has been much discussion in the press and beyond about the importance of Science, Technology, Engineering and Math (STEM) subjects at high school and at university. In particular, the Commonwealth Government’s National Innovation and Science Agenda has focused exclusively on STEM disciplines. However, that discussion misses the central importance of creativity, inventiveness and innovation.
By Dr Kate Tregloan and Professor Kit Wise — Interdisciplinarity has been widely recognised as a valuable response to the wicked problems of our time. The ability to collaborate across disciplinary boundaries brings together different perspectives and expertise, and allows entirely new approaches and solutions to emerge. To prepare students and graduates for the complex challenges of the twenty first century we need good quality interdisciplinary programs. But how do we know what is ‘good’?
By Professor Estelle Barrett and Professor Barbara Bolt — At a roundtable at the Australian Council of University Art Schools (ACUADS) annual conference in 2014, panelists were asked to address the following question: What impact are higher degree research programs having on emerging trends and themes in contemporary art? Whilst the panel felt that the development of higher degree research programs in creative arts did not lead to better “art” they did agree that it has profoundly affected the way art is framed and understood both within the academy and beyond.
By Professor Margaret Gardner AO — The Australian Government’s Federal Budget announcement in May was confirmation that funding for the Office for Learning and Teaching would be discontinued after this year. The news, though not unexpected, represented a blow to funding for teaching and learning scholarship in Australia.
By Dr Tim Cahill and Professor Julian Meyrick — ‘In God we trust.  All others bring data,’ quipped US statistician, W. Edwards Deeming.  As he implied, measurement is an inherently conservative occupation.  Units of appraisal have to be agreed in advance, while the aim of measuring something is usually to compare it with something that already exists.

By Julie Hare There are a lot of things that happen in universities that the majority of the population don’t know about. Research is one of them. The average punter – even the average undergraduate – would have little idea as the scope, scale and importance of research that takes place. And having a scientist […]

By Lynn Churchill and Jill Franz, IDEA (Interior Design Interior Architecture Educator’s Association) — IDEA comprises 12 International Institutions providing a minimum four-year Bachelor degree in the disciplines of Interior Design (ID), Interior Architecture (IA) and Spatial Design (SD). Most include an Honours program and the opportunity to undertake further research in Masters and PhD programs in compliance with the object of IDEA - excellence in ID/IA/SD education and research. Academic Research is a significant requirement for most academics in these disciplines.
By Associate Professor Denise Ferris and Professor Marie Sierra, Australian Council of University Art and Design Schools (ACUADS) — The National Innovation and Science Agenda, launched in December 2015, has significant consequences for tertiary institutions, and in particular, for the art and design disciplines, as well as the broader arts, humanities and social science (HASS) fields.