NiTRO Creative Matters

Perspectives on creative arts in higher education

The Rough Magic of Engagement Measurement

By Dr Tim Cahill and Professor Julian Meyrick — ‘In God we trust.  All others bring data,’ quipped US statistician, W. Edwards Deeming.  As he implied, measurement is an inherently conservative occupation.  Units of appraisal have to be agreed in advance, while the aim of measuring something is usually to compare it with something that already exists.

Then there is the problem of the real-world relations to which numbers are supposed to correspond. As Australian universities pour over their ERA results, they may be wondering how an entire spectrum of research effort has been squeezed into a cardinal scale of 1 to 5.

Here, measurement is at best rough magic. What makes it meaningful are the good faith intentions behind the evaluative process

The more qualitative a phenomenon, the more the correspondence with quantitative indicators breaks down. Here, measurement is at best rough magic. What makes it meaningful is the good faith intentions behind the evaluative process.  Many things elude exact computation.  But the effort we make to count them brings a degree of insight, and sometimes important change.

It is for this reason that the new assessment of ‘research engagement’ currently under consideration by the ARC is to be welcomed, albeit with caveats.

The government’s proposal to dilute the link between university research funding and peer reviewed publications has been greeted with initial gloom, especially by researchers in the Humanities, Arts and Social Sciences (HASS), who some see as dependent on this research output.

But actually HASS has no particular claim on that score. Based on the latest ERA data, these disciplines publish less than the medical and health sciences, biological sciences, information and computing sciences, and engineering. Three of the five smallest 2-digit disciplines as measured by publications are HASS.

A bigger problem is universities’ reliance on the three-year grant cycle, which is antithetical to the kind of long-term, enquiry-led research HASS is good at. Under our dual funding system, not only is around 50% of support delivered via large ARC and NHMRC grants, but these are also the principal inputs into the research block grant funding formulae – a legacy of the Dawkins reforms of the 1980s.

Meanwhile, income generated from working with public, private and not-for-profit sector partners, including income in kind, barely rates a mention.

These formulae hard-wire in certain academic practices, forcing researchers to focus on large grant-getting – which is predicated on journal articles to boost track records – at the expense of other activities. Are peer-reviewed academic articles the acme of research excellence?  The reality is surely more complicated.

For HASS the fit is poor because its research is often not well suited to large grant-getting. While STEM can spend endlessly on research assistants and specialised equipment, the majority of HASS costs are for teaching relief and travel. HASS researchers face a system of incentives that does not meet their real needs.  What they need is time and space to engage a diversity of problems in a plurality of ways.

To derive maximum value from publicly funded research requires two conditions.  First, it must be accessible, linguistically and physically.  Second, there must be a user capable of deriving value by applying it.  In many cases, this may mean writing an academic journal article. But a monocular focus on producing articles aimed at other academics is rife with assumptions antithetical to unlocking research value, including the hazard of impenetrable language, and the profit motives of academic publishers who lock knowledge away behind steep pay walls.

Professor Cameron Neylon addressed just this problem in a recent speech for Open Access Week when he argued

Knowledge is not a public good.  It is a club good.  [It]… is excludable by the simple expedient of not telling anyone else.  Through communication and dissemination we make it less exclusive… but we can never entirely eliminate exclusion.  We can only ever invest in reducing it… [This] recentres the question of how best to invest limited resources in making things more open, more public and less exclusive.  Which audiences should we target?  How best to enable unexpected contributions?  How to maximize network benefits?

In the creative arts disciplines, home of the non-traditional output, these issues are well understood.  The engagement agenda is potentially a way of recognising a broad range of research equivalent activities, many of which have sizeable audiences and impact. Architecture and design are the obvious areas in which community outcomes do not typically correspond to countable outputs in a way that promotes maximum social benefit. But the cultural industries as a whole – museums, theatres, galleries, cinema and so on – are all based on the idea of broad public engagement.

A rebalancing of funding to provide more recognition of income derived from public, private and not-for-profit sectors, plus delivering a larger portion of funding through block grants, would go a long way to democratising university research.

But there is a caveat.  For the engagement agenda to work requires we do not use it solely as a synonym for commercialisation.  It must be considered in the wider context of all the social, cultural, economic and environmental benefits it generates. This will be hard in a climate where the bottom line is increasingly the only line, and governments are focused on short-term economic factors to the exclusion of almost everything else.

Yet it ought not to hold up consideration of the overall aim of research that has for too long seen HASS measure itself in ways that do not reflect its original contributions.

Engagement is potentially not only an expansive but a progressive move. The challenge is how to ensure it is not used as a ‘thin’ indicator or as a proxy for business savvy.

Dr Tim Cahill is Chief Data Scientist with The Conversation, Director, Research Strategies Australia and an Adjunct Research Fellow at Swinburne University of Technology.  Professor Julian Meyrick is Strategic Professor of Creative Arts at Flinders University and Artistic Counsel, State Theatre Company of South Australia.

More from this issue

More from this issue

Independent artists are faced with a challenging and transforming landscape that requires adaptive resilience in order to thrive creatively, today and in the future. How do we, as tertiary educators, empower and enable artists to build strong and flexible, professional contemporary art practices? To address this issue, my current research draws models of praxis from artist-run initiatives (ARI) in the Visual Arts industry, specifically from my experience as director of Boxcopy Contemporary Art Space.

By Dr Jenny Wilson. DDCA’s Research officer Jenny Wilson caught up with Henk Borgdorff in Amsterdam in April 2016, hot on the heels of his recent speaking tour of European and UK universities, art and music schools, to find out more about artistic research and European experiences of the politics of art and higher education.
By Professor Jeri Kroll Since the Strand report (1998), scholars have been unpacking the manifold ways in which creative works can be research. Explaining the usefulness of questions to doctoral candidates not only keeps supervisors honest, but also keeps at the forefront of everyone’s mind why theory is unavoidable.
By Professor Paul Draper and Professor Scott Harrison Communities of profession, the old academy and the new academy, intimately rub up against each other and while some research may still be considered ‘more equal’ than others for now – this evolving mix can only positively impact on the rise of artistic research, its acceptance in society and its measurement by governments and universities.
By Associate Professor Cheryl Stock AM — The narrative of knowledge is almost always underpinned by the cognitive but how we know the world is often through the experiential. Whilst we have moved a long way in redefining knowledge in research terms to include the processes and outcomes of our practices (artistic, creative, professional) and importantly have privileged the artist’s voice as the expert in this recasting of what a knowledge claim might look like, some art forms prove more problematic than others in this endeavour.
By Dr Danny Butt — During the 1990s and 2000s, as readers of NiTRO know well, an intensive debate took place among art and design academics as to whether their practices and those of their graduate students could be called research, and if so what “contribution to knowledge” might be made by the creative output, as distinct from the writing that has traditionally accompanied submissions in higher degrees in creative arts.
By Professor Margaret Sheil — On my last outing in an ACUADS conference, I was described by Flinders University’s Julian Meryick as the “artist’s ideal of a scientist… impatient with the reduction of everything down to short term utility.” So as I venture once again into the creative arts domain, I draw on a scientific analogy. The principle of chemical equilibrium refers to a system in which the rate of consumption of inputs is the same as that at which outputs are produced so that the system is in a stable state of consumption and production.
By Professor Ross Woodrow — The decision by the Australia Research Council (ARC) to achieve the long-mooted merging of the Higher Education Research Data Collection (HERDC) and the Excellence in Research for Australia (ERA) exercise by adeptly disappearing the HERDC has been welcomed by many discipline leaders, and not just those in the creative arts. With the inclusive ERA becoming the singular evaluation of research quality across Australia, there couldn’t be a better time to rethink the classification of research in universities.
By Associate Professor Robert Burke and Dr. Andrys Onsman — Criticism of the scientific methods of doing research has increasingly pointed out that all experimental research involves some sort of creative leap. In the performing arts such creative leaps are fundamental to artistry.
By Dr Leo Berkeley — The creative practice of filmmaking, understood as a form of academic research, has been growing in scale and significance within Australian universities for several years. While doctorates involving the making of a film have been occurring for decades, it is only relatively recently that the academic screen production community has been seeking to more systematically establish how the production of a film can lead to the discovery of new knowledge.
By Dr Kate Tregloan and Professor Kit Wise — Interdisciplinarity has been widely recognised as a valuable response to the wicked problems of our time. The ability to collaborate across disciplinary boundaries brings together different perspectives and expertise, and allows entirely new approaches and solutions to emerge. To prepare students and graduates for the complex challenges of the twenty first century we need good quality interdisciplinary programs. But how do we know what is ‘good’?
By Su Baker, President, Australian Council of Deans and Directors of Creative Arts — At the DDCA annual conference in Adelaide in 2015, a group of 25 leaders in the creative arts engaged in rigorous and expansive discussion following a series of highly astute commentary and presentations by invited colleagues. Our goal was to determine how to advance our profession amidst the volatility of the higher education sector.
By Jenny Wilson — Welcome to the first edition of NiTRO, DDCA’s dedicated space for views and news in the tertiary creative arts community. Every six weeks we explore an issue of particular interest to creative artists practicing in the university sector. Our first edition focuses on the changing higher education landscape as we ask: Watt’s next for creative arts?
By Professor Margaret Gardner AO — The Australian Government’s Federal Budget announcement in May was confirmation that funding for the Office for Learning and Teaching would be discontinued after this year. The news, though not unexpected, represented a blow to funding for teaching and learning scholarship in Australia.

By Julie Hare There are a lot of things that happen in universities that the majority of the population don’t know about. Research is one of them. The average punter – even the average undergraduate – would have little idea as the scope, scale and importance of research that takes place. And having a scientist […]

By Lynn Churchill and Jill Franz, IDEA (Interior Design Interior Architecture Educator’s Association) — IDEA comprises 12 International Institutions providing a minimum four-year Bachelor degree in the disciplines of Interior Design (ID), Interior Architecture (IA) and Spatial Design (SD). Most include an Honours program and the opportunity to undertake further research in Masters and PhD programs in compliance with the object of IDEA - excellence in ID/IA/SD education and research. Academic Research is a significant requirement for most academics in these disciplines.
By Associate Professor Denise Ferris and Professor Marie Sierra, Australian Council of University Art and Design Schools (ACUADS) — The National Innovation and Science Agenda, launched in December 2015, has significant consequences for tertiary institutions, and in particular, for the art and design disciplines, as well as the broader arts, humanities and social science (HASS) fields.
By Dr Lynda Hawryluk, Australasian Association of Writing Programs — The ever-changing higher education landscape affects all disciplines and their related industries in a variety of ways, and the creative arts discipline is not immune to these changes and challenges.
By John Cumming and Craig Batty, Australian Screen Production, Education and Research Association (ASPERA) — Australia is engaged in completing a transition that will result in academic endeavor being placed squarely within the ‘triple helix’ of university-industry-government. Priority research projects will be those that can secure funding from ‘end-users’. The challenge for researchers is now shifting from publication and peer review to benchmarks of impact and industry engagement.