Peer Review in Filmmaking: A Contested Process

By Dr Leo Berkeley — The creative practice of filmmaking, understood as a form of academic research, has been growing in scale and significance within Australian universities for several years.  While doctorates involving the making of a film have been occurring for decades, it is only relatively recently that the academic screen production community has been seeking to more systematically establish how the production of a film can lead to the discovery of new knowledge.  

As part of this process, there has been interest in developing an approach to the peer review of screen works that evaluates the research in the creative work, leading then to some form of audiovisual publication.  In Australia, the Sightlines Journal is a recent example of this, following international precedents like The Journal of Artistic Research, Screenworks and Audiovisual Thinking.

Within many established academic research disciplines, the concept of peer review has been seen as central to the process by which new research is recognised and validated.  While it has its critics, peer review is widely supported as a means by which appropriate standards are reached prior to the publication of research, in relation to criteria of originality, rigour and significance to the field.  However, there are both conceptual and practical challenges in applying this approach when the research artefact is something like a fiction film.

The process of peer review is one that cannot simply be transferred without modification from text-based publishing to screen production practice.  Requirements that the ‘author’ is anonymous, that the creative work is modified in response to peer review feedback and even the criteria under which the research should be evaluated, are all contested issues. For example, there are often financial and logistical obstacles to removing a filmmaker’s name from the credits, as there are to reworking a film after it has been completed, in response to peer feedback.

As with other forms of creative arts practice, an original contribution to knowledge in a film is often implicit, or related to the production process, and so not clearly evident through a viewing of the finished work.  It is rarely communicated through an explicit rational, logical argument. . .

Furthermore, as with other forms of creative arts practice, an original contribution to knowledge in a film is often implicit, or related to the production process, and so not clearly evident through a viewing of the finished work.  It is rarely communicated through an explicit rational, logical argument but through other means: through the use of framing, movement, light and colour; through the fleeting expression on a person’s face; through the selection and ordering of moving images and sounds in complex patterns of relation; and in many other ways that provoke a response in viewers.  Understanding this as a process involving the communication of knowledge requires a broadening of the concept to include sensory and affective knowing, and there is a substantial body of scholarship in philosophy, social sciences, cultural studies and the creative arts that supports this position.  However, conceiving of knowledge in this way does not overcome the ambiguity seemingly inherent in audiovisual communication and so does not address the difficulties for peer reviewers when evaluating creative screen works as research.

The most common way to address these difficulties is through the use of written text, in the form of a statement that points to the research contribution and significance.  There is some resistance to this approach, a feeling that the meaning conveyed through moving images and sounds cannot adequately be translated into text-based language.  Some creative practitioners also argue that, in an increasingly audiovisual world, the broader academic research community and the formal institutional structures that evaluate research should be more open to diverse modes of expression.  However, a sizeable group of screen practice academics recognise that written text has a role to play in the external recognition of screen production research and that, within the discipline, there is a need to develop better ways to articulate on what basis a creative screen work can be regarded as research.

The first Sightlines event, a hybrid conference/film festival focused on filmmaking in the academy, debated this issue in late 2014.  The decision was made to establish a refereed audiovisual journal to explore and trial approaches to evaluating filmmaking as research.

The Sightlines Journal was established in 2015, with submitted films being peer reviewed with an optional research statement.  An approach was taken to use peer review in a deliberately open manner, to encourage a wide exchange of views and ideas with the objective of developing shared understandings of screen production research within the academic discipline.  Sightlines is running again later this year, with an impressive line-up of films but with questions still remaining around the issue of peer review.

Creative work is submitted for peer review after it is finished and the process is robbed of much of its potential as a form of discussion within the discipline around research. Changing this situation requires a culture change within the academic community. . .

Some of these questions result from the fact that, even among filmmaker-academics, making a film as a film comes before making the film as research.  So the creative work is submitted for peer review after it is finished and the process is robbed of much of its potential as a form of discussion within the discipline around research.  Changing this situation requires a culture change within the academic community, where peer review is seen as a more prominent part of the process when films are made in an academic context, with films being submitted for review before they are finished.  Encouraging this change is one objective of the 2016 Sightlines event and can be seen elsewhere, such as in the Journal of Media Practice’s current ‘Disrupted’ issue.

If peer review is to suit the needs of non-traditional, non-text-based forms of research inquiry, new academic publication platforms need to be explored and developed.

Leo Berkeley is a senior lecturer within the School of Media and Communication at RMIT University in Melbourne, Australia.  He also has considerable experience as an independent filmmaker, having written and directed the feature film, Holidays on the River Yarra, which was an official selection for the Cannes Film Festival in 1991.  His current research and production interests are in the practice of screen production, low and micro-budget filmmaking, improvisation, essay films, community media, mobile media and machinima. See leoberkeley.com for more details.

More from this issue

More from this issue

Independent artists are faced with a challenging and transforming landscape that requires adaptive resilience in order to thrive creatively, today and in the future. How do we, as tertiary educators, empower and enable artists to build strong and flexible, professional contemporary art practices? To address this issue, my current research draws models of praxis from artist-run initiatives (ARI) in the Visual Arts industry, specifically from my experience as director of Boxcopy Contemporary Art Space.

By Su Baker, President, Australian Council of Deans and Directors of Creative Arts — Over 2 decades the creative art academic community has grown and matured as a sector - so have the questions of method and purpose of publically funded research, that influence the processes of evaluation. Discussions around impact and ‘end-user’ value is a live issue at the ARC and we look forward to the new thinking that will shortly emerge. The creative arts depend almost entirely on end-user experience, and the impact of these experiences aspire to have real and meaningful impact on peoples lives.
By Dr Jenny Wilson. DDCA’s Research officer Jenny Wilson caught up with Henk Borgdorff in Amsterdam in April 2016, hot on the heels of his recent speaking tour of European and UK universities, art and music schools, to find out more about artistic research and European experiences of the politics of art and higher education.
By Professor Graeme Sullivan Visual arts has no singular function because it can be called on to do just about anything. Arts’ usefulness is because it is edgeless and homeless—art is masterful at shape shifting and form fitting
By Professor Jeri Kroll Since the Strand report (1998), scholars have been unpacking the manifold ways in which creative works can be research. Explaining the usefulness of questions to doctoral candidates not only keeps supervisors honest, but also keeps at the forefront of everyone’s mind why theory is unavoidable.
By Professor Paul Draper and Professor Scott Harrison Communities of profession, the old academy and the new academy, intimately rub up against each other and while some research may still be considered ‘more equal’ than others for now – this evolving mix can only positively impact on the rise of artistic research, its acceptance in society and its measurement by governments and universities.
By Associate Professor Cheryl Stock AM — The narrative of knowledge is almost always underpinned by the cognitive but how we know the world is often through the experiential. Whilst we have moved a long way in redefining knowledge in research terms to include the processes and outcomes of our practices (artistic, creative, professional) and importantly have privileged the artist’s voice as the expert in this recasting of what a knowledge claim might look like, some art forms prove more problematic than others in this endeavour.
By Dr Danny Butt — During the 1990s and 2000s, as readers of NiTRO know well, an intensive debate took place among art and design academics as to whether their practices and those of their graduate students could be called research, and if so what “contribution to knowledge” might be made by the creative output, as distinct from the writing that has traditionally accompanied submissions in higher degrees in creative arts.
By Professor Margaret Sheil — On my last outing in an ACUADS conference, I was described by Flinders University’s Julian Meryick as the “artist’s ideal of a scientist… impatient with the reduction of everything down to short term utility.” So as I venture once again into the creative arts domain, I draw on a scientific analogy. The principle of chemical equilibrium refers to a system in which the rate of consumption of inputs is the same as that at which outputs are produced so that the system is in a stable state of consumption and production.
By Professor Ross Woodrow — The decision by the Australia Research Council (ARC) to achieve the long-mooted merging of the Higher Education Research Data Collection (HERDC) and the Excellence in Research for Australia (ERA) exercise by adeptly disappearing the HERDC has been welcomed by many discipline leaders, and not just those in the creative arts. With the inclusive ERA becoming the singular evaluation of research quality across Australia, there couldn’t be a better time to rethink the classification of research in universities.
By Associate Professor Robert Burke and Dr. Andrys Onsman — Criticism of the scientific methods of doing research has increasingly pointed out that all experimental research involves some sort of creative leap. In the performing arts such creative leaps are fundamental to artistry.
By Dr Jenny Wilson As many in creative arts grappled with the amalgamation challenges of the 90s, few were aware that the Dawkins reforms also had increased the centrality of research to university funding. This ‘blissful ignorance’ was not to last.
By Professor Brad Buckley and Associate Professor John Conomos — Recently, there has been much discussion in the press and beyond about the importance of Science, Technology, Engineering and Math (STEM) subjects at high school and at university. In particular, the Commonwealth Government’s National Innovation and Science Agenda has focused exclusively on STEM disciplines. However, that discussion misses the central importance of creativity, inventiveness and innovation.
By Dr Kate Tregloan and Professor Kit Wise — Interdisciplinarity has been widely recognised as a valuable response to the wicked problems of our time. The ability to collaborate across disciplinary boundaries brings together different perspectives and expertise, and allows entirely new approaches and solutions to emerge. To prepare students and graduates for the complex challenges of the twenty first century we need good quality interdisciplinary programs. But how do we know what is ‘good’?
By Professor Estelle Barrett and Professor Barbara Bolt — At a roundtable at the Australian Council of University Art Schools (ACUADS) annual conference in 2014, panelists were asked to address the following question: What impact are higher degree research programs having on emerging trends and themes in contemporary art? Whilst the panel felt that the development of higher degree research programs in creative arts did not lead to better “art” they did agree that it has profoundly affected the way art is framed and understood both within the academy and beyond.
By Professor Margaret Gardner AO — The Australian Government’s Federal Budget announcement in May was confirmation that funding for the Office for Learning and Teaching would be discontinued after this year. The news, though not unexpected, represented a blow to funding for teaching and learning scholarship in Australia.
By Dr Tim Cahill and Professor Julian Meyrick — ‘In God we trust. All others bring data,’ quipped US statistician, W. Edwards Deeming. As he implied, measurement is an inherently conservative occupation. Units of appraisal have to be agreed in advance, while the aim of measuring something is usually to compare it with something that already exists.

By Julie Hare There are a lot of things that happen in universities that the majority of the population don’t know about. Research is one of them. The average punter – even the average undergraduate – would have little idea as the scope, scale and importance of research that takes place. And having a scientist […]

By Lynn Churchill and Jill Franz, IDEA (Interior Design Interior Architecture Educator’s Association) — IDEA comprises 12 International Institutions providing a minimum four-year Bachelor degree in the disciplines of Interior Design (ID), Interior Architecture (IA) and Spatial Design (SD). Most include an Honours program and the opportunity to undertake further research in Masters and PhD programs in compliance with the object of IDEA - excellence in ID/IA/SD education and research. Academic Research is a significant requirement for most academics in these disciplines.
By Associate Professor Denise Ferris and Professor Marie Sierra, Australian Council of University Art and Design Schools (ACUADS) — The National Innovation and Science Agenda, launched in December 2015, has significant consequences for tertiary institutions, and in particular, for the art and design disciplines, as well as the broader arts, humanities and social science (HASS) fields.