NiTRO Creative Matters

Perspectives on creative arts in higher education

Directions in Dance Technology Research

By Professor Kim VIncs — As I write, I’m in London, having spent the last day as a member of the User Board for EU Horizon 2020 project, WhoLoDance[i]. WhoLoDance is developing a motion capture data library of dance movement across the genres of ballet, contemporary dance, flamenco and Greek folk dance, and a suite of new technological tools for searching, matching, documenting, learning and sharing dance.

Movement based search engines, movement generation engines, holographic dance displays and augmented and virtual reality systems (AR and VR) for dance preservation and teaching are just some of the aims of this project. Effective knowledge transfer is essential to any contemporary discipline, and this is particularly true of the arts, since innovation, which is a core value of artistic practice, is predicated on exchanging, iterating and creatively building on current developments. Projects like WhoLoDance are demonstrating that knowledge transfer in dance takes on an entirely new set of meanings and possibilities in the context of recent developments in movement-based immersive interactive technologies such as virtual, augmented, and mixed reality (VR, AR, MR).

Technologies such as AR, VR and MR embed human movement behaviour directly within interactive systems. They enable the user to explore and interact with a 3D environment simply by moving. Force feedback systems enable users to literally feel the force of virtual objects in their hands and robotic systems complement and augment human movement capabilities. These technologies make it possible to exchange information movement-to-movement, as opposed to movement-to-language or language-to-movement. These systems enable, literally, a new dimensionality and level of embodied engagement with information and data. At the same time, advances in data processing and machine learning are creating a new capacity to meaningfully process the complexity of human movement data so that human/machine interaction can be based on sophisticated movement concepts and actions. As a result, embodied engagement and movement behaviour are now core design elements for new generation HCI interfaces.

This is a ‘moment’ for dance. From an effectively ephemeral form that disappears even as it is created,[ii] 3D motion capture technologies render dance capturable and immersive 3D visualisation technologies (e.g. AR, VR) render it re-mediatisable in ways that preserve dance’s three-dimensional spatiality and dynamics. These shifts open up new possibilities for documenting and transmitting dance through multisensory movement-based media. Conversely, scaffolding movement-based HCI on the ability of dance artists to create, understand and work creatively with movement forms offers the prospect of novel solutions to HCI design issues, along with a fine-grained sensory approach to designing embodied systems that would otherwise not be possible.

For many years, dance technology artists – myself included – have been investigating, exploring, developing and refining cyber-physical systems that visualise and sonify movement, and create the conditions for exchange through AI and machine learning techniques. This work has formed a foundational understanding of how to link movement performance with computational systems in ways that are not merely mechanical, but which leverage the embodied understanding of the body/self dyad inherent in dance practices. In 2001, Scott deLahunta’s ‘Software for Dancers’ project explored the potential of digital media and computation to help choreographers “…record and manipulate visual and audio data for sketching, indexing, commenting and preserving, presenting to its users (choreographers) the sense of an infinite mutability of such data.”[iii] This project re-conceptualized the relationship between dance and software as a dialogue between ways of thinking, and thereby re-positioned software as a tool for making as much as a means of representing dance movement.

These two approaches are converging in projects such as WhoLoDance, representing what I see as a maturation of practice based research methodology in the dance field. What I call practice-based dance research ‘1.0’ was about the realisation and recognition that knowledge creation is inherent in making art. This has been the backbone of the last twenty years of practice based artistic research. This realisation, and the various ways in which artists have sought to recognise, excavate and highlight the new knowledge created through and by their work, forms the conceptual foundation for ‘non-traditional research outputs,’ which are now the dominant form of research publication in the creative arts (> 50% in FOR19 in the last ERA exercise). This development has been invaluable, cementing the role of artistic practice within knowledge production, and positioning artform innovation as the first role of creative practice research. However, this knowledge tends to be specific to specific artists’ practices, and is not easily generalised. One might visualise this knowledge structure as a set of ‘spheres of influence’ emanating from individual nodes of practice.

In a digitally augmented world, knowledge is distributed, linking not through ‘spheres of influence’ but through direct communication one-to-many, many-to-one, many-to-many, through a network in which points are connected regardless of proximity. I see what I call practice based research ‘2.0’ as this kind of distributed model. 2.0 is about integrating artform innovation with innovation beyond the artform, so that knowledge generated links not only to other arts practices and to the context of the art world, but seamlessly with other domains, other communities, other industries and other knowledge systems.

Linking dance technology research with movement-based HCI development effects exactly this kind of shift, creating the conditions for simultaneous artform innovation and innovation beyond the artform. While this shift may displace ‘the artist’ from the centre of practice based research, it creates the conditions for placing ‘artists,’ as a group, at the centre of research that impacts both dance and the world beyond. I see this shift as a positive, even necessary step, not only for taking forward embodied HCI research with benefits within and beyond dance, but also as a structural shift in artistic research from a node-based to a distributed knowledge model.

Kim Vincs is Professor of Interactive Media, and Research Director within the Department of Film and Animation at Swinburne University of Technology. She is a leading researcher in the creative arts, with 6 Australian Research Council grants (2 Discovery, 3 Linkage and 1 LIEF), 35+ industry partnerships, and 20+ arts/science collaborations across fields including motion capture, game development, robotics, haptics, app design, 3D stereoscopy, artificial intelligence, virtual reality, augmented reality, cognitive psychology, biomechanics, mathematics, architecture and exercise science. Vincs’ industry partnerships include national and International companies such as Autodesk, Motion Analysis, Act3animation, Iloura, Alt.vfx, Arts Access Victoria, Victorian Opera and Australian Dance Theatre. She has commercial motion capture credits for several computer games, television commercials and film projects, including the Cannes Silver Lion winning Nocturnal Migration. She is currently developing a new centre for Transformative Technologies at Swinburne University of Technology with Professor Angela Ngdalianis

[i] WhoLoDance, (accessed 30 June 2017)

[ii] Phelan, P., (1993) Unmarked: The Politics of Performance, Routledge, New York & London.

[iii] Salter, C., (2010) Entangled: Technology and the Transformation of Performance, foreword by Peter Sellars MIT Press, Cambridge, Massachusetts, London, England, pp. 265–266.

More from this issue

Make Anew

By Oliver Smith — Art and technology. Creativity and invention. Curiosity and innovation. The artist can confidently claim a preeminent

Read More +

More from this issue

By Jenny Wilson — Collaborations across arts, science and technology are growing. It appears that the more that non-arts academics become involved with artists, the more they see how art can provide deeper insight into their own disciplines.
By Jenny Wilson — The first edition of NiTRO went live on 30 June 2016 with a focus on the future for creative arts amidst the flurry of government research and higher education reports.
By Professor Robyn Sloggett AM — Art practice and scientific enquiry have the same fundamental epistemological requirements; a broad and deep knowledge of the context in which they operate, the ability to contest and extend existing ontologies, and through this to be able to verify propositions in order to expand knowledge.
By Professor Frank Millward — Living the dream immersed in data...
By Professor Patricia Aufderheide — Australia has one of the most restrictive copyright regimes in the world. Australian creators have fewer ways to access unlicensed copyright material than almost anywhere else. 
By Professor Oron Catts — The life sciences are rapidly shifting into an engineering pursuit. This means that a new, problematic, challenging and performative palette of artistic possibilities opens up.
By Oliver Smith — Art and technology. Creativity and invention. Curiosity and innovation. The artist can confidently claim a preeminent position as a generator of new knowledge. Within the Academy the artist can play an important role in unlocking unforeseen research potential by imaginatively engaging with burgeoning technology.
By Dr Svenja J. Kratz and Anita Gowers — Since the establishment of UWA’s dedicated art and science lab  SymbioticA in 2000,  there has been a growing interest in fostering connections across art and science. An increasing number of academic staff and HDR students are working across the nexus of art and science at major Australian Universities including UNSW, Curtin, QUT, Monash and UTAS.
By Dr Jessica Seymour — Being a writer was always a romantic idea for me. I was a terribly antisocial child, and I liked the idea of hiding away in my bedroom for days – even months – on end and bashing away at a keyboard until a masterpiece came out. I was born in 1990, so a keyboard was part of my fantasy.
By Dr Kylie Pappalardo — Managing intellectual property can be challenging at the best of times, and university IP policies can add an extra layer of complexity for academics producing scholarly and creative works. This primer provides a short overview of the general legal principles likely to apply to creative arts practitioners working in the university sector.