Movement based search engines, movement generation engines, holographic dance displays and augmented and virtual reality systems (AR and VR) for dance preservation and teaching are just some of the aims of this project. Effective knowledge transfer is essential to any contemporary discipline, and this is particularly true of the arts, since innovation, which is a core value of artistic practice, is predicated on exchanging, iterating and creatively building on current developments. Projects like WhoLoDance are demonstrating that knowledge transfer in dance takes on an entirely new set of meanings and possibilities in the context of recent developments in movement-based immersive interactive technologies such as virtual, augmented, and mixed reality (VR, AR, MR).
Technologies such as AR, VR and MR embed human movement behaviour directly within interactive systems. They enable the user to explore and interact with a 3D environment simply by moving. Force feedback systems enable users to literally feel the force of virtual objects in their hands and robotic systems complement and augment human movement capabilities. These technologies make it possible to exchange information movement-to-movement, as opposed to movement-to-language or language-to-movement. These systems enable, literally, a new dimensionality and level of embodied engagement with information and data. At the same time, advances in data processing and machine learning are creating a new capacity to meaningfully process the complexity of human movement data so that human/machine interaction can be based on sophisticated movement concepts and actions. As a result, embodied engagement and movement behaviour are now core design elements for new generation HCI interfaces.
This is a ‘moment’ for dance. From an effectively ephemeral form that disappears even as it is created,[ii] 3D motion capture technologies render dance capturable and immersive 3D visualisation technologies (e.g. AR, VR) render it re-mediatisable in ways that preserve dance’s three-dimensional spatiality and dynamics. These shifts open up new possibilities for documenting and transmitting dance through multisensory movement-based media. Conversely, scaffolding movement-based HCI on the ability of dance artists to create, understand and work creatively with movement forms offers the prospect of novel solutions to HCI design issues, along with a fine-grained sensory approach to designing embodied systems that would otherwise not be possible.
For many years, dance technology artists – myself included – have been investigating, exploring, developing and refining cyber-physical systems that visualise and sonify movement, and create the conditions for exchange through AI and machine learning techniques. This work has formed a foundational understanding of how to link movement performance with computational systems in ways that are not merely mechanical, but which leverage the embodied understanding of the body/self dyad inherent in dance practices. In 2001, Scott deLahunta’s ‘Software for Dancers’ project explored the potential of digital media and computation to help choreographers “…record and manipulate visual and audio data for sketching, indexing, commenting and preserving, presenting to its users (choreographers) the sense of an infinite mutability of such data.”[iii] This project re-conceptualized the relationship between dance and software as a dialogue between ways of thinking, and thereby re-positioned software as a tool for making as much as a means of representing dance movement.
These two approaches are converging in projects such as WhoLoDance, representing what I see as a maturation of practice based research methodology in the dance field. What I call practice-based dance research ‘1.0’ was about the realisation and recognition that knowledge creation is inherent in making art. This has been the backbone of the last twenty years of practice based artistic research. This realisation, and the various ways in which artists have sought to recognise, excavate and highlight the new knowledge created through and by their work, forms the conceptual foundation for ‘non-traditional research outputs,’ which are now the dominant form of research publication in the creative arts (> 50% in FOR19 in the last ERA exercise). This development has been invaluable, cementing the role of artistic practice within knowledge production, and positioning artform innovation as the first role of creative practice research. However, this knowledge tends to be specific to specific artists’ practices, and is not easily generalised. One might visualise this knowledge structure as a set of ‘spheres of influence’ emanating from individual nodes of practice.
In a digitally augmented world, knowledge is distributed, linking not through ‘spheres of influence’ but through direct communication one-to-many, many-to-one, many-to-many, through a network in which points are connected regardless of proximity. I see what I call practice based research ‘2.0’ as this kind of distributed model. 2.0 is about integrating artform innovation with innovation beyond the artform, so that knowledge generated links not only to other arts practices and to the context of the art world, but seamlessly with other domains, other communities, other industries and other knowledge systems.
Linking dance technology research with movement-based HCI development effects exactly this kind of shift, creating the conditions for simultaneous artform innovation and innovation beyond the artform. While this shift may displace ‘the artist’ from the centre of practice based research, it creates the conditions for placing ‘artists,’ as a group, at the centre of research that impacts both dance and the world beyond. I see this shift as a positive, even necessary step, not only for taking forward embodied HCI research with benefits within and beyond dance, but also as a structural shift in artistic research from a node-based to a distributed knowledge model.
Kim Vincs is Professor of Interactive Media, and Research Director within the Department of Film and Animation at Swinburne University of Technology. She is a leading researcher in the creative arts, with 6 Australian Research Council grants (2 Discovery, 3 Linkage and 1 LIEF), 35+ industry partnerships, and 20+ arts/science collaborations across fields including motion capture, game development, robotics, haptics, app design, 3D stereoscopy, artificial intelligence, virtual reality, augmented reality, cognitive psychology, biomechanics, mathematics, architecture and exercise science. Vincs’ industry partnerships include national and International companies such as Autodesk, Motion Analysis, Act3animation, Iloura, Alt.vfx, Arts Access Victoria, Victorian Opera and Australian Dance Theatre. She has commercial motion capture credits for several computer games, television commercials and film projects, including the Cannes Silver Lion winning Nocturnal Migration. She is currently developing a new centre for Transformative Technologies at Swinburne University of Technology with Professor Angela Ngdalianis
[i] WhoLoDance, http://www.wholodance.eu/ (accessed 30 June 2017)
[ii] Phelan, P., (1993) Unmarked: The Politics of Performance, Routledge, New York & London.
[iii] Salter, C., (2010) Entangled: Technology and the Transformation of Performance, foreword by Peter Sellars MIT Press, Cambridge, Massachusetts, London, England, pp. 265–266.