By Associate Professor Sandra Gattenhof
Within the increasingly neo-liberalist world there is an obsession with numerical data to enable governments, businesses, NGOs and learning institutions to tell a story about value and impact of spending monies, both public and private. Such data is converted into easily communicated percentages, budget lines or graphs to demonstrate return on investment (ROI). Goldbard (2015) refers to this practice as ‘Datastan’ and argues that within the arts and culture there is a need to capture a more fulsome picture of arts participation and engagement by individuals and communities that moves beyond numbers on a page.
The debate in Australia, and internationally, about the best way to report value and impact of arts and cultural engagement by individuals and communities is not new. Belfiore (2015) notes that that a universal issue is situated around measurement of value attributed to arts and culture, particularly arts products and experiences supported through public funding. This creates a problem that ‘lies in the way in which the attribution of value to the outcome of aesthetic encounters has become part of the technocratic machinery of cultural policy-making’ (Belfiore, 2015, p. 97) and highlights that ‘arts and culture gives rise to forms of value that cannot be captured within the framework of mainstream, neo-classical economics’ (Carnwath & Brown 2014, p. 8). Arlene Goldbard (2015) says that arts and cultural impact measures are plagued by ‘the metrics effect’. Yet, it is such studies that are viewed more favorably by governments because as Goldbard (ibid.) notes they appear to take place within, ‘… a scientific framework, far less ‘soft’ than arguments from pleasure, beauty or meaning’. Most often the data captured is anchored in consumer behaviour around events allied to an arts experience – ticket buying, attendance figures, transport costs, food and accommodation – that provides little real evidence of the value or impact of an arts and cultural experience for individuals or communities. Goldbard (2015) believes that such studies may falsify results in that they are not providing evidence for what they say they are investigating, in this case engagement in an arts experience. What evaluations need to do is a build a more comprehensive picture of the ‘alterations in the quality of life’ (Brown and Trimboli, 2011, p. 617).
According to Radbourne, Glow & Johnson (2013) there has been some shift in governments and evaluation approaches in response to the requirement to evaluate and report on arts and cultural engagements. They believe that ‘there is now a movement to identify and measure the intrinsic qualities of the arts, whether these by artistic excellence, innovation or vibrancy’ (p. 5). Holden (2004) believes this is a ‘missing ingredient’ (p.22) in the value and impact debate. In a speech delivered to the Australia Council of the Arts Marketing Summit, titled On the Brink of a New Chapter: Arts in the 21st Century, Ben Cameron argued that arts organizations needed to rethink their relationship with communities and individuals. Cameron (2009) suggested three questions that arts organizations must answer in relation to the notion of value if they are to survive and have impact. Cameron framed the questions as:
- What is the value my organization brings to my community?
- What is the value my organization alone brings or brings better than anyone else?
- How would my community be damaged if we closed our doors and went away tomorrow?
While such questions could be seen within an economic framework of value, Cameron, in these questions, is challenging arts organizations to rethink the nature of the cultural task. Cameron says that arts organizations can no longer afford to ‘think of themselves as producers or presenters of cultural product, rather they are orchestrators of social interaction with communities who are seeking opportunities for interactivity, participation, access and engagement’ (Cameron, 2009, np.). So, if we take Cameron’s position of redefining the cultural task beyond an economic transaction or, as Belfiore classifies it, ‘economic doxa’ (2014, p. 95) but move it to a platform for social interaction, then it is imperative that the value equation ascribed to arts and culture is represented beyond numerical reportage. Goldbard says this overemphasis on numeric records that try to capture value and impact of arts engagement has placed us into ‘Datastan – the empire of scientism’ (2015, p. 214).
Goldband speaks about usurping ‘Datastan’ by allowing ‘artists and cultural policy-makers to convey cultural value and meaning with the tools best suited for that purpose: story, image, metaphor and experience’ (Goldbard, 2015, p. 226). Within this ethos Goldbard is calling for the impact of arts experiences to be reported through art forms and processes rather than bean counting. Whilst text and numeric data are ‘dominant in academic research – vital for production, measurement and dissemination of research findings’ (Durose et al., 2011, p.8) there is equally an interest in ‘beyond text tools’ (ibid.) including storytelling, performance, art and photography. Perhaps taking a performative research approach, using non-text based tools, may be able to provide an answer to Goldbard’s call to arms. Within the creative practice as research field the findings of a project are reported as performative data. These delivered by the researcher though new artistic forms for performance and exhibition, or designs for user-led, online games (Haseman, 2006, p.3). For Goldbard the capture of arts experience may take the form of digital stories – short, first person sound-and-image video narratives – to capture participants’ experience and response … or a participatory photography project in which images and captions show what needs preservation and what needs development. (Goldbard, 2015, p. 226).
Current evaluation methods dominated by quantitative methods are, according to Holden (2004, p. 17), ‘increasingly being questioned, both in terms of the utility of methodologies employed and the extent to which the results illuminate our understanding’. Commentators such as Goldbard (2015) and Blomkamp (2015) suggest that ‘… a plurality of approaches to measuring culture and understanding cultural change may be desirable’ (Blomkamp, 2015, p. 22). I believe it is time to challenge not only what is valued as evidence, including what data collection tools are used to measure value and impact, but it is time to rethink the transmission process of reporting findings. Using arts methods and processes to report on arts engagement could be the new frontier in arts and cultural measurement.
References
Belfiore, E. (2015). Foreword. In MacDowall, L., Badham, M., Blomkamp, E. and Dunphy, K. (Eds.), Making Culture Count: The Politics of Cultural Measurement. (pp. ix-xi). Houndsmills, United Kingdom: Palgrave Macmillan.
Blomkamp, E. (2015). A Critical History of Cultural Indicators. In MacDowall, L., Badham, M., Blomkamp, E. and Dunphy, K. (Eds.), Making Culture Count: The Politics of Cultural Measurement. (pp. 11-26). Houndsmills, United Kingdom: Palgrave Macmillan.
Brown, S. & Trimboli, D. (2011). The real “worth” of festivals: Challenges for measuring socio-cultural impacts. Asia Pacific Journal of Arts and Cultural Management. 8(1), 616–29.
Carnwath, J.D. and Brown, A.S. (2014). Understanding the Value and Impacts of Cultural Experiences. Manchester, United Kingdom: Arts Council England. Retrieved July 27, 2014 from, http://www.artscouncil.org.uk/publication/understanding-value-and-impacts-cultural-experiences
Durose, C., Beebeejaun, Y., Rees, J., Richardson, J. and Richardson, L. (2011). Towards Co-production in Research With Communities, Swindon: AHRC. Retrieved from, http://www.ahrc.ac.uk/documents/project-reports-and-reviews/connected-communities/towards-co-production-in-research-with-communities/
Goldbard, A. (2015). The Metrics Syndrome: Cultural Scientism and Its Discontents. In MacDowall, L., Badham, M., Blomkamp, E. and Dunphy, K. (Eds.), Making Culture Count: The Politics of Cultural Measurement. (pp. 214-227). Houndsmills, United Kingdom: Palgrave Macmillan.
Haseman, B. (2006). A Manifesto for Performative Research. Media
International Australia incorporating Culture and Policy, theme issue “Practice-led Research” (118), 98–106.
Holden, J. (2004). Capturing Cultural Value: How Culture Has Become a Tool of Government Policy. London: Demos. Retrieved October 22, 2014, from www.demos.co.uk/files/CapturingCulturalValue.pdf
Radbourne, J., Glow, H. and Johanson, K. (2013). Knowing and Measuring the Audience Experience. In Radbourne, J., Glow, H. and Johanson, K. (Eds.), The Audience Experience: a critical analysis of audiences in the performing arts. (pp. 3–13). Bristol, United Kingdom: Intellect.
Sandra Gattenhof is Associate Professor and Discipline Leader (Dance, Drama, Music) in the Creative Industries Faculty, Queensland University of Technology. She is co-leader of the Creative Education and Creative Workforce theme in the newly established Creative Lab at QUT. Sandra has published widely on the outcomes of arts intervention programs in low socio-economic school communities, and is regarded as a leader in the field of the evaluation of cultural programs, having completed studies on, regional arts engagement programs, national arts festivals, and international performing arts markets. Sandra’s latest book Evaluating Impact: Models for Evaluation in the Australian Arts and Culture Landscape was published this year by Palgrave.