NiTRO Creative Matters

Perspectives on creative arts in higher education

Challenging the prevalence of ‘Datastan’ in arts and cultural evaluation

Within the increasingly neo-liberalist world there is an obsession with numerical data to enable governments, businesses, NGOs and learning institutions to tell a story about value and impact of spending monies, both public and private. Such data is converted into easily communicated percentages, budget lines or graphs to demonstrate return on investment (ROI). Goldbard (2015) refers to this practice as ‘Datastan XE "Datastan"

By Associate Professor Sandra Gattenhof

Within the increasingly neo-liberalist world there is an obsession with numerical data to enable governments, businesses, NGOs and learning institutions to tell a story about value and impact of spending monies, both public and private. Such data is converted into easily communicated percentages, budget lines or graphs to demonstrate return on investment (ROI). Goldbard (2015) refers to this practice as ‘Datastan’ and argues that within the arts and culture there is a need to capture a more fulsome picture of arts participation and engagement by individuals and communities that moves beyond numbers on a page.

The debate in Australia, and internationally, about the best way to report value and impact of arts and cultural engagement by individuals and communities is not new. Belfiore (2015) notes that that a universal issue is situated around measurement of value attributed to arts and culture, particularly arts products and experiences supported through public funding. This creates a problem that ‘lies in the way in which the attribution of value to the outcome of aesthetic encounters has become part of the technocratic machinery of cultural policy-making’ (Belfiore, 2015, p. 97) and highlights that ‘arts and culture gives rise to forms of value that cannot be captured within the framework of mainstream, neo-classical economics’ (Carnwath & Brown 2014, p. 8). Arlene Goldbard (2015) says that arts and cultural impact measures are plagued by ‘the metrics effect’. Yet, it is such studies that are viewed more favorably by governments because as Goldbard (ibid.) notes they appear to take place within, ‘… a scientific framework, far less ‘soft’ than arguments from pleasure, beauty or meaning’. Most often the data captured is anchored in consumer behaviour around events allied to an arts experience – ticket buying, attendance figures, transport costs, food and accommodation – that provides little real evidence of the value or impact of an arts and cultural experience for individuals or communities. Goldbard (2015) believes that such studies may falsify results in that they are not providing evidence for what they say they are investigating, in this case engagement in an arts experience. What evaluations need to do is a build a more comprehensive picture of the ‘alterations in the quality of life’ (Brown and Trimboli, 2011, p. 617).

Most often the data captured is anchored in consumer behaviour around events allied to an arts experience – ticket buying, attendance figures, transport costs, food and accommodation – that provides little real evidence of the value or impact of an arts and cultural experience for individuals or communities.

According to Radbourne, Glow & Johnson (2013) there has been some shift in governments and evaluation approaches in response to the requirement to evaluate and report on arts and cultural engagements. They believe that ‘there is now a movement to identify and measure the intrinsic qualities of the arts, whether these by artistic excellence, innovation or vibrancy’ (p. 5). Holden (2004) believes this is a ‘missing ingredient’ (p.22) in the value and impact debate. In a speech delivered to the Australia Council of the Arts Marketing Summit, titled On the Brink of a New Chapter: Arts in the 21st Century, Ben Cameron argued that arts organizations needed to rethink their relationship with communities and individuals. Cameron (2009) suggested three questions that arts organizations must answer in relation to the notion of value if they are to survive and have impact. Cameron framed the questions as:

  • What is the value my organization brings to my community?
  • What is the value my organization alone brings or brings better than anyone else?
  • How would my community be damaged if we closed our doors and went away tomorrow?

While such questions could be seen within an economic framework of value, Cameron, in these questions, is challenging arts organizations to rethink the nature of the cultural task. Cameron says that arts organizations can no longer afford to ‘think of themselves as producers or presenters of cultural product, rather they are orchestrators of social interaction with communities who are seeking opportunities for interactivity, participation, access and engagement’ (Cameron, 2009, np.). So, if we take Cameron’s position of redefining the cultural task beyond an economic transaction or, as Belfiore classifies it, ‘economic doxa’ (2014, p. 95) but move it to a platform for social interaction, then it is imperative that the value equation ascribed to arts and culture is represented beyond numerical reportage. Goldbard says this overemphasis on numeric records that try to capture value and impact of arts engagement has placed us into ‘Datastan – the empire of scientism’ (2015, p. 214).

…if we take Cameron’s position of redefining the cultural task beyond an economic transaction . . .but move it to a platform for social interaction, then it is imperative that the value equation ascribed to arts and culture is represented beyond numerical reportage.

Goldband speaks about usurping ‘Datastan’ by allowing ‘artists and cultural policy-makers to convey cultural value and meaning with the tools best suited for that purpose: story, image, metaphor and experience’ (Goldbard, 2015, p. 226). Within this ethos Goldbard is calling for the impact of arts experiences to be reported through art forms and processes rather than bean counting. Whilst text and numeric data are ‘dominant in academic research – vital for production, measurement and dissemination of research findings’ (Durose et al., 2011, p.8) there is equally an interest in ‘beyond text tools’ (ibid.)  including storytelling, performance, art and photography. Perhaps taking a performative research approach, using non-text based tools, may be able to provide an answer to Goldbard’s call to arms. Within the creative practice as research field the findings of a project are reported as performative data. These delivered by the researcher though new artistic forms for performance and exhibition, or designs for user-led, online games (Haseman, 2006, p.3). For Goldbard the capture of arts experience may take the form of digital stories – short, first person sound-and-image video narratives – to capture participants’ experience and response … or a participatory photography project in which images and captions show what needs preservation and what needs development. (Goldbard, 2015, p. 226).

Current evaluation methods dominated by quantitative methods are, according to Holden (2004, p. 17), ‘increasingly being questioned, both in terms of the utility of methodologies employed and the extent to which the results illuminate our understanding’. Commentators such as Goldbard (2015) and Blomkamp (2015) suggest that ‘… a plurality of approaches to measuring culture and understanding cultural change may be desirable’ (Blomkamp, 2015, p. 22). I believe it is time to challenge not only what is valued as evidence, including what data collection tools are used to measure value and impact, but it is time to rethink the transmission process of reporting findings. Using arts methods and processes to report on arts engagement could be the new frontier in arts and cultural measurement.


References

Belfiore, E. (2015). Foreword. In MacDowall, L., Badham, M., Blomkamp, E. and Dunphy, K. (Eds.), Making Culture Count: The Politics of Cultural Measurement. (pp. ix-xi). Houndsmills, United Kingdom: Palgrave Macmillan.

Blomkamp, E. (2015). A Critical History of Cultural Indicators. In MacDowall, L., Badham, M., Blomkamp, E. and Dunphy, K. (Eds.), Making Culture Count: The Politics of Cultural Measurement. (pp. 11-26). Houndsmills, United Kingdom: Palgrave Macmillan.

Brown, S. & Trimboli, D. (2011). The real “worth” of festivals: Challenges for measuring socio-cultural impacts. Asia Pacific Journal of Arts and Cultural Management. 8(1), 616–29.

Carnwath, J.D. and Brown, A.S. (2014). Understanding the Value and Impacts of Cultural Experiences. Manchester, United Kingdom: Arts Council England. Retrieved July 27, 2014 from, http://www.artscouncil.org.uk/publication/understanding-value-and-impacts-cultural-experiences

Durose, C., Beebeejaun, Y., Rees, J., Richardson, J. and Richardson, L. (2011). Towards Co-production in Research With Communities, Swindon: AHRC. Retrieved from, http://www.ahrc.ac.uk/documents/project-reports-and-reviews/connected-communities/towards-co-production-in-research-with-communities/

Goldbard, A. (2015). The Metrics Syndrome: Cultural Scientism and Its Discontents. In MacDowall, L., Badham, M., Blomkamp, E. and Dunphy, K. (Eds.), Making Culture Count: The Politics of Cultural Measurement. (pp. 214-227). Houndsmills, United Kingdom: Palgrave Macmillan.

Haseman, B. (2006). A Manifesto for Performative Research. Media

International Australia incorporating Culture and Policy, theme issue “Practice-led Research” (118), 98–106.

Holden, J. (2004). Capturing Cultural Value: How Culture Has Become a Tool of Government Policy. London: Demos. Retrieved October 22, 2014, from www.demos.co.uk/files/CapturingCulturalValue.pdf

Radbourne, J., Glow, H. and Johanson, K. (2013). Knowing and Measuring the Audience Experience. In Radbourne, J., Glow, H. and Johanson, K. (Eds.), The Audience Experience: a critical analysis of audiences in the performing arts. (pp. 3–13). Bristol, United Kingdom: Intellect.


Sandra Gattenhof is Associate Professor and Discipline Leader (Dance, Drama, Music) in the Creative Industries Faculty, Queensland University of Technology. She is co-leader of the Creative Education and Creative Workforce theme in the newly established Creative Lab at QUT. Sandra has published widely on the outcomes of arts intervention programs in low socio-economic school communities, and is regarded as a leader in the field of the evaluation of cultural programs, having completed studies on, regional arts engagement programs, national arts festivals, and international performing arts markets. Sandra’s latest book Evaluating Impact: Models for Evaluation in the Australian Arts and Culture Landscape was published this year by Palgrave.

More from this issue

More from this issue

Call for Artists Closes: 3 July 2017Event: 19 – 23 February 2018Event Location: Brisbane, AustraliaWebsite: Link | Link APAM is Australia’s leading internationally focused industry event for contemporary performing arts – with a key focus on increasing international and national touring outcomes and business development opportunities for the sector. The event, which will be held in Brisbane in […]

Dr Jenny Wilson — 2016 was a year of discussion and consultation on the future of higher education.  Yet despite all the effort and a flurry of statements reinforcing the Australian government’s preferences, actual detail on how these will be implemented is trickling out at best with much still being debated.
By Professor Su Baker, President, Australian Council of Deans and Directors of Creative Arts — With a quick scan of the status of higher arts education around the country, and indeed the world, we see some hopeful signs and many looming dangers and this allows for flights of fancy and doses of reality.
By Professor Laurene Vaughan — For some time now I have been focused on a series of questions about the future of the university and what value it brings to the world now and for the future.

Project Anywhere was conceived as one possible response to these distinct yet interrelated challenges. Project Anywhere is a global exhibition model in which the role of curator is replaced with the type of peer review model typically endorsed by a refereed journal

In their narratives of art education, Pevsner (1940) and Goldstein (1996) trace a complex history from the medieval guilds to the 20th Century art schools. This narrative is separate from that of mainstream education, since art schools are independent institutions answerable only to themselves.

When I first enrolled at university to study music, I never doubted for a moment I would find a career when I finished. It was the halcyon days of the late 1960s, there was full employment in most parts of Australia, and an expanding appetite for the arts, fuelled as arts councils and arts centres came into being. I wanted to teach music, and with a rapidly expanding schools system crying out for teachers of any subject, recruiting even those had not graduated, professional employment in my field was a certainty.

...when government funding has been used to found a cloistered institution, as in the case of academic research, and this is overlaid with a thick coating of market logic, at some point someone will ask, ‘what are you actually doing over there?’