{"id":958,"date":"2013-02-04T10:19:11","date_gmt":"2013-02-04T15:19:11","guid":{"rendered":"http:\/\/www.artsjournal.com\/fieldnotes\/?p=958"},"modified":"2013-02-04T10:19:11","modified_gmt":"2013-02-04T15:19:11","slug":"stories-from-the-field-measuring-impact","status":"publish","type":"post","link":"https:\/\/www.artsjournal.com\/fieldnotes\/2013\/02\/stories-from-the-field-measuring-impact\/","title":{"rendered":"Stories from the Field: Measuring Impact"},"content":{"rendered":"
\"Calipers\"<\/a>

Photo: Thomas Claveirole via Flickr<\/p><\/div>\n

Over the past three or four years that I\u2019ve worked in the US cultural sector, I\u2019ve noticed that arts organizations are experimenting with the way that we count.\u00a0 Or, perhaps it is more accurate to say that we are developing more holistic ways of measuring what really counts.\u00a0 The recent Counting New Beans<\/a> study challenged theaters to go beyond economic impact or numbers of \u201cbutts in seats\u201d to measure the intrinsic value of their work.\u00a0 Similarly, many organizations are striving to measure intangibles such as attitudinal change or increased empathy in customers.\u00a0 My colleague Gail Crider and I spent the summer looking at different ways that cultural organizations are measuring the impacts and outcomes of their programs as well as the frameworks they used.\u00a0 We presented our findings at a meeting of the minds in Baton Rouge called Social Theory, Politics and the Arts<\/i>.<\/p>\n

One of my favorite things we discovered were the collectives of organizations that came together to create a large-scale study of impact.\u00a0 Some of these collaborations, like Dallas\u2019 Thriving Minds<\/a> initiative, included a range of social service and cultural organizations concentrated in a particular community and others were diverse in terms of geography, but included organizations with very similar goals.\u00a0 The Association of Zoos and Aquariums<\/a> (AZA) is a great example of the latter category, and their story provides a useful example of a practice that any type of organization could employ.<\/p>\n

It all started with an assumption about impact.\u00a0 The zoos and aquariums represented in the AZA have an important aspect in common: they are all accredited institutions that are bound by ethical standards in terms of animal care, public presentation and recreation.\u00a0 The leaders of these institutions always felt strongly that zoos and aquariums contributed to large-scale conservation efforts because exhibitions taught visitors about animals and their habitats, and an assumption was made that this knowledge would eventually lead to shifts in attitudes and behaviors.\u00a0 It was a good theory, but it had never been proven.\u00a0 As scientists themselves, the leaders weren\u2019t completely satisfied speaking from an educated \u201chunch,\u201d and wanted to test their theory to ensure that, as a field, they were having an impact on conservation.<\/p>\n

Leaders in all organizations make decisions based on both analyzing data and strong gut feelings, and I do believe that it\u2019s important to find a balance between the two.\u00a0 However, just as it\u2019s important to question the validity and quality of data, it\u2019s also important to question your hunches.\u00a0 There are aspects of our business as cultural organizations that we assume to be true.\u00a0 As Andrew Taylor of American University pointed out in a presentation, the assumption that we still need a physical box office space persists despite the proliferation and popularization of online ticketing platforms.\u00a0 Some of these assumptions \u2013 including physical spaces, relationship with customers, and business models \u2013 are deeply seeded in the DNA of our organization and it takes a lot of courage to call them into question.<\/p>\n

For the zoos and aquariums, the prevailing assumption had to do with impact on conservation.\u00a0 Member organizations from across the country came together to design and implement a study on the impact of their work on conservation efforts called \u201cWhy Zoos and Aquariums Matter: Assessing the Impact of a Visit to a Zoo or Aquarium\u201d<\/a> \u00a0with support from the National Science Foundation and in partnership with Institute for Learning Innovation and Monterey Bay Aquarium.\u00a0 This was an elegant, multi-layered study that used (among other methods) immediate pre- and post- visit surveys to measure any change in visitor knowledge and attitudes about conservation.\u00a0 Later on, follow up interviews were also conducted with a portion of these visitors to see whether any of their actions<\/i> had changed since visiting the zoo or aquarium.\u00a0 Sure, visitors now knew that turning off the water contributed to habitat preservation, but did they actually do it?<\/p>\n

The participating organizations were pleased with results of the study, which indicated that zoos and aquariums do<\/i> contribute to conservation efforts.\u00a0 Gathering this data was useful in countering harmful claims from detractors \u2013 such as animal rights activists \u2013 who claimed that the zoos and aquariums did more harm than good to animals and their habitats.\u00a0 It also helped organizations communicate their value to the community, visitors and donors.\u00a0 The results pointed leaders towards the types of activities they should do more of in order to maximize impact.\u00a0 What\u2019s more, the multi-institutional study resulted in an evaluation tool that was useful for the AZA\u2019s individual member organizations.<\/p>\n

Ohio\u2019s Akron Zoo<\/a> is a great example.\u00a0 At Akron Zoo, Executive Director Pat Simmons and her team are planning a new exhibit called \u201cGrizzly Ridge,\u201d which seeks to raise awareness of the human interventions that threaten grizzlies and their habitats.\u00a0 I had a chance to catch up with Simmons earlier this fall and discuss how the AZA\u2019s study has provided a helpful, ready-to-use tool for her organization.\u00a0 She and her team have designed the exhibition with visitor education in mind, drawing on the lessons learned about what works and what doesn\u2019t from the study\u2019s findings. To evaluate their effectiveness in changing people\u2019s attitudes and actions, they plan to use the survey methods from \u201cWhy Zoos and Aquariums Matter.\u201d\u00a0 Not only will they use the survey results as evidence for funders and other external audiences, they will also rely on evaluation data to inform how they can continuously improve the exhibit\u2019s education materials.<\/p>\n

Overall, Simmons noted that the model presented in the AZA study has been a useful tool for her own organization as well as other cohort institutions.\u00a0 Too often, great studies get set aside because the lessons coming out of them aren\u2019t digestible.\u00a0 AZA, on the other hand, published its research not to be filed away, but to be put into action by smart leaders like Simmons.<\/p>\n

Digging deeper into the meaning of this study, I also considered how the simple act of questioning assumptions can strengthen an organization.\u00a0 The leaders in this story put a deeply-held belief under the microscope \u2013 a very difficult thing to do \u2013 and were able to affirm that their theory of change<\/a> was correct.\u00a0 Furthermore, the leaders were better able to direct their efforts once they understood the why and how behind their success in mission achievement.\u00a0 What do we know to be true about our work, and when do we operate off of a hunch?\u00a0 What\u2019s at stake if we question our assumptions?\u00a0 These are important questions and while you will need to muster up a great deal of courage, it\u2019s not necessary to have a service organization or a professional evaluator on board to ask them.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Over the past three or four years that I\u2019ve worked in the US cultural sector, I\u2019ve noticed that arts organizations are experimenting with the way that we count.\u00a0 Or, perhaps it is more accurate to say that we are developing more holistic ways of measuring what really counts.\u00a0 The recent Counting New Beans study challenged […]<\/p>\n","protected":false},"author":5,"featured_media":963,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[23],"tags":[147,145,146],"coauthors":[22],"class_list":{"0":"post-958","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-stories-from-the-field","8":"tag-assumptions","9":"tag-aza","10":"tag-impact","11":"entry"},"_links":{"self":[{"href":"https:\/\/www.artsjournal.com\/fieldnotes\/wp-json\/wp\/v2\/posts\/958","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.artsjournal.com\/fieldnotes\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.artsjournal.com\/fieldnotes\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.artsjournal.com\/fieldnotes\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.artsjournal.com\/fieldnotes\/wp-json\/wp\/v2\/comments?post=958"}],"version-history":[{"count":0,"href":"https:\/\/www.artsjournal.com\/fieldnotes\/wp-json\/wp\/v2\/posts\/958\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.artsjournal.com\/fieldnotes\/wp-json\/wp\/v2\/media\/963"}],"wp:attachment":[{"href":"https:\/\/www.artsjournal.com\/fieldnotes\/wp-json\/wp\/v2\/media?parent=958"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.artsjournal.com\/fieldnotes\/wp-json\/wp\/v2\/categories?post=958"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.artsjournal.com\/fieldnotes\/wp-json\/wp\/v2\/tags?post=958"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.artsjournal.com\/fieldnotes\/wp-json\/wp\/v2\/coauthors?post=958"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}