Greg Sandow offers some tough love to Chorus America, and others who engage research to promote the public value of the arts. Although it seems that his post is tough for him as well:

So why am I sad to talk about this? Because the study suffers from an elementary misuse of statistics. But it's so eager, so hopeful, and so well-intentioned that it's almost painful to tell the poor choral people that their work is flawed.

He suggests that the noble intent of the organization's Choral Impact Study, released last month, is trumped by the disconnects in its methods and its conclusions. In short, he feels that the authors make a mad dash to choral participation as the driver of the correlation, without adjusting for other possible variables.

The criticism mirrors similar concerns about economic impact studies for the arts, which tend to ignore all other activities that might lead to similar (or stronger) economic effects.

Of course, we've all seen that even soft research can have an impact in the public realm. Council members, mayors, governors, and the like who already favor support for the arts like to have some talking points to reinforce their existing bias. And there's likely only a small handful ready and able to critique research methods and conclusions. But as more and more industries struggle for public attention and support -- professional sports, casinos, and others -- the skepticism about such advocacy-based research is beginning to grow.

But even as soft research can work, I'd agree with Greg that the long term success, health, and engagement of arts organizations with civic conversation requires a whole new level of rigor, curiosity, and balanced questioning.