The following article was originally printed in The Nonprofit Quarterly on March 2, 2011. The original can be found here.
Points of Contention: Ken Berger Defends Impact Measurement
March 02, 2011
Ken Berger and Robert Penna
Editor's Note: Two weeks ago we published an op-ed from Steven Lawry, the Senior Research Fellow at the Hauser Center for Nonprofit Organizations at Harvard University called, "When Too Much Rigor Leads to Rigor Mortis: Valuing Experience, Judgment and Intuition in Nonprofit Management." In this article, Lawry questions the growing trend among funders to require an increasing amount of assessment and measurement from their grantees.
In the following letter Ken Berger, president and CEO of Charity Navigator, and author, Robert Penna, take exception to many of Lawry's assertions. Berger and Penna believe that "charities owe both their donors and those they serve reliable demonstrations of positive impact."
At a time when the state of public discourse can be experienced as a blast of simultaneous echo chambers, we offer this space to surface strongly held – but thoughtfully articulated – disagreements. In a sector that tends to shy away from self-reflection, we hope a safe place to respectfully disagree will help us learn from each other and grow.
We must take rigorous exception to Steve Lawry’s column, “When Too Much Rigor Leads to Rigor Mortis.” We believe that, at best, it is shortsighted; and at worst may be used by some as a dangerous justification for backsliding on the gains being made in the all-important area of performance management and measurement of nonprofit organizations (especially public charities).
It is unfortunate that Lawry, to begin with, did not make clear that his focus was on international efforts sponsored by Western social investors for application in other parts of the world. Lawry’s contention that large Western donors often lack knowledge of the local context in other countries may be correct. Whether through simple ignorance, cultural chauvinism, misguided faith in their own omnipotence or for other reasons, instances probably can be found where Western donors have incorrectly prescribed an intervention based upon faulty data or a misreading of the information at hand. This is a situation that indeed does call for greater sensitivity and trust in local partners. However, it has been our experience that, at least in the case of some larger international NGO’s, they are much further along in their development and utilization of impact measurement tools than many other parts of the nonprofit sector. Therefore, even in this instance we remain unconvinced that the problem is as pervasive as Lawry characterizes it.
More troubling, however, is the fact that this is not where Mr. Lawry began; but rather with a criticism of the notion of sound, verifiable impact assessment. “Wise nonprofit leaders,” Lawry wrote, “know that the problems they work on are not susceptible to simple measurement.” This opinion marks a very perilous road for the nonprofit sector to travel. Good measurement is what we all should be working toward. Measurement tools such as primary constituent feedback, volunteer reviews, expert reviews, independent in-depth research and analysis, among others, all have value and provide different perspectives that can lead to a rich, multifaceted view of a charity’s performance. Some are simple measurement tools and others more complex. They vary in rigor, but all have a place in the spectrum of tools available to look more comprehensively at how a charity is performing. It is disheartening therefore, to hear Lawry claim “rigor mortis” when the “body” of tools is just beginning to catch hold and come alive!
Since it first emerged as a force in the nonprofit sector in the early 1990s, the outcomes movement has been dogged by numerous counter-claims that are, at their heart, excuses for not recognizing what many experts have been saying for some time: namely, that nonprofits have a critical obligation to produce nothing less than measurable, verifiable results on behalf of those they exist to serve. But rather than accept this idea as self-evident, apologists for poor performance have variously argued that the work was too complex, the issues too intractable, the clients too burdened by multiple issues, and the cost of measurement too high for results measures to be applied to the efforts of nonprofits. Instead, these voices continue to fall back on traditional indicators of activity to show how much they care and how busy they are.
Lawry cites certain “powerful donors” who have concluded that nonprofits do not, overall, make adequate use of impact assessment tools. This belated realization is not something to be either feared or pushed back against. Rather, it should be a clarion call to all charities, those operating both here and abroad, to stop hiding behind excuses and finally show donors – individual, institutional, and governmental – precisely what they are getting for their money, and to demonstrate unequivocally what the American people are getting in return for the tax-exempt status these charities enjoy. What Lawry accomplishes now by suggesting that impact measures will stifle creativity is to offer yet another excuse for those who want it taken on faith that they are accomplishing anything.
We and many of our colleagues believe, in contrast, that this approach represents the thinking of the past. The future must include offering informed donors information regarding not just the work a charity is doing, but the results it is achieving. There are various efforts currently underway that give us confidence that this future will become a reality and we will not return to the dark days of excuses and obfuscation.
Lawry is correct when he writes that formal impact measures are often hard to come by. He is correct when he says that the good judgment of experienced managers is absolutely necessary in assessing impacts. But the tools for helping those managers identify, work toward, and assess meaningful, sustainable, and verifiable impact targets do exist. Rather than offering yet one more rationale for ignoring these tools, Lawry would better serve the sector by joining those of us who believe that charities owe both their donors and those they serve reliable demonstrations of positive impact, by helping us find ways of making these tools available to more nonprofits, and by welcoming the fully informed donor as the best donor any charity could have.
Ken Berger is president & CEO of Charity Navigator. Dr. Robert Penna is the author of "The Nonprofit Outcome Toolbox," consultant to nonprofits, and advisor to Charity Navigator. For a deeper look into this ongoing discussion see this blog posting by Steven Lawry.