Our next blogger for the Open Forum on Outcomes is Dr. David Hunter. Dr. Hunter has some 30 years of experience running and consulting to human services organizations. He pioneered the use of performance management methods and developed a unique approach to helping organizations implemement Strategic Performance Management. He is also a member of the Alliance for Effective Social Investing and has developed an assessment tool (along with Steve Butz) to measure the potential for a nonprofit to create social value.
Ken Berger and Robert Penna*: Should charities be assessed on their use of outcomes?
David Hunter: I'm not sure what the question means. Should nonprofits track their outcomes? Absolutely...how else can they know they are doing a good job?! Should investors use outcomes to assess how good a return they are likely to get by investing in a nonprofit? Sure...but impacts are even better!!! And to be confident that outcomes/impacts will continue to be produced reliably, investors should also look at the nonprofit's internal performance data.
(Of course for some nonprofits one can expect only outputs...e.g., for a soup kitchen. Here the issue is tracking indicators of output quality.)
KB & RP Comments: The issue the question was meant to frame is whether nonprofits should be held accountable by both institutional and individual investors (donors with their eyes wide open) for the “return” they deliver on the monetary investments made in their work. We wholeheartedly agree with David that impacts are better than outcomes. To achieve impact however, an organization will need to be able to rule out any other variable for causation of the benefit provided. This is a very costly endeavor that most organizations will not be able to undertake. Therefore, for most, measuring outcome indicators is as good as it is likely to get. We also agree with David that it is essential for nonprofits to have an internal performance management system that provides data that tracks their outcomes. In other words, investors should ask whether a nonprofit is doing what it claims to be doing and effecting the changes it claims.
KB & RP: What is the real state of the outcomes movement? Is it spreading and growing, has it hit a plateau, or is it declining due to the economic downturn or other factors??
DH: I think it is undeniable that "outcomes" have become a hot topic. I don't know if one can say there is an "outcomes movement" or not...I think the economy has had no effect on the nature of outcomes discussions...they are ubiquitous...
Of course, there is many a slip between speechifying and action...and lots of outcomes talk remains just that. I think it is fair to assert that many funders are, in general, still amazingly unconcerned about the matter in practice .
KB & RP Comments: Even before the contraction in the economy took place, many nonprofits were already making the claim that they could not “afford” to identify, manage toward, track or verify outcomes. Many said they could not afford the training necessary to understand outcome concepts or to master the tools available for putting outcomes into practice, that they could not afford to devote resources toward tracking outcomes and certainly could not afford any verification. A tightened economy has only increased these visceral reactions. Counterintuitive though it may be, these tough financial times are precisely when both nonprofits and funders ought to be making the necessary investments in implementing outcomes:
1. It is especially in times of tightened fiscal resources that nonprofits should want to demonstrate their true effectiveness;
2. These times more than ever should dictate that funders start providing the wherewithal nonprofits need to master and implement outcomes.
Unfortunately, what we have seen instead is a retrenchment on the part of many organizations, away from any investments in outcome training or quality control. Across the economy the current hard times are forcing forward-looking for-profit organizations to push ahead with improvements in products and efficiency that had been too long delayed. In the nonprofit sector, by contrast, the urge instead seems to be to move backwards.
KB & RP: What are the “costs” associated with outcomes, and who should bear them? Are funders doing enough to enable nonprofits to fully implement outcome measurement?
DH: [The first question you ask is] the wrong question. The costs associated with outcomes are simply the costs of high performance. Yes it often (but not always!!!) costs more to work at high levels of quality and effectiveness -reliably and sustainably - than to work without any concern for such matters. But so what? The issue is not cost per unit of service or any other such exclusively cost-based metric. The issue is the social value that is produced...and whether it makes sense to pay what it costs to produce it - both sides of the equation: costs and value must always be considered in relation to each other. Running an inexpensive program that helps nobody is, arguably, much more expensive (all things considered...including opportunity costs and costs to the wider society) than running an expensive program that profoundly improves peoples lives and prospects! Are you serious? Who should pay for outcomes? Anybody who wants them!! [As for the second question, the answer is] “No.” The nonprofit sector is the only social domain where people don't think they should pay fair market value for what they are buying...hence grants with little or no overhead (and the self-satisfied crowing that accompanies them).
KB & RP: Comments: We agree wholeheartedly with the closing sentiments in David’s reply. The issue of high performance is inexorably tied to capacity: in order to do its job correctly, a nonprofit need more than simply the threadbare ability to “run” a program. Unfortunately, for many institutional investors, (barely) providing the resources for running a program is as far as they are willing to go; and this is especially true of governmental investors. We agree with David when he says that that the nonprofit sector is the only social domain where people often don't think they should pay fair market value for what they are buying. We also concur that many (most?) grants are approved with little or no allowance for overhead. The type of investments that would allow a nonprofit to truly implement outcomes are often simply not made. Funding for staff training, for guidance in rethinking their approach to the issues they tackle, tools for tracking and managing towards good, sustainable outcomes, and the ability to verify what they have accomplished, are very often the types of investments funders should make, but don’t. In fact, as soon as distressing ripples become apparent on the economic horizon, these are exactly the kinds of investments and expenditures that are jettisoned first. Institutional funders, philanthropic and governmental, are being both unfair and shortsighted when they ask for outcomes but refuse to provide the means whereby the nonprofits they support might actually accomplish and produce what the funders are asking for. The result is a game of misdirection and obfuscation that too closely resembles the old Soviet joke describing the Workers’ Paradise: “The government pretends to pay us, and so we pretend to work.” For many nonprofits today the situation might most aptly be described as “Funders pretend to want outcomes, and so we pretend to produce them.”
“Who should pay for outcomes?” David asks. The answer, he says, is “Anybody who wants them!!” We wonder whether the sector as a whole really wants them.
KB & RP: What are investor/funder organizations doing, beyond those asking for outcomes, to truly foster the movement and its spread throughout the sector?
DH: Damned if I know.
KB & RP: Comments: We share David’s sense of frustration, but would hazard the guess that the answer to the question is actually “very, very little.”
KB & RP: Has the outcomes field, its jargon, concepts and suggestions, gotten “too complicated?
DH: Yes; [because of] too many narcissists insisting on the primacy of their own locutions (cynically I'd say: efforts to control the jargon are part and parcel of strategies to control the consultation market!)
KB & RP Comments: We believe that the issue has to do with the fact that so few practitioners in the nonprofit field have any idea what those consultants are talking about.
Since the early nineties there has been a significant amount of growth in the number of outcome frameworks available to the nonprofit sector. This number is still growing and we suspect will only become larger as corporate-based approaches like the Balanced Scorecard and Six Sigma make their way across the great divide and begin to gain acceptance in the charitable sector. Yet while all this growth and refinement in the field is taking place, many practitioners still do not understand the essential difference between an output and an outcome.
As we stated in our response to David’s last reply, some of the responsibility for this situation rests with the funders who are failing to provide the means whereby practitioners in the nonprofit field could get up to speed –and stay there- on the practices and thinking currently being debated and developed in the outcomes field. We also see some nonprofits who are trying to avoid the matter entirely.
KB & RP: Alternatively, is there a danger the field will become co-opted for PR value but lose its critical value, meaning and depth?
DH: The degeneration of anything good into a morass of mediocrity or worse is an ever-present possibility...if not a likelihood. Where I weigh the probability in this case depends on what I ate for breakfast on the day you ask me the question.
KB & RP: Answering this question is one of the primary rationales for Charity Navigator’s intention to report upon the use of outcomes in the nonprofit sector. As a hedge against the danger David sees, of the current focus on outcomes sliding into a “morass of mediocrity,” we intend to report on nonprofits’ use of outcomes in as much depth as possible. We believe that investors, particularly individuals and those institutional investors with significantly limited resources, deserve to know whether the nonprofits asking for their money are actually accomplishing anything. Over the course of a long time, many nonprofits have gotten quite good at seeking donations based upon need (“The problem is SO big…”), or upon activity (“We’re working SO hard…”). Now many are becoming equally good at using the right jargon and seeking support based upon claims of effectiveness. We believe that investors have a right -and a responsibility- to know the truth, and so we intend to do all we can to provide guidance on this issue. Watch this space for more information on this soon.
*Dr. Robert Penna is an independent outcomes consultant and is assisting Charity Navigator in managing this Open Forum. He is the author of Outcome Frameworks and the forthcoming Outcomes Toolbox.