Monday, July 20, 2009

Alan Sorkin on Outcomes - Open Forum Participant

The first participant in our Open Forum is Mr. Alan Sorkin, Incoming President of Social Venture Partners International


Ken Berger: What do you see as the value of outcome measurement and the practices associated with it for nonprofits and social investors?

Alan Sorkin: We believe that they are absolutely essential. Funders, for far too long, have been giving without guidelines of effectiveness. With funds drying up and the needs even greater, funders and social investors must be concerned with providing funds only to organizations that can demonstrate impact and effectiveness. A dollar given to an ineffective organization is a dollar wasted that could have been given to an effective organization. Within SVP, accountability and results are among our shared principles. Partners are mutually accountable to each other, their Investees [our grant recipients] and community. They achieve and document measurable results, both in their own work and through their nonprofit partnerships.

KB: Why are so few nonprofits actually using outcome measurement? Is it that so few donors are asking for it, or is it something else getting in the way?

AS: The short answer is that outcome measurement is difficult. Social Venture Partner nonprofit investees understand that they have a much better chance of funding if they can talk about their outcomes and impact in a meaningful way. As long as funders do not require clearly defined outcomes, nonprofits will lag behind.

KB: Are funders doing enough to enable nonprofits to fully implement outcome measurement?

AS: No. Many funders still persist in underwriting programs without providing funds to measure the effectiveness of those programs. They should also be funding capacity building so that increases in effectiveness can be tracked and measured.

KB: What are funding organizations doing, beyond asking for outcomes, to actually foster the movement and its spread throughout the sector?

AS: I can’t speak for other organizations, but SVP is tracking our member organization outcomes in the areas of both philanthropic development and nonprofit effectiveness. By way of example, we recently had a USC outside evaluation measuring our impact. We are constantly looking to improve our own performance and that of our investees.

SVP is a network of individuals who care passionately about making the world a better place. Fundamental to the SVP approach is engagement. Partners give of their time, professional skills and creativity, bringing together the worlds of grant making, volunteerism, nonprofit capacity building, and philanthropic education. SVP-International is the network association of the 25 SVP community organizations across the U.S., in Canada and Japan.

*****

Ken’s Commentary: In thinking about Mr. Sorkin’s responses, a number of points come to mind:

  1. Mr. Sorkin mentioned that many funders still persist in underwriting programs without providing funds to measure the effectiveness of those programs. At the same time, he noted that without measurement, they could be wasting their investment. Could it be that, just as we observed with public charities (see A Scary Finding on Outcome Measurement), less than 10% of foundations fund the systems, services, and staff needed to measure and manage performance?
  2. Of those foundations that do require measurement, how many actually help the agency receiving the funding look at the bigger picture of agency performance, as opposed to performance just of the grant? Any?
  3. Are the measurements that are conducted truly meaningful? We have heard that for many funders, as long as something is filled in on the grant application and the final report (under the heading “outcomes”), no closer look happens. Garbage in and garbage out. So just as with the nonprofits they fund, even if a foundation says it measures outcomes, they may not actually do so. They’re using the latest jargon, but nothing has really changed about the way they do business.
  4. How many foundations measure their own performance in a meaningful way?
  5. Mr. Sorkin's short answer to why more nonprofits are not effectively managing their performance (and perhaps why more funders are not asking for them) is that working with outcomes is difficult. There may be more going on here than just technical difficulty with measurement. I am told the most common reason nonprofits don't do it is because they don't have to. As long as funds can be secured the old fashioned way - telling a sad story, why should nonprofits go through the hard work of managing their performance?
  6. In conclusion, I suspect the reasons why more nonprofits are not managing their performance fall into the following categories:
    a. Ignorance of how to do it
    b. Fear of what they might discover
    c. Lack of resources to do it
    d. Lack of commitment to doing it
    e. Conscious resistance to doing it

    So, that do you think? We’re eager for your thoughts….

16 comments:

Arloa Sutter said...

Ken: At Breakthrough we have been working hard to produce useful outcome measurements but it is difficult. Some of the things we value, like the number of people in the networks of the kids in our programs or the number of adults in our programs who take meaningful steps toward sobriety and stability, might seem inconsequential to some of our donors, but the reality is, these seemingly small steps can be huge for our guests.

Tracking outcomes takes a lot of time and discipline as well as the establishment of an overarching system of data input and analysis. I think you are right in that these administrative activities are more difficult to fund than direct program activities. Thanks for advocating for funding for overhead and capacity building. As you know, I think funding for these types of activities is crucial in order for mid-cap npos to make it to the level of scale needed to truly make a significant impact. Thanks for your thoughtful post.

Ken Berger said...

Arloa,

I believe that in the long run, your donors will see truly meaningful steps as very consequential. Part of our obligation is to educate them to what matters most - outcomes and impact. I believe that is the future of the nonprofit sector if it is to thrive.

I used to believe that small is beautiful but from experience see that "mid-size is better". There appears to be some strong evidence (see the book Billions of Dollars in Millions of Buckets) that Mid-cap or midsized charities have the best chance to balance mission and money to get to scalable impact.

Ken

Ken Berger said...

Arloa,

I believe that in the long run, your donors will see truly meaningful steps as very consequential. Part of our obligation is to educate them to what matters most - outcomes and impact. I believe that is the future of the nonprofit sector if it is to thrive.

I used to believe that small is beautiful but from experience see that "mid-size is better". There appears to be some strong evidence (see the book Billions of Dollars in Millions of Buckets) that Mid-cap or midsized charities have the best chance to balance mission and money to get to scalable impact.

Ken

Arloa Sutter said...

I agree. I am on page 169 of that book. It is an important read. Breakthrough is at $3.6M operating and conducting a $15M capital campaign. We need large investments in order to fulfill a comprehensive vision to bring lasting change to a distressed community, but we really believe it can be done. It's an exciting adventure. I applaud the notion of combining efforts to fund mid-size high impact organizations. Measuring impact is key.

Ken Berger said...

Arloa,

Indeed it is.

Ken

Holly Sue said...

Hi Ken,

In my opinion, one of the key elements to changing the attitude in charities about results will have to be a change in the attitude of the public.

I work for In Our Own Quiet Way. We recently conducted a survey on our website asking people how often they'd like to receive updates of the results of their donation. Forty percent of the respondents said they'd like to hear from us once a month, while another 35% replied that they only wanted a notification once a quarter.

In the end, I think it boils down to the fact that people prefer to be "donors" instead of "social investors" because it's less work. The problem with this, though, is that if the public doesn't demand accountability there will be no point in supplying it.

The Guide to Effective Social Investing from alleffective.org's site says "...if we are to build a robust and effective social sector, there is a major cultural obstacle to be surmounted," (p. 6).

What types of things, then, DOES the public demand accountability for? Bill Easterly talks in White Man's Burden about garbage collection. When garbage piles up in a rich society, the citizens complain. If politicians don't respond, they'll lose the next election. The task is handled effectively and efficiently because the public demands it. "Active civic organizations and political lobbies operate from the bottom up to hold leaders and bureaucrats accountable, correcting missteps and rewarding positive ones," (p. 16).

The public needs to hold the social sector accountable.

At Quiet Way, we're chewing on this concept. The public seems highly emotionally invested in the outcome of American Idol, and Hell's Kitchen. Why wouldn't they be interested in the results of social aid?

It's simply a question of supply and demand.

I sincerely believe that once the demand exists, the supply will naturally be created. But as Bill Easterly explains, this has to happen from the bottom-up. We, the members of the social sector, can lead the public to water but they won't drink unless they're thirsty.

And so, I wonder if the answer doesn't lie in bringing this issue to the public's attention (and I may be stoned for saying this) via popular, young, wave-making means, that will result in a demand?

We also asked in our survey for people to rate the following statement: "I would watch a TV channel dedicated to showcasing the results of humanitarian efforts."

19% Strongly Agreed
48% Agreed

That's 67% of our respondents.

Interesting?

Ken Berger said...

Holly,

Great observations. I agree that the public needs to demand this information for it to truly catch fire. However, I also believe that we have an obligation to advocate, educate and aggitate for the public to better understand the importance of this information. Companies are always attempting to convince the public of their "need" for things they don't care about before the publicity campaign. When something really IS important and vitally needed to make the right investment in a nonprofit, isn't it vital that we try to move public consciousness down the road?

Jonah Halper said...

As a fundraiser in the world of next generation philanthropy in the New York Jewish community, this shift to outcomes-based funding and transparency is absolutely critical and will give organizations that function under those guidelines a real edge against other charities fighting for the same dollars and marketshare.

Ken Berger said...

Jonah,

You got it! Most importantly of all, those you serve will get the biggest bang for the buck as will those who donate to you!

Ken

Bill Huddleston said...

A different point of view:

The Good Samaritan & “Performance Measurement”
by Bill Huddleston

Currently, there’s a lot of hype in the world about being “results oriented” and the culture of “performance management” has seeped its way into almost every realm of American life, including business, government and now, the non-profit world as well.

Well, why shouldn’t it? Doesn’t it sound like it’s the only way to be, after all, who could be “against results” or against “performance measurement.” It sounds great, but like the question, “When did you stop beating your wife (or husband)?” it sets the stage in an extremely negative, and skewed fashion.

Let’s use a historical example, the story of the good Samaritan from the Bible is one that I believe is so widely known that it qualifies as a societal story, not just a religious one.

To recap, in the parable a traveler is robbed, beaten, stripped of his clothes and left for dead. Two different people walk by, leaving the robbery victim alone. Then a man from Samaria (the Good Samaritan) comes upon the man, and even though the two different groups hated each other, he stops to render aid. The Samaritan takes pity on the victim, bandages him, pours oil and wine on his wounds, then puts the victim on his donkey and takes him to an inn and takes care of him. The next day, the Good Samaritan gives the innkeeper two dineri (this was two days wages at the time) and tells the innkeeper, “Look after him, and when I return I will reimburse you for any extra expense you have.” (The story is from Luke 10:29-35).

Now let’s apply modern performance measurement and outcome techniques to this story. With 2000 years of history the story still resonates, how many people have been helped because someone remembered the story of the Good Samaritan and acted in a way that was not perhaps their first impulse? We will never know, and to the performance management crowd, this incident would be recorded today as “too expensive” and “ineffective” – after all, the Samaritan only helped one person. We don’t know if the Samaritan ever came back and paid those extra expenses, and it was two day’s earnings to help just this one person.

It would also received the rating of : “Results Not Demonstrated” - we don’t know if the victim ever recovered, was permanently injured, or had mental impairment due to his injuries. All we know is that he had the crap beat out of him, multiple people walked by, until the “unclean” Samaritan stopped to help.

According to the performance measurement tools, the Good Samaritan “program” was a failure and had no impact.

I think not.

Copyright Bill Huddleston, All rights reserved.
www.cfcfundraising.com
Blog: www.cfctreasures.wordpress.com
BillHuddleston@verizon.net

Ken Berger said...

Bill,

To follow your analogy, two thousand years ago science had not yet discovered medicine to cure many diseases that we can today. Similarly, the current science available to measure the performance of Nonprofits can move us tremendously forward in helping those in need. Should we turn our back on it because it did not exist 2000 years ago? I say, "God forbid".

Ken

Ken Berger said...

Bill,

One more thing,the measurement tools we are talking about relate to the efforts of ORGANIZATIONS not individuals. With performance measurement systems in place for nonprofits, you do not throw logic out the window if you as an individual see someone in crisis on the street.

Isaac Castillo said...

Bill, I think your parable of "The Good Samaritan & “Performance Measurement” is a good example.....

of people not having a good understanding of outputs and outcomes and when they are valuable.

I agree, that in your example, it is hard to measure meaningful outcomes. But meaningful outputs are easily measured in your example (and in others). And in some instances that is enough (or all that can be measured).

For example, the clearest measure in your example is that the traveler gets a safe place to sleep for the night. That in and of itself is a good thing, and something dozens of homeless shelters measure every day. True, there is no way to know if the traveler (or homeless individuals) get safe places to live on a long term basis, but that is not a reason to not offer the service. Nor is that a reason to critique the concept of outcomes measurement.

Another output of the Samaritan's work is the reciept of first aid services by the traveler. This is similar to emergency room aid services provided. Again, both services lead to short term benefits, and there is no guarantee that they will lead to anything long term. But that shouldn't be construed as a potential justification to NOT provide first aid to someone (or to shut down emergency rooms).

Ultimately, in some instances, measuring just outputs is enough. The complication comes in trying to use outputs alone to justify the continuation of a program or service when outcomes would be more appropriate.

Again, not taking anything away from the Good Samaritan, but don't you think the traveler would have been better served if the Samaritan took the traveler to:

An organization or group that offered temporary housing that focused on victims of crime? And that offered long term aftercare? And had a trained medical professional on hand to help with the wounds? And had demonstrated outcomes on the effectiveness of their work?

Isaac.

Ken Berger said...

Isaac,

Well said.

Ken

Maris Jost Edgar said...

2 thoughts:

Would outcomes measurement increase the percentage of expenses on administration compared to programs, making the organization look less attractive to potential funders or under-informed individual donors?

Could the lack of outcomes measurement come from lack of business background at the top? An Accountant, I know people with business degrees learn much about and understand outcome/ progress management, but I suspect that is not true of all college degrees. I am thinking of really small charities, like single- location homeless shelters and food pantries likely not run by, or even employing, someone with a business background to see the importance of outcome measurement.

Maris Jost Edgar

Ken Berger said...

Maris,

Good questions! On the issue of whether administrative costs could increase because of the cost of a performance management system, I think the answer is maybe. The initial cost to implement the program may require an intensive investment of funds, however over time it could well decrease the cost. Presumably, a performance driven organization will become more efficient over time. However, CN would let the data tell us the answer to the question.

As to lack of expertise and small scale being the cause of a lack of performance management systems (and by extension outcome measurement) - I have been told by colleagues that an organization of any size should be capable of putting at least some systems in place if the leadership gets it and truly wants to do it. Others have suggested that if an organization does not have the capacity to do it, how can we hold them accountable?