Tuesday, December 1, 2009

The Worst (and Best) Way to Pick A Charity This Year

Experts have noted that a one dimensional focus on nonprofit finances, if not supplemented by other information, can lead a donor/funder to make the wrong decision as to which nonprofit they support. They are correct. Further, they have noted that an overemphasis on overhead is misleading. That is also true. Many people are shocked that we would admit these things but, to some degree we always have. Perhaps we have not been saying it as clearly as now, but it was always in the narrative of a number of our web site pages (Holiday Giving Tips, Tips for Savvy Donors- item #8, etc.). We now realize that many, were not reading the narrative. It is in that spirit that we signed on to the press release that comes out today with the same title as this blog entry (link here). We do not agree with everything that is stated in this press release (we think overhead does have a place in rating charities, yet agree it should not be primary or overly emphasized) but we do concur with the fundamental truth that the most critical dimension in evaluating a nonprofit has to do with achieving meaningful results (we call them outcomes).

It is also an important reason why we are now on a course to change our rating system so that, even if you do not read the narrative and only look at the star results when you use our rating system search engine, you will be much more likely to make a well informed social investment decision. As we have been saying for some time now, we believe three dimensions must be considered for a social investor (donors/funders with their eyes wide open) to have the critical information needed to make a wise decision:
  1. Financial health – Is the nonprofit sustainable? Does it have robust financial strength to survive in good times and bad? Is the overhead not at the extreme end of the continuum?
  2. Accountability – Does the organization have ethical practices, good governance and transparency? Is it accountable to its constituents?
  3. Outcomes – Can the nonprofit supply information about meaningful and lasting change in the communities and lives of the people it serves? Can they show evidence that these changes are as a result of their efforts? Do they have systems and processes in place to effectively manage their performance?
We intend to help social investors get the answers to these and related questions and use the same zero to four star system we always have used to get there. However, I suspect that the definition of the stars will change in two ways. First, embedded in the star system will be all of the three dimensions noted above in some type of weighted amounts with meaningful results (outcomes) getting the biggest piece of the pie. Second, whereas currently the star system benchmarks nonprofits according to how they compare to industry standard (from exceptionally poor for zero stars to exceptional for four stars), in the new rating system investment risk is likely to be the defining concept. In other words, a zero star nonprofit would be a very high risk social investment and a four star would be low risk. This more accurately reflects the reality of the situation that a social investor must consider. For example, if in the accountability review, it is discovered that a nonprofit is under investigation by a state agency for Medicaid fraud, the risk would increase. Although it is true that the nonprofit should be considered innocent until proven guilty, for a social investor the risk of a possible bad result for their support has increased!

As a start toward the changes described above, we have convened an Advisory Panel of experts and nonprofit leaders to help us. They all had to agree to three things to join the panel: (1) That a star system that rates nonprofits on a macro scale like ours is acceptable to them as a model to evaluate these organizations (2) That the aforementioned three dimensions are the appropriate components to include within that star system, and (my favorite – the no “kvetch rule”) (3) That if they have a criticism of something we are doing (currently or in our suggested changes to our rating system) they also MUST suggest a concrete solution that meets (1) and (2)!!!

We are in the final stages of developing the first draft of the accountability dimension and hope to have it reviewed, revised, tested and approved for implementation by the Spring of 2010. We then plan to modify our financial dimension to address some of the criticisms (regarding overemphasis on overhead versus sustainability, etc.) and hope to have it through the process by the Fall of 2010. Lastly, we anticipate submitting a prototype to the Panel for the outcome component in the Winter of 2010 with the hope that the entire new rating system may be up and running by the Spring of 2011. Some have said we are crazy (not on the Advisory Panel!) and that it is impossible given the complexity and scope of the nonprofit sector to do something like this. Our response is that we believe there are scalable tools that are macro enough to cover 5,500 nonprofits per year and meaningful enough to get to the essence of the question as to what level of risk a social investor will bear in making their decision to support an organization. Further, we believe it must be done.

Some question whether people who are “casual” (i.e. not super rich) investors, care about this sort of information. Our answer is proven by data. Over 3 million unique users come to our site for information each year and it is estimated to help them decide on where billions of dollars is going. That is with our existing, admittedly one dimensional rating system. Can you imagine what the new one will do?
Many experts and charity leaders have told us they are excited and emboldened by our plans. They also like that we are acknowledging the weaknesses of our existing system. That is what I learned over the 25 some odd years that I worked in homeless shelters, doing outreach on the street and a variety of other jobs in the helping professions. We need to be transparent and accountable for our actions. Charity Navigator will not just talk the talk, we will walk the walk. We are about to go through our own theory of change process and develop outcome indicators for our own work. We will be rating ourselves too! Hopefully, there will be a good outcome to our process. If not, we will keep trying until there is one.

Finally, let me tell you why we are so passionate about all of this. We believe like many others that this is a critical battle for the very soul of the nonprofit sector. We MUST get past the notion of doing the “good work” with no accountability. We MUST get past the idea that nonprofits are too complex or unique to be measured. I have seen it close up for years and it is not a pretty picture. The nonprofit sector must get its act together and make sure it is really helping provide meaningful change in communities and peoples lives. It is life or death for many of those we serve whether we are effective or not. So let’s work together to measure, manage and deliver what is really important to make our world a better place.

40 comments:

Sean Stannard-Stockton said...

Ken, you deserve a standing ovation for the transformation you are taking Charity Navigator on. In a sector where we are suppose to take the moral stand even when it is not in our personal best interest, you've just set a shining example. Bravo!

Ken Berger said...

Sean,

Thank you so much for your continuing insights and wise counsel along the way.

Anonymous said...

Thank Goodness. Way to be a visionary Ken.

Rich Polt said...

Ken -- really thrilled to read the joint announcement you issued today. While the message you put forth is clearly the most important aspect of the press release, I am also very encouraged by the spirit of collaboration (not competition) that permeates this effort. The fact that Charity Navigator, Guidestar, Givewell, et al all stand behind these words is what makes them so powerful. Congrats.

Ken Berger said...

Anonymous

Thanks!

Ken Berger said...

Rich,

Collaboration is critical for high impact!

Ken

James Edward Dillard said...

What exactly is the place for overhead ratios? I'm curious.

Kudos for signing on anyway and for taking on the large task of making a four star system that fits all charities.

One thing I wondered if you considered: Are most gifts more like investments or purchases? I would argue that almost all gifts are people purchasing impact rather than investing in impact. Changing the metaphor I think does a lot for how we structure things like rating systems.

Carrie Varoquiers said...

Ken- This joint announcement is such an important step in beginning to measure what matters. Kudos to you and your fellow collaborators!

Pete Smith said...

Ken, this is tremendous progress. Very thoughtful approach -- keep up the good work.

Benjamin Light said...

Ken,

I am thrilled to see that you will be revamping the way you evaluate non-profit organizations. I do not believe this will be an easy task and wish you lots of luck. I was someone who was initially critical of the methods used by Charity Navigator and I look forward to seeing what you come up with.

I would be happy to assist in this process if you are seeking input from non-profit professionals.

Sincerely,

Ben Light
light@touchedbycancer.org

Robert Egger said...

Now THAT'S leadership.

Ken, you've done GREAT work in your tenure at CN. Kudos to you and your Board, as well as to ALL the other groups involved. Finally, we are moving the dialogue forward, faster.

Ken Berger said...

Carrie, Pete, Benjamin and Robert,

Thanks to all of you for your words of encouragement. It means more to me than you will ever know.

Ken Berger said...

James,

The blog entry was meant to provide a general outline of where we hope to go. On the particulars of what the new financial metric will look like, we are not yet there. We MAY "dial down" the significance of overhead as a factor. Perhaps it will be a pass/fail rather than a ratio with failure only if you are an extreme outlier (far outside the norm for overhead)? I am not sure at this point.

The more critical issue is that, if you have good results, that is what weighs most heavily in the rating.

Investment vs. purchase? My initial reaction is that using the concept of purchase doesn't make sense to me as the way to go. Purchase implies a specific product. Investment is in an organization (not a product or program) that shows overall it is providing social value. Whether many people are thinking the way we are suggesting or not isn't the key issue. The key is, what is the best way to view your support for an organization. We need to educate all concerned about what's best to help most!

Deyan Vitanov said...

Ken,

I want to join the rest of the community in congratulating you on making such bold steps and announcing a much needed change for the better. The whole team at Philanthropedia (and I personally) cannot wait to see what comes out of this. We are also excited by the opportunities for collaboration and being part of a better future.

Deyan

Charles Maclean said...
This comment has been removed by a blog administrator.
James Edward Dillard said...

Ken --

I look forward to hearing more. As I said, I'm open to seeing CN experiment with a new system, but I'd love to see you make the case (perhaps here on the blog, when you get a chance) for overhead percentage.

Unless a charity is acting merely as a redirecting agent (in the case of a foundation or a fundraiser), I don't understand how it matters.

Especially for non-profits that are trying to provide outcomes, I think it makes no sense... after all, many of the things that are needed to maximize consistent and long term impact are defined as overhead.

Jeff Raderstrong said...

Mr. Berger,

I am very excited about this. I think this can create some major, badly-needed reform in the philanthropic industry. Thank you for your commitment to this.

My commentary about this is here: http://changecharity.blogspot.com/2009/12/1212009.html , if you are interested.

Thanks again,

Jeff Raderstrong

jk said...

Mr. Berger,
Congratulations on striking out in this positive re-direction. The excessively narrow focus on expense ratios has oversimplified the public's understanding of whether or not a charity is "good," and it is high time to broaden the conversation about what makes a charity worth supporting. Thank you for your work, and best wishes with your endeavours. Lastly, in the hope of encouraging you, i share a motto ascribed to an Oregon Trail settler: The cowards never started, and the weak died along the way.
j. kim
donor and fundraiser
Toronto, ON

David said...

Mr Berger, I strongly believe you're on the right track. Improvements in this area will result in a stronger non-profit sector that will both make better use of the current donation stream, and also substantially grow the donation stream by making the social value of non-profits more clear to all.

Charles Maclean said...

“Ken you’ve made the unmentionable mentionable at a time when trust in nonprofits needs strengthening. Your integrity, rigor and candor in more deeply assessing nonprofit financial health, accountability and outcomes mark a new chapter in maturing nonprofits and making the public informed partners in giving and volunteering.
What if savvy nonprofits began to openly rate themselves around these same three criteria? What if donors had an opportunity to submit “reputation points” for nonprofits like e-Bay does for sellers? And what if recipients of nonprofit services had a structured opportunity to do the same?
A multi-level assessment, while it may be bumpy and imperfect in its inception, could as the Kenny Rogers song says, inform nonprofits and their stakeholders to “Know when to hold’em, know when to fold’em , know when to walk away and know when to run . . . the secret to surviving 'is knowin' what to throw away and knowing what to keep.”
Charles Maclean
PhilanthropyNow

Elizabeth Share said...

Ken,
For years I have been in discussion with staff at Charity Navigator about concerns over the algorithm used to determine the number of stars a nonprofit receives. Today I applaud you wholeheartedly for your leadership, real commitment to the sector and vision. I can't wait to see you roll out a new system of ratings that will help refocus donors on what really matters.

Ingvild Bjornvold said...

Way to go, Ken. You'll be making a real difference to the sector.

Ken Berger said...

James,

As to overhead, you will have to wait and see. I have already told you that we believe it is overemphasized and the critical measure is meaningful results.

I am perfectly comfortable with your wait and see attitude on our proving results! The devil is indeed going to be in the details. I'm confident we will be a great help to people in becoming social investor. I hope you come to the same conclusion at the end of this.

MarcusMakaiwi said...

Ken,
Perhaps we have historically focused on overhead ratios and executive salaries because they are much quicker to measure and easier to compare. Whereas, measuring Outcomes and Impact require a lot more rigor and in many cases may require additional administrative cost to prove over time (sometimes over numerous life times) that there is verifiable evidence that positive changes in condition, behavior, attitude or status of an individual or group served by a program.
I believe your efforts will add value to donors, sponsors, and ultimately to those we are desperate to serve.

Tosca Bruno-van Vijfeijken said...

Ken,

I join the many, many others who congratulate you on the rigor and courage you have in improving the methodology for rating not for profits and NGOs! The collaboration among the different rating websites showcased in the press release is also very, very encouraging. However, when I look at your advisory panel for the review, I am struck by the fact that the expertise represented is almost solely domestic, with the exception of David Bonbright of Keystone. International NGOs (INGOs), though a minority in your population of rated NGOs, are just as much impacted by CN's ratings. The new criteria, indicators and proxies that your team and advisory panel will come up with need to be designed in such a way that they speak as much to the (often very different) context and realities of international NGOs as they do to domestic ones. This was illustrated in the InterAction Forum this past July when you and I, with others, participated in a panel on INGO outcome measurement and rating systems. I hope you will take this into account.

Ken Berger said...

Tosca,

Hi! Thanks for your kind words.

I agree that we will need to have additional expertise in the international area. InterAction will definitely be advising us, whether on the panel or not.

Have a great holiday.

Ken

G Steven said...

My greatest concern is that your new system take into account the time--or timing--dimension in fundraising costs. New charities with good boards have high operating and fundraising costs; established institutions conducting campaigns have high early cost ratios and in the years at the end, their ratios become very low because of pledge payments. Not taking account of time skews any evaluation of administrative and fundraising costs.

Yet another point is that an organization can be underinvested in fundraising, i.e. its expenditures in that area are not connected significantly to the money it gets.

But, complicated as is the task you've set yourself, keep at it. Its worth doing.

Erich said...

Ken, as a donor, thank you.

From my perspective, this is not a new approach, it is a continuing process of improvement.

The search for a single measurement tool to define the effectiveness of a non profit seems mythical to me.

The overhead ratio is a very important, first, basic step. The next time granny gets a phone call asking for a credit card number, if she thinks to check, much good can come of it.

Karl Metzner said...

Ken,
Your plan truly hits the nail on the head, and many will support your quest for a more comprehensive ratings scheme. I second Tosca's concern about the international context. If people and institutions outside the United States find your methodology applicable (or at least adaptable) to their circumstances, the impact of the new approach will be just that much stronger.
Best,
Karl

Erich said...

From the press release: "Experts explain that overhead ratios and executive salaries are a red herring"

From CN's blog, today: "Pay At The Top Comes Under Scrutiny"

As someone who always respected Trent Stamp, especially when he was attacked by the non profit consulting industry, this is so disappointing.

Ken Berger said...

Erich,

I am sorry to hear that you feel that way. I actually believe we are about to make Charity Navigator better than ever. Trent Stamp accomplished amazing things when he led the organization and we see our efforts as building on his great work.

Erich said...

Any user of CN should be disappointed at conflicting information.

I am very glad CN is not abandoning the obvious.

Erich said...

And, for the benefit of donors who do not understand the real issue:

Don't ask paid non profit fundraisers and consultants to opine on whether overhead ratios matter.

Ken Berger said...

Erich

I am not aware of any conflicting information. Remember that we did not agree with everything in the press release.

Ken Berger said...

Erich

To my knowledge, the other three rating groups in the press release do not get funding from the charities they evaluate. Neither do we.

G Steven said...

At the risk of diverting the comments section from CN's worthy plans, I would like to suggest that if CN is to grasp the differences as well as similarities in overhead costs and outcomes of the various kinds of nonprofits, almost certainly it needs to involve the insights of nonprofit fundraisers and consultants.

The potential for conflict of interest is not evidence of or proof of any conflict and there are many people of the highest integrity working in both areas.

Keep moving ahead, don't neglect overhead, and listen to, among others, those who labor in nonprofit vineyards. Your results will be much better and your system much more helpful to thoughtful donors and the good nonprofits.

Once more, this is worth the effort and you are to be congratulated for making it.

Nancy Roach said...

I'm both a donor and a non-profit founder. Wearing both of those hats, I am glad to see that you are looking at OUTCOMES.

My feeling is that many nonprofits focus on doing what they love instead of figuring out how to change their world and then doing it. Awareness campaigns have their place, but awareness without impact is meaningless (imho).

Anonymous said...

While we are at it, could we all please get past the notion that the charity's CEO should work for minimum wage? There are so many comments for every charity complaining about what the CEO makes. An effective responsible CEO SHOULD be well paid; it's a damn hard job, and all those self-righteous complainers need to give it a rest.
Rating charities on their effectiveness instead of their CEO's salary is a splendid idea, and Mr. Berger is to be commended.

Ken Berger said...

Anonymous,

Btw, although we LIST CEO salaries for the organizations we evaluate, we have NEVER included CEO salaries in our rating system.

Pete Smith said...

Anonymous's comments about CEO pay are right on. For more thoughts on this, check my post at http://www.nonprofitmusings.com/2009/11/theyre-at-it-again.html.