Thursday, June 21, 2012

The Challenge of Measuring What Matters Most

(This brief article, by Ken Berger, was originally published in Philanthrofiles - the Association of Small Foundations' blog.)

Four years ago when I began working at Charity Navigator, I went on a “listening tour” to see what concerned experts in the field about our rating system. The fundamental concern expressed by many was that we were not factoring in what matters most: the results (especially outcomes) of the work of the charities we analyze. That feedback, among other things, led us to make a commitment to upgrade our rating system over time in the direction they had counseled.

After a few years of research, funded by one large and a number of small foundations, we came to the conclusion that there is a fundamental problem with the experts’ suggestion. Essentially, for the vast majority of charitable causes, there is no publically available information on results. In other words, most charities either do not currently compile such information or if they do, they are unwilling to share it publicly.

The traditional nonprofit culture is to not make waves (unless you are an advocacy organization) and keep a low profile. Nonprofits don’t want to give stakeholders any reason to weaken their trust in them. Nor do they want to give competitors any leg up by learning sensitive information about the vulnerabilities of internal operations. Therefore, the increasing emphasis on transparency about performance is resisted by many.

So what can be done? We believe that we all must reward those courageous early adopters who understand that the excuses are unacceptable. That is why Charity Navigator is currently adding a new dimension to its rating system that will reward those charities that provide the best results reporting. In other words, we realize that it will probably be quite a few years from now before there will be adequate comparable and standardized data on charity performance to benchmark the results of one organization against another. However, we believe the first step to take to get there is to encourage and incentivize public sharing of results data by charities.

The fundamental reason that charities exist is to provide a public benefit. If they do not measure and manage their performance to assure they are getting the job done, how can anyone be certain they are using precious resources as efficiently and effectively as possible? Furthermore, how can they truly be held accountable by their stakeholders including funders like you? Finally, how can those being served get the best possible assistance?

We hope you will join with us in helping to transform the nonprofit sector from “duck and cover” to a transparent and performance-driven orientation focused on results. We believe that in doing so, many more people and communities will be helped in a meaningful way and the world will be a much better place.


Sherrie Murphy said...

Ken Berger: Excellent initiative: getting charities to
TRY to provide best results reporting. Every donor wants to know whether the donated funds are being used intelligently and effectively.
Sherrie Murphy (
Certificate in Fundraising, 2011
Heyman Center for Philanthropy and Fundraising
New York University

Ken Berger said...

Thanks Sherrie!

Ed Boyer said...


Thank you on the continued effort to encourage organizations to publish results. Our organization (Mercy Medical Airlift) has long had results data available via a "button push" from the web site home page. We are upgrading those reports and adding a "daily report" showing charitable patient transports taking place that day all over the United States.

We now have a QR code we are putting on CFC handout literature to be distributed in "CFC Fairs" around the country this fall. One can simply use the CR code to bring up the "Daily Results" details instantly - formatted for whatever device has made the request.

When we get the project complete I'll send you an example.

We are 100% with you on the "results reporting" emphasis - and have been for a couple of years.

Thanks much.

Edward R. Boyer
Mercy Medical Airlift
Virginia Beach, Virginia

RF Elk Valley said...

Do you measure -- or if you don't are you familiar with guidelines or rules of thumb about -- the level of Board giving as a percentage of total fund raising (or total income)? I'm trying to find some quantification to use as a starting point for our board.

Chris Shaida said...

Do you measure or are you familiar with (even at a rough or rule of thumb level) the level of total board giving in a given organization as a percentage of either total funds raised or total income? I am looking for a general starting point along the lines of 'In general, a well-functioning Board will contribute approximately x (or in the range of x and y)'

Ken Berger said...


Congratulations on being an early adopter of this critically important effort. I hope that your data is very outcome focused rather than simply outputs. We are looking to see evidence of meaningful change in people's lives vs. activity counts and the like.


Ken Berger said...

RFK Elk Valley and Chris,

We do not measure the Board giving levels nor are we aware of a prevailing benchmark for it.

I believe the size of your organization and the "stage in lifecycle" (i.e. how old is the organization and what is the past expectations of Board members) are two critical factors to take into account. Most Board's do have a give or get policy for their Board members and some funders look for 100% Board participation but usually not a set % of overall revenues.


HButler said...

Would you be willing to compare non-profit hospitals?

H.E. Butler III M.D., FACS

Ken Berger said...


Nonprofit hospitals are not currently being added to our database for at least two reasons:
1) The vast majority of individual donors who donate to such organizations have a personal connection to it and do not rely upon our ratings, and
2) The unusual financial makeup of nonprofit hospitals (ex. huge reserves of working capital, etc.) make them skew our rating metrics as currently constituted.

If we were to reconsider including them in our database, we would need:
1)Funding to explore an alternative way of rating these organizations before we could add them to our database.
2) Conducting a donor survey to see if our current assumptions are in error and that a large enough pool of donors would be making these decisions with our ratings rather than their personal connections.