On March 20th, I debated William Schambra (Director of the Bradley Center for Philanthropy and Civic Renewal at The Hudson Institute) on the whether it is appropriate to measure the performance of charities and, by extension, rate them. This 80 minute debate occurred during the Closing Plenary at the Grant Managers Network's 8th Annual Conference. You can view a recording of the debate and access my PowerPoint slides below.
The U.S. has the largest nonprofit sector in the history of the world. It is estimated to contain over 1.1 million public charities, which account for somewhere around $1.5 trillion per year in revenues; and the nonprofit sector as a whole employs over one out of every 10 workers in this country. Therefore, with massive government budget deficits, tax increases on top earners, along with payroll taxes hitting millions of workers, America is at a critical juncture and needs to know that charitable revenues are being used as effectively as possible. Even more amazing is a fact most of the general public does not realize -- roughly 1 percent of the charities in the USA (perhaps 15,000 organizations) garner over 85 percent of that $1.5 trillion that comes into the sector each year.*
While we feel better for our efforts, the truth is that we can't actually measure the social impact that most of these expenditures lead to because the vast majority of charities (those nonprofits that have 501 (c) 3 status, and provide you with a tax deduction for your donation) do not publicly report meaningful information on the results of their work. Any private investment of this magnitude would demand a measurable indicator of the result, what about measures of the results for the non-profit sector to create social value (meaningful change in communities and peoples' lives)? Indeed, a substantial number do not have anything to report at all, even privately, because even they do not know whether they are truly effective. As a consequence, both philanthropists and average donors are often forced to use proxies to make their social investing/charitable giving decisions. These proxies typically include financial metrics and other indicators of best practices. Although we believe strongly that financial management and best practices are important and should be considered carefully in making an informed decision about which nonprofits to support, it is clearly not the same as knowing directly about whether the charity is meeting its mission and truly helping people with its programs. This state of affairs is completely unacceptable. In a day and age when we can access data on virtually anything in nanoseconds, it is long past time for charities to get serious about reporting on the results of their work. Thankfully, there are some bright stars that are showing the way through the fog that covers the sector. Groups such as the Harlem Children's Zone, headquartered in NYC, ROCA in Boston, and Nurse Family Partnership in Denver have built state of the art systems to manage toward and measure their results. As a consequence, they have adapted over time, as they learn ways to become more efficient and effective. They are proving that many more people and communities can be helped in a much more significant and measurable way than many of their fellow service providers have been able to accomplish. Not surprisingly, there is a chorus of opponents that offer a host of arguments for why managing, measuring, and reporting on results cannot and should not be done. They say it is too expensive to do so; they claim that they are unique and their services are not measurable; they claim that being held accountable is too much of a burden for both the nonprofits and those they exist to serve... and on and on go the excuses. There is a kernel of truth to some of these arguments; but there is also a hidden reality that most Americans do not realize that cloaks the situation: Almost half of all charities in the U.S. have $25,000 or less in annual revenues (and garner less than 1 percentof the revenues that come into the sector each year).** Obviously we are not anticipating a robust performance management and measurement system from these groups! Rather, we are talking about the charities at the other end of the spectrum -- that 1 percent we mentioned earlier. It is simply disingenuous for these larger charities to hide behind the excuse that it is too expensive to know whether or not they are having a meaningful impact. Furthermore, even if they cannot find measures to track all the services they provide, surely they can start somewhere.
As a long time nonprofit evaluator and colleague of ours often says, "If you do not manage your performance, you cannot measure it, and if you cannot measure it, how can you be accountable"? We believe that it is high time that the larger charities, in particular that 1 percent at the top tier of funding, become accountable for all the resources they collect and the efforts they claim those resources support.
Another challenge facing donors of all types is to be found in the claims of those charities that say they are managing and measuring toward results when, in fact, they are not. Rather, they use the jargon of the outcomes field to suggest that they are effective; but actually provide little if any real evidence to support the claim. For example, an employment program might tell you that over 1,000 people graduated from their employment program last year. Unfortunately, that is merely a measure of activity, and speaks not at all to effectiveness. A much more meaningful result could be determined by learning how many people secured jobs as a result of the training... and kept them for a period of time after they graduated from the employment program. Sadly, as calls increase for charities to report on the results of their work to get funded, we also see an increasing amount of false claims of such results. Our collective challenge is to break the long-standing reality in which "the charity that does the best marketing wins" and replace it with "the charity that provides reliable evidence of the most meaningful and lasting change in people's lives wins".
Indeed, we believe the country is currently witnessing a battle for the very soul of the nonprofit sector. The final outcome of this battle will be determined by the amazingly generous American public which donates two to three times more than any other country in the world. With a growing amount of information available, the donating public (we hope), will soon realize that the vast majority of nonprofits seeking their support cannot provide evidence that they are providing meaningful and lasting change that helps people and communities. When this realization becomes common knowledge, our society will truly be at a crossroads. The end result will be that either charities provide meaningful, reliable evidence of their performance, or their donors will abandon them in disgust and frustration.
We often hear from angry donors who, upon repeatedly reading headlines about charity scams and scandals, become even more cynical and skeptical when they learn of the lack of evidence of charity effectiveness. They understandably wonder whether any charity is doing any good. For the nonprofit sector, this is a disastrous conclusion for donors to be arriving at. There are great charities out there doing amazing work; the public simply needs to get the relevant information to find them. But donors also need to remember that giving should not just come from the heart (although it is a great place to start); but also from the head (the place where you can confirm through research that the charity meets the heartfelt impulse). Even a little research can go a long way in finding the high performing charities out there.
Charity Navigator has been and intends to continue to help donors in their research to find those great charities as quickly and easily as possible. As more donors focus on giving to the best charities based on research, more and more charities are striving to become high performers. We know this because within the first year and a half after we launched a revised rating system in 2011 (which we called CN 2.0), roughly 50 percent of the charities we evaluate have made changes to their governance and other practices to assure they met our new standards. This year we launched CN 3.0 which begins to evaluate the quality of how charities report the results of their work. This is the central question that must be answered so that we, the donating public, get the best return on our "social investment" and so that those people who need to be helped, get the best services possible. If we win this "battle" for the soul of the nonprofit sector, many more people will be helped and the world will be a much better place. That's the outcome that really counts! *The Urban Institute estimates that of the approximately 1.1 million public charities (i.e. 501(c)3 organizations that file with the IRS), only 366,000 report data to the IRS each year. Of those 366,000, 4 percent expend (and also receive revenue) 86 percent of the $1.5 trillion each year. The remaining roughly 800,000 public charities get such a minimal amount of funds it does not even enter into the calculation. Therefore 1 percent of the 1.1 million public charities are garnering roughly 86 percent of revenues/expenses. **The Urban Insitute report on the Nonprofit Sector (http://www.urban.org/UploadedPDF/412674-The-Nonprofit-Sector-in-Brief.pdf) notes that one-half of one percent of expenses (and also revenues) of the 366,000 public charities that report to the IRS are 100,000 or less. In the full report they indicate that roughly half of nonprofits are less than $25,000 in annual revenues. The amount is therefore, significantly less than 1 percent of revenues and expenses for this group.
In the following blog post, which was originally published on MarketsForGood, David Bonbright, founder and Chief Executive of Keystone Accountability, looks underneath supply and demand at the current state of constituent feedback and the need to increase and sustain the market conditions that can, in turn, generate the feedback we need for high-quality programming and impact in the nonprofit sector.Towards the end of the post, Mr. Bongbright makes note of Charity Navigator's new Results Reporting methodology. ************************************************************************ Stimulating Demand For Constituent Feedback By David Bonbright Reading the recent contributions on the theme of “Beneficiary Feedback” is both enlivening and sobering. In his summative post our curator, Eric Henderson, rightly highlights the “encouraging efforts” to cultivate feedback for greater impact. But I am also mindful of how rare these efforts are, and how limited the resources are to support them. What we have today are a few inspiring pioneers who are getting on with it against the odds, and without much support. There is a groundswell of awareness about the importance of feedback, a few early pioneering and shining examples, but no commensurate follow through.* If we want action – organizations listening and responding to their constituents in a way that raises performance and enhances impacts – we need to stimulate the demand for feedback and provide the professional support infrastructure to meet that demand. This raises a couple of important strategic questions in the field of what is coming to be known as Constituent Voice. How do we get an appropriate infrastructure for a nuanced and rapid scale-up of Constituent Voice and what would it look like? Perhaps the answers to these questions are to be found in two familiar analogues: Social Entrepreneurship and Customer Satisfaction. I remember a conversation I had nearly thirty years ago with Bill Drayton, the father of the modern Social Entrepreneurship movement. In the mid 1980s, social entrepreneurship was a shiny new idea and Ashoka was the only organization of its kind. In those days, Bill described Ashoka as the world’s first “venture support” firm for social entrepreneurs. Today, Ashoka is the leading star in a vibrant constellation of financial, educational, service, and research organizations supporting social innovators. The case of Customer Satisfaction is even more to the point since it has done for businesses what Constituent Voice aspires to do for social purpose organizations. Customer Satisfaction as a business practice began in the 1960s and has grown into a multi-billion dollar global industry that enables millions of companies to listen and respond, effectively and profitably, to their customers. Nurturing customer loyalty is a well-developed craft that is taught in business schools and debated in a vibrant professional literature (two of my favourite books are The Ultimate Question 2.0 and Satisfaction: How Every Great Company Listens to the Voice of the Customer). There are formal professional qualifications, and a steady stream of conferences and professional associations. Consumers can compare user ratings on virtually any product, for free. If this is an indication of what we need to get full value from listening to beneficiaries, then who is going to pay for it? Down the road, the answer has to be the organizations seeking and using feedback. But for our sector, at least during the early and growth stages, this means that funders will have to play a leadership role. They can do this in three ways:
First, they can stimulate the demand for feedback by asking grantees and investees to report on what their primary constituents think about their work along the lines of the template sketched out below.
Second, they can make sure they provide adequate financial support required to collect and use this feedback. At little or no extra cost they can transform dry formal evaluations into dynamic feedback-based dialogues for learning, improving and relationship building. (Every survey on the subject shows funders typically do not pay for the full costs of the evaluative requirements they impose.)
Third, they can invest in the development of the wider infrastructure of support for Constituent Voice.
By providing a chronicle of the emergence of Constituent Voice practices, the Markets for Good initiative is itself a high-level example of this support infrastructure. With new communications technologies dramatically reducing the cost and effort required to collect feedback, now is the moment to invest in Constituent Voice methodology, tools, data infrastructure and services. Perhaps the most important contribution to date comes from Charity Navigator, the world’s largest charity rater. Just last month it released a new version of its rating model, Results Reporting, which incorporates Constituent Voice at its heart. In the future, U.S. charities will be rated according to their answers to the following six questions.
Does the charity publish feedback data from its primary constituents?
Does the published feedback data include an explanation of how likely it is to be representative of all primary constituents?
Does the data include an explanation of why the organization believes the feedback is frank and honest?
Is that data presented in a way that shows changes over time going back at least one year?
Does the data include questions that speak to the organization’s effectiveness?
Does the organization report back to its primary constituents what it heard from them?
Consider how transformative it would be if most organizations could answer each of those questions in the affirmative. So I guess the war of feelings in my breast can quiet down. Yes, we are at the start of a long and challenging journey. But the value proposition is clear, and we have more friends than we imagined accompanying us along the way. *For a very recent article capturing the state of the field, see, Listening to Those Who Matter Most, the Beneficiariesin the Spring 2013 issue of SSIR.