Monday, November 16, 2009

Measuring Performance Course at Bergen Community College

I will be teaching a 3 hour session at NJ's Bergen Community College on November 25th at 6:30pm. Measuring Nonprofit Organizational Performance will review the tools and techniques that are currently in use to measure nonprofit organizational performance in the areas of efficiency, capacity, suggested best practices and outcomes.

For more information or to register, please contact Ilene Kleinman at 201-447-7488 or ikleinman@bergen.edu.

Monday, November 9, 2009

Debra Natenshon on Outcomes - Open Forum Participant

Our next participant in the Open Forum is Debra Natenshon, Chief Executive Officer at
The Center for What Works. Debra has extensive professional experience in organizational development, project management, sales and client services. She functions as a social entrepreneur engaged in creative problem-solving for the organization. In addition to her executive role with The Center, Debra volunteers as the founder and co-chair of the Chicago professional chapter of Net Impact, an active and inclusive global nonprofit network that uses the power of business for positive social change.


I) Ken Berger & Robert Penna[1]: What is the real state of the outcomes movement? Is it spreading and growing, has it hit a plateau, or is it declining due to the economic downturn or other factors?

Debra Natenshon: I'm not sure I would describe it as a "movement" as much as a heightened awareness. For some organizations, this awareness comes at them from external forces such as funders, donors or a new metrics-driven board member. For others, there is a reflective management, an executive director or other key person who understands that it is high-time to better understand the results of the work in addition to the anecdotal stories that nonprofits are so good at telling. Both awareness scenarios are spreading and the economic downturn has only made the need that much more poignant. When resources are scarce in the giving or the receiving, people want to ensure that their money is well-spent. The only way to truly know that results are achieved is to implement a solid, ongoing outcome measurement process and then managing by the results.

KB&RP: We concur that the scenarios contributing to the awareness of outcomes are spreading. The language of outcomes is everywhere in the charitable sector at this point. However, actual practice, from what we have seen, is lagging seriously behind.

The irony is that the current economic circumstances are providing an excuse for many nonprofits to not invest in the implementation of outcomes. At the very moment when, as Debra points out, the need is greatest to truly know what results are being achieved, budgetary concerns are instead causing many sector leaders to conclude that this is not the time to focus on quality, performance…or outcomes.


II) KB&RP: Outcome measurement, where it is used, tends to reflect specific programs. How does the sector move to assessing the outcomes of organizations themselves?


DN: The achievements of an organization are an aggregation of the program results to a large extent. There are various process, management, governance and financial measures that need to be assessed on an organizational level to show the potential for solid outcomes to be achieved. In our work, we strongly recommend that organizations engage in a measurement process as a complement to any organizational strategic planning. Organizations need to adopt a new model of strategic planning that focuses on the implementation piece and this piece is achieved most successfully through the identification of solid outcomes and indicators.

KB&RP: The soundness of a social investment in any nonprofit should be determined by examining a number of dimensions: an organization’s outcomes, its financial soundness, and its accountability practices. Traditionally, CN has tried to get at financial soundness through an analysis of the information available on nonprofits’ IRS 990 submissions. In the very near future, we will add a rating that reflects organizational accountability through an examination of transparency and agency practices. Within the next two years, we also intend to begin rating charities on their outcomes through an examination of the indicators they establish and their success in achieving these targets. All three variables, we believe, should be utilized in assessing not just the effectiveness of an organization, but the ongoing capacity of an organization to deliver meaningful social value.

Beyond this, however, it should become more common for funders to require an assessment of organizations’ structural soundness through the use of such tools as the Capacity Assessment Tool (CAT) designed by McKinsey & Company for Venture Philanthropy Partners and modified for Social Venture Partners. The ability of any organization to deliver its intended and promised outcomes is largely dependent upon, in addition to functional and implementation capacity, its structural capacity…in other words, how well it is governed and managed. The best, most promising program in the world cannot long succeed if implemented by an organization that has serious flaws in its governance or management. While any number of strategic planning models hold the potential for examining these crucial elements, the widespread use of a tool such as CAT would go a long way toward giving the sector a common basis for assessing the inherent strength of nonprofits. We believe it is an approach the sector should seriously consider.[2]


III) KB&RP: Can “outcomes” really be applied across the board throughout the charitable sector? Are there areas where “outcomes” simply don’t apply? If so, how is the value of investments in those areas to be determined?


DN: Outcomes are the results of the work that is done to benefit a target group (individuals or other organizations). From that definition, yes, outcomes can be applied generally. For examples of some common outcomes, readers may refer to our draft Nonprofit Taxonomy of Outcomes, co-published with the Urban Institute.

Some program areas may only have early outcomes that can truly be attributed to their work, but I believe there are very few exceptions. We recently developed an outcomes training for Feeding America, a network of food banks across the country. Even there, where the traditional model is to count the output of pounds of food distributed, we can imagine a day where food banks and distribution agencies will systematically account for an increase in health as measured by the percentage of nutritious food/quality of food distributed. There are many other outcomes in this arena as well. Outcome measurement often takes a reflective process to break out of false norms and initial conclusions that it's impossible.

As others have noted, the issue of harm is truly a risk for organizations and programs who do not measure the results of their work.

KB&RP: In addition to Debra’s excellent example, there are areas – such as advocacy, the arts, and cultural institutions - that are just within recent years joining the outcomes discussion that has been going on in the realm of direct services for a much longer period of time.

IV) KB&RP: What’s next in the outcomes field?

DN: United Way succeeded in bringing awareness of the terminology and initial buy-in for the need for outcomes. The Logic Model is a flawed process though, and there remain too many organizations and funders who continue to have misconceptions about how to do outcome measurement well. The Center for What Works is actively working on building the capacity of nonprofits and funders to better understand the terminology, access tools and increase the ongoing process of measuring, reporting and learning.

There is a groundswell of interest and we envision the field of implementation to expand quickly in the next few years. Organizations that report on their outcomes, including the reasons why certain outcomes were not achieved, will be the norm rather than the exception. Currently, there are several impressive measurement efforts underway - many using the term "Impact Measurement" – [even if] in reality much of the measurement activity remains process- and output-focused. There is a real possibility and need for true outcome measurement and we see clear signs of buy-in from forward-thinking nonprofits and funders to advance this into the very fabric of the sector.

KB&RP: Recent work in the field is going beyond the view and treatment of outcomes first codified by the Logic Model and other early models. These efforts are seeing the basics of outcome thinking being applied to such areas as communications, reporting, HR, and the continuing need to capture the lessons of anecdotal evidence in a meaningful and useful way. Of even greater potential impact, we think, will be the eventual migration of several important corporate sector outcome and analysis tools to the wider social sector of nonprofits, philanthropy and government. As demonstrated in Dr. Penna’s forthcoming book, The Outcomes Toolbox, these tools and approaches hold exciting and promising potential for the field. [3]

The key to any of this, of course, is the commitment of the donor community to make the resources available so that nonprofits can learn about and master these techniques. As we have maintained throughout the discussions in The Forum, if the donor community does not make the necessary resources available, the practice of outcomes will neither spread nor realize its powerful potential.




[1] Dr. Robert Penna is an independent outcomes consultant and is assisting Charity Navigator in managing this Open Forum. He is the author of Outcome Frameworks and the forthcoming Outcomes Toolbox.
[2] For further information on CAT or its use, readers may either contact Dr. Penna at RMPC52@aol.com, or Peggy Kidd, Executive Director of San Diego Social Venture Partners at mailto:peggy@sdsvp.org?subject=SDSVP%20Inquiry
[3] For further information regarding any of the tools or approaches mentioned here, readers may contact Dr. Penna at their convenience.