All posts by Maria

If I Started Blogging Today…

Today is a theme writing assignment for the 2012 WordCount Blogathon. The challenge is to reflect on what I would do differently if I started my blog today. Since I only really started adding content to my blog during this month, all I can say is I would have…

…Started sooner!

I have been an independent consultant for nearly three years now. I’ve been very fortunate to never struggle to find clients and make a good living in this time. However, now that I am getting content loaded onto the blog, attracting visitors from search engines, through Twitter and even iTunes, I know that if I had done this two years, I could have had a much better known blog, educated more small and mid-sized nonprofits on the workings of evaluation, and probably met some great clients to work with.

No point in dwelling on all that, though. I am thrilled that the Blogathon has pushed me to post every day and make up for some lost time. I am also excited about how far the site has already come and where I will take it in the near future. I hope you will join me, either by checking the site frequently, or subscribing through Feedburner or email using the boxes on the right side of this page.

photo by: Tim J Keegan

Neighborhood Associations and Evaluation

My podcast partner, Ruth Terry recently interviewed me for a story about neighborhood associations and the unique ways they can use evaluation. The article is titled Making the most of it: Three ways neighborhood associations do a lot with a little.

As happens in meeting deadlines and word limits, some of my comments were left out, so I wanted to share them here.

Ruth’s question: What are three low-cost tools/resources related to data and evaluation that every neighborhood association should be aware of and why?

  1. The simplest tool neighborhood associations can use is doing surveys at events, door to door, and online through their newsletters. Because associations are all about personal connections, recruiting volunteers to give surveys and getting residents involved by asking their opinions builds social capital and connectedness across the group.
  2. Google Docs is great for collecting surveys and will even do a lot of the chart making and summarizing for you, all for free.
  3. For old fashioned mail surveys, the U.S. Post Office has a great nonprofit rate for bulk mail. You can mail a postcard survey for as little as $0.08 a piece.
photo by: exfordy

Evaluation Haiku

Today is Haiku Day in the 2012 WordCount Blogathon. In classical Japanese form, a Haiku is a 3 line poem, where the first line contains 5 syllable, the second line is 7, and the third line is 5 again. Because I haven’t written a Haiku since middle, and because “Evaluation” conveniently has 5 syllables, I’ve decided to try my hand at poetry writing.

Haiku #1:


You want truth, you get data


Haiku #2


They say it is all in the


Haiku #3


We paid how much for this #$%^?


All you poets out there, feel free to contribute your own evaluation inspired haiku!


Evaluating Change

Over two years ago now Heather Stagl of Enclaria LLC interviewed me for her radio show, The Change Agent’s Dilemma. We talked about ways to measure the progress your organization is making toward a planned change. I am reposting the interview because the topic is timeless and a source of frustration for many people seeking to create change in an organization.

There is a point in change management where all the plans are made, key people are on board, and the resources are available. (See my post It’s So Meta: Managing Change in the Public Sector for detail on these steps.) All that’s left to do is the work – which can take months or even years!

How do you keep progressing toward a change over the long haul?

One part of the answer is measurement. In the interview, Heather and I discuss ways to find appropriate measurements, how to monitor them, and how to use them to keep your change on track.

You can listen to the show using the player below. I also created a worksheet to help with this process, which is available for download.

Listen to internet radio with EnclariaRadio on Blog Talk Radio

How to Cross the Gap From Knowing to Doing

Do you have parts of your life that you want to change? Do you know exactly what you need to do to change them? Do you go to bed every night knowing you did nada to make that change happen?

I’ve been there. I’ve watched others get there. I even developed a model to explain what happens and why. And the best part is, this model can help you go from a good idea to a lasting life change.

From Data to Knowledge to Action

Most of the mental “stuff” of the world starts out as raw data. These are things like our personal observations, scientific research, financial reports, political speeches and lots of other collections of “bits.” Most of it is meaningless noise that our brains naturally filter out. For example, if you live in an urban area, you probably see thousands of advertisements every day . Yet, at the end of the day you may remember one or two.

For data to become knowledge (What I Know) it has to go through some processing and contextualization. The good news is that there are many sources that do this for us automatically. The nightly news, doctors, the Federal Reserve and even this blog are all examples. The bad news is that you have to decide if you trust what those sources have to say. Even so, let’s assume that most data aggregation and reporting sources are 75% reliable. That means that the majority of your data to knowledge transformation work is already done for you.

For knowledge to become action (What I Do) well, YOU actually have to DO something. This could mean performing an action once, or it could mean establishing a life long habit. No one can do this for you.

Inertia is the Enemy

So why is it so hard to act on our knowledge? I like to place the blame firmly on the shoulders of inertia. For those who never took physics, Merriam-Webster defines inertia as:

  1. a property of matter by which it remains at rest or in uniform motion in the same straight line unless acted upon by some external force
  2. indisposition to motion, exertion, or change

Sound familiar?

What this means (see, I’m processing data for you and helping you transform it into knowledge) is that our very being is resistant to change. In order to get it moving, we need to apply external force. Here are a few ways to do that.

How to Overcome Inertia

  • Create Obstacles. Make it more difficult to engage in the old behavior. People with gambling problems can voluntarily add themselves to a list that effectively bans them from going into a casino. You can do the same by placing controls between you and your old behavior. In the quitting smoking example, don’t just get rid of all of your cigarettes, throw away your lighters and matches.
  • Create Social Pressure. Call, email or talk to EVERYONE you know and tell them the change you’re making. Tell all the cashiers at the convenience store that you’re quitting smoking and ask them not to sell cigarettes to you. If possible, get featured in the local paper so that you’re REALLY screwed if you have a relapse!
  • Burn Your Bridges. There’s something very final about a fire. When you really want to say goodbye to a part of your past, find a physical representation of it and set it on fire. Burn an empty cigarette carton or some glossy ads. Throw in your vintage NASCAR Winston Cup hat if you must.
  • Take Baby Steps. Maybe you’re not ready for a slash & burn approach. That’s okay. Rapid change isn’t the only way to go. Start your change with just 1 small action. Just 1. Then tomorrow do 1 more. And then the day after do 1 more. Rinse & repeat a few times and in a few months you’ll be a New & Improved person.
  • “Put Out.” Get your mind out of the gutter. The Navy Seals use the phrase “put out” to mean giving everything you’ve got to an activity. Just like a freight train takes a lot of energy to get rolling, but is equally difficult to stop, someone who truly puts out will soon gain momentum and will soon achieve their goal.
  • Make Sure It’s What YOU Really Want. If you don’t REALLY want to change some aspect of your life, you might be able to grit it out and get over the first gap, but more and more gaps will appear. There may be a lot of great reasons to quit smoking, but if you don’t actually desire in the bottom of your soul to quit, it will be an exercise in frustration and failure. You’d be better off to apply your energy to something else.

Now It’s Up to You

I’ve just done everything I can do to help you cross the gap from knowledge to action. If you are truly committed to changing some part of your life, pick a strategy and start to use it. Don’t let inertia keep you from creating the life you desire.

How to Create a Theory of Change

In Part 1 of this series, I explained what a Theory of Change is and why it can be useful for changes on a personal and organizational level. In this post, I’ll explain how to create a theory of change, using the following format:

For this example, let’s say that I am overweight and out of shape. I want to improve my health, so I decide to construct a theory of change and action plan that will help me in that process.

Step 1: Desired Change Assumptions. In this step I look at all of the underlying causes of my less-than-stellar health and why I think I got that way. For this example, I’ll say that I want to change because I haven’t been feeling well and I want more energy. The underlying causes of my health problems are poor eating habits and a lack of exercise. Ideally, I would like to be fit enough to run in a 5k and want to eat at least 5 servings of fresh fruit and vegetables every day.

Step 2: Actions. This is the traditional action planning step that most of us are familiar with. In this step I’ll brainstorm all of the things I need to achieve my desired outcome. Because I think I need more exercise, I decide that I’ll join a gym and work with a trainer. I also need to learn more about nutrition, so I’ll go to the library and research good eating habits. In working with my trainer, we come up with a 5 times per week workout plan and an eating schedule of 5 small meals per day. I’ll continue in this plan for 3 months.

Step 3: Results & Reflection. Some changes can take place quickly, but many are an ongoing process – such as my health improvement example. When this is the case, it’s helpful to pause occasionally and assess the progress that you’ve made so far. Then you can determine if you want to continue doing the same actions, take a break, or rework your assumptions and actions for better results. After my 3 months of improved diet and exercise, I’ll assess my fitness by running in a 5k race. I’ll review my food logs to see if I’ve been eating better. Then I will decide if I want to work toward more improvement or come up with a maintenance plan.

The real power of a Theory of Change is that it forces you to examine the assumptions behind your thinking. My example is a pretty simple one, but in a more complex situation, this 1st step is very helpful. It is also a continuous improvement type of process that requires evaluation and at least thinking about ways to make your processes and plans better.

What is a Theory of Change?

In the nonprofit and charitable sector organizations exist for the explicit purpose of changing some aspect of their communities. There are usually many possible ways to create the desired change, so organizations have to choose how to use their limited staff and budgets to get the most “bang for their buck.” The most successful organizations usually have an underlying framework that they use to guide those decisions, rather than just making it up as they go along. One tool that we use to create such a framework is called a theory of change.

What is This and Why Should I Care?

The Ford Foundation, one of the largest and most respected charitable foundations in the U.S. defines a theory of change as:

“[A tool that] describes a process of planned social change, from the assumptions that guide its design to the long-term goals it seeks to achieve.”

This definition really only applies to nonprofit organizations who are trying to advance their missions. However, I think that the idea of Theory of Change has a lot of relevance in many situations all the way from individual personal improvement up to and including inter-governmental groups such as the United Nations. In my view, tweaking the definition to:

“A tool that describes any process of planned change, from the assumptions that guide its design to the to the long-term goals it seeks to achieve.”

In my slightly tweaked definition, a theory of change can incorporate core values, a change management plan, a behavior modification plan, or any other type of improvement.

How is This Different From an Action Plan?

On the surface, a theory of change looks like just another plan of action that we might use to: try to quit smoking, find greater efficiencies in our departments, or lobby for a new law. When it’s done right, though, a theory of change includes many elements not found in a traditional plan of action such as:

  • Core values. In other words, why do you want to create that particular change.
  • Plan of action. What steps are you going to take.
  • Assumptions. Why do you think the steps in your plan of action will work?
  • Desired outcome(s). What do you want to happen when you complete this process?
  • Measures of success. How will you know when you are done?

In Part 2 of this series, I’ll walk you through the process of creating a theory of change. In the meantime, if you want to read up on this concept, I recommend Grantcraft’s Mapping Change: Using a Theory of Change to Guide Planning and Evaluation.


photo by: Julia Folsom

Story by Numbers Episode 2: Disney’s Chimpanzee and Nonprofit Storytelling

In this episode of the Story by Numbers podcast, Ruth and Maria go to the movies. We discuss the Disney documentary Chimpanzee and find lessons that you can use in your nonprofit communications. The main ideas we discussed are:

  • Joseph Cambell’s Hero’s Journey and how it relates to your nonprofit’s programs.
  • Creating an idea bank and populating it with photos, videos, notes and client testimonials. Tools range from paper and pen to online systems like Evernote.
  • Why Tim Allen should never narrate a documentary. Grunting? Really?

We’re really excited to announce that the podcast is now available in iTunes in the Nonprofit section. If you subscribe there, please write a review. We want your feedback and suggestions for improvement!

The Long Causal Chain

YouTube Preview Image

If you own a television, you’ve probably seen this commercial or others like it recently. It’s a great example of an interesting phenomenon that takes place in nonprofit performance measurement. It’s what I’ve come to call the “Long Causal Chain, Easily Interrupted.” While this is intended to be a humorous post, it does have a serious point.

There have been times in my career when I’ve met with well meaning nonprofit professionals who have a somewhat… loose grasp on the idea of cause and effect. When we’ve discussed what types of outcomes their program can take credit for, they might say something like, “We’re going to build a park, which will encourage young children to play and get more exercise. Since exercise keeps children healthier and healthier children do better in school, I think we should measure neighborhood graduation rates as part of our program.”

I have honestly heard scenarios like this.

As an evaluator, my job is to suggest measures that the program actually can influence. In the case of building a park, I might suggest measuring park usage one Saturday per month or a door to door survey to ask neighbors if they like the park. I would definitely discourage a small program from conducting a 12 year study to see if kindergartners now will go on to graduate high school because of the increased health they have acquired from a playground!

Don’t end up in a roadside ditch. And don’t try to take credit (and responsibility) for things that you cannot possibly control!

It’s So Meta: Managing Change in the Public Sector

Maria’s note: I’ve spent much of my student and professional career reading thick academic articles and translating them into “regular people’s” English. From time to time, I will summarize articles that I find helpful on this website.

The academic article Managing Successful Organizational Change in the Public Sector (Fernandez & Rainey, 2006) is an extensive literature review of research on change management. The authors distill dozens of articles and books into eight steps.

  1. Ensure the Need. A change leader must create a vision for change and show that they are not promoting change for the sake of change. If necessary, they may even artificially create discomfort if that is the only way to create movement.
  2. Provide a Plan. If plans for change are left ambiguous, they leave room for too much discretion at the individual level. People who are not in favor of the change may find ways to ignore it or even oppose for many years. In some cases, they may just wait it out until management changes and the initiative is forgotten. A clear plan with goals and milestones is one way to clarify expectations and keep everyone on track.
  3. Build Internal Support for Change and Overcome Resistance. Make room for widespread participation. The more say individuals have in the plan for change, the more they are likely to support and work toward the plan.
  4. Ensure Top Management Support and Commitment. If management decides to oppose a plan for change, it’s often game over. At least one member at the executive level needs to be on the side of change.
  5. Build External Support. In the nonprofit and public sectors, external stakeholders such as funders and interest groups can help or hinder change. If a change is going to cost funding or attract lots of negative publicity, it may not work.
  6. Provide Resources. Change is never free. Employees or volunteers will need time to adjust to the change. Outside consultants and service providers may need to help out for a while. The change might require an investment in resources such as software, physical space or meeting time. Without resources, a change won’t succeed long term.
  7. Institutionalize Change. Incorporate the change(s) into daily routines. Set performance measures and monitor data to keep everyone on track over time. Give regular update reports so everyone can see where the organization is and how far they have left to go.
  8. Pursue Comprehensive Change. Examine all parts of the organization and see if the systems, staffing and training are consistent and support the change. If you want greater employee input and engagement, change policies to allow them to take more action without supervisor approval. If you want to encourage interaction, get rid of cubicles or create a lounge area.

While these eight steps read like a linear plan, that may not always be the case. For example, it may be important to secure top management support before presenting a vision for change to the rest of the organization. It’s also possible that change initiatives might backslide for a while and the change team may have to reconfigure the plan or build more internal support.

photo by: taberandrew