(You can read the first article in this series at http://www.clarity-research.net/2016/07/outputs-outcomes-measuring-impact-101/)
Measuring impact really comes down to seeing if your organization is making a difference, and the real key word here is difference. Simply put, a difference means that someone or something has changed over a period of time. In the world of Christian mission, we will often talk about this change as “transformation”. I have heard a lot of debate in Christian research circles about defining “transformation”, but Webster’s dictionary puts it simply and succinctly:
to change (something) completely and usually in a good way
Measuring impact will typically come down to measuring transformation that happens as a result of a ministry. But what do you look at to measure “transformation”? This can often seem daunting when we weigh it down with our biblical teaching of transformation. What can be measured in terms of being “transformed by the renewal of your mind” (Rom 12:2) or “being transformed into the same image from one degree of glory to another” (2 Cor 3:18)? In terms of research, I believe that measuring transformation is definitely possible.
I usually go about measuring transformation as a change in one of four[1] different areas:
- An increase in knowledge- does someone know more than they did before?
- An increase in ability- is someone able to do something that they could not do before?
- An increase in attitude- does someone have a better perspective than they had before?
- An increase in behavior- is someone doing something better or new that they were not doing before?
The first thing to keep in mind as you begin to measure changes is that you need to focus very clearly on the realistic scope and objectives of the ministry. For example, your ministry may have a vision for planting 1000 new churches in France and your means of accomplishing this is to train church planters in France. Now the temptation to measure impact for this type of ministry would be to count the number of churches planted in France. This would certainly measure impact, but how much of this is and is not the result of your ministry? What is your organization actually responsible for? Creating church planters.
Now, this may seem like we are making things easy and “overly gracious” to this hypothetical ministry. But recall the point of last month’s article on measuring impact. Showing up to training does not make you a church planter, actually planting a church makes you a church planter.
So how would we measure impact for this hypothetical ministry? We focus on the changes in the church planter. We can start with the change in knowledge- does the church planter actually know more about planting a church. On the one hand, measuring a change in knowledge is the weakest measurement of impact. For example, a trainee may now know how to study the Bible better, but is this enough? No, but a change in knowledge is often the gateway to other changes that happen.
Measuring a change in ability is a next step in understanding impact. Ability is simply assessing someone’s potential to move from knowledge to action. Yes, the trainee now knows how to study the Bible, but are they able to study the Bible better?
Measuring a change in attitude is another impact measure that gets at the heart of impact measurements. The trainee may know how to plant a new church and may even be very capable of planting that church. But how do they “feel” about planting the church? How motivated are they to put their knowledge and ability into action?
Measuring the changes in knowledge, ability, and attitude are important in assessing the impact on the “inner world” of the people being served by the ministry. However, it is measuring changed behaviors where we move the assessment of impact out into the broader world. Is someone actually reading the Bible more? Is someone actually sharing their faith? Are people actually going out to plant churches?
Changed behavior really is the highest standard for measuring impact; however, it does raise an important question. How much of this changed behavior is really a result of my ministry? Would they have chosen to behave differently regardless of my ministry? Could my ministry have even held them back from doing more? I will address these in next month’s article when I talk about the concept of “null hypothesis” and explain why I am a “control” freak.
[1] There are actually two other areas of change that are not addressed in this article for the sake of simplicity: changes in commitment and changes and influence.