By Jenni Wolfson
WITNESS has just published it’s annual performance dashboard covering 2010. We publish a dashboard twice a year that shows how we measure the impact of our work, both in qualitative and quantitative terms. This is important to the entire WITNESS team since we want to be sure that our efforts and passion travel in the right direction. By monitoring our progress and assessing how and where we are making the most difference, we can make important decisions about how to focus and realign our limited resources.
The Challenges of Measuring Impact
It can be hard for a human rights organization to show how it measures impact. It can take months, and often years, for systemic changes in human rights policies, practices and behavior to take effect. We are working in communities where people hold deeply entrenched beliefs and where traditional practices can result in human rights abuses. Additionally, we are dealing with perpetrators of abuse who are used to freely operating in a culture of impunity. Changing those environments for the better is no small feat.
Also, effective human rights organizations don’t go it alone. We collaborate with many other civil society organizations, NGOs, bi-lateral and multi-lateral organizations and other partners, so making direct attribution for the success we achieve may need to be articulated differently.
I sometimes envy development organizations who can show impact through figures easier for others to understand and connect to – 8,000 people were provided with clean drinking water or $25 can immunize a child for life. However, even Charity:Water is changing its messaging that $20 brings water for 20 years and I applaud their transparency on the question.
But in the human rights realm, how do you show that you have prevented 5,000 women and girls from being raped or that you have helped create an environment where there is less discrimination against a minority group? There are ways to do that, such as recording a decrease in the numbers of women and girls who come forward and report the abuse, or a reduction in the number of people seeking medical or psycho-social treatment. But are your numbers on how many women and girls were raped in the first place reliable? There is a stigma attached to reporting a rape and seeking help, not to mention a fear of reprisals from the very people to whom you are reporting. This can be a hard reality to explain to a potential donor who desires to know exactly how their money is going to directly impact change on the ground.
How Does WITNESS Measures Human Rights Impact?
We are accountable to the human rights defenders and local communities that we aim to serve, not to mention accountable to our generous supporters without whom we couldn’t do this important work. We not only believe that video advocacy does create short and long-term changes, but we also believe it’s possible to monitor and evaluate the impact, although challenging. In our dashboard, you will find a chart that we created to assess progress with our campaign partnerships.
Every campaign partnership starts out with a video action plan where the the advocacy goals for that project are clearly articulated. And then we measure those goals by assigning points that are calculated for completed activities (outputs) and results (outcomes and impact). Here are some examples:
- Outputs – We completed a video advocacy training for our partner. Our partner completed filming and editing. Strategic distribution of the video screened to targeted audiences.
- Outcomes – The video successfully influenced the targeted audience to introduce new legislation (e.g. the Elder Justice Act in the U.S.). A local community is openly discussing and approaching a human rights issue through fresh eyes (e.g. families no longer allowing their children to volunteer into the rebel armed forces in the DRC). If the video actually had an adverse affect on our goals, then we would give it a negative rating. Fortunately we haven’t experienced that so far.
- Impact – The most important of all. Did the video and the accompanying advocacy activities result in real change taking place? For example, a community forcibly evicted from their land is fairly compensated for its loss, or girls who are trafficked into prostitution are placed in protective services, treated as victims rather than perpetrators.
An important disctinction that we often make is that it’s not how many eyes see our videos that matter, but which eyes. Take our partnership with the Centre for Minority Rights & Development in Kenya. The target audience was just a few people on the African Commission on Human and People’s Rights since that is the group which has the power to find the Kenyan government guilty of violating the human rights of the Endorois people. The outcome was a positive one and a landmark ruling for other indigenous groups in Africa. The Endorois people are still waiting to have that ruling translate into real changes in their own lives but we believe it will. It takes time.
We also measure other aspects of our work by figures and charts, such as how many human rights defenders we train, how many people are watching our videos on YouTube, how many people are contributing to the discussion on our blogs, how many videos we sell or license. And we add analysis and feedback (from surveys, for example) from our various stakeholders including those we train, those using our tools, people engaging with our campaigns, etc.
New Directions, New Measurements
Part of our new strategic vision is about scaling impact in other ways. For example, partnering with coalitions and networked organizations, not just individual human rights partners, and developing online interactive toolkits that can be used by tens of thousands of people on the ground. We are currently defining new and revised indicators to monitor this work.
We are always trying to improve and clarify how we monitor our progress and achieve our impact, so please share your reactions and suggestions when you read the dashboard. We make our dashboard available under a Creative Commons license so that other organizations will learn from our experience and we can learn from theirs. Can you point us in the direction of any organization or tool that you think is really effective in this regard?
I wanted to expand on some of the factors that can make it hard to quantify advocacy work. I mentioned the challenges of measuring prevention of future human rights abuses. Some other challenging areas are also capacity building, awareness raising and reconciliation. There is also the reality that we work in local environments where it can be hard to control or influence the daily situations advocates face – force majeure. For example, deteriorating security conditions might prevent or limit someones ability to film. Just a couple of extra thoughts. I look forward to hearing yours.