Monitoring and Evaluation when Politics Matters: Notes from Program Experience

Chris Roche & Linda Kelly (2012) "Monitoring and Evaluation when Politics Matters: Notes from Program Experience" DLP Background Paper 12.

Everyone agrees that there is a crucial role for research and evidence in development policy-making. But the apparently simple claim that policy decisions should be based upon clear and rigorous evidence of value and effectiveness masks the inherently political – and normative -  nature of how we define what counts as ‘valid evidence’, and what this means for policy decisions. This is the argument put forward in the excellent recent paper by Andries du Toit. In it, he warns that the dominance of the Evidence Based Policy  (EBP) agenda has in some ways narrowed the value of research evidence in policy making because the “technocratic concern with ‘what works’” ignores the many other ways in which research and evidence can and should inform, illustrate or challenge policy.

As the first paper in this DLP series on ‘the politics of evaluation’ suggested, when it comes to monitoring and evaluation practice, the same practices tend to apply.  Despite the strength of evidence of the value of a ‘mixed methods’ approach to evaluation for social change – and for many of the reasons highlighted in du Toit’s critique of the EBP discourse (the desire for simple ‘take-home’ advice, institutional incentives, the emphasis on experimental evidence) – monitoring and evaluation practice remains focused on short-term results, narrow interpretations of value-for-money and an overemphasis on the value of experimental evidence.

It seems that we are on the frontier of a narrative shift: between a technical, rational, and scientific approach to development, and a recognition that politics matters; that poverty reduction is not a technical problem but requires significant social change, and that this social change is, and must be, both political and locally led. But how does monitoring and evaluation practice and the use of evidence in policy-making reflect and react to these shifting explanatory frameworks?  And, what can programs that are operating in this newly emerging niche of policy - focused on the politics of social change - do about it?

How can these programs and their evaluators navigate the narrow and tricky path between the pressure to meet existing evaluation and reporting requirements, on the one hand, and the desire to build a strong evidence-base to support the assertion that ‘working politically’ can produce stable and positive long-term development outcomes, on the other? How can they both remain engaged with the dominant value-for-money and results-oriented paradigms, and also push for the space necessary to test and validate the new and evolving ways of ‘thinking and working politically’? And what can donors and other development organisations do to support this?

This, the second paper in DLP’s series on ‘the politics of evaluation’, draws on the experience of the organisations that participated in the DLP ‘Politics Matters’ workshops, (all of which have a number of programs that are ‘working politically’) to offer some answers to these and other questions and to suggest some areas for further exploration.

Tags

Related items

Findings on the politics of service delivery presented at Danida seminar

Thursday 30th October 2014

Senior researcher Claire Mcloughlin has been sharing findings on the politics of service delivery this week with members of the Danish government's development cooperation staff. She spoke at a Danida seminar, which took place on 28-30 October.

Read more

Call for papers: DLP conference on the politics of inequality

Thursday 16th October 2014

How does politics foster or reduce inequalities? We welcome proposals for papers for the 2015 DLP conference.

Read more

Welcome to DLP's new website

Tuesday 3rd June 2014

DLP's Director, Dr Heather Marquette, introduces the new dlprog.org - gateway to the findings of more than five years of research on leadership and the politics of development.

Read more

DLP's core research themes

Friday 30th May 2014

Five core themes run through the heart of this new-look Developmental Leadership Program website. They are signposts to help visitors explore our research, and they are the building blocks of the new strategy that will guide our work over the next three years.

Read more

Public talk by DLP's Niheer Dasandi at UNDP, Singapore on successful policy reform and political-bureaucratic interaction

Monday 28th April 2014

On 28 April, DLP Research Fellow Dr Niheer Dasandi gave a talk at UNDP's Global Centre for Public Service Excellence in Singapore. He addressed the question: how can the interaction between politicians and bureaucrats help to foster successful policy reform?

Read more

Panel on primacy of politics in development at PSA 2014

Tuesday 15th April 2014

On 15 April 2014, DLP will host a panel on 'The Primacy of Politics in Development' at the Political Studies Association conference in Manchester, UK.

Read more

Adrian Leftwich Memorial Conference

Friday 24th January 2014

We were delighted that over 100 scholars and development practitioners joined us for the Adrian Leftwich Memorial Conference on 24 January. We commemorated DLP's founding Director of Research with a fascinating day of reflection and discussion.

Read more

Making Politics Practical II: Development Politics and the Changing Aid Environment

Friday 15th November 2013

The PSA Development Politics Group and the University of Birmingham's International Development Department jointly hosted a workshop on the 15th November 2013. This explored the '(re)politicisation' of development studies and the need to move beyond criticism to constructive engagement with policy makers.

Read more

Monitoring and Evaluation when Politics Matters: Notes from Program Experience

Monday 5th November 2012

How can programs that are focused on the politics of social change navigate the narrow and tricky path between the pressure to meet existing M&E requirements, on the one hand, and the desire to build a strong evidence-base to support the assertion that 'working politically' can produce stable and positive long-term development outcomes, on the other? And what can donors and other development organisations do to support this? This, the second paper in DLP's series on 'the politics of evaluation', draws on the experience of the organisations that participated in the DLP 'Politics Matters' workshops, to offer some answers to these and other questions and to suggest some areas for further exploration.

Read more

Advocating a mixed methods approach to the evaluation of politics

Friday 17th August 2012

Few people would want to argue against the 'results' and 'value for money' agenda that now dominates the current fashions in evaluation and monitoring. But are we clear about what is meant by 'value'? Value for whom? And, value over what period? Are all 'results' amenable to standard methods of evaluation? And how does one evaluate results that are intended or expected mature gradually or occur many years in the future? This new DLP paper by Chris Roche and Linda Kelly explores these issues, looking in particular at programmes and projects that are 'thinking and working politically' and argues that it is now vital for a 'mixed methods' approach to be adopted.

Read more

Bringing History Back in: Three Big Books

Wednesday 11th July 2012

Are you short on time, but feeling the pressure to keep up with the latest ideas in development theory? If so, you probably won't have had a chance to read fully the three recently published and important books on the politics of development: 'Violence and Social Orders' (by D.C North, J.J. Wallis and B.R. Weingast, 2009); 'The Origins of Political Order' (by Francis Fukuyama, 2011); and, 'Why Nations Fail' (by Daron Acemoglu and James A. Robinson, 2012). DLP may be able to help! This paper provides the core summaries of their main arguments and the supporting evidence, accompanied by a brief analysis of some common themes and questions.

Read more

A New Paper: The Politics of Free Public Services in Indonesia

Wednesday 11th May 2011

What explains the differences in quality and kind of service delivery in sub-national districts which are otherwise very similar? Exploring this question in the context of Indonesian decentralization, this paper found that the nature of district leadership was critical. Where district heads pursued strategies of 'political entrepreneurship', becoming dependent upon their electoral support to remain in power, district governments were more likely to promote free public services than where political leaders focused on consolidating patronage networks. These strategies in turn appear related to the political effects of the personal networks, alliances, informal coalitions and constituencies of local leaders.

Read more

New Paper: Yemen: Developmental Dysfunction and Division in a Crisis State

Monday 28th February 2011

Yemen is one of the countries in the Middle East currently experiencing profound turbulence. But what opaque internal politics has kept the regime entrenched for the last three decades? Why have its leaders and elites - like those of many other countries - been so ineffective in addressing serious threats to the viability of the state and to the wellbeing of its citizens? This original and path-breaking research paper by Sarah Phillips offers a detailed political analysis of the inner workings of the Yemen state.

Read more

New Paper: The Politics of a Women's Rights Coalition in South Africa

Wednesday 23rd February 2011

How do women's rights groups campaign for vital institutional reform of archaic laws on sexual violence in new democracies? How can they best 'work politically' to achieve positive outcomes? What lessons are there for donors and supporters? This research paper uses findings from a study of the National Working Group on Sexual Offences to demonstrate how civil society coalitions may draw on and expand their elite networks and exploit political and institutional arrangements to build developmental partnerships.

Read more

Workshop: Political dimensions of development programming

Tuesday 15th February 2011

The Developmental Leadership Program, in partnership with The Asia Foundation (TAF), ran the second in a series of workshops entitled 'Political Dimensions of Development Programming' in Manila, Philippines from 15-16 February 2011. The workshop brought together TAF Representatives working in the Philippines and the region, AusAID staff based in the Philippines, and selected non-government partners.

Read more

The Elephant in the Room: Politics and the Development Problem

Monday 13th December 2010

The Asia Research Centre, Murdoch University hosted a workshop on 13-14 December 2010 entitled "The Elephant in the Room: Politics and the Development Problem".

Read more

Politics into practice: A dialogue on governance strategies and action

Tuesday 7th December 2010

The Overseas Development Institute (ODI) hosted a round table on 7-8 December 2010 entitled "Policy into Practice: A dialogue on governance strategies and action in international development".

Read more

Political Dimensions of Development Programming

Thursday 2nd December 2010

A joint DLP-Asia Foundation workshop on the Political Dimensions of Development Programming was held in Canberra, Australia, on 2nd and 3rd December 2010 for staff of AusAID.

Read more

About DLP

The Developmental Leadership Program (DLP) is an international research initiative that explores how leadership, power and political processes drive or block successful development.

DLP focuses on the crucial role of home-grown leaderships and coalitions in forging legitimate political settlements and institutions that promote developmental outcomes, such as sustainable growth, political stability and inclusive social development.

Find out more

News

Findings on the politics of service delivery presented at Danida seminar

Thursday 30th October 2014

Senior researcher Claire Mcloughlin has been sharing findings on the politics of service delivery this week with members of the Danish government's development cooperation staff. She spoke at a Danida seminar, which took place on 28-30 October.

Read more

Heather Marquette and Caryn Peiffer present research at ANTICORRP Consortium meeting

Friday 24th October 2014

Heather Marquette and Caryn Peiffer are in Berlin this week to give a presentation at the second ANTICORRP Consortium meeting. They are presenting on research on integrity management in Bolivia and Rwanda.

Read more

Follow: @dlprog