Advocating a mixed methods approach to the evaluation of politics

Few people would want to argue against the 'results' and 'value for money' agenda that now dominates the current fashions in evaluation and monitoring. But are we clear about what is meant by 'value'? Value for whom? And value over what period? Are all 'results' amenable to standard methods of evaluation? And how does one evaluate results that are intended or expected to mature gradually or to occur many years in the future? Is that too risky? Should we only promote programmes and projects that intend to deliver, and hence that can be measured in terms of, immediate or short-term outcomes? Does that mean that donor concerns for embedding the political institutions of accountability, transparency, participation or inclusion, for instance - that always take a long time to mature - should be abandoned? Or just assessed 'in bits'? Or in different ways? 

This new DLP paper by Chris Roche and Linda Kelly explores these issues, drawing heavily on a DLP specialist evaluation workshop held in Canberra in October 2011. The authors identify the wide range of objectives that are commonly pursued by different programmes and projects, looking in particular at those that are increasingly beginning to 'think and work politically'. In these cases anticipated changes are likely to take time, intermediate steps may be hard to discern, and traditional or single-method evaluation approaches may fall short of providing a full and long-term picture of 'value' or 'results'. They argue, therefore, that it is now vital for a 'mixed methods' approach to be adopted to match this growing range of objectives, and to provide a better understanding of the long-term political processes that underpin all inclusive growth stories. A second paper, exploring the politics of evaluation in more detail will follow later in 2012.

Chris Roche and Linda Kelly (2012) "The Evaluation of Politics and the Politics of Evaluation" DLP Background Paper 11.

Related items

Welcome to DLP's new website

Tuesday 3rd June 2014

DLP's Director, Dr Heather Marquette, introduces the new dlprog.org - gateway to the findings of more than five years of research on leadership and the politics of development.

Read more

DLP's core research themes

Friday 30th May 2014

Five core themes run through the heart of this new-look Developmental Leadership Program website. They are signposts to help visitors explore our research, and they are the building blocks of the new strategy that will guide our work over the next three years.

Read more

Public talk by DLP's Niheer Dasandi at UNDP, Singapore on successful policy reform and political-bureaucratic interaction

Monday 28th April 2014

On 28 April, DLP Research Fellow Dr Niheer Dasandi gave a talk at UNDP's Global Centre for Public Service Excellence in Singapore. He addressed the question: how can the interaction between politicians and bureaucrats help to foster successful policy reform?

Read more

Panel on primacy of politics in development at PSA 2014

Tuesday 15th April 2014

On 15 April 2014, DLP will host a panel on 'The Primacy of Politics in Development' at the Political Studies Association conference in Manchester, UK.

Read more

Adrian Leftwich Memorial Conference

Friday 24th January 2014

We were delighted that over 100 scholars and development practitioners joined us for the Adrian Leftwich Memorial Conference on 24 January. We commemorated DLP's founding Director of Research with a fascinating day of reflection and discussion.

Read more

Making Politics Practical II: Development Politics and the Changing Aid Environment

Friday 15th November 2013

The PSA Development Politics Group and the University of Birmingham's International Development Department jointly hosted a workshop on the 15th November 2013. This explored the '(re)politicisation' of development studies and the need to move beyond criticism to constructive engagement with policy makers.

Read more

Monitoring and Evaluation when Politics Matters: Notes from Program Experience

Monday 5th November 2012

How can programs that are focused on the politics of social change navigate the narrow and tricky path between the pressure to meet existing M&E requirements, on the one hand, and the desire to build a strong evidence-base to support the assertion that 'working politically' can produce stable and positive long-term development outcomes, on the other? And what can donors and other development organisations do to support this? This, the second paper in DLP's series on 'the politics of evaluation', draws on the experience of the organisations that participated in the DLP 'Politics Matters' workshops, to offer some answers to these and other questions and to suggest some areas for further exploration.

Read more

Advocating a mixed methods approach to the evaluation of politics

Friday 17th August 2012

Few people would want to argue against the 'results' and 'value for money' agenda that now dominates the current fashions in evaluation and monitoring. But are we clear about what is meant by 'value'? Value for whom? And, value over what period? Are all 'results' amenable to standard methods of evaluation? And how does one evaluate results that are intended or expected mature gradually or occur many years in the future? This new DLP paper by Chris Roche and Linda Kelly explores these issues, looking in particular at programmes and projects that are 'thinking and working politically' and argues that it is now vital for a 'mixed methods' approach to be adopted.

Read more

Bringing History Back in: Three Big Books

Wednesday 11th July 2012

Are you short on time, but feeling the pressure to keep up with the latest ideas in development theory? If so, you probably won't have had a chance to read fully the three recently published and important books on the politics of development: 'Violence and Social Orders' (by D.C North, J.J. Wallis and B.R. Weingast, 2009); 'The Origins of Political Order' (by Francis Fukuyama, 2011); and, 'Why Nations Fail' (by Daron Acemoglu and James A. Robinson, 2012). DLP may be able to help! This paper provides the core summaries of their main arguments and the supporting evidence, accompanied by a brief analysis of some common themes and questions.

Read more

'Reform' Coalitions: A New Concept Paper

Wednesday 9th May 2012

'Reform' coalitions have been noted by many authors in a very diverse range of literatures. But what do we know about them? And should we not understand better their role in the politics of development? What are the circumstances of their provenance, and the political conditions and characteristics of successful ones? Can it be demonstrated that such coalitions have contributed directly to growth and poverty reduction? And, if so, can or should donors work politically to facilitate, encourage and promote their emergence and functioning? In this paper Caryn Peiffer identifies some of the common features of these coalitions and suggests some of the important questions that will be explored by further DLP research.

Read more

Leadership and the 'emerging' African 'success' stories

Thursday 9th February 2012

In his well-received book 'Emerging Africa: How 17 Countries Are Leading the Way', Steve Radelet distinguished between 'Emerging', 'Threshold' and 'Non-Emerging' countries in sub-Saharan Africa. He pointed out that the 'Emerging' countries had performed particularly well in terms of growth and democratization. This new paper, drawing on the DLP African Heads of State database, explores the potential contribution that leadership may have made to these stories. The analysis indicates that the Heads of State in 'Emerging' countries have in general had higher levels of education, are more mature, have a different and more diverse career history and less military experience than both their counterparts in their own countries before the mid-1990s and the leaders of 'Non-emerging' countries.

Read more

Trends and transformations in African political leadership since 1960: Some evidence

Monday 7th November 2011

In a recent speech, the former Prime Minister of the UK, Mr Tony Blair, has suggested that improvement in rates of economic growth and the stabilization of a variety of political orders in parts of Africa can in part be attributed to the role of leadership. New research about African political leadership since 1960, commissioned by DLP, provides some interesting evidence in support of Mr Blair's claim. In this thought-provoking paper, Monique Theron provides statistical trends and patterns over the last 50 years with respect to the profiles of different types of ruler, their educational qualifications, their field of tertiary study, their career histories before becoming Heads of State, their political backgrounds, their length of incumbency and how they gained and lost power.

Read more

A New Paper: The Politics of Free Public Services in Indonesia

Wednesday 11th May 2011

What explains the differences in quality and kind of service delivery in sub-national districts which are otherwise very similar? Exploring this question in the context of Indonesian decentralization, this paper found that the nature of district leadership was critical. Where district heads pursued strategies of 'political entrepreneurship', becoming dependent upon their electoral support to remain in power, district governments were more likely to promote free public services than where political leaders focused on consolidating patronage networks. These strategies in turn appear related to the political effects of the personal networks, alliances, informal coalitions and constituencies of local leaders.

Read more

Research and Policy Workshop in Frankfurt

Monday 4th April 2011

The Developmental Leadership Program (DLP) held its first Research and Policy Workshop in Frankfurt on 10th and 11th March, 2011, facilitated by the GIZ. The focus of the DLP is on the role of developmental leaderships and coalitions in the politics of sustainable growth, political stability and inclusive social development. The purpose of the workshop was to explore and elaborate the policy, programme and operational implications of the recently completed phase of DLP research and to map the way forward for future research, policy and communications activities.

Read more

New Paper: Yemen: Developmental Dysfunction and Division in a Crisis State

Monday 28th February 2011

Yemen is one of the countries in the Middle East currently experiencing profound turbulence. But what opaque internal politics has kept the regime entrenched for the last three decades? Why have its leaders and elites - like those of many other countries - been so ineffective in addressing serious threats to the viability of the state and to the wellbeing of its citizens? This original and path-breaking research paper by Sarah Phillips offers a detailed political analysis of the inner workings of the Yemen state.

Read more

New Paper: The Politics of a Women's Rights Coalition in South Africa

Wednesday 23rd February 2011

How do women's rights groups campaign for vital institutional reform of archaic laws on sexual violence in new democracies? How can they best 'work politically' to achieve positive outcomes? What lessons are there for donors and supporters? This research paper uses findings from a study of the National Working Group on Sexual Offences to demonstrate how civil society coalitions may draw on and expand their elite networks and exploit political and institutional arrangements to build developmental partnerships.

Read more

Workshop: Political dimensions of development programming

Tuesday 15th February 2011

The Developmental Leadership Program, in partnership with The Asia Foundation (TAF), ran the second in a series of workshops entitled 'Political Dimensions of Development Programming' in Manila, Philippines from 15-16 February 2011. The workshop brought together TAF Representatives working in the Philippines and the region, AusAID staff based in the Philippines, and selected non-government partners.

Read more

New paper on 'Growth Alliances: Insights from Egypt'

Wednesday 5th January 2011

A recent paper in Business and Politics explores the emergence, evolution and forms of 'growth alliances' in Egypt. Professors Abla Abdel-Latif, of the American University in Cairo, and Hubert Schmitz, of the Institute of Development Studies, show how largely informal and politically-negotiated relations helped to shape alliances between some business sectors and key policy-makers without apparently becoming abused for private gain.

Read more

The Elephant in the Room: Politics and the Development Problem

Monday 13th December 2010

The Asia Research Centre, Murdoch University hosted a workshop on 13-14 December 2010 entitled "The Elephant in the Room: Politics and the Development Problem".

Read more

Politics into practice: A dialogue on governance strategies and action

Tuesday 7th December 2010

The Overseas Development Institute (ODI) hosted a round table on 7-8 December 2010 entitled "Policy into Practice: A dialogue on governance strategies and action in international development".

Read more

Political Dimensions of Development Programming

Thursday 2nd December 2010

A joint DLP-Asia Foundation workshop on the Political Dimensions of Development Programming was held in Canberra, Australia, on 2nd and 3rd December 2010 for staff of AusAID.

Read more

About DLP

The Developmental Leadership Program (DLP) is an international research initiative that explores how leadership, power and political processes drive or block successful development.

DLP focuses on the crucial role of home-grown leaderships and coalitions in forging legitimate political settlements and institutions that promote developmental outcomes, such as sustainable growth, political stability and inclusive social development.

Find out more

News

Cash transfers: why elites' attitudes matter

Wednesday 24th September 2014

A new DLP Research Paper presents findings from Malawi on elites' attitudes to poverty and their opinions on how to reduce it. The findings question the sustainability of cash transfer programmes beyond donor funding.

Read more

Informing debate: why higher education matters for the post-2015 agenda

Thursday 11th September 2014

DLP-commissioned research is being fed into the continuing debate on the post-2015 agenda to establish development priorities beyond the Millennium Development Goals.

Read more

Follow: @dlprog