How to find surprising development successes

16th April 2018

Much of the governance and development literature looks at the role of institutions in securing progress. But cases where actors – whether leaders or coalitions – are working effectively against the odds are harder to identify. In a new paper we introduce an approach researchers can use to help uncover where such political agency might be at work, focusing on anti-corruption efforts.

Recently, researchers have identified ‘positive outliers’ as one way to study what works in low-quality governance environments. Research on such cases – also called ‘pockets of effectiveness’, examples of ‘positive deviance’, or ‘islands of integrity’ – highlights how institutions, reforms or individuals achieve developmental progress in incredibly challenging contexts. This research has had considerable impact: problem-driven iterative adaptation, for example, is one policy approach that has emerged.

However, the research often suffers from a methodological blind spot. In looking for cases to study, researchers and practitioners have relied on those that already have reputations as success stories. This is problematic, not just because ‘success’ may be misattributed or even misrepresented by charismatic development actors. It means that we miss hidden cases and can only learn from those that are relatively well known.

So how do we locate the positive outcomes that aren’t clearly attributable to a specific program or initiative? When positive development outcomes are unexpected, how do we ensure they don’t escape detection?

This methodology ... could also help identify surprising cases of subnational poverty reduction or service delivery progress, for example.

Our British Academy-funded anti-corruption research on ‘Islands of Integrity’ is inspired by work on positive outliers but has developed a new mixed-methods approach to case selection. In our pilot study, we looked for cases where bribery had unexpectedly – even unintentionally – been reduced. Then we went about discovering why.

To uncover hidden cases of bribery reduction, we first identified statistically significant positive outliers using a simple regression analysis. We examined sector-specific bribery rates from Transparency International’s Global Corruption Barometer to hone in on rates that had reduced far more than expected, given those in other sectors in the same country over the same period.

Second, we vetted 18 of these potential cases through a literature review and preliminary consultations with in-country experts to assess whether any should be excluded from further scrutiny. This enabled us to identify errors in the quantitative data that cause statistical tests to identify false-positives as outlying cases.

It also helped us to identify which cases involved reforms and which didn’t. We found, for instance, that bribery rates in schools in Sierra Leone and Liberia dropped significantly during the ebola year – most likely the result of school closures rather than effective anti-corruption policy. Bribery also reduced significantly in the Mongolian land services sector between 2009 and 2013 – when Ulaanbaatar experienced a dramatic housing and land market slowdown.

Third, we used qualitative fieldwork to check two cases in detail – Uganda’s health sector and South Africa’s police – to see if the statistical method had indeed alerted us to previously unknown positive outliers. Over 11 weeks we visited hospitals and clinics, police stations and government offices and interviewed health care workers, patients, police, government bureaucrats, journalists and academics. This enabled us to examine whether bribery really had reduced in these sectors (spoiler alert: it had) and to identify and work towards understanding the complex, and often politically contentious, causes.  

This methodology could be used in many other areas beyond anti-corruption work. It could also help identify surprising cases of subnational poverty reduction or service delivery progress, for example.

Using statistical analyses to identify unexpected improvements and then checking these cases through qualitative research can enable researchers to investigate the factors that lead to surprising developmental change much more fully.

And as we write up our pilot case studies, it strikes us that there may be another important reason not to rely on reputation alone for case study selection of positive outliers. Because we’ve not been primed to think of the cases as ‘successes’, it’s encouraged us to be critical of our findings, to see the failures that sit alongside the successes, and to begin to think through the (potential) negative consequences that might emerge. 

ImagePeaks through the clouds by flickr user pdvos (CC BY-NC-ND 2.0)

0 Comments

Leave a comment

The views expressed in Opinions posts are those of the author(s) and do not necessarily represent those of DLP, the Australian Government or DLP's partner organisations.

Author

Caryn Peiffer

Caryn Peiffer

Dr Caryn Peiffer is co-investigator on the 'Islands of Integrity' project funded by the British Academy/Global Challenges Research Fund Sustainable Development Grant programme. She is Lecturer in International Public Policy at the University of Bristol and a former DLP Research Fellow. 

Read more

Author

Rosita Armytage

Rosita Armytage

Dr Rosita Armytage is an anthropologist and the lead qualitative researcher on the 'Islands of Integrity' project. She is a Research Fellow at Durham University.

Read more

Related items

Time for a grown-up conversation about corruption

To combat corruption, we need to understand the deeper political realities, power dynamics and social structures that perpetuate it.

Opinion by Alina Rocha Menocal9th December 2014

Service delivery and state legitimacy: For better or for worse?

DLP research fellow Claire Mcloughlin challenges the widely held assumption that there is a self-reinforcing 'virtuous circle' between service delivery and state legitimacy. 

Opinion by Claire Mcloughlin24th November 2015

How does politically informed programming shape development outcomes?

A new 'thinking and working politically' community of practice aims to develop practical guidance for development practitioners based on evidence of what works in politically smart programming.

Opinion by Mark Robinson29th January 2016

Corruption research: Hunting for glimmers of light in the gloom

Why it can be hard to start a conversation with the people who might know what really works in the fight against corruption.

Opinion by Caryn Peiffer25th July 2017

Being 'there': Bermuda Triangulation

Fieldwork in fragile places - Part 2: Data difficulties. Adapting methodology to 'messy' contexts.

Opinion by Suda Perera6th November 2014

How to find surprising development successes

How mixed methods can unearth cases of positive deviance.

Opinion by Caryn Peiffer16th April 2018

Politics, risk and development: three takeaways

Reflections from last week's Australasian Aid Conference and DLP’s 2016 Annual Conference, both hosted at Australian universities. 

Opinion by Chris Roche19th February 2016

Our money, our projects: Demand-driven community development

Emerging lessons from the Central Land Council’s community development program to strengthen Aboriginal people’s participation in mainstream Australia.

Opinion by David Ross15th April 2016

Research methods and marshalling messy data: Dear Diary

The useful role a research diary can play in the assimilation and ordering of qualitative data. 

Opinion by Suda Perera2nd September 2015

Fixing aid: we can't turn off the tap at the first sign of corruption

Much 'petty' corruption is about the poor using what little power they have to stave off destitution. (Guest post for The Conversation)

Opinion by Heather Marquette10th November 2014

Do anticorruption messages work? Findings so far and what they could mean for Papua New Guinea

How do anticorruption messages influence people’s views about corruption and about anticorruption efforts?

Opinion by Caryn Peiffer23rd March 2017

Does talking about corruption make it seem worse?

Examining the reluctance of those working in development to engage with the public on the complexity of corruption in poor countries. Guest post for The Guardian's Global Development Professionals Network.

Being 'there': Reflections on fieldwork in the DRC

Fieldwork in fragile places - Part 1: The security dilemma. Staying safe while collecting the data that matters. 

Opinion by Suda Perera5th November 2014

Beyond perceptions of corruption?

Corruption is hard to define without straying into the subjective. It's also difficult to build robust methodologies to investigate it.

Opinion by Caryn Peiffer5th February 2015

Masculinity and sexual violence in India

The brutal rape and murder in December 2012 of a 23-year-old student in a Delhi bus has been the catalyst for rapidly evolving activism against sexual violence in India.

Opinion by Martin Rew16th September 2015

The practicalities of change: Positive deviance and land reform in Vanuatu

Anna Naupa's 2016 Adrian Leftwich Memorial Lecture discussed where most transformation happens - in drafting the rules, or in putting them into action.

Opinion by Anna Naupa13th April 2016

Developmental leaders, 'dirty hands', and the dark side of collaboration

Should donors support developmental leaders who gain or keep power through questionable means? 

Opinion by Niheer Dasandi11th December 2013

Corruption: is the right message getting through?

Anti-corruption messages aim to recruit citizens to the fight - but what do they actually hear?

Opinion by Caryn Peiffer12th August 2015

Neither 'good guys' nor 'bad guys': Positive engagement with armed groups

The final post in our short series on 'Power, Politics and Positive Deviance', the theme of our 2016 Annual Conference at La Trobe University, Melbourne, on Monday (8 February).

Opinion by Suda Perera5th February 2016

Is education a magic bullet for addressing corruption? Insights from Papua New Guinea

This post for Devpolicy unpacks the findings of a new Development Policy Centre/DLP paper.

Opinion by Grant Walton17th June 2015

Development - getting our story straight

Replacing the traditional aid narrative with a more grown-up - and more inspiring - development story. 

Opinion by Alex Frankel20th April 2016

Is fighting corruption like fighting zombies?

Corruption - a scary word. And our childish, simplistic view of it is hampering efforts to fight it. Guest post in The Guardian

Opinion by Heather Marquette13th October 2015

Peace processes after civil war: Choosing the right tools for the job

Why, despite the best of intentions and the investment of significant resources, do peace processes so often fail to lead to a stable and lasting peace after civil war?

Opinion by Jasmine-Kim Westendorf18th April 2016

Anti-corruption in Bolivia: fighting greed – or attitudes?

Social attitudes towards corruption may be shaped by beliefs and values, rather than facts.

Opinion by Nieves Zúñiga29th June 2015

Communicating anti-corruption messages in development

If donors are allowed to be open about the possibility of corruption, monitoring mechanisms and proportional responses can be planned - and may improve results. Guest post for the OECD.

Opinion by Heather Marquette9th March 2015

Identifying rebels with a cause (and effect)

The Developmental Leadership Program will host its 2016 Annual Conference at La Trobe University in Melbourne on 8 February. Its theme is Power, politics and positive deviance.

Opinion by Chris Roche1st December 2015

Positive deviance and Myanmar's telecoms revolution

A DLP research project looks at the politics of economic reform through the lens of Myanmar's remarkable transformation of its telecoms sector.

Opinion by Niheer Dasandi3rd February 2016

Education, development, and the problem with consensus

Why rethink the international consensus on 'quality basic education for development'?

Opinion by Michele Schweisfurth7th April 2014

Corruption: do we target the servant or the paymaster?

Corruption can only be fought effectively with a coherent strategy collectively supported by all actors. (Guest post for The Guardian on UK aid watchdog report)

Opinion by Heather Marquette5th November 2014

Bringing Political Economy Analysis in from the cold

Once seen as a 'transformative' tool to change donor thinking, does much PEA now do little to help staff think and work politically?

Opinion by Jonathan Fisher6th May 2014

Reforming FIFA: what can we learn from experience with (other) corrupt autocrats?

FIFA may not be a developing nation, but international football has its own complex political economy. Guest post for From Poverty to Power.

Opinion by Paul Jackson and Heather Marquette11th June 2015

Fiji's Roshika Deo - outlier, positive deviant or simply feisty feminist?

Priya Chattier will speak at DLP's 2016 Annual Conference. Her post here begins a short series on the conference theme of Power, politics and positive deviance.

Opinion by Priya Chattier 1st February 2016

Development cooperation and fighting corruption: thinking differently

Corruption is an emotive word and covers a huge range of behaviours - yet anti-corruption efforts still follow a one-size-fits-all pattern.  

Opinion by Heather Marquette24th June 2015

Corruption: unpacking the black box of political will

New thinking on the reasons why individuals engage in corruption - including the pragmatic calculation that, right or wrong, corruption may be the only solution to pressing difficulties. 

Opinion by Heather Marquette12th January 2015

Why are Africa's poor more likely than the rich to pay a bribe for public services?

The poor aren't simply 'easy targets' - they necessarily come into contact with corrupt state officials more often.

Opinion by Caryn Peiffer19th January 2017

Decentralisation and the potential for corruption in PNG

Guest post for Devpolicy on findings from a DLP-supported study on decentralisation and service provision.

Opinion by Grant Walton30th June 2017

Corruption? The developing world has bigger problems

More nuanced anti-corruption work should focus on results - and even put up with some corruption if things are working well. (Guest post for Prospect)

Opinion by Alina Rocha Menocal26th April 2016