Uncounted: has the post-2015 data revolution failed already?

12th May 2015

Counting matters. As the Stiglitz-Sen-Fitoussi report puts it: 

What we measure affects what we do; and if our measurements are flawed, decisions may be distorted…. [I]f metrics of performance are flawed, so too may be inferences we draw.

The UN Secretary General was told two years ago by the 2012–13 High Level Panel of Eminent Persons on the Post-2015 Development Agenda that any follow-up to the Millennium Development Goals (MDGs) had to include a data revolution.

In common with the UN global thematic consultation on inequality earlier in 2013, the High Level Panel recognised that challenging inequalities and better data collection are inextricably linked – because better data make it clear which goals are and are not being met, and because with better data we can all demand answers and action. 

"If there’s no recognition of the political nature of how we count, then we’d be fooling ourselves to expect any great change."

So the data revolution can only be about changing the balance of power. Yet much of the current discussion emphasises purely technical reforms instead.

I use the term ‘Uncounted’ to describe a politically motivated failure to count that reflects power. It ignores people and groups at the bottom of distributions whose ‘uncounting’ adds another level to their marginalisation. It ignores people at the top whose uncounting hands them even greater power.

Why do we fail to count well at the bottom? This figure shows three different series for primary school enrolment in Kenya. One comes from the Kenyan National Bureau of Statistics (KNBS); one from the Demographic and Household Surveys (DHS); and one from the Ministry of Education (MOE). MOE data come directly from schools and are used as the basis for funding decisions. 

Now, MOE trends tell you that progress is rapid and unsustained, while surveys look static. Which do you believe? If your children are in Kenyan state education, how well counted do you feel?

Not that survey data are perfect either. Six groups are systematically excluded from most household survey and census returns. Excluded by design are the homeless, those in institutions and nomadic populations. Ignored by undersampling are those living in fragile, disjointed households, in areas facing security risks and in informal settlements. These groups, thought to amount to around 250 million uncounted people – roughly 3.5% of today’s global population – obviously contain a disproportionate share of the world’s poorest people. They are being systematically failed even in the ‘best’ counting approaches we have.

"The world’s poorest people … are being systematically failed even in the ‘best’ counting approaches we have."

It’s no coincidence that people in poverty are excluded. Nor is it because of technical problems that Sudan’s government in Khartoum suppresses publication of data on regional development outcomes. Or that the deaths of those living with disabilities in the UK go uncounted. 

As for counting at the top, it’s equally no coincidence that high-income households are undersampled in surveys. Or that even when tax data are used to adjust the picture, major wealth - $8 trillion? $32 trillion? – remains uncounted. Or that the OECD, charged with measuring the ‘misalignment’ globally between the profits of multinational companies and the actual location of their economic activity, has so far been unable to lay its hands on the necessary data.

Our choice of measure is also important – and also political. Take a look at this chart which shows how two measures, the Gini coefficient and the Palma ratio, come up with radically different answers to the same question about income distribution. Has UK wealth inequality been flat across the crisis? Or did it fall sharply, then immediately rebound even more dramatically?

The Gini coefficient embodies such strong normative views (pp. 129–144) that it doesn't capture well changes in the top 10%, or in the bottom 40% where most poverty lies. It is very encouraging (to me!) that instead the Palma ratio has featured in recent drafts of the post-2015 indicators.

The Palma – which expresses the ratio of income shares of the top 10% to the bottom 40% – also embodies a normative view, but it’s absolutely explicit about it. The chart of UK wealth distribution across the financial crisis shows why the Gini gave rise to so many congratulatory headlines about stable inequality, and why they’re wrong.

What might an actual ‘data revolution’ look like? If there’s no recognition of the political nature of the problem, then we’d be fooling ourselves to expect any great change: the same people and the same things will continue to go uncounted.

What’s noticeable in the discussion so far is that there has been a great deal more attention paid to the uncounted at the bottom than at the top. There’s been precious little mention of Piketty’s proposal for a global wealth register, for instance, or of specific measures that would eliminate anonymous company ownership, require states to exchange tax information with each other (think SwissLeaks), or multinational companies to publish country-by-country reporting (think LuxLeaks). Yet if we don’t start counting things that make elites uncomfortable, then we’re not doing it right.

Data reforms are, broadly, welcome; but a revolution remains far off.  People and things go uncounted largely for political, not technical reasons.

That’s why a data revolution is so badly needed. And revolutions aren’t technical: they’re political. 

2 Comments

1.

Alex Cobham

13th May 2015 at 15:40

Thanks Catherine, you're completely right. In my defence for focusing on quantitative data, I would just say that the temptation to treat data as being universal and somehow neutral is strongest for quantitative data.

There's certainly value in thinking about how qualitative data could enhance the revolutionary nature of post-2015...

2.

Catherine Dom

13th May 2015 at 05:26

My colleague sociologist Pip Bevan sums it in one sentence: "data is not collected, it is made". For many this seems obvious when you talk about qualitative data. But it IS true too for quantitative data, as your blog shows well. This leads me to another point - there's very little practical ideas about using other than quantitative data in all the talk about the data revolution (people's and places' stories and histories...). Or do I miss something?

Leave a comment

The views expressed in Opinions posts are those of the author(s) and do not necessarily represent those of DLP, the Australian Government or DLP's partner organisations.

Documents

Author

Alex Cobham

Alex Cobham

Alex Cobham is Director of Research at the international Tax Justice Network. Over the last fifteen years Alex has held various policy and research posts, including as a research fellow at the Center for Global Development, as of chief policy adviser at Christian Aid and head of research at Save the Children (UK). Alex’s work has mainly focused on issues of taxation, horizontal and vertical inequality, and illicit financial flows. With Andy Sumner, he has proposed a new measure of inequality, the Palma. Alex was on the advisory group for the UN’s thematic consultation on inequalities in post-2015.

Read more

Related items

Why are Africa's poor more likely than the rich to pay a bribe for public services?

The poor aren't simply 'easy targets' - they necessarily come into contact with corrupt state officials more often.

Opinion by Caryn Peiffer19th January 2017

Beyond the limits: can we Think and Work Politically to achieve the SDGs?

How international development agencies need to change to confound the sceptics. (Guest post for the OECD's Institutions and Stability blog)

Opinion by Heather Marquette4th February 2016

Climate change and adaptation in the Pacific Islands: watering down women's security?

Beyond 'adaptability'? In this guest post, Nicole George highlights the work of women leaders who are challenging a narrow adaptation agenda.

Opinion by Nicole George7th March 2014

More room for politics in the inequality debate?

Inequality, a hot topic since Pikettymania, and the theme of the DLP conference earlier this year, has resurfaced with a vengeance.

Opinion by David Hudson24th June 2015

Political settlements: people and the landscapes of power

The inescapable conundrum that politics involves actual politicians is one reason why the subject of political settlements generates so much debate.

Opinion by Alan Whaites24th July 2015

Where do inclusive institutions come from? Lessons from Asia

Societies with more inclusive institutions are more peaceful and more resilient, and tend to be better governed - but how do they get there?

Opinion by Alina Rocha Menocal27th February 2017

Adding gender and power to the TWP agenda

Gender relations are full-blown power relationships. Yet in the development context, they are too often seen as value-neutral cultural arrangements. 

Opinion by Sally Moyle6th August 2015

From functional governance to sustainable peace: Making the space to reflect, learn and adapt

Learning how to balance the technically possible and politically feasible in volatile, conflict-affected contexts.

Opinion by Aditi Haté 22nd February 2017

The practicalities of change: Positive deviance and land reform in Vanuatu

Anna Naupa's 2016 Adrian Leftwich Memorial Lecture discussed where most transformation happens - in drafting the rules, or in putting them into action.

Opinion by Anna Naupa13th April 2016

The curious case of Indian autocracy and what it tells us about 'thinking and working politically'

The history of India’s largely forgotten shift to autocracy and its return to democracy can tell us much about how change happens.

Opinion by Niheer Dasandi25th June 2015

Bringing Political Economy Analysis in from the cold

Once seen as a 'transformative' tool to change donor thinking, does much PEA now do little to help staff think and work politically?

Opinion by Jonathan Fisher6th May 2014

Medellin - more than a miracle

Bad news sells. And for news editors looking for horror stories to recycle, Colombia's second largest city used to be a reliable source.

Opinion by Cheryl Stonehouse4th March 2014

International donors - aiding or abetting?

The importance of acknowledging the dilemmas donors may face when giving aid to developmental states.

Opinion by Niheer Dasandi10th September 2015

Using aid to strengthen Parliaments: fix the car, or worry about the driver?

Parliaments have always been treated as the poor cousins of democracy assistance efforts. (Guest post for From Poverty to Power)

Opinion by Alina Rocha Menocal24th November 2014

Innovation: transactional or transformative?

It's time to discuss how the word 'innovation' might mean different things to different audiences. 

Opinion by Chris Roche23rd March 2015

Reforming FIFA: what can we learn from experience with (other) corrupt autocrats?

FIFA may not be a developing nation, but international football has its own complex political economy. Guest post for From Poverty to Power.

Opinion by Paul Jackson and Heather Marquette11th June 2015

Gender analysis, and thinking and working politically – bridging the gap

Guest post on Devpolicy  introducing panels at this week's Australasian Aid Conference

Opinion by Chris Roche14th February 2017

Thinking about women and girls makes development work better for everyone

A look at what happens when gender analysis is placed more squarely at the heart of governance work. (Guest post in The Conversation)

Opinion by Orlanda Ward7th March 2017

How does politically informed programming shape development outcomes?

A new 'thinking and working politically' community of practice aims to develop practical guidance for development practitioners based on evidence of what works in politically smart programming.

Opinion by Mark Robinson29th January 2016

Identifying rebels with a cause (and effect)

The Developmental Leadership Program will host its 2016 Annual Conference at La Trobe University in Melbourne on 8 February. Its theme is Power, politics and positive deviance.

Opinion by Chris Roche1st December 2015

Don't give up on government

Can the World Bank's flagship World Development Report inspire a good governance revolution that delivers development gains?

Opinion by Dan Hymowitz3rd February 2017

Cancer and the links between medicine and development

DLP Senior Research Partner discusses what cancer has taught him about the links between medicine and development. Guest post for From Poverty to Power.

Opinion by Chris Roche15th April 2015

The politics of redistribution: we need you

Share your thoughts on episodes of redistribution that have helped redress inequality, and help us shape new research into the politics behind them. 

Opinion by David Hudson16th October 2014

'Sticky’ change: What international development can learn from adaptive management

Promoting and sustaining individual behavioural change is as important as building flexibility into development programming.

Opinion by Greg Power2nd December 2016

Breaking new ground in parliamentary strengthening

The importance of tailoring parliamentary support programmes to their context. (Guest post for openDemocracy)

Opinion by Alina Rocha Menocal29th March 2016

What's so 'African' about African leadership?

Does a focus on 'African' leadership obscure the rich and diverse nature of Africa's many states and get in the way of useful lessons from other parts of the world?

Opinion by Suda Perera1st April 2014

Uncounted: has the post-2015 data revolution failed already?

The Stiglitz-Sen-Fitoussi report tells us that how we measure affects what we do; and if our measurements are flawed, decisions may be distorted.

Opinion by Alex Cobham12th May 2015

What's in a name? Leadership as more than the 'big men' and 'big women' of history

A more nuanced understanding of good developmental leadership demands a shift away from the conventional focus on 'big' individuals.

Opinion by Heather Lyne de Ver11th February 2014

Welcome to DLP's blog

Welcome to DLP's new blog on politics, power, policy and developmental leadership

Opinion by Heather Marquette10th December 2013

DLP political settlements workshop: reflections

A practitioner considers how the intangible nature of power can be discussed and included in a policy framework. 

Opinion by Astrid Jamar22nd July 2015

When the stars align to tackle inequality

Reflections on the 2015 DLP annual conference.

Opinion by Alina Rocha Menocal18th February 2015

Taking the Results agenda to the next level?

In The Politics of Evidence and Results in International Development, Chris Roche and his fellow editors examine how practitioners have navigated, resisted and used the ‘results agenda’ for more transformational ends.

Opinion by Chris Roche15th July 2015

What are governance advisers missing with 'Political Economy Analysis'?

DLP's contribution to a new-style field guide for development practitioners. Guest post in FP2P

Opinion by David Hudson8th October 2015

Inequality – the politics behind the policies

On the eve of the 2015 DLP Conference, Deputy Director David Hudson kicks off discussion on the conference theme - the politics of inequality.

Opinion by David Hudson11th February 2015

Different development: walk the talk

The argument for asset-based approaches to development programming and practice that value communities' capacity, skills and knowledge.

Opinion by Gillian Fletcher14th April 2015

Gender - the power relationship that Political Economy Analysis forgot?

While most development research is well on the way to embedding gender analysis, PEA - many donors' key analytical tool - largely ignores it.

 

Opinion by Evie Browne13th February 2014

Connections, contradictions and the political economy of attention

Thoughts on two sources of new and useful ideas about the deeper structures that might shape creativity.

Opinion by Chris Roche7th May 2015

Gender in impact evaluation: norms as well as numbers

Sex-disaggregated data tells us little about the gender-related impact of an intervention, argues DLP research fellow Gillian Fletcher, since gender is a process of judgement linked to norms about femininity or masculinity.

Opinion by Gillian Fletcher27th November 2015

Political analysis as the practical art of the possible

Bringing politics back into PEA - a new paper with Adrian Leftwich

Opinion by David Hudson24th July 2014

Fragmentation of the Thinking and Working Politically agenda: Should we worry?

Many different paths, but all leading to similar destinations - and adding useful nuance to development thinking and practice.  

Opinion by Thomas Parks29th August 2016

Do donors have realistic expectations of their staff when it comes to 'thinking and working politically'?

Is learning to ‘think politically’ like learning a new language? 

Opinion by Heather Marquette9th June 2014

Parliamentary strengthening: the IDC report

Parliaments - and the political parties that populate them - are the institutions people trust least. Guest post for ODI's Shaping Policy for Development blog.

Opinion by Tam O'Neil9th February 2015

Fixing aid: we can't turn off the tap at the first sign of corruption

Much 'petty' corruption is about the poor using what little power they have to stave off destitution. (Guest post for The Conversation)

Opinion by Heather Marquette10th November 2014

Does talking about corruption make it seem worse?

Examining the reluctance of those working in development to engage with the public on the complexity of corruption in poor countries. Guest post for The Guardian's Global Development Professionals Network.

Gender and power: six links and one big opportunity

Donors have recently made great efforts to understand power in partner countries. Yet they have largely ignored one of the most pervasive power relations – gender.

Opinion by Diana Koester21st May 2015

Transparency and Accountability: learning through collaboration

How can the impact of transparency and accountability work be deepened? 

Opinion by Brendan Halloran10th June 2014

What do we do on Monday? Political settlements in theory and practice

Can donors, researchers, policymakers and practitioners all agree on what we mean when we talk about 'political settlements'?

Opinion by Edward Laws15th July 2015

Development cooperation and fighting corruption: thinking differently

Corruption is an emotive word and covers a huge range of behaviours - yet anti-corruption efforts still follow a one-size-fits-all pattern.  

Opinion by Heather Marquette24th June 2015

Developmental leaders, 'dirty hands', and the dark side of collaboration

Should donors support developmental leaders who gain or keep power through questionable means? 

Opinion by Niheer Dasandi11th December 2013

Politics, risk and development: three takeaways

Reflections from last week's Australasian Aid Conference and DLP’s 2016 Annual Conference, both hosted at Australian universities. 

Opinion by Chris Roche19th February 2016

Two remarkable transitions: lessons from Oman and Somaliland

What we can learn from areas of stability and calm in regions where near neighbours seem to be struggling to resolve strife.

Opinion by Sarah Phillips20th July 2015

Security and justice – the mismatch between policy and practice

What hinders more politically nuanced security and justice programming?

Opinion by Shivit Bakrania21st July 2014

Anthropology and elites: 'Studying up', politically

The parallels between - and ethical dilemmas of - anthropology's focus on context and international development's ‘thinking and working politically’ concept. 

Opinion by Paul Robert Gilbert10th March 2016

Oil reform in Nigeria: The ups and downs of channel-hopping programme delivery

How much do we really know about what 'thinking and working politically' can achieve – and where it might present dangers – in challenging political and sectoral contexts?

Opinion by Joanna Buckley27th July 2017