Uncounted: has the post-2015 data revolution failed already?

12th May 2015

Counting matters. As the Stiglitz-Sen-Fitoussi report puts it: 

What we measure affects what we do; and if our measurements are flawed, decisions may be distorted…. [I]f metrics of performance are flawed, so too may be inferences we draw.

The UN Secretary General was told two years ago by the 2012–13 High Level Panel of Eminent Persons on the Post-2015 Development Agenda that any follow-up to the Millennium Development Goals (MDGs) had to include a data revolution.

In common with the UN global thematic consultation on inequality earlier in 2013, the High Level Panel recognised that challenging inequalities and better data collection are inextricably linked – because better data make it clear which goals are and are not being met, and because with better data we can all demand answers and action. 

"If there’s no recognition of the political nature of how we count, then we’d be fooling ourselves to expect any great change."

So the data revolution can only be about changing the balance of power. Yet much of the current discussion emphasises purely technical reforms instead.

I use the term ‘Uncounted’ to describe a politically motivated failure to count that reflects power. It ignores people and groups at the bottom of distributions whose ‘uncounting’ adds another level to their marginalisation. It ignores people at the top whose uncounting hands them even greater power.

Why do we fail to count well at the bottom? This figure shows three different series for primary school enrolment in Kenya. One comes from the Kenyan National Bureau of Statistics (KNBS); one from the Demographic and Household Surveys (DHS); and one from the Ministry of Education (MOE). MOE data come directly from schools and are used as the basis for funding decisions. 

Now, MOE trends tell you that progress is rapid and unsustained, while surveys look static. Which do you believe? If your children are in Kenyan state education, how well counted do you feel?

Not that survey data are perfect either. Six groups are systematically excluded from most household survey and census returns. Excluded by design are the homeless, those in institutions and nomadic populations. Ignored by undersampling are those living in fragile, disjointed households, in areas facing security risks and in informal settlements. These groups, thought to amount to around 250 million uncounted people – roughly 3.5% of today’s global population – obviously contain a disproportionate share of the world’s poorest people. They are being systematically failed even in the ‘best’ counting approaches we have.

"The world’s poorest people … are being systematically failed even in the ‘best’ counting approaches we have."

It’s no coincidence that people in poverty are excluded. Nor is it because of technical problems that Sudan’s government in Khartoum suppresses publication of data on regional development outcomes. Or that the deaths of those living with disabilities in the UK go uncounted. 

As for counting at the top, it’s equally no coincidence that high-income households are undersampled in surveys. Or that even when tax data are used to adjust the picture, major wealth - $8 trillion? $32 trillion? – remains uncounted. Or that the OECD, charged with measuring the ‘misalignment’ globally between the profits of multinational companies and the actual location of their economic activity, has so far been unable to lay its hands on the necessary data.

Our choice of measure is also important – and also political. Take a look at this chart which shows how two measures, the Gini coefficient and the Palma ratio, come up with radically different answers to the same question about income distribution. Has UK wealth inequality been flat across the crisis? Or did it fall sharply, then immediately rebound even more dramatically?

The Gini coefficient embodies such strong normative views (pp. 129–144) that it doesn't capture well changes in the top 10%, or in the bottom 40% where most poverty lies. It is very encouraging (to me!) that instead the Palma ratio has featured in recent drafts of the post-2015 indicators.

The Palma – which expresses the ratio of income shares of the top 10% to the bottom 40% – also embodies a normative view, but it’s absolutely explicit about it. The chart of UK wealth distribution across the financial crisis shows why the Gini gave rise to so many congratulatory headlines about stable inequality, and why they’re wrong.

What might an actual ‘data revolution’ look like? If there’s no recognition of the political nature of the problem, then we’d be fooling ourselves to expect any great change: the same people and the same things will continue to go uncounted.

What’s noticeable in the discussion so far is that there has been a great deal more attention paid to the uncounted at the bottom than at the top. There’s been precious little mention of Piketty’s proposal for a global wealth register, for instance, or of specific measures that would eliminate anonymous company ownership, require states to exchange tax information with each other (think SwissLeaks), or multinational companies to publish country-by-country reporting (think LuxLeaks). Yet if we don’t start counting things that make elites uncomfortable, then we’re not doing it right.

Data reforms are, broadly, welcome; but a revolution remains far off.  People and things go uncounted largely for political, not technical reasons.

That’s why a data revolution is so badly needed. And revolutions aren’t technical: they’re political. 

2 Comments

1.

Alex Cobham

13th May 2015 at 15:40

Thanks Catherine, you're completely right. In my defence for focusing on quantitative data, I would just say that the temptation to treat data as being universal and somehow neutral is strongest for quantitative data.

There's certainly value in thinking about how qualitative data could enhance the revolutionary nature of post-2015...

2.

Catherine Dom

13th May 2015 at 05:26

My colleague sociologist Pip Bevan sums it in one sentence: "data is not collected, it is made". For many this seems obvious when you talk about qualitative data. But it IS true too for quantitative data, as your blog shows well. This leads me to another point - there's very little practical ideas about using other than quantitative data in all the talk about the data revolution (people's and places' stories and histories...). Or do I miss something?

Leave a comment

The views expressed in Opinions posts are those of the author(s) and do not necessarily represent those of DLP, the Australian Government or DLP's partner organisations.

Documents

Author

Alex Cobham

Alex Cobham

Alex Cobham is Director of Research at the international Tax Justice Network. Over the last fifteen years Alex has held various policy and research posts, including as a research fellow at the Center for Global Development, as of chief policy adviser at Christian Aid and head of research at Save the Children (UK). Alex’s work has mainly focused on issues of taxation, horizontal and vertical inequality, and illicit financial flows. With Andy Sumner, he has proposed a new measure of inequality, the Palma. Alex was on the advisory group for the UN’s thematic consultation on inequalities in post-2015.

Read more

Related items

Security and justice – the mismatch between policy and practice

What hinders more politically nuanced security and justice programming?

Opinion by Shivit Bakrania21st July 2014

When the stars align to tackle inequality: reflections on the DLP annual conference

From the Occupy Movement to Thomas Piketty to current proposals for a new set of Sustainable Development Goals, inequality has emerged as one of the most intractable challenges of our time, and everyone, from activists to academics to policymakers, is talking about it. 

Opinion by Alina Rocha Menocal18th February 2015

Resources and reflections on gender and thinking and working politically

Next week's meeting of the Thinking and Working Politically Community of Practice will focus on gender at an opportune time. It follows a spate of interesting papers, blog posts and talks about the relationship between 'thinking and working politically' and gender issues. 

Opinion by Chris Roche12th June 2015

International donors - aiding or abetting?

In September 2012, lawyers representing an Ethiopian farmer announced that they planned to sue the UK government for its role in human rights violations in Ethiopia. The farmer, named in court papers as “Mr O”, alleged that the Ethiopian government’s “villagisation” programme had involved the forced resettlement of thousands of families including his own.

Opinion by Niheer Dasandi10th September 2015

How does politically informed programming shape development outcomes?

Many well-intentioned development programmes founder in the face of resistance from entrenched elites who feel threatened by a potential loss of power and resources. Resources intended for the poor and disadvantaged benefit the rich and powerful. In response, development practitioners and academics have become keenly interested in the political factors that shape development outcomes over the past ten years.

Opinion by Mark Robinson29th January 2016

Why are Africa's poor more likely than the rich to pay a bribe for public services?

Research in Africa has consistently found that the poor are more likely than the better off to pay bribes to state officials for public services. This matters for all sorts of reasons, but from a state-building and developmental perspective, the crisis of trust that corruption can trigger can be devastating. When services are pushed just that bit further away by public-servants-turned-corrupt-gatekeepers, it is likely to colour the already jaundiced perceptions that hard-pressed communities may have of state institutions and of their legitimacy; and also, as Seligson puts it, of ‘the broader national governance frameworks in which they are located’.

Opinion by Caryn Peiffer19th January 2017
Opinion by Heather Marquette10th November 2014

Parliamentary strengthening: the IDC report

Having presented evidence to the UK's International Development Committee, what of the final report?

Opinion by Tam O'Neil9th February 2015

Two remarkable transitions: lessons from Oman and Somaliland

We tend to look through the political settlements lens only at places experiencing either conflict or deep poverty – or both. Yet we would know much more about how useful the lens is if we examined more successes with it. Areas of stability and calm, especially in regions where near neighbours seem to be struggling to resolve strife, might teach us something about how historical experiences do or don’t chime with contemporary donor practices.

Opinion by Sarah Phillips20th July 2015

Does talking about corruption make it seem worse?

Guest post for The Guardian's Global Development Professionals Network

DLP political settlements workshop: reflections

Serendipity, perhaps. I joined the Political Settlements Research Programme at the beginning of June; my first formal engagement was on June 17, at the Political Settlements Workshop organised by the Developmental Leadership Program. It was quite an induction day.

Opinion by Astrid Jamar22nd July 2015

Innovation: transactional or transformative?

Innovation has become a popular word in international development. In Australia today, Bjorn Lomborg helped to formally open DFAT’s development innovation hub innovationXchange, which is designed to ‘identify, trial and scale up successful approaches’. Other donors, including the US and the UK, are also promoting innovation through initiatives like the Development Innovation Ventures programme.

Opinion by Chris Roche23rd March 2015

Gender and power: six links and one big opportunity

Donors have recently made great efforts to understand power in partner countries. Yet they have largely ignored one of the most pervasive power relations – gender.

Opinion by Diana Koester21st May 2015

Don't give up on government

The World Bank launched its flagship World Development Report (WDR) this week, which boldly redefines how governance and policy interact to yield good or bad development outcomes. People are rightly praising the report for rejecting best practicesembracing adaptation and endorsing a focus on politics.

Opinion by Dan Hymowitz3rd February 2017

Anthropology and elites: 'Studying up', politically

Some strikingly parallel questions are being asked in my own discipline of anthropology and by those examining how donors and practitioners can think and work politically with developing communities.

Opinion by Paul Robert Gilbert10th March 2016

More room for politics in the inequality debate?

This month at DLP our focus is largely on corruption as we prepare for a collaborative event in London discussing 'Corruption and development'. Yet we could hardly fail to notice that inequality, a hot topic since Pikettymania and the theme of our conference earlier in the year, has resurfaced with a vengeance.

Opinion by David Hudson24th June 2015

The practicalities of change - positive deviance and land reform in Vanuatu

This guest post by Anna Naupa draws on her Adrian Leftwich Memorial Lecture, presented at the DLP Annual Conference 2016: Power, Politics and Positive Deviance. It is the perspective of Anna Naupa and not that of any organisation with which she is, or has been, affiliated.

Opinion by Anna Naupa13th April 2016

'Sticky’ change: What international development can learn from adaptive management

One of the most significant failings of international political assistance has been the tendency to focus too much on institutional structure and process, and not enough on culture and behaviour.

Opinion by Greg Power2nd December 2016

The curious case of Indian autocracy and what it tells us about 'thinking and working politically'

Today marks the fortieth anniversary of Indira Gandhi’s declaration of a national emergency in India, which led to an 18-month period of autocracy. Civil rights were suspended, political opponents and journalists were arrested without the right to trial, censorship was imposed, elections were cancelled, non-Congress state governments were dismissed, the constitution changed.

Opinion by Niheer Dasandi25th June 2015

Inequality – the politics behind the policies

Discussion starter for the #polinequality conference

Opinion by David Hudson11th February 2015

Do donors have realistic expectations of their staff when it comes to 'thinking and working politically'?

Is learning to ‘think politically’ like learning a new language? 

Opinion by Heather Marquette9th June 2014

Political analysis as the practical art of the possible

Bringing politics back into PEA - a new paper with Adrian Leftwich

Opinion by David Hudson24th July 2014

Fragmentation of the Thinking and Working Politically agenda: Should we worry?

Recently, I’ve read many articles and heard from many colleagues who are concerned about the apparent competition between the Doing Development Differently (DDD) network and the Thinking and Working Politically (TWP) community of practice.

Opinion by Thomas Parks29th August 2016

Gender in impact evaluation: norms as well as numbers

The World’s Women 2015, recently released by the UN, tells us that in 2015 women held 22% of parliamentary seats – almost double the level recorded in 1997 (12%).

Opinion by Gillian Fletcher27th November 2015

From functional governance to sustainable peace: Making the space to reflect, learn and adapt

Last month, UNDP co-hosted a Global Meeting in Jordan on supporting core government functions in fragile and conflict-affected settings. It brought together over 60 colleagues and practitioners from the UN system, World Bank, donors, and government representatives from around the world.

Opinion by Aditi Haté 22nd February 2017
Opinion by Alina Rocha Menocal29th March 2016

Development cooperation and fighting corruption: thinking differently

Everyone associates Brazil with football and the World Cup. Brazilians pouring out onto the street last summer to protest the competition being hosted in their country was last thing many of us expected to see.

Opinion by Heather Marquette24th June 2015

Different development: walk the talk

Spent the day at a ‘Doing Development Differently’ event recently and, while it offered a great opportunity to meet and hear from fascinating, dedicated, thoughtful people, I came away somewhat disheartened. Why? Because:

Opinion by Gillian Fletcher14th April 2015

Gender analysis, and thinking and working politically – bridging the gap

Guest post on Devpolicy  introducing panels at this week's Australasian Aid Conference

Opinion by Chris Roche14th February 2017

Identifying rebels with a cause (and effect)

The Developmental Leadership Program will host its 2016 Annual Conference at La Trobe University in Melbourne on 8 February. Its theme is ‘Power, Politics and Positive Deviance’.

Opinion by Chris Roche1st December 2015
Opinion by Alina Rocha Menocal24th November 2014

Developmental leaders, 'dirty hands', and the dark side of collaboration

new analysis of poverty in India from the World Bank challenges claims that India's reforms and striking economic growth have failed to help the poor and disadvantaged. But the deprivation and inequalities in India (highlighted by Jean Drèze and Amartya Sen in An Uncertain Glory) are stark.

Opinion by Niheer Dasandi11th December 2013

Taking the Results agenda to the next level?

On new book The Politics of Evidence and Results in International Development

Opinion by Chris Roche15th July 2015

Uncounted: has the post-2015 data revolution failed already?

Counting matters. As the Stiglitz-Sen-Fitoussi report puts it: What we measure affects what we do; and if our measurements are flawed, decisions may be distorted…. [I]f metrics of performance are flawed, so too may be inferences we draw.

Opinion by Alex Cobham12th May 2015

What's so 'African' about African leadership?

I'm often guilty of describing myself as an 'Africanist' even though my research focuses only on Rwanda and the eastern DRC.

When I teach courses on African security, I focus on the 'important states' – Nigeria, South Africa, Kenya, and the 'significant conflicts' – Biafra, the Rwandan Genocide, Sierra Leone, Liberia, the Congo Wars, the Sudans and Darfur.

Opinion by Suda Perera1st April 2014

Gender - the power relationship that Political Economy Analysis forgot?

It's easy to forget that 'gender' does not mean 'women' but actually means 'gender relations', or the relations between men and women. Gender analysis examines how power is distributed between women and men, how it operates, who can use it and for what purposes. It's best understood as a system which shapes everything around us – what we think, what we know, whose knowledge is privileged, and which values are supported.

Opinion by Evie Browne13th February 2014

Welcome to DLP's blog

Welcome to DLP's new blog on politics, power, policy and developmental leadership

Opinion by Heather Marquette10th December 2013

What's in a name? Leadership as more than the 'big men' and 'big women' of history

When most people hear the term leadership, they think of 'The Leader', of the powerful CEO or the strong President (also, most likely, a man). This may be in part because the body of research into leadership is overwhelmingly dominated by the fields of business or organisational psychology. Or because the processes of leadership are often conflated with the actions or vision of a single leader.

Opinion by Heather Lyne de Ver11th February 2014

What do we do on Monday? Political settlements in theory and practice

The political settlements framework can seem a distraction to some practitioners, many of whom have been thinking and working politically about development for a number of years. They find the term difficult to define with any precision and, in any case, quite unnecessary. In the real world, progress towards better understanding of and engagement with the political conditions which help and hinder development has been ticking along nicely, independently of the academic debates.

Opinion by Edward Laws15th July 2015

Cancer and the links between medicine and development

Guest post for From Poverty to Power

Opinion by Chris Roche15th April 2015

Political settlements: people and the landscapes of power

The problem with politics is that it involves people, and people do strange things. When development actors engage with power they often prefer to iron out the unpredictability of real politics in favour of the much neater lines of trends and social groups. We revere drivers of change studies because we can cope with the long-term, identity-based analysis of `deep’ politics. 

Opinion by Alan Whaites24th July 2015

Connections, contradictions and the political economy of attention

How can we encourage creativity, even in risk-averse organisations? How can we protect our attention resources?

I listened to two interesting LSE podcasts recently which got me thinking more about creativity following on from a recent blog I posted about the current interest in innovation. Some even suggest the ‘innovation imperative’ is a mega trend.

Opinion by Chris Roche7th May 2015

Medellin - more than a miracle

Bad news sells. And for news editors looking for horror stories to recycle, Colombia's second largest city used to be a reliable source.

Opinion by Cheryl Stonehouse4th March 2014

The politics of redistribution: we need you

Which are the key country cases? Help us shape new research.

Opinion by David Hudson16th October 2014
Opinion by Orlanda Ward7th March 2017

Adding gender and power to the TWP agenda

Thinking and Working Politically presents development to us as an endeavour embedded within power structures. This is so important.

It helps us see clearly that we need to understand domestic politics to deliver development outcomes. Who are the players? Who makes decisions? Who will stand to lose from a proposal and how can they block progress?

Opinion by Sally Moyle6th August 2015

Where do inclusive institutions come from? Lessons from Asia

Inclusion is the new buzzword in international development. The recently adopted Sustainable Development Goals are perhaps the most ambitious articulation of this consensus, with Goal 16 in particular calling for building more ‘effective, accountable and inclusive institutions at all levels’.

Opinion by Alina Rocha Menocal27th February 2017

Climate change and adaptation in the Pacific Islands: watering down women's security?

Beyond 'adaptability'? In this guest post, Nicole George highlights the work of women leaders who are challenging a narrow adaptation agenda.

Opinion by Nicole George7th March 2014

Politics, risk and development: three takeaways

Last week was a big one for the Australasian development community, particularly for those interested in the politics of development. The Australasian Aid Conference at the Australian National University (10-11 February) included a packed session on 'Putting political thinking into development practice'. And DLP’s Annual Conference at La Trobe University focused  on Power, Politics and Positive Deviance.

Opinion by Chris Roche19th February 2016