Q&A: Why do we need a philosophy for better evidence?
Stefano D'Errico talks about how better evidence can help decision makers make better policy, and help realise the Paris Agreement and the Sustainable Development Goals.
How do IIED researchers view evidence?
SD: As researchers and evaluators, 'evidence' is absolutely what we do: finding and collating information into evidence, analysing contrasting evidence, making sure decision makers get the evidence they need. But it goes further than that. The key is that evidence is used, that it influences positive change. That's what IIED's research evidence is all about, supporting locally-led transformational development.
So what is 'better' evidence?
SD: Better evidence is evidence that is gathered and analysed in the right way, at the right time, so that it can influence policy effectively. Evidence shouldn't be based only on 'theory' or technical data. It must be rooted in the experience and participation of communities who are affected by the issue or by any policy change.
It should use many different methodologies to check its assumptions; and bring together stakeholders from different areas for analysis and solution-building. And crucially it must recognise its own limitations and be focused on creating change.
What's the idea of a 'philosophy for better evidence'?
SD: We wanted to explore how researchers and evaluators can generate better evidence, so we brought together colleagues from across IIED and like-minded organisations to share their thoughts and experiences. Together we produced a checklist, a sort of philosophy for better evidence.
Better evidence should:
- Be well triangulated, using different sources of information and embracing multiple perspectives
- Be transparently based upon data and information processed through knowledge and experience, including local knowledge and collective experiences
- Address local issues and seek to influence discourse, policy and/or practice at national and global
- Be communicable, well communicated and listened to
- Be action-oriented towards creating positive change
- Challenge perceived wisdom and/or reinforce alternative narratives where relevant
- Be cognisant of its own limitations in being the best evidence under the circumstances (such as financial, contextual, cultural, political), and
- Have a high level of validity and reliability, and be transparent about its own methodological basis.
What does that mean in practice?
SD: Some research methods can be manipulative or extractive; when a consultant goes into a community to collect data for example, and treats people as passive 'objects' in their research.
'Better evidence' is inclusive; people are participants and co-creators in the process. They may help frame the research, gather data, discuss and analyse what it's saying and what it means for them. Inclusive methodologies incorporate a broader cross section of voices, and they may produce more valuable evidence because those affected by a problem are part of the process of finding a solution to it. That solution is far more likely to be implemented if it has buy-in from the start.
Can you give me an example?
SD: Participatory resource mapping is a great example, where residents have mapped their own localities and produced more accurate evidence of how many people live there, what resources they have, what areas are susceptible to flooding or good for development, and so on.
The community will also know its own priorities better than an 'outsider'. Maybe they don't need electricity for more hours a day, but just to know when the power outages might happen; or they know that women won't use latrines in a certain area because of the risk of violence.
What else makes it 'better evidence'?
SD: Sometimes it's about being timely. Evidence needs to be robust of course, but sometimes it's about collecting the evidence you can in order to feed into a particular decision-making process at the right moment. Sometimes evidence needs to be pragmatic if it's the best evidence under the circumstances.
Ultimately it's about gathering and interpreting facts, information and perceptions to challenge our own assumption, and unpack what has happened, how, and why. To do this, we need to be aware of the limitations of the methodology used, or how research and evaluation was conducted, and be transparent about this.
Why is it important?
SD: When decision makers don't have access to good evidence, policies go wrong. The big push on education in the Millennium Development Goals, for instance, led to more children in school, but evaluations have shown that in many instances this has produced less quality education. Evidence needs to be informed by realities on the ground, and to have a feedback loop so decision makers get early warning about the impact of their policies.
Can you tell us about the 'better evidence' methodology papers?
SD: The papers each cover a different methodology and offer a kind of 'toolkit' to others conducting or commissioning research. But the real value of them, I think, is the fact that we examine, and are open about, the limitations and 'blindspots' of each approach – and how they relate to sensitive issues of power, inequality and gender, enabling others to choose the right tool for the job.
Is 'better evidence' particularly important in the current context?
SD: Absolutely. We don't know what is going to happen with climate change: how fast it will happen or what the impacts will be. So we need an agile evidence loop. We need to constantly collect and review the evidence with those affected, and adapt our strategies accordingly.
Responding to climate change, achieving sustainable development – all the complex problems we face – need collaboration between multiple stakeholders based on the best and most relevant evidence. The 2030 (Sustainable Development Goals) Agenda recognises this and puts better evidence at the heart of its ambition.
So what's for next for this initiative?
SD: We want to talk to others to see what they think – to engage donors, communities, social movements and other NGOs in a conversation about 'better evidence', and how we can ensure that it has greater impact on policymaking, and ultimately will improve people lives.