Effective evaluation for the Sustainable Development Goals
Evaluation processes will play a key role in national and global review systems for the Sustainable Development Goals. IIED and EVALSDGs are publishing a series of briefings about evaluation designed to help promote effective conduct and use of evaluation in SDGs implementation, follow-up and review.
Head of monitoring, evaluation and learning (MEL), Strategy and Learning
The 2030 Agenda for Sustainable Development adopted by the United Nations in 2015 puts follow-up and review processes at the heart of global and natioanal efforts to achieve the 17 Sustainable Development Goals (SDGs). It says the follow-up and review processes will be informed by country-led evaluations and data which is high-quality, accessible, timely and reliable.
IIED and the EVALSDGs network have prepared a series of briefing papers looking at effective evaluation for the SDGs. The briefings make the case for greater use of evaluation and evaluative thinking in follow-up and review processes for the SDGs, especially at the national and sub-national levels.
They argue that effective evaluation goes beyond measurement, and provides a way to explain data trends and investigate whether progress is equitable, relevant and sustainable.
EVALSDGs (Evaluation – adding Value And Learning to the SDGs) is a network of policymakers, institutions and practitioners who advocate for evaluation of the SDGs and support integration of evaluation initiatives into national, regional and global SDG feedback and review systems.
EVALSDGs aims to form a strong evaluation platform to inform, support, measure and assess development efforts around the SDGs. IIED experts are engaged in wide range of work across the 2030 Agenda.
And in December 2021 this knowledge and learning from EvalSDGs , the German Institute for Development Evaluation (DEval) and IIED (with funding contributed by EvalSDGS through its membership of EvalPartners) was presented in a three-episode podcast series that discussed the details and challenges of assessing progress against the SDGs. The podcast covers:
Episode 1: Adaptive evaluation: considering climate risks in theory and practice
Episode 2: Defining and scoping an SDG evaluation – how do you get started?, and
Episode 3: Country-led evaluations: revealing the progress of Agenda 2030.
Monitoring, evaluation and the SDGs
To date, there has been considerable focus on how to measure progress using indicators and statistics. The briefing papers focus on the role of evaluation. They explain that evaluation differs from monitoring and indicators, but is complementary and adds substantial value for learning and adaptive management.
The briefings also unpack crucial lessons learnt during the implementation of the Millenium Development Goals, and how these should inform national evaluation agendas and systems for the SDGs.
One of the authors, IIED's Stefano D'Errico, says effective evaluation will play a vital role in implementing the SDGs: "There are many misconceptions about evaluation. People say 'we will do a survey', but evaluation involves much more than that.
"Evaluation is about establishing value, worth and merit. It means using critical thinking, asking questions, analysing arguments and assessing claims. This can help to make difficult decisions in complex environments."
Building evaluation capacity is a key task for evaluation professionals as well as for evaluation commissioners, and users. They need to select evaluation questions, understand what is the most appropriate methodology and ultimately get the greatest benefit possible from evaluation findings.
The 2030 Agenda (PDF) commits to engaging in systematic follow-up and review of the implementation of the agenda over the next 15 years. It says that the follow-up processes will operate at national, regional and global levels and will be country-led.
Building the capacity of national policymakers and parliamentarians to use evaluative thinking will be an important task. The agenda documents note that follow-up and review processes "will require enhanced capacity-building support for developing countries, including the strengthening of national data systems and evaluation programmes, particularly in African countries, least developed countries, small island developing states, landlocked developing countries and middle-income countries." (Paragraph 74H)
The new briefing papers are designed to be accessible and useful to a wide audience: they use clear and simple language, and avoid technical jargon. Each briefing will be translated initially into French, Portuguese and Spanish and other languages when possible.
The briefings were developed in partnership between IIED and EVALSDGs, with support from the Government of Finland, UK aid from the UK Government, Sida (Sweden), Danida (Denmark), Irish Aid and the United Nations Development Programme.
Voluntary national reviews (VNRs) are an important tool, rolled out globally, for demonstrating how far countries are progressing toward realising the Sustainable Development Goals (SDGs); but VNRs represent just one approach to evaluation in the 2030 Agenda. Countries can look beyond evaluation in the context of global reporting through VNRs, to embedding evaluation at the national and local level. In this way, evaluation can be a strong entry point for countries to achieve truly transformative development, reflecting the Agenda’s ambition. This briefing examines the idea of ‘transformation’ as promoted in the 2030 Agenda and looks at how it can be applied to evaluation. We argue that when embedded in national planning, evaluation assumes additional roles and functions that go far beyond technical target measurement and can even assess political processes and activities.
This briefing paper argues that measurement is not enough: monitoring must be accompanied by evaluation that addresses the complexity of the SDGs and how they are achieved. It says that to maximise their usefulness to policymakers and citizens, the follow-up and review processes for the 2030 Agenda must incorporate rigorous, country-led evaluations that examine policy and programme implementation and effectiveness, and build well-reasoned and supported cases for claims of progress.
The paper looks at the importance of considering whether progress is equitable, relevant and sustainable. The paper provides an overview of six key considerations for effective evaluation. These are:
- Measurement is not enough
- Evaluation addresses the complexity of the SDGs and their achievement
- Evaluative thinking is indispensable for informed choices
- National policy evaluation is essential
- Evaluation builds solid evidence for claims, and
- Building capacity for evaluation is crucial.
Counting critically: SDG 'follow-up and review' needs interlinked indicators, monitoring and evaluation
The second briefing in the series introduces key considerations for the use of indicators, monitoring and evaluation of SDG implementation, review and follow-up at the national level.
The paper argues that maintaining stakeholder involvement in follow-up and review processes will help national governments select the most relevant indicators, inform their decisions and will foster widespread ownership of the development agenda.
The paper also suggests using an adaptive management approach – exploring emerging and alternative ways to meet objectives and targets, and using knowledge from monitoring and evaluation to develop new understanding and new ways of acting.
This paper highlights the importance of grounding national evaluation systems in a philosophy and practice that matches the interconnected nature of the SDGs. The MDGs highlighted the benefits of adopting a complex systems perspective: this means understanding that goals and solutions are diverse, hoslistic, transnational and can unfold in non-linear ways.
This briefing gives an overview of the complex systems perspective. It also sets out five considerations that can help resource-constrained countries to set national evaluation agendas and maximise the value of evaluation. The five points are: thinking beyond single policies, programmes and projects; examining macro forces influencing success or failure; having a nuanced understanding of 'success'; recognising the importance of culture; and adopting evaluative thinking and adaptive management.
Realising the SDGs by reflecting on the way(s) we reason, plan and act: the importance of evaluative thinking
The fourth briefing paper focuses on the importance of evaluative thinking for effective follow-up and review frameworks and mechanisms.
This briefing describes evaluative thinking, outlines what promotes and inhibits it, and explains how it is intimately connected to adaptive management.
The paper argues that evaluative thinking is an essential component of adaptive management and is indispensable for all decision makers, organisations and communities working towards the SDGs.
This briefing paper examines four challenges to developing national evaluation capacities informed by the SDGs, and highlights areas to consider.
The four challenges are: developing a national evaluation policy; setting up the institutional processes; securing adequate evaluation capabilities; and engaging with partners. The challenges affect both the supply of sound evaluations for development plans, and the demand for their relevant and useful evidence, which in turn informs national policy development.
This briefing paper reports on an study of voluntary national reviews (VNRs) – status reports by countries on how they are implementing national-level follow-up and review frameworks for the SDGs. The first 22 countries presented their VNRs at the 2016 UN High Level Political Forum. The study reviewed these VNRs, examining how they addressed the role of evaluation.
It found that most VNRs showed little awareness of what evaluation is and how it can be used to support the 2030 Agenda. The briefing notes that there is still time to "put more E into M&E", and offers five recommendations to enhance future VNRs.
National parliaments are crucial in ensuring the 17 Goals in the 2030 Agenda for Sustainable Development are meaningfully translated into national policies and programmes that improve citizens’ lives. Parliamentarians are responsible for making national government accountable and should therefore be strong advocates for using evaluation in policymaking. They can also help lead efforts to develop each country’s wider capacity for evaluation. This briefing discusses parliamentarians’ efforts to build national evaluation capacity and use evaluation effectively. Such tasks are not without challenges, but these, we argue, can be reframed as opportunities.
This briefing analyses 43 'Voluntary National Reviews' (VNRs) of progress towards the Sustainable Development Goals. Extending a 2016 analysis, it finds that monitoring is strong but evaluation systems and processes often remain missing or misunderstood. Sporadic good practice is emerging, and countries still to submit their first VNR could build on these examples.
The paper also recommends action for the UN Secretary General, the UN Evaluation Group, national governments, international organisations and professional evaluators to jointly address the significant gaps.
This briefing is also available in Spanish/español.
Update: meeting on 'Tracking development progress and evaluating development partnerships in the post-2015 era'
In December 2016 D'Errico joined decision makers, scholars and evaluators at a two-day meeting focusing on the implications of the SDGs for development and evaluation. The event at Wilton Park examined the key policy issues and evaluation challenges that would need to be addressed in the post-2015 era.
The meeting organisers asked five participants for their views on the state of the SDGs in 2030 – how would we know when the goals have been achieved? You can view their answers below and on Wilton Park's YouTube channel.
From what works to what will work. Integrating climate risks into sustainable development evaluation – a practical guide
Publication, 08 July 2022
The climate crisis: four ways evaluations can strengthen policymaking
Publication, 01 September 2020
Equity-focused, gender-responsive evidence: a blind spot in VNR reporting
Publication, 01 February 2019
Embedding evaluation in national plans and policies to foster transformative development
Publication, 01 May 2018
VNR reporting needs evaluation: a call for global guidance and national action
Publication, 01 January 2018
The 2030 Agenda and evaluation: opportunities and challenges for parliamentarians
Publication, 01 July 2017
Evaluation: a missed opportunity in the SDGs’ first set of Voluntary National Reviews
Publication, 01 May 2017
Developing national evaluation capacities in the sustainable development era: four key challenges
Publication, 01 December 2016