On the Way to Measuring the SDGs: Implications for Project Monitoring

Johanna Hartmann's picture

To discuss the role of the Sustainable Development Goals (SDGs) and their implication on monitoring and evaluation (M&E) of projects, the working group Development Policy of DeGEval, the German Evaluation Society, met end of June 2017 in Bonn. Around 70 persons of German consultancies, civil society, BMZ, GIZ and the federal statistical office participated in the event.

We learnt about the status of developing the indicators for the set of goals. Discussions revealed how they could or will influence the work of development projects in designing, monitoring and evaluating in the development and humanitarian aid sector.

SDG as a network of targets

Imme Scholz from DIE (the German Institute for Development) gave an introduction into the SDG indicators. From a scientific point of view, the SDG are very correlated. She presented the SDGs as a network of targets (see picture). Interactions could have positive, but also negative effects. See also the recommended blog of Caroline Heider of IEG Worldbank about "Policy coherence and SDGs".

The group assessed the implications of the SDGs on planning, monitoring and evaluation. Regarding monitoring, people expect little change for development projects. 1) Since monitoring depends on the planning phase, maybe specific aspects of monitoring will be highlighted like disaggregation of data or impacts on other sectors. 2) The SDGs may provide a normative framework for networked monitoring and aggregation for institutional monitoring. 3) The relevance of capacity development for monitoring the SDGs will probably grow: Monitoring in the partner countries will have implications for donor organisations and the relevant data set to be collected.

One example of evaluation to compare climate impacts is the international climate initiative (IKI). The evaluation of 115 IKI projects (2011-2013) by GFA consulting group revealed the following recommendations and demonstrate that there is room for improvement of M&E: it is very important to formulate measurable indicators; donors should request an effective M&E system; furthermore, cohesion among donors was considered to be important. The next IKI evaluation (2017-2021) will include 425 projects and will profit from the lessons learnt of the first evaluation round. GFA manages the evaluation and tries to guarantee the comparability of single, impact and cluster evaluations. Criteria are internationally recognised (DAC). However, similar to the SDG process, many different departments and stakeholders are involved – an exciting process!

During the presentations and especially in the discussion, several aspects of M&E of the SDG were highlighted:

  • One aspect that needs to be figured out for M&E will be how to measure and deal with the impacts of interactions over the course of time. What focus should projects give to each single SDG? E.g. should we still construct fossil power stations in least developed countries to reach SDG 7 vs. climate change implications of SDG 13?
  • However, projects do not have to claim to contribute directly to SDG data collection.
  • Detailed data sets are only available in some countries: while research in Germany already answers many questions regarding the SDG indicators, this is not yet true for developing countries. Therefore, the SDGs provide an international framework to focus on the global aspect of development.
  • For each SDG there is a custodian agency of the UN. However, 35% of all SDG indicators still need a methodology and no data is yet collected. Furthermore, the process of harmonisation how the data is collected and by whom is still ongoing and should lead to standardisation of methods and estimations.
  • The civil society has also information needs in order to monitor the progress of their initiatives. We need to explore different data collection tools (e.g. based on dialog) and accept that there are cultural differences in (self-) evaluation. There is a need to train skilled people to analyse the data. Participation of local stakeholders in evaluations is very important, as the raising number of joint evaluations proofs.

Final discussion round with Michaela Zintl (BMZ), Sven Harten (DEval), Sven Kaumanns (federal statistical office), Pedro Morazán (Südwind e.V.) and Susanne von Jan (Hanns Seidel Foundation/AK-Epol)

To sum it up: All agreed that the process is still at a very early stage of SDG measurement. All relevant organisations and ministries involved are about to start identifying how they can contribute to measuring the SDGs. Everybody expects to experience the implications of the SDGs on their daily work, starting soon. It is however clear that only countries have to report to the SDG indicators, not projects. Donor or partner countries will then influence planning, monitoring and evaluation of development projects according to the SDG indicators. The SDG process will hopefully enhance the coherence among all stakeholders involved and the understanding of the SDG as a network of 17 interdependent targets.