Advancing Resilience Measurement Consultation Report
The demand for resilience evidence has grown exponentially as conflict, COVID-19 and the accelerating impacts of climate change have reversed development gains on a massive scale and pushed hundreds of millions of people into crisis levels of poverty and hunger.
Image
Over the last decade, resilience has continued to be elevated as an analytic, programmatic, and organizing concept in development discourse and practice. In line with this, approaches to measuring resilience have proliferated, giving rise to a nascent evidence base on both the impact of resilience programming and the sources of resilience that explain why some households, communities, systems and countries fare better in the face of shocks and stresses than others. Despite clear progress, significant challenges and gaps in resilience measurement and evidence remain. The demand for resilience evidence has also grown exponentially as conflict, COVID-19 and the accelerating impacts of climate change have reversed development gains on a massive scale and pushed hundreds of millions of people into crisis levels of poverty and hunger.
On May 17-18, the University of Arizona, the Global Resilience Partnership and USAID convened a group of 50 experts and development practitioners at the University of Arizona, D.C. Center for Collaboration and Outreach in Washington, D.C. with the aim of advancing resilience measurement and setting a common agenda for addressing these challenges and gaps. The group of experts and development practitioners included representatives from USAID, the State Department’s Special Envoy for Climate, United Nations agencies, the World Bank, private foundations, universities and research institutions, nongovernmental organizations and governments and regional institutions, including the Government of Kenya and the Sahelian West Africa Permanent Committee for Drought Control.
Objectives
The objectives of the consultation were to identify and affirm core principles and priorities for resilience measurement and evidence with a focus on four critical themes. The four themes were selected on the basis that they represent critical frontier issues in resilience measurement that also have significant, unresolved challenges that must be addressed.
The first theme, demand-driven resilience measurement and evidence, was selected in recognition of both the growing demand for and diversity of evidence needs and the extent to which current measurement and evidence are not meeting all of those needs. The second theme, psychosocial resilience measurement, was selected due to the growing recognition that psychosocial resilience and well-being are foundational to resilience and development in ways we are just beginning to understand. The third theme, systems-level resilience measurement, was selected because of the importance of systems in managing shocks and stresses that go beyond the capacity of households and communities to manage on their own. Finally, the fourth theme, climate adaptation, was selected because of the urgency and growing global momentum to support communities and countries to adapt to the accelerating impacts of climate change. The session on climate adaptation did not attempt to tackle the full spectrum of climate adaptation measurement, including measuring climate finance and climate action as part of the global stocktaking. Rather, the discussion focused more narrowly on lessons learned from resilience measurement that can inform climate adaptation measurement.
For the purposes of the consultation, resilience was defined as the ability of people, households, communities, countries and systems to mitigate, adapt to and recover from shocks and stresses in a manner that reduces chronic vulnerability and facilitates inclusive growth. Principles were defined as shared beliefs about each measurement theme and how it should be approached or applied, both conceptually and operationally. Priorities were defined as the most pressing measurement issues to address within each measurement theme within the next three to five years.
Principles and Priorities by Theme
Demand-Driven Resilience Measurement and Evidence
There is growing recognition of the diversity of resilience evidence needs and the extent to which current evidence and measurement approaches are not meeting all of these needs. A better understanding of who needs what type of evidence and when is critical to ensuring that resilience measurement evolves to meet this demand more effectively.
Principles:
-
Engaging evidence users — including communities — in the coproduction of knowledge is critical for ensuring resilience measurement and evidence are demand driven and contribute to strengthening individual and collective agency. This requires balancing the top-down evidence needs of government policymakers and donors with the more localized bottom-up needs of communities, local authorities and project implementers in recognition that all of these stakeholders need resilience evidence for decision-making. It also requires sustained processes for involving communities, local actors and project implementers at all stages of research design, implementation and analysis, including program design and the cocreation of resilience theories of change that inform measurement. Integrating indigenous knowledge systems and strengthening the capacity of communities to use evidence generated by other development actors are both critical in this regard. Finally, it requires feedback loops that provide evidence to users in a timely manner and form that meets their needs.
-
Different evidence users have different evidence needs that require different resilience measurement methods and approaches. A range of methods and approaches are required to meet the growing diversity of resilience evidence needs. This includes qualitative and quantitative methods; objective and subjective measures; a range of novel data collection approaches, such as phone surveys, remote sensing and earth observation; and various types of artificial intelligence (AI)-supported analyses. A common feature of resilience measurement across these methods and approaches is the use of longitudinal data, often panel data. These data are collected at relatively high frequency to assess the impact of shocks and stresses on well-being, as well as the extent to which households, communities, countries and/or systems are resilient and able to mitigate, adapt to and recover from shocks and stresses without compromising their future well-being.
-
Mixed methods and approaches help meet diverse evidence needs and reinforce trust and confidence in findings. Different resilience evidence users have different levels of trust and confidence in findings generated through different methods and approaches. For some, Randomized Control Trials (RCT) constitute a gold standard against which other methods are judged. For others, qualitative evidence they can see and hear themselves inspires more trust and confidence. Still, others recognize that all methods and approaches have strengths and limitations. Identifying the appropriate mix of methods and approaches can help leverage these strengths, address limitations and ensure the responsible use of data and evidence, including anecdotal evidence. It can also help balance the need for rigor and relevance in relation to the specific needs of different evidence users.
Priorities:
-
Develop a map and typology of resilience evidence users, uses and needs at different scales. There is a need for both a generic map and typology to guide our collective approach to demand-driven resilience measurement and context-specific maps that define evidence users and their needs in relation to a specific context and evidence-generating exercises. A process for developing the latter would provide a transparent means of assessing whose evidence needs are and aren’t being met. These context-specific mapping exercises should be done in consultation with a range of stakeholders, including communities, local authorities and other local evidence users.
-
Connect data and evidence from multiple sources in a resilience information network that, collectively, meets a diversity of evidence needs. Given that no single method or approach of even a single mixed-methods, evidence-generating exercise can meet all evidence needs, it is useful to conceive of each exercise as contributing to a resilience information network. These networks can exist at a global scale to help curate global resilience evidence. However, they are also needed at a local scale to provide insights and evidence to inform local actors and action, and this should be prioritized. Regional institutions and governments at a national and local level have a critical role to play in this regard, as do local research institutions, universities and other actors. Resources and support for developing these networks must align behind locally led efforts.
-
Operationalize learning from monitoring and evaluation in (closer to) real time. Time lags between when evidence is needed and when evidence is generated continue to undermine the use of resilience evidence for decision-making. There is an opportunity to ensure evidence is both rigorous and relevant by reducing these time lags and ensuring data collection and analysis are generated in (closer to) real time. The longitudinal, high-frequency data collection approaches that are commonly used in resilience measurement are tailor-made for interim analysis of this kind. However, a commitment by donors, governments and other stakeholders to prioritizing time-bound relevance of the data being collected and analyzed is still required.
-
Improve visualization and translation of resilience evidence in consultation with evidence users to meet their evidence needs. The form that evidence takes is as important as the evidence itself. Improved visualization and translation of evidence into actionable policy and programming recommendations is required. A process of consultation to ensure this is demand-driven is also required, both generally to guide our collective approach and in relation to a specific context, evidence use and users, and evidence-generating exercises.
-
More fully leverage existing, longitudinal data to meet evidence needs. There is a collective recognition that the amount of longitudinal data now available is far greater than it was 10 years ago when efforts to measure resilience in the context of international development began. However, there is also a collective recognition that the data is underutilized and constitute an important resource for meeting demand-driven evidence needs. As part of the process of mapping global and local evidence users and uses described above, existing longitudinal data should be prioritized as a cost- and time-effective means of meeting unmet evidence needs.
-
Better demonstrate the effectiveness of resilience projects and programming. Current approaches to resilience measurement prioritize expanding our understanding of the sources of resilience that explain why some households, communities, countries and systems fare better in the face of shocks and stresses than others. There is a demand to balance this with the related but distinct need to measure the effectiveness of resilience interventions. There is a similar demand to ensure evidence on effectiveness not only captures the collective impact of projects in a resilience portfolio, but the specific contribution of individual projects. Finally, there is an urgent need to prioritize generating additional evidence on the “value for money” of investing in resilience projects and programming expressed in terms of averted humanitarian assistance needs and declines in well-being.
Psychological Resilience and Well-Being
The rise of resilience as an analytic, programmatic and organizing concept in the field of international development, as well as an increase in conflict, insecurity and displacement, have led to a greater interest and focus on psychosocial factors in recent years. In turn, this resulted in a growing, but still nascent, evidence base on their importance for resilience and development.
Principles:
-
Psychosocial well-being is a critically important well-being outcome, an enabling condition that facilitates improvement in other well-being outcomes and a source of resilience that protects other well-being outcomes in the face of shocks and stresses. The importance of psychosocial well-being in each of these three respects has been undervalued and is only beginning to be recognized and understood. Improving and further incorporating measurement of psychosocial well-being is, therefore, critical to expanding our understanding and further substantiating its value as an outcome in its own right, as an enabler of other outcomes and as a source of resilience.
-
Culture and context shape people’s perceptions and understanding about psychosocial constructs and this must be reflected in how they are measured. Many psychosocial concepts and constructs reflect a Western bias. Great care must be taken in measuring and translating these concepts and constructs in non-Western cultures and communities to reduce bias and avoid false equivalence. Even where cross-culturally validated tools are used (e.g., Center for Epidemiologic Studies Depression Scale (CES-D) and Perceived Stress Scale (PSS)), these must be complemented with qualitative inquiry to ground understanding in the local culture and context. Cultural and contextually specific evidence and learning about psychosocial well-being and resilience should take precedence over cross-context and cross-culture comparison. In-depth exploratory and participatory qualitative inquiry with communities are critically important in this regard.
-
Measuring psychosocial resilience and well-being can be sensitive and requires (pre)caution and adherence to do-no-harm principles. Improving and further incorporating the measurement of psychosocial well-being and resilience is critical for addressing profound gaps in our understanding about resilience and development. However, measuring psychosocial well-being and resilience can itself cause discomfort, distress and trauma for respondents and participants. Therefore, care and precautions must be taken, including using skilled enumerators and making psychosocial support available, particularly where respondents have experienced trauma, conflict and violence. The use of in-depth exploratory and participatory qualitative inquiry with communities, as suggested above, can help establish trust and reduce the potential for discomfort, distress and trauma.
-
Psychosocial resilience and well-being (like other forms of resilience and well-being) exist and can be measured at different scales. Individual and community-level psychosocial resilience and well-being are both important in the context of international development for reasons stated above. They are related, but distinct. Understanding the relationship between individual and community-level psychosocial resilience and well-being is an important frontier issue for resilience measurement.
Priorities:
-
Form a technical working group for advancing psychosocial resilience measurement. The primary aims of this Psychosocial Resilience Technical Working Group (PR-TWG) should be to continue to advance measurement practice, generate evidence to further validate the importance of psychosocial well-being for resilience and international development and translate this evidence into actionable policy and programming recommendations. The PR-TWG should include both practitioners and academics — including psychologists, sociologists and other social scientists working in academia and the field of international development — as well as practitioners that use resilience evidence to make sure efforts to advance psychosocial resilience measurement are demand driven and locally informed.
-
Conduct a systematic review of psychosocial resilience and well-being measurement tools, methods and evidence. A priority task for the proposed PR-TWG is to conduct a systematic review of tools, measurement methods and evidence on psychosocial resilience and well-being in the context of international development. This should also include an assessment of the impact of different interventions on psychosocial well-being where possible, including but not limited to, graduation programming. Given the scope and scale of this review, external funding will be required.
-
Further validate the use of existing psychological scales in contexts relevant to resilience and international development, including with communities themselves. There are many scales currently available and in use. Most have been developed in Western contexts. Even for scales that have been validated in different cultural and country contexts, there is a need for further validation among individuals and communities relevant to resilience and international development, including in areas of recurrent crises and among refugees or internally displaced persons (IDPs). A secondary aim of this validation exercise is greater standardization by identifying a more limited set of scales appropriate for use in these contexts.
-
Explore the potential for standardization of concepts, tools and scales, while also allowing for cultural and contextual tailoring. A lack of common understanding of key psychosocial concepts and constructs and how to measure them has the potential to create a lack of coherence at the very moment further clarity is required. There is a need to identify five to seven key psychosocial concepts and constructs and develop guidance on their meaning and measurement in the context of international development. Similarly, the wide range of scales and tools being used to measure psychosocial resilience and well-being is creating a lack of coherence and the very moment when more clarity is needed. The potential for further standardizing tools and scales must be explored.
Systems Resilience
Systems thinking has provided new insights into the complex ways in which components and actors within market, ecological and social systems interact and interconnect, as well as the importance of systems to achieving development outcomes at an individual, household and community scale. Drought, conflict and COVID-19 have further demonstrated how shocks reverberate through systems — the latter on a global scale — and reaffirmed the importance of both understanding and strengthening the resilience of systems.
Principles:
-
Defining a system, its components and boundaries is a subjective exercise. As such, it is critical to engage a broad range of actors within the system in the process of doing so, including the most vulnerable. It is a challenge to find the right balance between including a broad range of actors and keeping systems mapping exercises manageable. However, the inclusion of a variety of actors is needed to fully understand different perspectives on a system’s components and boundaries and how actors in a system interact. Care must also be taken to limit the inherent biases introduced by those facilitating the systems mapping exercise. Incentivizing participation can also be a challenge, particularly among private sector actors that may not see inherent value in their participation.
-
Defining the determinants or sources of a system’s resilience is also subjective and difficult to validate in the absence of systems-level well-being outcomes. A key measurement principle in the foundational 2014 Food Security Information Network (FSIN) Resilience Measurement Principles, is that resilience is a capacity that should be indexed to a development (well-being) outcome. This presents an unresolved challenge for systems-level measurement where such outcomes are either difficult to identify or measure, do not adequately represent a system’s well-being or have a high probability of masking unintended consequences and maladaptation.
-
A primary objective for measuring systems-level dynamics and resilience is to understand their impact on household and community resilience and well-being. Understanding the relationship between these scales and how structures and processes impact households and communities is critical for understanding household and community resilience, as well as the systems and institutions needed to manage shocks and stresses that go beyond the capacity of individuals, households and communities to manage on their own. Resilience at one scale must not be conflated as conferring resilience at another scale unless analytically substantiated. Interscalar analysis to examine these relationships is currently at a nascent state and remains a frontier issue in resilience measurement.
-
Systems-level resilience measurement must be highly sensitive to and anticipate unintended consequences and maladaptation. This holds true for resilience measurement at all scales, as emphasized in the principles on measuring climate adaptation. However, the combination of an uncertain future being shaped by climate change and the complexity and connectedness of systems further exacerbates both the threat of unintended consequences and maladaptation and the potential for them to be masked in systems-level measurement.
Priorities:
-
Make systems and systems resilience measurement more accessible to a broader range of development stakeholders. Many development stakeholders are uncomfortable with systems thinking and terminology or are skeptical about its utility. This is in part a reflection of the complex, dynamic systems themselves. However, it also signals a need to better translate systems thinking and systems-level resilience measurement into terms that those not accustomed to or comfortable with the language and concepts associated with systems thinking can understand. Conceiving of a system as a network and using network mapping and social network analysis may provide one way forward in this regard.
-
Develop a typology of systems and focus future consultations on specific systems to enable a deeper and context-specific discussion. Systems differ enough in their qualities that efforts to measure resilience in one type of system may have little in common with measuring resilience in another type of system. An agreed-upon typology will be useful in guiding future discussions and work, particularly as interest in systems-level resilience measurement extends beyond market systems, social systems and ecological systems to health systems, food systems and other systems.
-
Develop a shared typology or categories for the determinants (or sources) of resilience used in relation to specific systems. The most obvious starting point for this is market systems resilience measurement which has seen significant advances in recent years. Despite these advances, the proliferation of tools and approaches has resulted in different typologies or categories for conceptualizing determinants (or sources) of resilience. In turn, this reduces the ability to compare between methods and studies and aggregate evidence. It also has the potential to cause confusion among evidence users who are not steeped in the nuance of market systems resilience measurement. This shared typology must also be dynamic both in light of new evidence and because systems change over time.
-
Consolidate lessons learned on processes for systems mapping, including the incorporation of risks associated with shocks and stresses and determinants or sources of resilience within the system. The process of mapping a system can be challenging given the complexities and feedback loops involved, as well as the range of actors needed to effectively map out a system’s components and boundaries. A consolidated guide to best practices and lessons learned from different organizations engaged in systems-level resilience measurement would help manage these challenges and lower the barriers to entry for those seeking to incorporate systems-level resilience measurement into their work.
-
Demonstrate the value of systems-level resilience and systems-level resilience measurement to achieving development outcomes. There is a growing appreciation in international development that systems thinking, and systems approaches, including in relation to resilience, provide a means of grappling with complexity of systems and achieving development outcomes at scale. However, that value has yet to be demonstrated to policymakers in relation to the centrality of systems-thinking and systems approaches for achieving development outcomes. Doing so is a priority for sustained interest and investment in systems thinking and systems approaches.
Resilience and Climate Adaptation
Advances in resilience measurement over the last decade provide valuable insights for measuring climate adaptation. However, to effectively inform and contribute to adaptation measurement and decision-making, resilience approaches to measuring climate adaptation must overcome several conceptual and practical measurement challenges.
Principles:
- Resilience measurement frameworks and approaches provide a means of measuring climate adaptation that complements other approaches and fills a critical gap, particularly at the individual, household and community scales. Other approaches to climate adaptation measurement, including measuring climate actions and climate finance, remain critical for gauging progress, including at national and international scales. Greater coordination and collaboration among communities of practice implementing these different approaches to climate adaptation measurement will improve coherence and ensure these complementarities are maximized.
-
Resilience approaches to measuring climate adaptation measure capacities that enable people, households, communities, countries and systems to adapt to and manage the impacts of climate change without compromising current and future well-being. This focus on capacities mirrors current approaches to resilience measurement with additional emphasis on measuring how these capacities protect and enable future well-being. It also includes a more explicit recognition that actions taken to protect current well-being can be maladaptive and compromise future well-being, highlighting the need to account for this in the way climate adaptation is measured. Given the timescales involved and uncertainties about the future, resilience and adaptation capacities and their relationship to future well-being must be analyzed in relation to a range of potential adaptation futures.
-
Climate shocks and stresses occur in complex risk environments in which a range of shocks and stresses are interacting and compounding one another. Conflict, COVID-19 and the accelerating impacts of climate change have demonstrated in stark terms how shocks and stresses and a cascade of downstream effects interact at local and global scales. In turn, this makes isolating the impact of a particular shock or stress (climate or otherwise) increasingly difficult and detached from a complex reality. The compound nature of shocks and stresses that households and communities faced must be incorporated into resilience and climate adaptation measurement. Embracing this complexity also provides a natural bridge for linking climate adaptation action and measurement to efforts and issues at the nexus of these challenges, including the Humanitarian-Development-Peace (HDP) nexus.
-
Flexibility to adapt to a range of potential adaptation futures is a source of resilience and form adaptation in the face of an uncertain future. This is in line with the concept of adaptation pathways, as well as the first and second principles highlighted above on measuring capacities and future well-being in relation to a range of potential adaptation futures in compound and complex risk environments. As such, measures of flexibility must be more effectively incorporated into resilience and climate adaptation measurement.
-
Sustained processes for involving communities in the coproduction of resilience and climate adaptation evidence is required. Communities are already adapting, and local and indigenous knowledge and actors are central to the adaptation evidence enterprise. In line with demand-driven resilience measurement and locally led adaptation principles, engaging communities in all stages of research design, implementation and analysis is critical. This includes engaging communities in developing theories of change that inform measurement and program design. It also includes feedback loops that provide timely evidence to local actors and decision-makers in a form that meets their evidence needs.
Priorities:
- Develop measurement and analytic innovations to address unresolved challenges to measuring climate adaptation using a resilience measurement approach. Foremost among these challenges is how to define adaptation success in terms of adaptation capacities and their relationship to future well-being, including whether well-being has been compromised (maladaptation), given the timescales of climate change and uncertainties about the future. A technical working group made up of resilience measurement experts and climate adaption measurement experts would help speed this much needed innovation. Predictive modeling under different climate scenarios will likely play a critical role and these models should not only consider extreme scenarios.
-
Refine and improve measures of resilience capacities to better reflect the uncertainty and future orientation of climate adaptation. Resilience and resilience measurement have tended to focus on the ability to manage current shocks and stresses. The time horizon and uncertain future associated with a longer-term perspective on climate change demands measuring whether changes are occurring now to better equip people, households, communities, countries and systems to future impacts under different climate scenarios. This includes measuring whether the transformational changes to systems, structures and institutions needed to manage future impacts are occurring now. It also includes measuring flexibility in the face of an uncertain future as a source of resilience and form of adaptation.
-
Greater collaboration and convergence between resilience and climate adaptation measurement communities of practice. Efforts to measure resilience and climate adaptation have emerged in parallel from largely separate communities of practice. Different terminology, timescales, evidence priorities and a lack of understanding of approaches developed by the “other” impede opportunities for greater collaboration, coherence and complementarity and must be overcome.
-
Distill the various approaches used in resilience measurement and climate adaptation for practitioners. One specific recommendation to enhance is to develop a practitioner-oriented overview that explains various measurement approaches and how they differ and complement one another.
-
Conduct measurement exercises with joint resilience and climate adaptation measurement teams. Another recommendation to accelerate collaboration is to create joint resilience and climate adaptation measurement teams to support measurement efforts in a specific country, either in the Horn of Africa and Sahelian West Africa.
-
- Further elevate local and indigenous knowledge and actors in resilience and climate adaptation evidence enterprises. Locally led adaptation principles, as well as several principles outlined in this report, provide strong statements on the need to do so. However, effectively and meaningfully doing so requires confronting long-standing patterns of power that shape how we perceive and value evidence. Greater intentionality in creating space for diverse perspectives on resilience and climate adaptation evidence is required.
Next Steps
The University of Arizona, the Global Resilience Partnership and USAID are committed to working with participants and others to advance the principles and priorities identified during this consultation. This includes socializing the principles and priorities presented in the preceding sections of this report more broadly through a donor briefing and a webinar hosted by USAID’s ResilienceLinks. It also includes using the outputs of this consultation to inform and shape future Advancing Resilience Measurement (ARM) events being planned for the 2022 United Nations Climate Change Conference (COP27) and USAID’s Resilience Evidence Forum in 2023. Ensuring these events build upon one another is critical for sustaining momentum and continuing to grow and support the resilience measurement community of practice. The more specific calls for technical working groups, systematic reviews, follow-on consultations in the global south and other priority actions identified under each theme will be pursued under the auspices of the Global Resilience Partnership’s (GRP) Resilience Knowledge Coalition. Readers interested in collaborating on these follow-up actions are encouraged to join the coalition’s listserv: