Policy engagement is key, but… how to assess its impact?

By Tomás Rosada, IFAD’s Regional Economist for LAC and Ed Heinemann, Lead Technical Specialist – Policy in PTA.

Policies have an impact on every dimension of poor rural people's lives. They shape economic opportunities, provide a ladder for people to climb out of poverty or can prevent them to make it to that ladder.

For IFAD, policy engagement is a core part of our business. It is the only way to create the conditions for larger numbers of rural people to move out of poverty, above and beyond the impact of individual projects.

IFAD-funded projects and programmes are increasingly becoming laboratories to test new ways of tackling rural poverty. When lessons and learning from our investment activities are scaled up and incorporated into policy framework at a national level, the projects achieve their ultimate goal and targets.

Policy engagement happens in many different ways:

• Supporting the creation and strengthening of public institutions working on rural development policies and rural people’s organizations
• Promoting fora for sharing experiences and concerns between governments, rural people and private sector players
• Operationalizing policy programmes at local, regional and national levels.

It is hard to tell when a change in policy will have a positive impact on the lives of rural people, and it is even more difficult to assess the overall contribution policy engagement. Yet answering these questions is key if we want to be able to assess our performance in this field.

Policy engagement: A central part of development work

Some days ago, IFAD hosted the event Assessing the Impact of Policy Engagement. This gathering provided a unique opportunity for IFAD and its partners to reflect on these questions.

IFAD’s Senior Policy Technical Specialist Lauren Phillips, – set the scene: "for IFAD, policy engagement is the process by which IFAD, and IFAD-supported projects and grants, work with governments and other national actors to create, reform, implement or monitor policies.

"The final goal is to influence policy (the combination of laws, regulations, institutional approaches and practices) and to shape it in a way that allows poor rural people to overcome poverty", said Phillips.

John Young, Head of the Research and Policy in Development Programme at ODI ©IFAD

John Young is Head of the Research and Policy in Development Programme at the Overseas Development Institute (ODI), a well-known British think tank. For him, policy engagement is a “central part” of international development work.

“Outright policy change is rare and the causes that lead to it are often unique and rarely able to be replicated.”

However, a number of frameworks and approaches can help to overcome conceptual and technical difficulties when it comes to assessing the impact of policy engagement. All of them involve developing a theory of change: a logical model or roadmap of how one envisages policy changes.

“The earlier and clearer you have your theory of change, the easier it is to check it against delivery.”

Review what works and what doesn’t

Ignacia Fernández, Lead Researcher at the Latin-American Centre for Rural Development (RIMISP), talked about a successful experience of policy engagement in four countries of Latin America. The IFAD-funded project Knowledge for Change has allowed RIMISP to set up Rural Dialogue Groups (GDR) in Mexico, El Salvador, Colombia and Ecuador.

The GDRs are fora where representatives of these countries’ authorities and smallholder organizations discuss how to create a political environment for poor people to overcome poverty. They have facilitated the adoption of policies in favour of family farming in the areas of biodiversity, adaptation to climate change and agricultural innovation.

Susana Márquez and Ignacia Fernández, first and second on the left ©IFAD

After three years of experience, RIMISP was able to set up an evaluation strategy for GDRs based on a model called steps of influence. Its aim is to identify which actors contribute to policy changes -and how.

“This has allowed us to learn more and better about which mechanisms led to positive outcomes and which mechanisms failed and have to be redesigned.”

Susana Márquez, Planning and Strategy Manager at the Unit for Rural Change (UCAR), Argentina’s national branch of the Specialized Meeting on Family Farming (REAF) illustrated some of their achievements of this policy dialogue body of the Common Market of the South (MERCOSUR, the free trade area formed up by Argentina, Brasil, Paraguay, Uruguay and Venezuela).

REAF has functioned as a political forum for representatives of family farmers’ organizations, academia, civil society and policy-makers from MERCOSUR countries to discuss public policies targeting family farming. REAF policy recommendations have been implemented at a national level, including both pro-family-farming legislation and financial support programmes.

Márquez made clear that REAF “has no crystal ball. Yet we know now some approaches work and some do not.”

And what does work? “The adoption of a strict methodology, the definition of clear but broad objectives and, above all, flexibility. To create policy engagement you have to be flexible, trusted by both governments and civil society.”

National strategies to define policy engagement targets

Óscar García, director of IFAD's Independent Office of Evaluation pointed to the Results-Based Country Strategies (COSOPs) as an essential pillar to build on policy engagement.

“The COSOP preparation process is where IFAD and governments agree on how they can collaborate to promote rural development. To evaluate what that collaboration brings about is impossible if we don’t know what we want to achieve.”

Left to right: Edward Heinemann, Óscar García, Paolo Silveri and Lisandro Martín ©IFAD

He also shared a proposed framework for evaluating policy engagement – one that recognises the complex policy cycle and the need to distinguish between policy dialogue, adoption, implementation and outcomes.

Lisandro Martín, IFAD’s Senior Portfolio Manager, reminded us of the strong correlation that exists between good policies and good project outcomes. “Thus, it is critical for IFAD to have a specific focus on policy engagement as part of its self-assessment system.”

Martín underscored that COSOPs are the natural space to capture policy engagement objectives and activities, but “we need to be conceptually clear as to what can be attributed to IFAD’s policy engagement and the outcomes and impact to which it contributes.”

As we closed this half-day event, the following lessons emerged: First, there is no single right method to assess the impact of policy dialogue. Second, and more importantly, every path starts by knowing where you want to go. Otherwise, is impossible to know whether you have arrived or not.