5.2.1 Theory of change

The theory of change is a type of logical thinking exercise that occurs primarily during the development of an intervention but is also helpful during its implementation.

A theory of change describes and explains how and why a result or desired change is expected to happen in a particular context. It focuses on mapping out what a programme or change initiative does (its activities) and how these lead to results (outputs, outcomes, objectives). In this way the theory of change articulates a hypothesis about how change happens by explaining the connection between an intervention and its effect.

It does so by surfacing the logic and rationale for an intervention and articulating the assumptions inherent in the approach.⁴⁶

The theory of change is particularly suited for interventions seeking social or community-based change or those related to empowerment initiatives. It can also be used to measure the complexity of transformation and change, because it acknowledges that social change is not linear but dynamic and complex. Given the fact that reintegration interventions (at individual, community and structural levels) are complex and aim to cover multiple dimensions at economic, social and psychosocial levels, a theory of change can be a useful tool for defining the rationale behind the expected process of change brought about by reintegration interventions.

It is recommended to develop the theory of change using a participatory approach that includes all actors involved in reintegration. It is a collaborative process that can encourage discussion around questions such as:

1. Why do we think this change will happen?
2. What evidence is there to support this?
3. Is this logical?
4. What assumptions are we making?

This will also help all involved clearly understand the link between M&E activities and desired results.

The theory of change helps reveal assumptions to be ‘tested’ through an intervention’s actions. Assumptions therefore play a central role in developing a theory of change. Generally, a theory of change can be articulated using the “If X, then Y, because of Z” formula. That is, “If X action occurs, then Y result will occur, because of Z assumption(s).” The process of surfacing underlying assumptions helps both identify where logical jumps are being made and identify missing key steps in the change process.

Understanding how a theory of change works helps better monitor and evaluate an intervention. A common challenge when designing an intervention are logical leaps and gaps. Often there is a disconnect between strong problem analysis and seemingly unrelated activities meant to address the problem. This is reflected in a causal pathway with weak links between objectives, outcomes, outputs and activities. Through surfacing underlying assumptions, the theory of change is a bridge between analysis and programming.

There are multiple pathways that can lead to a specific objective or the highest level of change. While there may be many other reasons for a specific change to occur, not all of these can be addressed through one single intervention. A theory of change identifies the multiple pathways to change and the most realistically achievable pathway.

A fully developed theory of change clearly spells out the sequence in which outcomes are likely to happen, and how early and intermediate outputs relate to outcomes. Sometimes outcomes are closely related, but they can also occur independently. These changes and connections are often represented visually, for example through a chart or a set of tables (see Table 5.2).

Once results are framed in a theory of change, indicators for each of these can be formulated. As explained, monitoring a theory of change focuses on assessing whether or not the assumptions hold true. Therefore, when developing indicators for monitoring, it is important to take the assumptions of the theory of change into account. (See the “Results’ Monitoring Framework” section for more on indicators and how to formulate them.)

Theory of change diagrams are generally flexible in format and may be simple or complex. They can be vertical, horizontal or circular. The chart below is just one of many ways of illustrating a theory of change. It illustrates an example of what a theory of change for an integrated approach to reintegration could include. It articulates an overall holistic vision of the intended impact of each reintegration intervention, while also spelling out conditions that should be in place for this impact to occur.

5.2.2 Results framework

A results framework or logical framework (“logframe”) clearly formulates intended results, outlines targets and specifies how to plan for success and achieve results.

A logframe helps identify an intervention’s operational design and is therefore the foundation of M&E for that intervention. It is a summary of an intervention’s intended approach to attain results and is based on the situation and problem analysis undertaken during the conceptualization stage. It summarizes the logical sequence in which an intervention aims to achieve desired results and identifies the inputs and activities required to achieve these results. It also provides indicators and sources of verification to measure progress towards achieving results.

A logframe is mostly used in the form of a matrix, which encourages linear thinking about change. It is often viewed as a management instrument for planning, monitoring and evaluation.

The table below is a sample template results’ matrix. The columns are further described in section 5.2.4.

5.2.3 Types of monitoring

Different M&E approaches can be considered for assessing results at each level of intervention (individual, community, structural). The appropriate monitoring approach depends on the overall programme theory of change, main stakeholders, the indicators developed in the results framework and the programme timeline (short or long term).

While there are many more types of monitoring, for the purpose of this Handbook, the following most relevant types are mentioned:

• Programme monitoring tracks progress and performance throughout the entire reintegration programme (covering project activities, results, budget and expenditure, and risk).
•  Beneficiary monitoring tracks individuals’, communities’, governments’ and other relevant stakeholders’ perceptions of an ongoing or completed intervention. Beneficiary monitoring is a way to include beneficiaries in monitoring. It assesses beneficiary satisfaction or dissatisfaction, the level of participation and inclusion, access to resources, how they were treated and their overall experience of change. This type of monitoring is recommended (and particularly useful) for generating qualitative data (narratives of reintegration) from beneficiaries or even any stakeholder. This gives realistic feedback for reintegration interventions and can be used as a tool for programme visibility.
• Reintegration governance assessment assesses at national and regional levels the reintegration ecosystem. This includes the level of engagement of various stakeholders (including migrants, diaspora groups, local authorities and relevant organizations), potential livelihoods’ possibilities and mechanisms for durable solutions. At this level, collaboration of multiple stakeholders is required to assess whether implemented reintegration interventions have made any impact. This should happen over a longer term, at least 16–18 months after the reintegration intervention begins.

As with all programming, it is important to set up clear financial monitoring procedures, as well as risk monitoring.

When designing a reintegration initiative, resources should be allocated specifically for M&E. An overall range for M&E as recommended by the evaluation community is 5–10 per cent of the total budget, with 2–4 per cent for evaluation and 3–6 per cent for monitoring. However, this is purely indicative. Similarly, M&E activities should  be reflected in the initiative’s workplans to support consistent and effective monitoring practices.

5.2.4 Results-monitoring framework

The logical framework can be used as a basis for setting up a results-monitoring framework. This framework enables both all members of the implementing team and all stakeholders, to track progress being made towards achieving intended results.

As a monitoring tool, the results-monitoring framework can be used alongside a detailed work plan, financial reporting tools and a risk management plan to create a more holistic monitoring approach.

What follows is a sample results-monitoring framework based on the theory of change or result matrix outcomes. It outlines the questions that the framework’s components aim to respond to. This should be developed for all outputs and outcomes and for the objective(s). Further explanation on indicators, baseline and target, means of verification, the data collection method and the timeline is provided in the following sections.

Indicators

Indicators are measurable pieces of information that help assess how work or activities lead to results. They show progress towards targets and whether a result is achieved. During monitoring, indicators are meant to measure outputs and outcomes, and for evaluation they can be used at the impact level.

Data source and collection method

Based on the indicators selected, data sources identify where and how information is gathered for the purpose of measuring the specific indicators. The data collection method identifies the method(s) to be used to collect the data. Commonly used methods include:

• Document or desk review
• Observation
• Surveys (mini and formal)
• Interviews (including key informant and exit interviews, see section 2.7)
• Focus group discussions
• Testing or direct measures
• Mapping (for example, community maps)

Data collection sources can include questionnaires, checklists, topic guides,⁴⁷ or project administrative documents such as handover certificates, case file documents, and so on.

When creating a data collection tool, remember to:

• Include fields that record the name of the data collector and the date and location of data collection, biodata and contact information of the respondent.
• Include free and informed-consent and confidentiality clause in the personal data collection instrument if the tool is not anonymous (see section 5.1.1).
• Address data-management requirements for the specific data collection tool. This can include budgeting for resources or staff time to develop and use the tool, as well as databases or systems that may need to be set up and maintained.

Language in data collection tools should be neutral and objective. Consider the data collection skills and technology available in the country. Different tools require different skills and failure to match capacity with the tool creates data bias and error. It is recommended to pre-test the data collection tool.

When it comes to generating feedback through monitoring beneficiaries,⁴⁸ sampling as a method can be specified at the planning stage of monitoring or evaluation.⁴⁹ This method is particularly useful, as often it is unrealistic to meet every beneficiary or visit every project site. Instead, use of a smaller group of beneficiaries, their geographical coverage, allocated resources and security context are all key aspects to be considered. Hence sampling is useful to:

1. Minimize data bias and improving data quality;
2. Reduce the time and money spent on data collection.

Sampling involves a variety of techniques. The choice of technique depends on the context, type of population, information available, data collection method and type of data collected by the project. All techniques provide different answers on:

• Representation: the degree to which the sample “represents” the larger group;
• Sample selection: how the people or places are chosen;
• Sample size: how many people, services and so on to include in the sample.
• If sampling is planned, programme M&E officers with skills in this area should be recruited or trained.

Data analysis

How the data will be analysed will depend on the data collection method. Different tools are needed based on the type of analysis required. Some data collection methods can be analysed for both qualitative and quantitative information. For example, if the indicator is “presence of legislation that reflects international best practice”, the data source would be where the information (data) comes from (copy of the legislation), while the data collection method would be a document review (review of the legislation). Data analysis can be qualitative in nature, for example an expert undertaking an assessment of the degree to which the legislation is in line with international best practices.

Frequency

The timing and frequency of data collection should be clearly defined from the outset of planning. Reintegration programme implementation often takes place in varied geographical places and with various partners, something crucial to consider when deciding the frequency of data collection, because this has budget implications. For example, if the indicator being measured is “referral to psychosocial support”, then it would make sense to monitor the number of persons being referred on a regular basis, such as monthly or quarterly.

Normally the results-monitoring framework is transferred to a clear workplan, where monitoring steps and their frequency are outlined.

Person responsible

There should be clear roles and responsibilities for data collection, verification and storage (see sections 5.3.2 and 5.3.3), especially when multiple stakeholders are involved. There should also be a data controller for personal data who ensures that data protection principles are being followed.

Baseline and target

A baseline provides a foundation against which to measure change over time. The baseline is the first measurement of an indicator; it assesses conditions pre-implementation and sets the conditions against which future change will be measured. A baseline study can have budget implications but can also be based on a previous evaluation or a desk review. When budget is limited, or when security constraints or other factors do not allow for a baseline study, the monitoring visit in which a specific indicator is measured for the first time can be considered the baseline.

The target is what the intervention hopes to achieve and is usually defined in relation to the baseline.

⁴⁶ IOM definition of theory of change adapted from the Center of Theory of Change, What is Theory of Change? (2017).

⁴⁷ A topic guide is an outline of key issues and areas of questioning used to guide a qualitative interview or group discussion.

⁴⁸ Beneficiaries include returnees, community members and local stakeholders.

⁴⁹ A sample is a part of the population, used to describe the whole group. Sampling is the process of selecting units from a population, to describe or make inferences about that population; that is, to estimate what the population is like based on the sample results.