- GLOSSARY OF PROJECT TERMS
- I. THE BENEFITS OF EVALUATION
- II. PREPARING THE EVALUATION
- III. DATA COLLECTION
- IV. SPECIFICS OF EVALUATION IN PROJECTS
AIMED AT YOUTH AGED 15-24
- V. DATA ANALYSIS
- VI. RESEARCH CONCLUSIONS
I. THE BENEFITS OF EVALUATION
There are many ways to understand evaluation. According to the approach applied in the Youth Impact project the main goal of evaluation is to assess the effects of project activities. The assessment is based on facts and methodology of social sciences with regard to the change to be exerted by the project.
Our approach largely refers to the methods specific to impact evaluation. It is a data-based reflection on the activities and their real effects. It allows you to understand the determinants of the ongoing change, and focus on the sustainability of the achieved outcomes and the impact of the project that goes beyond its direct recipients. This evaluation also allows for the formulation of recommendations supporting project management, which contribute to the effective and efficient implementation of its objectives, as well as the mission of the organization, and take into account the needs of various stakeholders.
Such approach to evaluation enables to determine the value of a given project and to understand the reasons for successes and failures in its implementation. It is also a good management tool for organizations focused on social mission and other “learning” institutions.
BENEFITS OF THE WELL-DONE EVALUATION:
- it allows you to PREDICT different DIFFICULTIES before you start the project (ex ante evaluation) or notice problems at an early stage of its implementation (ongoing or periodic evaluation), and also allows you to plan actions minimizing identified risks.
- PROVIDES RECOMMENDATIONS concerning an ongoing or completed project: it indicates HOW TO IMPROVE a given project to better meet the needs of its recipients, ACHIEVE more durable and useful OUTCOMES, to HAVE A WIDER IMPACT, to FULFILL THE PLANNED OBJECTIVES using less resources.
|Example: The use of the efficiency criterion in the evaluation makes it possible to assess whether, for example, a sufficient number of people were involved in the implementation of the project, and sufficient time and financial resources were allocated to the implementation of individual activities. On this basis, a decision can be made to strengthen the team in terms of staff or to properly plan project tasks.|
- INCREASES THE MOTIVATION OF EMPLOYEES – involving the project team in evaluation, especially at the design stage, as well a in discussing the results of the evaluation study, increases the sense of agency, emphasizes the relationship between the work performed and the goals of the project, the mission of the organization and one’s own values.
- INCREASES THE KNOWLEDGE OF EMPLOYEES – from issues related to project management, through substantive aspects related to its subject, to knowledge of the mechanisms of the changes taking place.
- influences INCREASING THE LEVEL OF CONFIDENCE AND COOPERATION WITH PARTNERS (ALSO IN FUTURE PROJECTS), thanks to taking into account the PERSPECTIVE OF EXTERNAL stakeholders in the EVALUATION.
- enables to demonstrate the achieved results and improves cooperation with SPONSORS and GRANT SUPPLIERS, encouraging them to finance subsequent projects.
|Example: In the application for co-financing, justifying the need for the project, you can quote the results of the evaluation that concerned an earlier / similar project. Providing reliable data may help you convince funders that your project will be effective, and therefore worth funding.|
- serves to PROMOTE THE ORGANIZATION.
|Example: The case studies developed as part of the project evaluation can be used on social media to promote the activities of the organization. These can be, for example, stories of young people who, thanks to your support, acquired new competences and then found a satisfying job or successfully run their own business.|
- IT ALLOWS YOU TO ASSESS IF AND TO WHAT SCALE THE EXPECTED EFFECTS OF THE PROJECT WERE REALLY A RESULT OF PROJECT ACTIVITIES. Moreover, it makes it easier to decide whether a given PROJECT IS WORTH REPEATING, DISSEMINATING, or directed to a different target group.
Overall, evaluation has many benefits. Introducing it to everyday work can be a very useful support for managing an organization – educating and motivating staff, raising funds, strengthening credibility and improving the image, and above all, the effective fulfillment of the assigned mission.
II. PREPARING THE EVALUATION
“You can’t do “good” evaluation if you have a poorly planned program”.
Beverly Anderson Parsons (1999)
This toolkit, we concentrate on impact evaluation. We present practical ways of conducting evaluation focusing primarily on the effects of activities undertaken within a given project. The subject of the analysis are the effects of project activities (outputs, outcomes, impact) and their compliance with the project theory. The project theory defines the concept and plan of a project, including its objectives, activities, expected outputs, outcomes and impact, and how they will be measured, and what resources are needed to achieve them.
The most important element of the project theory is the so-called logic model of change that compiles information on what the project organization needs to accumulate (inputs / resources), the work it needs to do (project activities), and the effects it intends to achieve. The logic model of change for individual projects is created according to the following scheme.
The methods of measuring the outcomes of the project and the related assumptions are sometimes specified in a separate table called the project logic matrix. The logic model and logic matrix should be part of the project documentation.
In practice, it happens that the logic matrix or even the logic model of change have not been developed or are very selective. Lack of assumptions indicating what and how to measure to determine the success of the project makes it impossible to carry out a correct evaluation and thus verify whether the planned change took place and if it did – whether it occurred as a result of the implementation of project activities.
What to do if there is no logic model of change in the project documentation?
In such a situation, it is necessary to recreate the logic of change behind the project, e.g. on the basis of interviews with the management and project staff, as well as already existing documents such as strategy / project implementation plan, justification for its implementation, application for co-financing, partnership agreement, etc. The following table may help you to reconstruct the logic of the project.
Breaking down the logic of the project in such a way allows reflection on the ways of demonstrating the level of achieved effects (outputs, outcomes and impact). This goal is served by defining the indicators by which we will measure the progress of the project. An indicator is an observable attribute (feature) that enables the measurement of the phenomenon to be measured. Each indicator has a measure (quantitative, qualitative) which informs about the degree / intensity of the occurrence of a given phenomenon. In order to measure the change that has occurred as a result of the project implementation, you should determine the values of a given indicator at the beginning and at the end of the project, i.e. the baseline value and the final value. It is also good to know the minimum value of the indicator, if such a value was defined, which at the beginning of the project was considered as confirmation of the achievement of a given project outcome. More information on indicators can be found in the online course (module 2).