III. DATA COLLECTION
Data Collection
3.1. What are the major types of evaluation research methods?
In order to estimate the value and quality of the project in relation to the chosen criteria and answer the evaluation questions, you should correctly collect the necessary information. Research methods and tools serve this purpose. Research methods mean a specific way of collecting information – qualitative or quantitative – with the use of specially developed tools, such as interview scenarios, observation sheets or questionnaires. Let’s look the differences between these methods and research tools.
Qualitative methods enable the collection of data in an in-depth and flexible manner, but they do not allow you to assess the scale of the studied phenomena as these methods cover only a small number of people from the groups involved in the project (e.g. selected recipients). On the contrary, quantitative methods are used in the case of large groups that consist of several dozen people. In the case of more numerous groups (e.g. more than 400-500 people) these methods enable the generalisation of conclusions drawn from the survey of a representative, randomly selected sample of people for the entire population, i.e. the community that is of interest to the researcher, including people who did not participate directly in the particular study. This generalisation must be carried out in a specific way that will ensure that the sample of people subjected to the study is representative, i.e. maximum similarity in various socio-demographic characteristics to the population from which they were selected.
Comparison Of Qualitative And Quantitative Methods Of Evaluation Research
Both of these types of methods have some strengths and weaknesses, therefore you should always use both qualitative and quantitative methods in the evaluation study. This approach is in line with the triangulation principle aimed at ensuring the high quality of the information collected. Triangulation means using various sources of information, types of collected data and analytical techniques, theories explaining the identified relationships / mechanisms, as well as people conducting the evaluation (whose competences should complement each other). Providing diversity of this elements triangulation enables:
- comprehensive knowledge and understanding of the studied object,
- taking into account various points of view and aspects of the phenomenon studied,
- supplementing and deepening the collected data,
- verification of collected information,
- increasing the objectivity of formulated conclusions.
3.2. What methods and tools are typically used in evaluation research?
To facilitate the choice of methods and tools most appropriate for a particular evaluation, below are the characteristics of the most popular of them:
- Qualitative methods
- desk research,
- individual in-depth interviews (IDI),
- focus group interviews (FGI),
- observation,
- case study.
- Quantitative methods (surveys)
-
- survey conducted without the participation of an interviewer – self-administered paper surveys, computer-aided web interview / online survey (CAWI), central location (simultaneously surveying all respondents),
- questionnaire interviews conducted with the support of a pollster – paper and pen interview (PAPI), computer-assisted personal interview (CAPI) and computer-aided telephone interview (CATI).
- Active / workshop methods (mixed, i.e. qualitative and quantitative).
3.2.1. DESK RESEARCH
In the case of desk research existing data is used, i.e. data that was generated regardless of the actions taken by the evaluator.
The existing data includes internal data (generated for the needs of the evaluated project) and external data:
- Internal data is information created during the preparation and implementation of project activities (e.g. project application, training scenarios, attendance lists, contracts, photos, videos and materials about the project posted on the website, posts and responses on social media). In the case of training projects for young people looking for a job, these may also be the results of measuring the competences of the beneficiaries at the beginning and at the end of participation in the training (knowledge tests, skills tests, attitudes tests, etc.)
- External data is information that may relate to the studied phenomenon, processes or target group, but has been collected independently of the evaluated project (e.g. statistics, data repositories, reports, articles, books, videos, and other materials available on the Internet). In the case of the evaluation of employment projects, it is worth using information on similar projects, as well as data available to labour offices, social insurance institutions, national statistical offices, regarding the employment of young people living in a particular town.
Documentation analysis is the basic method of collecting information on a given project, also providing some knowledge about the needs of its recipients and the context of the evaluated project.
CONDITIONS OF APPLICATION:
Public institutions provide administrative data in accordance with the principle of transparency in the operation of public institutions and civic participation (open government concept). However, it is important to assess the data reliability and accuracy based on the methodological information provided in the source documentation.
ADVANTAGES:
- accessibility (especially regarding information available on the internet),
- large variety (you can use any data / materials related to the conducted evaluation),
- no costs – most documents and data are available free of charge,
- no evaluator’s effect on data in the case of external data.
DISADVANTAGES:
- different levels of data credibility – you need to take into account the credibility of the source and the context of data acquisition (under what conditions, who collected and analysed the data and why),
- restrictions on the access and use of internal information due to the protection of personal data, copyright and property rights.
3.2.2. INDIVIDUAL IN-DEPTH INTERVIEW (IDI)
An individual interview takes the form of a direct conversation between the interviewer and the respondent, usually conducted using a scenario. The interview allows you to obtain extensive, insightful and in-depth information, get to know opinions, experiences, interpretations and motives of the interviewee’s behaviour, examine facts from the interviewee’s perspective, as well as gaining a better understanding of their views.
IMPORTANT TIP
The language of the interview should be adapted to the respondent. In interviews (especially with young people) use simple language and avoid specialistic vocabulary (e.g. project jargon), that may cause misunderstanding of the questions asked and intimidate the interviewees. |
CONDITIONS OF APPLICATION: Individual interviews should be conducted in quiet rooms that guarantee discretion. Interview recording is a common practice, but the respondent does not always agree – in such cases the researcher should take notes during the interview and complete them immediately after the meeting. It is recommended that the interview be conducted by an external expert to avoid situations in which the interviewee feels uncomfortable expressing honest opinions.
ADVANTAGES:
- the possibility to discuss complex and detailed issues,
- better understanding of the interviewee’s point of view (“getting into his/her shoes”),
- getting to know facts in the situational context,
- flexibility – the possibility to adapt to the interviewee and to ask additional questions not included in the scenario.
DISADVANTAGES:
- unwillingness of some interviewees to express honest opinions due to lack of anonymity,
- the impact of the interviewee’s personality traits on the findings obtained, e.g. difficulty in obtaining information from people who are taciturn, shy or introvert.
RESEARCH TOOL: the interview may be supported by an interview scenario, containing a list of questions or issues to be discussed. The interviewer can change the order of questions or add some questions during an interview if it is needed to better understand the issue.
Examle Of IDI Scenario For The Project Team
Individual IDI Scenario
3.2.3. FOCUS GROUP INTERVIEW (FGI)
A focus group is a conversation between about 6-8 people supported by a moderator who gives the group issues for discussion and facilitates its course. FGI participants are selected according to specific assumptions set by the researcher and their knowledge of studied issues.
IMPORTANT
In the case of young people, the discussion should be divided into shorter forms, involving all the participants, so that they do not get bored too quickly. It is worth using multimedia tools, elements of gamification or non-standard solutions, e.g. a paper cube with questions, thrown by the participants themselves. It is helpful to write down a group’s opinions on a flipchart and record the group discussion. |
CONDITIONS OF APPLICATION: The basic condition for the success of a group interview is correctly selecting people with specific information that they are ready to share. It is important to guarantee that the participants are comfortable by organising the interview in a quiet room of the right size with comfortable seating, a large oval / square table and a flip chart.
ADVANTAGES:
- learning about different points of view, taking into account different opinions,
- mutual verification and supplementation of information about the facts discussed by different persons,
- the opportunity to observe interactions between participants,
- obtaining relevant information from several people in a relatively short time.
DISADVANTAGES:
- dynamics of group processes, including pressure on group consensus / cohesion, may lead to minority opinions not being disclosed, e.g. due to the group being dominated by a natural peer group leader,
- risk of transferring to group conflicts or bad interpersonal relations, reducing the effectiveness of the research and the reliability of the findings obtained,
- organisational difficulties (the need to gather a group of people at a particular place and time and to provide a properly equipped room)*.
RESEARCH TOOL: the tool used by the moderator for this method is an FGI scenario, which includes the principles of group discussion, specific issues / questions and guidelines regarding various forms of activity in which the moderator is to involve the participants.
FGI Scenario
*Both IDIs and FGIs can be conducted by remote means using online communicators.
3.2.4. OBSERVATION
This method is based on careful observation and listening to the studied objects and situations (phenomena, events). The observation may be participant, partially participant or non-participant, depending on the degree of involvement of the researcher, who may act as an active participant in the events he or she observes or as an external, uninvolved observer. The observation can be carried out in an overt, partially overt or covert way*, i.e. the participants of the event may know that they are being watched or selected persons (e.g. trainer and / or training organiser) or only an observer know about it.
CONDITIONS OF APPLICATION: if the observation is non-participant, the observer should not come into contact / relations with the people being observed as this carries the risk of affecting the course of the observed events and behaviours.
ADVANTAGES:
- providing information about a particular event / process during its course,
- reporting facts without their interpretation by the participants (examination of actual behaviour, not declarations,
- facilitating the interpretation of investigated events,
- the opportunity to learn about phenomena usually hidden or unnoticeable or that people are reluctant to discuss.
DISADVANTAGES:
- possible influence of the researcher on the course of events (the respondents’ awareness that they are being observed may change their behaviour),
- limited scope of observation range, difficulty in accessing all events,
- the risk of subjectivity (the researcher may assume some stereotypes, perceive and interpret events for the benefit of the observed group).
RESEARCH TOOL: The observation may be conducted using a research tool which is the observation sheet. Its use focuses the observer’s attention on selected issues and enables the recording of important information (e.g., the behaviour of people participating in the observed events), which may be not only qualitative, but also quantitative (the checklist).
Training Observation Sheet
* With regard to evaluation studies, we do not recommend covert observation, i.e. one that is not known to the people who are its subject.
3.2.5. CASE STUDY
This is an in-depth analysis of the studied issue using information from different sources and collected by various methods. Its findings can be presented in a narrative form. The analysed “case” could be a person, group of people, specific activities, a project or a group of projects.
The case study is used to:
- get to know thoroughly and understand a particular phenomenon along with its context, causes and consequences,
- illustrate a specific issue using a realistic example with a detailed description,
- generate hypotheses for further research,
- present and analyse best / worst practices to show what is worth doing and what should not be done.
CONDITIONS OF APPLICATION: This method requires time to collect and analyse various data regarding the phenomenon / object being studied, its context, processes, and mechanisms. Case studies are best used as a complementary method to other research methods.
ADVANTAGES:
- is a source of comprehensive information on a given topic,
- uses different points of view, which gives the description and analysis a wider perspective,
- takes into account the context of the phenomena studied.
DISADVANTAGES:
- usually requires the use of various sources of information, sometimes difficult to access,
- it requires a lot of work and is time-consuming,
- provides incomplete data results with low credibility of the described case.
3.2.6. SURVEYS CONDUCTED BY INTERVIEWERS
Quantitative methods are a standardised measurement method. Standardisation enables the collection and counting of quantitative data in a unified way, and also enables their statistical analysis. Standardisation covers:
- Research tool (interview questionnaire) – the order, content and form of questions put to respondents,
- The manner of recording respondents’ responses by selecting one option (on the scale) or several options from the “cafeteria” (a set of ready answers),
- Behaviour of interviewers (pollsters) who are obliged to follow the instructions contained in the questionnaire during the interview.
Respondents’ opinions are transformed into numbers and saved in the database. Then, this information is analysed using statistical methods.
Questionnaire interviews are conducted by trained pollsters who read the respondents’ questions from the questionnaire and write down the answers that were obtained. There are the following techniques for this type of research:
- Paper and Pencil Interview – (PAPI),
- Computer-Assisted Personal Interview – (CAPI),
- Computer-Aided Telephone Interview (CATI).
3.2.6.1. Paper And Pencil Interview (PAPI) and Computer-Assisted Personal Interview (CAPI)
Both of these techniques are field-based and are implemented in direct contact of the respondent with the pollster using a paper (PAPI) or electronic version of the interview questionnaire displayed on a laptop or tablet (CAPI). The pollsters read out the questions included in the questionnaire and then mark the answers given by the respondent.
CONDITIONS OF APPLICATION: a wide range of topics and a direct (F2F) meeting between the interviewer and the respondent is required. The best place for the interview is a place isolated from noise and the presence of third parties (in home / work conditions, make sure that bystanders, such as family members or colleagues, do not influence the respondents’ answers).
ADVANTAGES:
- personal, close contact with respondents (the possibility to observe non-verbal signals, respond to misunderstanding of the question or tiredness of the respondent),
- greater readiness of respondents for a longer interview and more difficult questions than during CATI,
- with CAPI data is automatically saved during the interview.
DISADVANTAGES:
- higher costs, including time and cost of travel and arranging a personal meeting with the respondent,
- lack of a sense of anonymity of the respondent,
- uncontrolled influence of the pollsters on the respondent’s answers (the interviewer’s effect*)
- With PAPI the interviewer must manually enter the data from the questionnaire into the database after the interview, which is time-consuming, adds costs, and involves the risk of mistakes.
* This is the influence that the interviewer exerts on the respondent during the survey. The respondent unconsciously interprets the interviewer’s social characteristics (e.g. gender, age), assuming what is expected of him/her. The interviewer may also unknowingly send signals to the respondent suggesting the “right” answers.
3.2.6.2. Computer-Assisted Telephone Interview (CATI)
This type of interview is carried out by phone. The interviewer reads the questions displayed on the computer screen, and after receiving the answers marks them in the electronic questionnaire on his/her computer.
CONDITIONS OF APPLICATION: studying established opinions and attitudes, with the use of questions that do not require longer reflection due to the short duration of this interview (max. 10-15 minutes), as well as a specific channel of transmission and reception of information (no possibility of reading it several times at own pace).
ADVANTAGES:
- shorter time and lower cost of reaching the respondent compared to face-to-face interviews (PAPI, CAPI),
- time flexibility (the possibility to adjust the interview time to the respondent’s preferences, to stop the interview and continue it at a convenient time for the respondent),
- easy management and control of pollsters’ work,
- automatic saving (coding) of data during the interview.
DISADVANTAGES:
- possible difficulty in obtaining respondents’ phone numbers (due to the lack of access and / or protection of personal data), and in the case of employers, no personalised contacts (having only the reception / headquarters phone numbers),
- interview time limited to 10-15 minutes (due to shaky concentration and short duration of the respondents’ involvement),
- the tendency of the respondents to choose extreme answers, or the beginning and end points on the scale (resulting from a specific channel of information transfer which enhances the ‘priority effect’ and the ‘freshness effect’).
3.2.7. SELF-ADMINISTERED SURVEYS
In self-administered surveys, the respondents read and mark the answers in the questionnaire on their own (without the pollsters’ participation).
CONDITIONS OF APPLICATION: these surveys can be carried out as a paper or online questionnaire (i.e. Computer-Assisted Web Interview – CAWI). In the case of the latter, respondents receive a link to the website with the questionnaire which they can complete on a computer, tablet or smartphone. After answering, the data is sent to the server where it is automatically saved in the database.
A very effective method of collecting quantitative data is a central location, which relies on questionnaires being filled in by people who are at the same time in one room, e.g. after completion of a training, workshop or conference. It is necessary to ensure that the respondents fill in the questionnaires themselves (without support from other people).
ADVANTAGES:
- short time it takes to obtain information (especially in the case of a central location),
- lower cost compared to questionnaire interviews conducted by pollsters,
- sense of anonymity in people completing the survey,
- no interviewer’s effect .
DISADVANTAGES:
- respondents’ motivation to complete the questionnaire may decrease with no interviewer presence,
- lack of control over the process of completing the survey*,
- risk of consulting responses with other people**.
PRACTICAL TIP
The survey questionnaire must:
- be short, easy, visually attractive to encourage a response,
- have all necessary explanations, which in other methods are given by the interviewer,
- have clear instructions (paper version) or algorithms (electronic) leading the respondent to the relevant questions (based on previous answers, irrelevant questions are filtered and omitted).
|
Questionnaire For Training Participants
* Instead of the right respondent, the survey may be completed by another person, which disrupts the representativeness of the sample.
** Especially in the case of a central location conducted without the researcher’s supervision.
3.2.8. ACTIVE / WORKSHOP METHODS OF GROUP WORK WITH YOUNG PEOPLE
Below we present additional active methods of collecting data (mainly qualitative), which can be particularly useful in group work with young people, because these methods are engaging, they integrate the team, facilitate cooperation and support the development of soft skills.
Active methods are workshop methods of collecting information that can complement the “classic” methods of evaluation research. They allow you to get quick feedback on a particular action, learn about the ratings, feelings and impressions of the participants as well as develop recommendations. These methods are worth using during workshops, training or conferences, in order to make the meeting more attractive, get to know the participants and better adapt the project activities to their needs.
ADVANTAGES:
- speed – you receive instant feedback during the classes / meetings,
- casual atmosphere,
- the projective nature of tasks / questions makes it easier to formulate critical opinions and propose new solutions,
- possibility to jointly collect qualitative and quantitative data,
- stimulating self-reflection,
- a positive impact on the well-being of participants (satisfying the need for expression, acceptance, integration).
DISADVANTAGES:
- you cannot generalise the obtained opinions to a wider community (not participating in the meeting),
- the need for an experienced trainer / moderator to moderate / facilitate,
- the lack of anonymity of the participants in the case of group reporting and discussion (threat to mental well-being and group relations for people who are particularly vulnerable or have a weak position in the group).
Below you can find examples of active methods implemented in the form of a workshop.
CLOTHESLINE
The purpose of this tool is to get to know the expectations of the project audience. It is a visual method of collecting qualitative data.
Each participant receives drawings with clothes (e.g. shirt, underwear, trousers, socks), which symbolise the type of expectations they have towards the project – they may be, for example, hopes, fears, needs, suggestions, etc. Participants are given sufficient time to reflect and complete individual drawings / garments. After writing down their ideas, each of them “hangs their clothes” on a string hung or drawn in the room. Participants can read their expectations aloud and look at others’ “laundry”.
TELEGRAM
This tool allows you to quickly summarise part of the meeting (workshop, training) to learn about the mood in the group.
The participants are asked to think about a particular fragment of the classes and describe their reflections with three words: positive, negative and summative (e.g. intense – tiredness – satisfaction). Each person reads their words, which allows for a joint summary of the activities (you can write them down on post-its and stick them on a flipchart, etc.).
HANDS
The purpose of this tool is to find out opinions on selected aspects of the project or part of it (e.g. training, internship), as well as to summarise the course and effects of the classes. People participating in the workshop receive sheets of paper on which to draw their hands. Each of the fingers is assigned one assessment category, e.g.:
- On the thumb – what was the strongest / best side of the training / project,
- On the index finger – what I will tell my friends about,
- On the middle finger – what was the weakest point of the training / project,
- On the ring finger – what I would like to change (element needing improvement),
- On the little finger – what I have learned or found out.
Participants enter their opinions on each of the fingers in accordance with the above categories. The exercise can be used to find out about the opinions of individuals and / or for group discussion.
EVALUATION ROSE
This method is used to gather feedback on many aspects of a project / activity at the same time. It is a visual method that allows you to collect quantitative data – assessments of various aspects of the assessed object using a joint scale.
Participants receive cards with an “evaluation rose” drawn. The drawing is inspired by the “wind rose” – instead of the directions of the world, it presents various aspects of the evaluated object (e.g. the usefulness of the training, how attractive the method of conveying the content is, the appropriate amount of time spent on training). Divide the axes into sections and assign to them selected values (e.g. scale 1-5, where 1 is the weakest grade and 5 – the best). Participants are asked to indicate their views on each axis of the “evaluation rose”. Then you can combine the points and get a visually attractive picture of your opinions (the final effect resembles a radar chart).
TALKING WALL
The purpose of this method is to gather opinions on the value of a particular project activity or the entire project. Thanks to its application, you can obtain qualitative data (types of opinions) and quantitative data (how many people share a particular opinion).
Hang five large sheets of paper on the wall. On each of them, put a question about the conducted activities, e.g.:
- Sheet 1: What new things did you learn during the training?
- Sheet 2: How will you use the knowledge acquired during the training?
- Sheet 3: What did you like the most about the training?
- Sheet 4: What did you like least about the training?
- Sheet 5: What would you change in this training?
Participants write down their answers on each sheet or – if the opinion is already on them – add a plus / dot next to it. At the end, the facilitator summarises the entries and encourages the group to discuss them and develop their recommendations. This form of collecting opinions encourages more openness, participants gain a sense of agency and overcome reluctance to speaking in public.
RUBBISH BIN AND SUITCASE
With this method, you can get a summary of training or other project activity. It allows you to collect information on elements that were useful, redundant or considered missing for the participants.
Draw a suitcase, rubbish bin and sack on the blackboard / flipchart. Each of the figures symbolises one category of opinion about the evaluated activity:
- Suitcase: “What do I take with me from the training?” (what will be useful to me, what will I use in the future)
- Rubbish bin: “What was unnecessary during the training?” (what is not useful to me, what was redundant),
- Sack: “What was missing?” (what should be added to the next training).
Then you can ask the participants to speak or write down their opinions on sticky notes or directly on the pictures on a flipchart.
PRACTICAL TIPS FOR CONDUCTING GROUP ACTIVITIES
It is good for the participants to sit in a circle so that everyone can see each other. To increase their involvement, you can propose that they themselves indicate the next person to talk, e.g. by throwing a ball (this solution can be used provided that no one in the group is discriminated against). Oral statements should be noted down – this can be done by the person conducting the classes while they are taking place (e.g. on the blackboard, flipchart) or by their assistant. |
3.3. How to choose appropriate research methods
Research methods must fit well with the evaluation concept and plan. To make the right choice, consider whether the methods are relevant to:
- The purpose, subject, scope and type of evaluation, as well as the criteria and evaluation questions – will these methods provide you with the information necessary to answer your evaluation questions?
- The data sources from which you plan to obtain information – will it be appropriate to provide information on the groups that will take part in the evaluation research?
- The characteristic of the interviewees / respondents – do the methods take into account group size, their perceptive capabilities, communication abilities, health condition, etc.?
- The circumstances of the data collection – will all the necessary data and interviewees / respondents be available at a particular moment? Will the chosen method suit the place of data collection?
- The resources you have access to? – does the method require availability of qualified or independent researchers and other resources (organisational, technical, financial and time)? Will you be able to use the method on your own? Do your resources make you able to use it?
Knowledge of research methods (quantitative and qualitative) and related tools will help in preparing the second part of the evaluation concept (see chapter 2.4, tool 4), which will be supplemented with methodological issues. This element enables you to gather information to answer evaluation questions.
Tool 6: Logic Matrix Of The Evaluation Research
3.4. How to design research tools
A common mistake is to start an evaluation by creating research tools, e.g. a questionnaire for project recipients. You must remember that you will not be able to choose the right research methods or prepare the right measurement tools (e.g. scenarios, questionnaires, observation sheets) in isolation / detached from the overall concept of evaluation. Therefore, start constructing research tools after determining:
-
- The subject, scope and purpose of the evaluation,
- Evaluation criteria and questions,
- Studied groups of people and research methods.
Without referring to the above elements, you are not able to create correct research tools, because you may include questions that are unrelated to the purpose of the research, making it impossible to answer evaluation questions and respond to evaluation criteria. “Bad” tools contain useless questions, are overloaded or incomplete, do not provide relevant information and do not allow for the formulation of meaningful recommendations.
The questions included in the research tools are a particularisation of the evaluation questions. Remember that these questions evaluators ask themselves, not the respondents! These two types of questions should not be confused as they are formulated in languages adjusted to the needs of:
- Evaluators / evaluation stakeholders → evaluation questions,
- Studied groups of persons (interviewees, respondents)→ questions in research tools.
If you are not sure whether a particular question should be put to the interviewees / respondents, consider whether they will be able to answer it, and the information obtained will allow you to answer the evaluation questions and formulate useful recommendations.
HOW TO ASK QUESTIONS
- The number of questions included in the tools should be appropriate to the purpose and duration of the research.
- Research tools should have a transparent structure, with the main issues identified (e.g. “reasons for joining the project”, “assessment of different types of support”, “effects of participation in the project”). Topics should be grouped thematically (e.g. organisational issues).
- Questions should be asked in a specific order. Put preliminary questions (relatively easy) at the beginning of your tool. They should be followed by introductory questions in the subject (not very difficult), then main questions (key for the purpose of the research). Put the most difficult questions in the middle of the tool. Finally, ask summary and closing questions.
- Questions should be asked in a logical order that cannot surprise or confuse the research participants. Each question should follow on from the previous one or – in the case of an interview – refer to the respondent’s statements.
- The language of an interview should be easy to understand: use as short sentences as possible, use a language close to the research participants – without foreign words, specialised terminology, jargon, abbreviations.
- Questions should be formulated precisely – e.g. there should be no doubt what period of time they relate to (don’t ask “whether recently …”, but “whether in the last week / month / year …”)
- Do not ask for several issues in one question (“what are the strengths and weaknesses of the project?”) and do not use negative questions (“shouldn’t you …”, “don’t you prefer …”). Each of these errors makes it difficult to understand the questions and interpret the answers.
- Questions and proposed answers must not be sensitive to the research participants – they cannot lead to the disclosure of traumatic experiences, declaration of behaviour or beliefs contrary to the law or morality. When anonymity is not guaranteed, do not ask about property status, family matters or health issues.
- Do not ask questions suggesting an answer – do not present any of the options as being in accordance with the rule of law or morality, do not refer to the authorities or the opinion of the majority.

The differences between quantitative and qualitative research tools, the structure / construction of scenarios and questionnaires and the most common mistakes in their design are discussed in the online course.