INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION

Présentation au sujet: "INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION"— Transcription de la présentation:

1 INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION

2 Interventions psychosociales
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Axe de travail de cette session d'atelier Interventions psychosociales Suivi Évaluation Qu'est-ce que les indicateurs ? Pourquoi sont-ils importants ? Qui y est associé ? Planification Méthodes Indicateurs In the first part of this workshop session we will be looking at Monitoring and Evaluation in psychosocial interventions. We will look at the two activities separately, exploring what they are, why they are important and who is involved in these activities. Then we will discuss planning of monitoring and evaluation activities, and the different methods that can be used for psychosocial interventions. Throughout the workshop session we will also talk about indicators, which the measures that are used to enable monitoring and evaluation.

3 INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION
Qu'est-ce que le suivi ? Notez un mot ou une phrase que vous associez au terme « suivi » sur les cartes que vous avez reçues. Lorsque vous aurez terminé, affichez vos cartes sur le tableau/espace mural. Si quelqu'un a écrit le même mot que vous, placez votre carte sur la sienne. 1.Give the participants pieces of card (1/4 of an A4 sheet), markers and sticky putty to display their cards. Ask them what words come to mind then they think of the term ‘monitoring’ and ask them to write these words on their pieces of card, writing one word on each piece of card – for example: ‘control’, ‘quality’, ‘indicators’, ‘outputs’, ‘outcomes'. After a few minutes when they have finished, ask them to place their cards on an assigned board or wall space. If they see someone else has written the same word as them, they should just put the card on top of that card.

4 INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION
Qu'est-ce que le suivi ? Le suivi est constitué du processus régulier et continu de collecte et d'analyse des données permettant d'évaluer les avancées et le développement. 1.Give the participants pieces of card (1/4 of an A4 sheet), markers and sticky putty to display their cards. Ask them what words come to mind then they think of the term ‘monitoring’ and ask them to write these words on their pieces of card, writing one word on each piece of card – for example: ‘checking’, or ‘control’, or ‘quality’. After a few minutes when they have finished, ask them to place their cards on an assigned board or wall space. If they see someone else has written the same word as them, they should just put the card on top of that card.

5 Axés sur le processus Deux principaux types de suivi - CRCR
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Deux principaux types de suivi - CRCR Axés sur le processus Avancées et développement de réponse Activités mises en œuvre comme prévu ? Utilisation des ressources Problèmes ? Comment doivent-ils être abordés ? Nouvelles opportunités d'affiner la réponse ? There are two main types of monitoring activities. The first is on the process of the response – looking at the progress and development of the response. This explores whether activities are being implemented as planned And whether resources are being used in the way they were supposed to – and if not, why not? And at whether any problems have developed in terms of implementation, and if so, how can these problems best be dealt with This type of monitoring also looks at whether new opportunities have arisen to improve the progress of the response.

6 INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION
Les deux principaux types de suivi au sein des CRCR Axés sur les résultats Résultats directs des interventions Pertinence des activités en cours et des activités planifiées Les objectifs sont-ils encore réalistes/pertinents ? Changements dans la population cible / l'environnement (impact) Faut-il plus d'informations ? There are two main types of monitoring activities. 2. The second is results oriented and measures direct results of the implemented activities, such as whether activities, present and planned, are relevant to the needs of the population. - Are the objectives of the response still realistic and relevant or do they need changing? - Are there changes in the targeted population or the external environment have occurred that affect the planned activities - New information is needed to enhance understanding of the presenting situation - Both types of monitoring are useful and are required for reporting purposes.

7 La réponse est-elle encore adaptée aux besoins de la population ?
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Qu'est-ce que le suivi ? La réponse est-elle encore adaptée aux besoins de la population ? Conserve la trace des contributions, productions et résultats à l'aide d'indicateurs In very simple terms, monitoring helps us to keep a track of whether the response is still relevant to the needs of the population. This is done by keeping track of the inputs, outputs and outcomes of the implemented interventions by using indicators.

8 Que sont les contributions, les productions et les résultats ?
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Que sont les contributions, les productions et les résultats ? Production : réussites mesurables qui ressortent ou qui découlent de l'intervention Ex. : nombre de personnes formées, nombre de réunions tenues, etc. Contributions Toute ressource qui alimente le programme Ex. : fonds, matériels, personnel, temps Résultats : Changements survenus du fait du programme Ex. : amélioration du bien-être psychosocial, des aptitudes et des connaissances liées au SPS, Ask the participants what these terms mean, one by one Use the explanations given to summarize or add to the responses of the participants – one by one. (for e.g. ask them what inputs are – when they have given their responses, CLICK and finish discussing inputs before asking what Outputs) Each arrow comes with a CLICK

9 Qu'est-ce que les indicateurs ?
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION INDICATEURS Réflexion sur : Qu'est-ce que les indicateurs ? 1. Ask the participants what indicators are? Ask them to just say whatever they think they are, and write their explanations as simply as possible in a flip chart or a board. Summarise with the definition on the next slide.

10 Qu'est-ce que les indicateurs ?
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Qu'est-ce que les indicateurs ? Critères utilisés pour mesurer les données recueillies Mesurer les changements associés à la mise en œuvre Exemples d'indicateurs (de résultats) pour les réponses PS : Diminution des symptômes liés au stress chez la population A Augmentation des connaissances et aptitudes dans la délivrance de PSP Les enfants retrouvent l'envie de jouer Les bénéficiaires retrouvent la capacité d'avoir des relations avec autrui In summary of the explanations of indicators given by the participants, explain: Indicators are criteria that are used to measure the data collected. They measure the changes that have been caused by the implementation of the intervention In other words: indicators are what show us if anything has changed. We choose specific indicators for our programs, depending on what we hope will change. Below are some examples of indicators that have been used in psychosocial responses - as you can see, these measure will INDICATE that the different activities were successful, and helped to achieve the overall aim or goal of the program.

11 INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION
Exemples d'indicateurs de contributions, de productions et de résultats Indicateurs de productions : NOMBRE de personnes formées ; NOMBRE de réunions tenues ; NOMBRE de kits PS distribués Contributions Indicateurs : MONTANT EUR/$ dépensé ; NOMBRE de membres du personnel actifs ; NOMBRE d'heures Indicateurs de résultats : NOMBRE d'enfants qui jouent (quantitatif) ; ACCROISSEMENT auto-déclaré du désir d'interaction sociale (qualitatif) ; NIVEAU d'amélioration des aptitudes de SPS Here are some examples of input, output and outcome indicators. Read the indicators in each category before CLIKCING to the next arrow. As you can see, the indicators are the measureable criteria, such as numbers, amounts, levels – and they are directly related to a particular activity that is part of the psychosocial response. Explain we will talk more about indicators later.

12 Conserver la trace des contributions, productions et résultats des activités - Un exemple
Approche de cadre logique Indicateurs But général Diminuer la souffrance et le risque de développement de traumatismes sévères au sein de la Population A Adultes et enfants montrent des signes sains d'adaptation à l'impact de la crise Activité 1 Formation en PSP Contributions Ressources permettant la formation Quantité d'argent ; personnel ; manuels de formation ; matériels Production La formation a eu lieu Nombre de personnes formées Résultats Les bénévoles sont capables d'administrer les PSP Le niveau de compétences en PSP a augmenté Activité 2 : Ateliers avec des enfants Ressources pour les ateliers Productions Réalisation des ateliers Nombre d'ateliers réalisés ; nombre d'enfants concernés ; nombre de formateurs Les enfants s'adaptent mieux Enfants plus enjoués, plus grande confiance en soi et en les autres Here is an example of two activities in a psychosocial intervention, presented in an LFA table – a Logical Framework Approach table. This is a simple and effective way of keeping track of inputs, outputs and outcomes for activities. At the top, you can see what the overall goal of the program is – and on the right there is the indicator that will show if the program has achieved it’s overall goal. Then, for each of the activities, there are examples of what the expected inputs, outputs and outcomes will be – and on the left, the indicators that will be used to measure whether or not the activities were successful. The LFA is a very common monitoring tool – and many organizations have standard LFA tables that have to be filled out for all sectoral responses. This enables the central management team to monitor all activities and programs across sectors.

13 Pourquoi le suivi est-il important ?
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Pourquoi le suivi est-il important ? Pertinence Responsabilité et communication Permet de rendre des comptes Ensuring that an intervention is relevant to the needs of the population is an obvious indication of how important it is to monitor a response. Ask the participants: for what other reasons is monitoring important? When they have shared some of their answers – add the following two reasons with explanations: It is very common that psychosocial responses are implemented far away from the headquarters of the National RCRC Society. Monitoring reports therefore become one of the most important communication tools between the field, where the response is implemented, and the headquarter offices, that have the ultimate responsibility for the planning and implementation of the response. This is one of the reasons that staff and volunteers have to be trained well in monitoring activities, and the development of the monitoring tools has to be done carefully to include all possible data that could be needed. A third important reason for doing monitoring is that of accountability. The Red Cross or Red Crescent Society involved in the response are first and foremost accountable to the affected population they are assisting. Continuous monitoring enables the RCRC Society to regularly change and adapt the program, and in this way make sure they are helping the population in the best way possible. The National Society is also accountable to the donors that support the program – both in terms of fulfilling their expected aims, and in terms of financial spending etc. Simply put – monitoring helps us to assess if we are on track, in terms of what we planned to do – and more importantly – are we on the right track? Plus simplement : Sommes-nous sur la voie ? Sommes-nous sur la BONNE voie ?

14 Qui est associé au suivi ?
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Qui est associé au suivi ? Responsabilité Y compris suivi et évaluation Autres partenaires Basically, everyone who is involved in a psychosocial response, is also involved in monitoring the progress and development of the response. CLICK: The managing programme staff at national and regional level have the ultimate responsibility to make sure that there are both time and resources allocated to enable successful monitoring activities. They are the supervise the entire programme planning and implementation, which includes the monitoring and evaluation activities. CLICK CLICK: Next,, the local programme staff, who are working at the field level, - together with the community leaders or cluster leaders - have the responsibility for practical arrangements and also for exploring and deciding what the most appropriate methods of data collection would be with the various informants. CLICK 2 times: The actual data collection is usually done by community facilitators and volunteers. Since volunteers are often from the affected population themselves, they are very well placed to collect data in culturally appropriate and respectful ways. It is naturally the affected population who are the key informants in a psychosocial response, as they are the ones who provide the data needed. CLICK: A last group of important participants are other partners that may be contributing to or collaborating in the psychosocial response. These can be internal partners from other sectors of the National Society, or external partners who either also provide psychosocial support, or attend to other needs of the population. Collecte de données

15 Qu’est-ce que les évaluations ?
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Qu’est-ce que les évaluations ? En groupes de réflexion de 2 ou 3, discutez de ce que sont les évaluations et de ce en quoi elles diffèrent du suivi. Essayez de rédiger une définition simple et précise du rôle de l'évaluation dans la réponse psychosociale. Give the participants a few minutes, in buzz groups, to discuss “what evaluations are and how they differ from monitoring?”. When they are finished, move on to the next slide.

16 Que sont les évaluations ?
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Que sont les évaluations ? « Les évaluations permettent d'établir si les interventions ont atteint les visées et buts généraux de la réponse psychosociale. » Points de débat : Les évaluations sont censées être OBJECTIVES – Qu'est-ce que cela signifie et pourquoi est-ce important ? Show the participants the above definition of an evaluation of a psychosocial intervention. Ask them if they agree with this definition, or if they have anything to add or feel should be taken out or rephrased. Work with the participants to adjust the definition until everyone agrees. Use the points below if there are no comments, to encourage discussion: Important to state it is an OBJECTIVE assessment. Discuss what this means and why it is important.

17 Évaluation intermédiaire
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Que sont les évaluations ? Évaluations = Suivi + davantage Évaluation finale Évaluation intermédiaire M1 M2 M3 M4 M5 M6 M7 M8 In simple terms, evaluations are similar to monitoring, only more detailed and advanced in terms of analysis and reporting. This means they are also more costly and time-consuming. Evaluations often consider both the raw data and the information developed from the regular monitoring activities, together with more data and generated information from additional data collection activities. The diagram shows that the mid-term evaluation will consider the data and information from the monitoring that has taken place so far – and additional; whilst the final evaluation will consider all the monitoring data and information, and that from the mid-term evaluation, and if needed, additional data gathered.

18 Évaluations axées sur l'estimation du BUT/VISÉE GÉNÉRAL(E)
Approche de cadre logique Indicateurs But général Diminuer la souffrance et le risque de développement de traumatismes sévères dans la Population A Adultes et enfants montrent des signes sains d'adaptation à l'impact de la crise Activité 1 Formation en PSP Contributions Ressources permettant la formation Quantité d'argent ; personnel ; manuels de formation ; matériels Productions La formation a eu lieu Nombre de personnes formées Résultats Les bénévoles sont capables d'administrer les PSP Le niveau de compétences en PSP a augmenté Activité 2 : Ateliers avec des enfants Ressources pour les ateliers Réalisation des ateliers Nombre d'ateliers réalisés ; nombre d'enfants concernés ; nombre de formateurs Les enfants s'adaptent mieux Enfants plus enjoués, plus grande confiance en soi et en les autres CLICK: Evaluations are thus particularly interested in assessing if the OVERALL AIM or GOAL of the response was achieved CLICK: However, assessing this ALSO involves a full consideration and analysis of the monitoring results of each activity and smaller component of the implemented response.

19 Types d'évaluations Évaluations en temps réel Précocité de la réponse
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Types d'évaluations Évaluations en temps réel Précocité de la réponse Personnel interne Axe : questions de processus / d'opérations Évaluations intermédiaires Évaluer l'impact Rendre des comptes (budgets/admin) Adaptations nécessitant un guide Évaluations finales Impact et processus Réussites et difficultés principales Informations précieuses pour les prochaines interventions PS There are three main types of evaluations. The first are real time evaluations. These are usually done very early in a response to see if the responders are ‘on the right track’. They are usually very short in length (time taken) and typically focus on organizational aspects of the response, such as administrative issues (employees, logistics etc); coordination and cooperation with other partners; and any other practical issues that may influence the success of the planned intervention. They are done in ‘real time’ meaning that they provide immediate feedback to the organization, and opportunities for immediate action and needed adaptation. The second are – as mentioned, the mid-term evaluations. These are also called reviews – and they are not always done, but highly recommended. They are a good way of consolidating all the information generated by the monitoring activities to date, and help to give and overview of the administrative progress and the impact of the intervention. The final evaluation is, for most, very important. It is usually the results of the final evaluation that are shared with all stakeholders involved in the intervention, including the donors. The final evaluation not only reports what has taken place, and whether the intervention was a success, but it also includes an analysis of the challenges and gives recommendations either to future activities with the targeted population, or in future psychosocial responses with other populations.

20 Évaluer l'impact Comment mesurez-vous l'impact d'une intervention ?
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Évaluer l'impact Comment mesurez-vous l'impact d'une intervention ? De quoi avez-vous besoin ? To measure the impact of a psychosocial intervention you will need data, from the beginning of an intervention, during an intervention, and at the end of the intervention. This is true whether it is for a short term immediate intervention (CLICK) or a long term intervention (CLICK). It is very important that the nature of the data is well planned – so that it measures the same indicators from beginning to end of the intervention. (CLICK 2 TIMES) This will enable comparison of results, which in turn will show whether any change took place. Données Données MÊMES INDICATEURS

21 Comment définir les indicateurs psychosociaux
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Comment définir les indicateurs psychosociaux Évaluer - Comment la population est-elle touchée ? Quel est le type de changement désiré ? (Buts/objectifs/activités de réponse) Quelle est la définition locale du bien-être psychosocial ? Choosing the right indicators is a very important task, since the indicators are what are going to show whether or not the psychosocial intervention is working. It often takes a lot of consultation, data collection, analysis and time to choose the right indicators – but this should not deter those working with psychosocial interventions. There are some pretty simple steps to follow. CLICK: First, the results from the assessments that show how the population has been affected, will help to guide what kind of change is desired. CLICK: This is the first step in determining indicators. – finding out what change you want to measure CLICK: In programmatic terms, this is measuring the goals, objectives and activities of the response. The most difficult indicators to define are usually the most important ones – the ones that show whether or not the populations’ psychosocial wellbeing has improved. CLICK: To choose the right indicators for this one first needs to know what the local definition of psychosocial wellbeing is.

22 Indicateurs du bien-être psychosocial
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Indicateurs du bien-être psychosocial Ask the participants:" On a piece of paper in front of you – write three things that about yourself or your life that you think show to others that you are feeling good and doing well. Now think about how others could measure whether you felt better from one day to the next. Write these down”. Now ask the participants to share some of their indicators of wellbeing. There should be a good variety of indicators, and also some indicators that may be the same for the whole group. Use this discussion to highlight that different people may have different things that indicate their psychosocial wellbeing, but if you spent some time with the group, you would probably be able to find some common indicators that everyone would agree would be both general, and specific, enough to determine the groups’ psychosocial wellbeing. Explain: Indicators of psychosocial wellbeing focus both on how the individuals are doing, and on how the group is doing. This is the nature of psychosocial – that there is a focus on the individual’s psyche and on the social interaction and network.

23 Indicateurs psychosociaux
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Indicateurs psychosociaux 1. Écrivez trois choses sur vous-même ou sur votre vie qui, selon vous, montrent aux autres que vous vous sentez bien et que vous allez bien. 2. Maintenant réfléchissez et écrivez ce qui, selon vous, pourrait permettre aux autres de mesurer si vous vous sentez mieux de jour en jour. 3. En plénière, partagez quelques-uns de vos indicateurs de bien-être. Nous cherchons à obtenir des indicateurs très divers, mais aussi certains indicateurs qui soient valides pour l'ensemble du groupe. Choosing the right indicators is a very important task, since the indicators are what are going to show whether or not the psychosocial intervention is working. It often takes a lot of consultation, data collection, analysis and time to choose the right indicators – but this should not deter those working with psychosocial interventions. There are some pretty simple steps to follow. CLICK: First, the results from the assessments that show how the population has been affected, will help to guide what kind of change is desired. CLICK: This is the first step in determining indicators. – finding out what change you want to measure CLICK: In programmatic terms, this is measuring the goals, objectives and activities of the response. The most difficult indicators to define are usually the most important ones – the ones that show whether or not the populations’ psychosocial wellbeing has improved. CLICK: To choose the right indicators for this one first needs to know what the local definition of psychosocial wellbeing is. 23

24 Indicateurs psychosociaux
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Indicateurs psychosociaux Les éléments qui témoignent du bien-être psychosocial des gens sont différents, toutefois, en approfondissant un peu, nous déterminerons certainement des indicateurs courants à la fois généraux et spécifiques, assez en tout cas pour déterminer le bien-être psychosocial du groupe. Les indicateurs du bien-être psychosocial sont axés sur : Comment vont les personnes Comment va le groupe. Le psychosocial s'intéresse, par nature, au psychisme de l'individu, à l'interaction et aux réseaux sociaux. Choosing the right indicators is a very important task, since the indicators are what are going to show whether or not the psychosocial intervention is working. It often takes a lot of consultation, data collection, analysis and time to choose the right indicators – but this should not deter those working with psychosocial interventions. There are some pretty simple steps to follow. CLICK: First, the results from the assessments that show how the population has been affected, will help to guide what kind of change is desired. CLICK: This is the first step in determining indicators. – finding out what change you want to measure CLICK: In programmatic terms, this is measuring the goals, objectives and activities of the response. The most difficult indicators to define are usually the most important ones – the ones that show whether or not the populations’ psychosocial wellbeing has improved. CLICK: To choose the right indicators for this one first needs to know what the local definition of psychosocial wellbeing is. 24

25 Indicateurs psychosociaux
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Indicateurs psychosociaux JE SAIS... Demandons-leur comment ILS savent qu'ils se sentent mieux, comme ça nous pourrons trouver des moyens de mesurer cette amélioration… Nous voulons que les enfants se sentent mieux…. Mais comment SAURONS-nous s'ils se sentent mieux ? 1. Here is a very simple explanation of what needs to be done to define indicators for psychosocial wellbeing.

26 Pourquoi l'évaluation est-elle importante ?
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Pourquoi l'évaluation est-elle importante ? Pertinence (réponse pertinente aux besoins de la pop.) Efficacité (calendrier, coût) Impact (réussi ou non) Efficience (objectif du programme atteint) Durabilité (avantages qui se prolongent au-delà du programme) Like monitoring, evaluations also help to assess if interventions are relevant to the needs of the population – and to the needs of the planned response (real time evaluations). Mid-term evaluations, in particular, are often used to guide adaptations to the interventions, if the results of the evaluation indicate that the population needs have changed or are simply not being met. Final evaluations can help to highlight if at all the response succeeded in meeting the needs, in retrospect – but they do not play a part in guiding adaptations for activities in the response. However, recommendations are usually made for future responses – either to the same population or to others. When exploring efficiency, evaluations look at whether the psychosocial response activities are or were implemented succesfully in terms of intended time-frame and cost. As mentioned, evaluations are very important tools in assessing the impact of the response. They explore, in detail, what has changed as a result of the psychosocial response, and if the desired outcome of improved psychosocial wellbeing has been achieved. Evaluations also assess whether the objectives set for the program were met – and in this way determine if the program was effective. Finally, by not only looking backwards, but also focusing on future needs of the population, evaluations explore whether the benefits of the program are likely to continue beyound the life of the program. This is especially important for psychosocial support interventions, as a common goal in most psychosocial responses is helping the affected population become strong enough to take care of themselves. Evaluations help to assess this.

27 INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION
Qui est associé aux évaluations outre le personnel, les bénévoles, les partenaires ? Responsabilité Consultants extérieurs Y compris suivi et évaluation Autres partenaires Evaluations involve the same people as are involved in monitoring, as discussed earlier. However, it is very common that external consultants (CLIKC) are hired to conduct evaluations. Ask the participants why they think this is a good practice? Explain: External consultants are usually more objective and able to provide honest and constructive feedback, without the fear of losing their jobs or upsetting anyone, if, for example, they report inefficiency of administrative staff, or other issues that may be taken personally. Hiring external consultants is often quite pricey, and this should be budgeted for in all psychosocial responses. It is definitely a best practice to hire local external consultants – who have a solid understanding and knowledge of the culture and history of the affected population. If this is not possible, then an international external consultant should definitely have opportunities to consult closely with local representation to develop some of this understanding. Collecte de données

28 Échantillonnage Population ciblée Large & varié Large & varié
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Échantillonnage Population ciblée Large & varié CLICK: In the immediate aftermath of a disaster or before starting a psychosocial response to an ongoing crisis, it is important to obtain data from a sample that is as broad and varied as possible. This will enable a holistic assessment and understanding of how the different subpopulations have been impacted, allow identification of vulnerable groups; and help to identify resources outside the affected population that may be able to assist in the psychosocial response. CLICK: However, when the target groups for the psychosocial intervention have been chosen, the sampling for the population who provide data for monitoring especially, becomes more narrow. Focus is now on collecting data from the targeted population and from those who work directly with them The exception may be for the mid-term evaluation, where, like in the final evaluation (CLICK) – the sampling again INCLUDES information from a broad and varied group – AS WELL as from the targeted population who provided data throughout the program. Large & varié

29 Quand réaliser un suivi et quand évaluer
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Quand réaliser un suivi et quand évaluer Monitoring schedules will vary according to whether a program is short term or long term, and also according to the program design. In a short term immediate response, monitoring may be done on a daily basis, and then later on a weekly basis. In a long term programme, monitoring may be done weekly in the beginning, and then later be extended to monthly, or even three-monthly. It may be that different aspects of the psychosocial response will need monitoring at different intervals or times. For example, logistic inputs and outputs may need monthly monitoring, whilst it may be decided that impact on psychosocial wellbeing should only be assessed bi-monthly. What is important is that monitoring is carefully planned as a regular, ongoing and structured series of activities. Monitoring activities should be budgeted for, both in terms of the time the activities take, and the resources – human and financial. Evaluations are not on-going activities like monitoring, but are usually scheduled to take place around the mid-term mark of the program life, and at the end. As mentioned earlier, real time evaluations take place in the very early stages of a response, and are usually scheduled very quickly in response to a need for them. They are typically multi-sectoral and do not focus, for example, only on psychosocial support or any other sector.

30 Deux catégories principales de méthodes de collecte de données
INTERVENTIONS PSYCHOSOCIALES ÉVALUATION Comment réaliser une évaluation ? Deux catégories principales de méthodes de collecte de données Quantitatives Qualitatives Enquêtes, Questionnaires Outils psychométriques (mesurer réactions, comportements, sentiments, à l’aide d’une échelle graduée 1,2, =, %, a : b Entretiens de témoins privilégiés Discussions dirigées de groupe Observations Description terminologique There are two main types of data collection methods, (also explained in chapter and presentation on assessments). The methods used will depend on the nature of the data you want to collect, and what you are going to do with it. Quantitative data is collected through surveys or questionnaires or other tools, that have scaled measures. Such data is usually reported in numeric terms – as counts, ratios or percentages. Sometimes psychometric tools are also used in psychosocial interventions. These are tools that measure people’s reactions, behaviour and feelings, using a scaled measure. They are often self-reporting measures. For example, a psychometric tool may measure signs of depression, by asking a person to indicate from a scale of 1 – 5 how sad they feel, with 1 being ‘not sad at all’ and 5 being ‘so sad I cry all the time’. Whilst quantitative data is usually COUNTED, qualitative data is usually DESCRIBED WITH WORDS, and is collected through TALKING and LOOKING. Examples are focus group discussions, key informant interviews or behavioral observations. It is common to use both quantitative and qualitative data collection methods, because most monitoring and evaluation activities need both kind of data to develop the needed information.

31 Faut-il des données pour ces indicateurs quantitatifs ou qualitatifs ?
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Faut-il des données pour ces indicateurs quantitatifs ou qualitatifs ? Approche de cadre logique Indicateurs But général Diminuer la souffrance et le risque de développement de traumatismes sévères dans la Population A Adultes et enfants montrent des signes sains d'adaptation à l'impact de la crise Activité 1 Formation en PSP Contributions Ressources permettant la formation Quantité d'argent ; personnel ; manuels de formation ; matériels Productions La formation a eu lieu Nombre de personnes formées Résultats Les bénévoles sont capables d'administrer les PSP Le niveau de compétences en PSP a augmenté Activité 2 : Ateliers avec les enfants Ressources pour les ateliers Réalisation des ateliers Nombre d'ateliers réalisés ; nombre d'enfants concernés ; nombre de formateurs Les enfants s'adaptent mieux Enfants plus enjoués, plus grande confiance en soi et en les autres Ask the participants to go through each of the indicators on this LFA example, and to tell you whether the data needed for these indicators is quantitative or qualitative. See below for the correct answers: Amount of money – quantitative Number of personnel – quantitative Number of training manuals and materials – quantitative Number of people trained: - quantitative Skill level in PFA – could be quantitative or qualitative – discuss why? Skill level could be measured on a self-reporting scale, or it could be observed through, for example, role plays. Number of workshops, children attending, trainers – quantitative Increase in children’s playfulness, self-confidence and trusting of others – again could be quantitative or qualitative – as above, it could be measured on a measured scaled or through observation. These are more likely to be qualitative

32 Principes éthiques en matière de collecte de données
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION La collecte de données Principes éthiques en matière de collecte de données Justifiable et correctement planifiée Coordination Expliquer les visées et les procédures Participative et collaborative PLEASE NOTE THE FOLLOWING TWO SLIDES ARE ALSO INCLUDED IN THE PRESENTATION ON ASSESSMENTS. IF THE PARTICIPANTS HAVE ALSO RECEIVED THIS PRESENTATION YOU DO NOT NEED TO INCLUDE THESE TWO SLIDES IN THIS PRESENTATION, UNLESS FOR REMINDER PURPOSES. There are a number of things one has to be aware of when planning to collect data in assessments. The first is that the data collection is well planned and justifiable. This means there has to be a good reason to collect the data – which there usually is if a disaster has taken place and a population has been hurt or displaced. The data collection activity has to be well planned – the data collectors have to be well trained, and adequate time and resources should be budgeted for the data collection activities. Secondly, it is important to coordinate data collection with others, both in your own organization, and with other organizations and government departments – for example, at coordination meetings. This will prevent the same people from being interviewed over and over again, which can be traumatic for them. It is also less confusing for the affected population. Third, during training of data collectors, it is important that the aims and the ethical and expected procedures are carefully explained and then followed. All data collectors should be able to explain the aims of the data collection they are doing clearly to the people they talk to. Explaining the aims of data collection is connected to this next point, since those who are interviewed should be given the opportunity to understand completely what the data is going to be used for – and then the option of whether they want to contribute with data. This reduces any unrealistic expectations.

33 Principes éthiques en matière de collecte de données
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION La collecte de données Principes éthiques en matière de collecte de données Groupes de comparaison Conduite et consentement Respect de l'intimité et confidentialité Anticiper les conséquences négatives In scientific experiments it is usual that one has a control group, or a comparison group, that was not included in the scientific experiment. This is done to show if the experiment worked on the group that was included. This is NOT ethical and should be avoided when planning psychosocial interventions. It is not ethical to deprive some people of psychosocial support purely for comparison purposes. The only way this sometimes works, is if a psychosocial response is implemented with a time delay in some groups – then one could make comparisons with groups that were waiting to receive support. It is very important that all people who collect data do so in a culturally appropriate and respectful manner. All persons who provide data should also give their informed consent at the beginning of the data collection activity. Whilst giving informers the opportunity to give consent, it should also be explained to them that all the data will be private and confidential, and their names will not be made public or shared with others, except for the purposes of assisting them with psychosocial support. When people have gone through a difficult experience, it may be painful and disturbing for them to talk about it again to others. Data collectors should be trained to deal with this, especially when they are collecting information on psychosocial needs. It is best data collectors expect strong reactions, so they know what to do.

34 Planifier le suivi et l'évaluation
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Planifier le suivi et l'évaluation Travail de groupe : Que faut-il planifier pour garantir la qualité du suivi et de l'évaluation ? Explain: Now that we have talked about what monitoring and evaluation are, how they are done and why they are so important, I want you to consider what needs to be done to plan good monitoring and evaluation. Ask the participants to sit in groups and discuss the given question. Give them flip charts to record their findings and when they have had adequate time ask them to present their group work to plenary and move on to the next slides as a summary.

35 Planifier le suivi et l'évaluation
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Planifier le suivi et l'évaluation Calendrier Durée de la réponse (court terme contre long terme) Moment opportun pour la pop. touchée (ex. ne pas prévoir d'activités de suivi auprès d'enfants scolarisés pendant la période des examens, etc.) Durée de planification, de mise en œuvre, d'analyse, de suivi (allouer temps et ressources) Formation Gestion du SPS Collecteurs de données Équipe du programme – analyse et rapport Two important aspects of planning for good M and E are timing and training. Regarding timing, it is first and foremost important to know and plan according to whether you are going to implement a short term or long term response. This will determine the appropriate intervals for monitoring activities, and planning of evaluations. Secondly, when planning data collection activities, it is of course important to plan these activities when it is convenient for the targeted population. For example, it would not be a good idea to plan data collection with in-school children during exam time, or during their holidays. Third, both monitoring and evaluation take TIME. The amount of time taken to conduct these activities well should NOT be underestimated. Time should be allocated and budgeted for, in terms of financial and human resources. Regarding training: It may be necessary for top management staff to receive some training, or at least, orientation on what psychosocial support is, how it will be provided, and what needs to be done in terms of M and E. The reason for this is that PSS is not a cross-cutting field YET, at some management staff may not yet understand exactly what it is. To enable budgeting for M and E, they need to understand what it is first. Second, data collectors need careful training in data collection methods and ethics. Third, the program team who have responsibility for the M and E activities also need to be well-trained and skilled in data collection, analysis and appropriate reporting.

36 Planifier le suivi et l'évaluation
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Planifier le suivi et l'évaluation Ressources Personnel et bénévoles Finances Temps Logistique Partage de l'information Interne - normes et formats Externe – différents auditoires (bénéficiaires/donateurs/publics) As mentioned on the previous slide, good planning for M and E requires adequate resource allocation. The final important part of planning, is making sure that monitoring tools are well-prepared and carefully filled out – according to the required standards and formats. This will make reporting easier and more successful As results of monitoring and evaluation activities are achieved, decisions also have to be made on how to share the results. There may be a need to present the results in different ways, according to who is going to read them. Donors might, for example, want financial reporting, whereas the beneficiaries and the public may be more interested in results of the impacts of the interventions, and recommendations for the future.

37 Rôle et importance du S&E dans le SPS
INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Rôle et importance du S&E dans le SPS Nous aide à comprendre la signification locale - et pertinente - du bien-être psychosocial Nous maintient sur la bonne voie Nous aide à faire de notre mieux, actuellement et à l'avenir In summary, ask the participants why they think monitoring and evaluation is so important, particularly for psychosocial responses. When they have finished, present this simple summary with the three given points. M and E help to keep us on the right track throughout the program period. They also help us to understand what the local meaning of psychosocial wellbeing is, for the given population. This is very important as we increasingly recognize that psychosocial wellbeing is experienced differently by different people, in different situations. Successful data collection, reporting and information sharing in monitoring and evaluation are also invaluable tools for research in the field of psychosocial support – which is definitely still needed. The results of research, and of every monitoring and evaluation activity, help us to improve psychosocial responses, now and in the future.