2Interventions psychosociales INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONAxe de travail de cette session d'atelierInterventions psychosocialesSuiviÉvaluationQu'est-ce que les indicateurs ? Pourquoi sont-ils importants ? Qui y est associé ? Planification MéthodesIndicateursIn the first part of this workshop session we will be looking at Monitoring and Evaluation in psychosocial interventions.We will look at the two activities separately, exploring what they are, why they are important and who is involved in these activities.Then we will discuss planning of monitoring and evaluation activities, and the different methods that can be used for psychosocial interventions.Throughout the workshop session we will also talk about indicators, which the measures that are used to enable monitoring and evaluation.
3INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Qu'est-ce que le suivi ?Notez un mot ou une phrase que vous associez au terme « suivi » sur les cartes que vous avez reçues. Lorsque vous aurez terminé, affichez vos cartes sur le tableau/espace mural. Si quelqu'un a écrit le même mot que vous, placez votre carte sur la sienne.1.Give the participants pieces of card (1/4 of an A4 sheet), markers and sticky putty to display their cards. Ask them what words come to mind then they think of the term ‘monitoring’ and ask them to write these words on their pieces of card, writing one word on each piece of card – for example: ‘control’, ‘quality’, ‘indicators’, ‘outputs’, ‘outcomes'. After a few minutes when they have finished, ask them to place their cards on an assigned board or wall space. If they see someone else has written the same word as them, they should just put the card on top of that card.
4INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Qu'est-ce que le suivi ?Le suivi est constitué du processus régulier et continu de collecte et d'analyse des données permettant d'évaluer les avancées et le développement.1.Give the participants pieces of card (1/4 of an A4 sheet), markers and sticky putty to display their cards. Ask them what words come to mind then they think of the term ‘monitoring’ and ask them to write these words on their pieces of card, writing one word on each piece of card – for example: ‘checking’, or ‘control’, or ‘quality’. After a few minutes when they have finished, ask them to place their cards on an assigned board or wall space. If they see someone else has written the same word as them, they should just put the card on top of that card.
5Axés sur le processus Deux principaux types de suivi - CRCR INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONDeux principaux types de suivi - CRCRAxés sur le processusAvancées et développement de réponseActivités mises en œuvre comme prévu ?Utilisation des ressourcesProblèmes ? Comment doivent-ils être abordés ?Nouvelles opportunités d'affiner la réponse ?There are two main types of monitoring activities.The first is on the process of the response – looking at the progress and development of the response.This explores whether activities are being implemented as plannedAnd whether resources are being used in the way they were supposed to – and if not, why not?And at whether any problems have developed in terms of implementation, and if so, how can these problems best be dealt withThis type of monitoring also looks at whether new opportunities have arisen to improve the progress of the response.
6INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Les deux principaux types de suivi au sein des CRCRAxés sur les résultatsRésultats directs des interventionsPertinence des activités en cours et des activités planifiéesLes objectifs sont-ils encore réalistes/pertinents ?Changements dans la population cible / l'environnement (impact)Faut-il plus d'informations ?There are two main types of monitoring activities.2. The second is results oriented and measures direct results of the implemented activities, such as whether activities, present and planned, are relevant to the needs of the population.- Are the objectives of the response still realistic and relevant or do they need changing?- Are there changes in the targeted population or the external environment have occurred that affect the planned activities- New information is needed to enhance understanding of the presenting situation- Both types of monitoring are useful and are required for reporting purposes.
7La réponse est-elle encore adaptée aux besoins de la population ? INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONQu'est-ce que le suivi ?La réponse est-elle encore adaptée aux besoins de la population ?Conserve la trace des contributions, productions et résultats à l'aide d'indicateursIn very simple terms, monitoring helps us to keep a track of whether the response is still relevant to the needs of the population.This is done by keeping track of the inputs, outputs and outcomes of the implemented interventions by using indicators.
8Que sont les contributions, les productions et les résultats ? INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONQue sont les contributions, les productions et les résultats ?Production : réussites mesurables qui ressortent ou qui découlent de l'interventionEx. : nombre de personnes formées, nombre de réunions tenues, etc.ContributionsToute ressource qui alimente le programmeEx. : fonds, matériels, personnel, tempsRésultats : Changements survenus du fait du programmeEx. : amélioration du bien-être psychosocial, des aptitudes et des connaissances liées au SPS,Ask the participants what these terms mean, one by oneUse the explanations given to summarize or add to the responses of the participants – one by one.(for e.g. ask them what inputs are – when they have given their responses, CLICK and finish discussing inputs before asking what Outputs) Each arrow comes with a CLICK
9Qu'est-ce que les indicateurs ? INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONINDICATEURSRéflexion sur :Qu'est-ce que les indicateurs ?1. Ask the participants what indicators are? Ask them to just say whatever they think they are, and write their explanations as simply as possible in a flip chart or a board. Summarise with the definition on the next slide.
10Qu'est-ce que les indicateurs ? INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONQu'est-ce que les indicateurs ?Critères utilisés pour mesurer les données recueilliesMesurer les changements associés à la mise en œuvreExemples d'indicateurs (de résultats) pour les réponses PS :Diminution des symptômes liés au stress chez la population AAugmentation des connaissances et aptitudes dans la délivrance de PSPLes enfants retrouvent l'envie de jouerLes bénéficiaires retrouvent la capacité d'avoir des relations avec autruiIn summary of the explanations of indicators given by the participants, explain:Indicators are criteria that are used to measure the data collected.They measure the changes that have been caused by the implementation of the interventionIn other words: indicators are what show us if anything has changed. We choose specific indicators for our programs, depending on what we hope will change.Below are some examples of indicators that have been used in psychosocial responses - as you can see, these measure will INDICATE that the different activities were successful, and helped to achieve the overall aim or goal of the program.
11INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Exemples d'indicateurs de contributions, de productions et de résultatsIndicateurs de productions : NOMBRE de personnes formées ; NOMBRE de réunions tenues ; NOMBRE de kits PS distribuésContributionsIndicateurs :MONTANT EUR/$ dépensé ; NOMBRE de membres du personnel actifs ; NOMBRE d'heuresIndicateurs de résultats : NOMBRE d'enfants qui jouent (quantitatif) ; ACCROISSEMENT auto-déclaré du désir d'interaction sociale (qualitatif) ; NIVEAU d'amélioration des aptitudes de SPSHere are some examples of input, output and outcome indicators. Read the indicators in each category before CLIKCING to the next arrow.As you can see, the indicators are the measureable criteria, such as numbers, amounts, levels – and they are directly related to a particular activity that is part of the psychosocial response.Explain we will talk more about indicators later.
12Conserver la trace des contributions, productions et résultats des activités - Un exemple Approche de cadre logiqueIndicateursBut généralDiminuer la souffrance et le risque de développement de traumatismes sévères au sein de la Population AAdultes et enfants montrent des signes sains d'adaptation à l'impact de la criseActivité 1Formation en PSPContributionsRessources permettant la formationQuantité d'argent ; personnel ; manuels de formation ; matérielsProductionLa formation a eu lieuNombre de personnes forméesRésultatsLes bénévoles sont capables d'administrer les PSPLe niveau de compétences en PSP a augmentéActivité 2 :Ateliers avec des enfantsRessources pour les ateliersProductionsRéalisation des ateliersNombre d'ateliers réalisés ; nombre d'enfants concernés ; nombre de formateursLes enfants s'adaptent mieuxEnfants plus enjoués, plus grande confiance en soi et en les autresHere is an example of two activities in a psychosocial intervention, presented in an LFA table – a Logical Framework Approach table. This is a simple and effective way of keeping track of inputs, outputs and outcomes for activities.At the top, you can see what the overall goal of the program is – and on the right there is the indicator that will show if the program has achieved it’s overall goal.Then, for each of the activities, there are examples of what the expected inputs, outputs and outcomes will be – and on the left, the indicators that will be used to measure whether or not the activities were successful.The LFA is a very common monitoring tool – and many organizations have standard LFA tables that have to be filled out for all sectoral responses. This enables the central management team to monitor all activities and programs across sectors.
13Pourquoi le suivi est-il important ? INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONPourquoi le suivi est-il important ?PertinenceResponsabilité et communicationPermet de rendre des comptesEnsuring that an intervention is relevant to the needs of the population is an obvious indication of how important it is to monitor a response.Ask the participants: for what other reasons is monitoring important?When they have shared some of their answers – add the following two reasons with explanations:It is very common that psychosocial responses are implemented far away from the headquarters of the National RCRC Society. Monitoring reports therefore become one of the most important communication tools between the field, where the response is implemented, and the headquarter offices, that have the ultimate responsibility for the planning and implementation of the response. This is one of the reasons that staff and volunteers have to be trained well in monitoring activities, and the development of the monitoring tools has to be done carefully to include all possible data that could be needed.A third important reason for doing monitoring is that of accountability. The Red Cross or Red Crescent Society involved in the response are first and foremost accountable to the affected population they are assisting. Continuous monitoring enables the RCRC Society to regularly change and adapt the program, and in this way make sure they are helping the population in the best way possible. The National Society is also accountable to the donors that support the program – both in terms of fulfilling their expected aims, and in terms of financial spending etc.Simply put – monitoring helps us to assess if we are on track, in terms of what we planned to do – and more importantly – are we on the right track?Plus simplement : Sommes-nous sur la voie ? Sommes-nous sur la BONNE voie ?
14Qui est associé au suivi ? INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONQui est associé au suivi ?ResponsabilitéY compris suivi et évaluationAutres partenairesBasically, everyone who is involved in a psychosocial response, is also involved in monitoring the progress and development of the response.CLICK: The managing programme staff at national and regional level have the ultimate responsibility to make sure that there are both time and resources allocated to enable successful monitoring activities. They are the supervise the entire programme planning and implementation, which includes the monitoring and evaluation activities. CLICKCLICK: Next,, the local programme staff, who are working at the field level, - together with the community leaders or cluster leaders - have the responsibility for practical arrangements and also for exploring and deciding what the most appropriate methods of data collection would be with the various informants.CLICK 2 times: The actual data collection is usually done by community facilitators and volunteers. Since volunteers are often from the affected population themselves, they are very well placed to collect data in culturally appropriate and respectful ways.It is naturally the affected population who are the key informants in a psychosocial response, as they are the ones who provide the data needed.CLICK: A last group of important participants are other partners that may be contributing to or collaborating in the psychosocial response. These can be internal partners from other sectors of the National Society, or external partners who either also provide psychosocial support, or attend to other needs of the population.Collecte de données
15Qu’est-ce que les évaluations ? INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONQu’est-ce que les évaluations ?En groupes de réflexion de 2 ou 3, discutez de ce que sont les évaluations et de ce en quoi elles diffèrent du suivi. Essayez de rédiger une définition simple et précise du rôle de l'évaluation dans la réponse psychosociale.Give the participants a few minutes, in buzz groups, to discuss “what evaluations are and how they differ from monitoring?”. When they are finished, move on to the next slide.
16Que sont les évaluations ? INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONQue sont les évaluations ?« Les évaluations permettent d'établir si les interventions ont atteint les visées et buts généraux de la réponse psychosociale. »Points de débat :Les évaluations sont censées être OBJECTIVES – Qu'est-ce que cela signifie et pourquoi est-ce important ?Show the participants the above definition of an evaluation of a psychosocial intervention. Ask them if they agree with this definition, or if they have anything to add or feel should be taken out or rephrased. Work with the participants to adjust the definition until everyone agrees. Use the points below if there are no comments, to encourage discussion:Important to state it is an OBJECTIVE assessment. Discuss what this means and why it is important.
17Évaluation intermédiaire INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONQue sont les évaluations ?Évaluations = Suivi + davantageÉvaluation finaleÉvaluation intermédiaireM1M2M3M4M5M6M7M8In simple terms, evaluations are similar to monitoring, only more detailed and advanced in terms of analysis and reporting. This means they are also more costly and time-consuming.Evaluations often consider both the raw data and the information developed from the regular monitoring activities, together with more data and generated information from additional data collection activities.The diagram shows that the mid-term evaluation will consider the data and information from the monitoring that has taken place so far – and additional; whilst the final evaluation will consider all the monitoring data and information, and that from the mid-term evaluation, and if needed, additional data gathered.
18Évaluations axées sur l'estimation du BUT/VISÉE GÉNÉRAL(E) Approche de cadre logiqueIndicateursBut généralDiminuer la souffrance et le risque de développement de traumatismes sévères dans la Population AAdultes et enfants montrent des signes sains d'adaptation à l'impact de la criseActivité 1Formation en PSPContributionsRessources permettant la formationQuantité d'argent ; personnel ; manuels de formation ; matérielsProductionsLa formation a eu lieuNombre de personnes forméesRésultatsLes bénévoles sont capables d'administrer les PSPLe niveau de compétences en PSP a augmentéActivité 2 :Ateliers avec des enfantsRessources pour les ateliersRéalisation des ateliersNombre d'ateliers réalisés ; nombre d'enfants concernés ; nombre de formateursLes enfants s'adaptent mieuxEnfants plus enjoués, plus grande confiance en soi et en les autresCLICK: Evaluations are thus particularly interested in assessing if the OVERALL AIM or GOAL of the response was achievedCLICK: However, assessing this ALSO involves a full consideration and analysis of the monitoring results of each activity and smaller component of the implemented response.
19Types d'évaluations Évaluations en temps réel Précocité de la réponse INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONTypes d'évaluationsÉvaluations en temps réelPrécocité de la réponsePersonnel interneAxe : questions de processus / d'opérationsÉvaluations intermédiairesÉvaluer l'impactRendre des comptes (budgets/admin)Adaptations nécessitant un guideÉvaluations finalesImpact et processusRéussites et difficultés principalesInformations précieuses pour les prochaines interventions PSThere are three main types of evaluations.The first are real time evaluations. These are usually done very early in a response to see if the responders are ‘on the right track’. They are usually very short in length (time taken) and typically focus on organizational aspects of the response, such as administrative issues (employees, logistics etc); coordination and cooperation with other partners; and any other practical issues that may influence the success of the planned intervention. They are done in ‘real time’ meaning that they provide immediate feedback to the organization, and opportunities for immediate action and needed adaptation.The second are – as mentioned, the mid-term evaluations. These are also called reviews – and they are not always done, but highly recommended. They are a good way of consolidating all the information generated by the monitoring activities to date, and help to give and overview of the administrative progress and the impact of the intervention.The final evaluation is, for most, very important. It is usually the results of the final evaluation that are shared with all stakeholders involved in the intervention, including the donors. The final evaluation not only reports what has taken place, and whether the intervention was a success, but it also includes an analysis of the challenges and gives recommendations either to future activities with the targeted population, or in future psychosocial responses with other populations.
20Évaluer l'impact Comment mesurez-vous l'impact d'une intervention ? INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONÉvaluer l'impactComment mesurez-vous l'impact d'une intervention ?De quoi avez-vous besoin ?To measure the impact of a psychosocial intervention you will need data, from the beginning of an intervention, during an intervention, and at the end of the intervention. This is true whether it is for a short term immediate intervention (CLICK) or a long term intervention (CLICK).It is very important that the nature of the data is well planned – so that it measures the same indicators from beginning to end of the intervention. (CLICK 2 TIMES) This will enable comparison of results, which in turn will show whether any change took place.DonnéesDonnéesMÊMES INDICATEURS
21Comment définir les indicateurs psychosociaux INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONComment définir les indicateurs psychosociauxÉvaluer - Comment la population est-elle touchée ?Quel est le type de changement désiré ?(Buts/objectifs/activités de réponse)Quelle est la définition locale du bien-être psychosocial ?Choosing the right indicators is a very important task, since the indicators are what are going to show whether or not the psychosocial intervention is working. It often takes a lot of consultation, data collection, analysis and time to choose the right indicators – but this should not deter those working with psychosocial interventions. There are some pretty simple steps to follow.CLICK: First, the results from the assessments that show how the population has been affected, will help to guide what kind of change is desired. CLICK: This is the first step in determining indicators. – finding out what change you want to measureCLICK: In programmatic terms, this is measuring the goals, objectives and activities of the response.The most difficult indicators to define are usually the most important ones – the ones that show whether or not the populations’ psychosocial wellbeing has improved. CLICK: To choose the right indicators for this one first needs to know what the local definition of psychosocial wellbeing is.
22Indicateurs du bien-être psychosocial INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONIndicateurs du bien-être psychosocialAsk the participants:" On a piece of paper in front of you – write three things that about yourself or your life that you think show to others that you are feeling good and doing well.Now think about how others could measure whether you felt better from one day to the next. Write these down”.Now ask the participants to share some of their indicators of wellbeing. There should be a good variety of indicators, and also some indicators that may be the same for the whole group.Use this discussion to highlight that different people may have different things that indicate their psychosocial wellbeing, but if you spent some time with the group, you would probably be able to find some common indicators that everyone would agree would be both general, and specific, enough to determine the groups’ psychosocial wellbeing.Explain: Indicators of psychosocial wellbeing focus both on how the individuals are doing, and on how the group is doing. This is the nature of psychosocial – that there is a focus on the individual’s psyche and on the social interaction and network.
23Indicateurs psychosociaux INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONIndicateurs psychosociaux1. Écrivez trois choses sur vous-même ou sur votre vie qui, selon vous, montrent aux autres que vous vous sentez bien et que vous allez bien.2. Maintenant réfléchissez et écrivez ce qui, selon vous, pourrait permettre aux autres de mesurer si vous vous sentez mieux de jour en jour.3. En plénière, partagez quelques-uns de vos indicateurs de bien-être. Nous cherchons à obtenir des indicateurs très divers, mais aussi certains indicateurs qui soient valides pour l'ensemble du groupe.Choosing the right indicators is a very important task, since the indicators are what are going to show whether or not the psychosocial intervention is working. It often takes a lot of consultation, data collection, analysis and time to choose the right indicators – but this should not deter those working with psychosocial interventions. There are some pretty simple steps to follow.CLICK: First, the results from the assessments that show how the population has been affected, will help to guide what kind of change is desired. CLICK: This is the first step in determining indicators. – finding out what change you want to measureCLICK: In programmatic terms, this is measuring the goals, objectives and activities of the response.The most difficult indicators to define are usually the most important ones – the ones that show whether or not the populations’ psychosocial wellbeing has improved. CLICK: To choose the right indicators for this one first needs to know what the local definition of psychosocial wellbeing is.23
24Indicateurs psychosociaux INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONIndicateurs psychosociauxLes éléments qui témoignent du bien-être psychosocial des gens sont différents, toutefois, en approfondissant un peu, nous déterminerons certainement des indicateurs courants à la fois généraux et spécifiques, assez en tout cas pour déterminer le bien-être psychosocial du groupe.Les indicateurs du bien-être psychosocial sont axés sur :Comment vont les personnesComment va le groupe.Le psychosocial s'intéresse, par nature, au psychisme de l'individu, à l'interaction et aux réseaux sociaux.Choosing the right indicators is a very important task, since the indicators are what are going to show whether or not the psychosocial intervention is working. It often takes a lot of consultation, data collection, analysis and time to choose the right indicators – but this should not deter those working with psychosocial interventions. There are some pretty simple steps to follow.CLICK: First, the results from the assessments that show how the population has been affected, will help to guide what kind of change is desired. CLICK: This is the first step in determining indicators. – finding out what change you want to measureCLICK: In programmatic terms, this is measuring the goals, objectives and activities of the response.The most difficult indicators to define are usually the most important ones – the ones that show whether or not the populations’ psychosocial wellbeing has improved. CLICK: To choose the right indicators for this one first needs to know what the local definition of psychosocial wellbeing is.24
25Indicateurs psychosociaux INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONIndicateurs psychosociauxJE SAIS...Demandons-leur comment ILS savent qu'ils se sentent mieux, comme ça nous pourrons trouver des moyens de mesurer cette amélioration…Nous voulons que les enfants se sentent mieux….Mais comment SAURONS-nous s'ils se sentent mieux ?1. Here is a very simple explanation of what needs to be done to define indicators for psychosocial wellbeing.
26Pourquoi l'évaluation est-elle importante ? INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONPourquoi l'évaluation est-elle importante ?Pertinence (réponse pertinente aux besoins de la pop.)Efficacité (calendrier, coût)Impact (réussi ou non)Efficience (objectif du programme atteint)Durabilité (avantages qui se prolongent au-delà du programme)Like monitoring, evaluations also help to assess if interventions are relevant to the needs of the population – and to the needs of the planned response (real time evaluations). Mid-term evaluations, in particular, are often used to guide adaptations to the interventions, if the results of the evaluation indicate that the population needs have changed or are simply not being met. Final evaluations can help to highlight if at all the response succeeded in meeting the needs, in retrospect – but they do not play a part in guiding adaptations for activities in the response. However, recommendations are usually made for future responses – either to the same population or to others.When exploring efficiency, evaluations look at whether the psychosocial response activities are or were implemented succesfully in terms of intended time-frame and cost.As mentioned, evaluations are very important tools in assessing the impact of the response. They explore, in detail, what has changed as a result of the psychosocial response, and if the desired outcome of improved psychosocial wellbeing has been achieved.Evaluations also assess whether the objectives set for the program were met – and in this way determine if the program was effective.Finally, by not only looking backwards, but also focusing on future needs of the population, evaluations explore whether the benefits of the program are likely to continue beyound the life of the program. This is especially important for psychosocial support interventions, as a common goal in most psychosocial responses is helping the affected population become strong enough to take care of themselves. Evaluations help to assess this.
27INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATION Qui est associé aux évaluations outre le personnel, les bénévoles, les partenaires ?ResponsabilitéConsultants extérieursY compris suivi et évaluationAutres partenairesEvaluations involve the same people as are involved in monitoring, as discussed earlier.However, it is very common that external consultants (CLIKC) are hired to conduct evaluations.Ask the participants why they think this is a good practice?Explain: External consultants are usually more objective and able to provide honest and constructive feedback, without the fear of losing their jobs or upsetting anyone, if, for example, they report inefficiency of administrative staff, or other issues that may be taken personally.Hiring external consultants is often quite pricey, and this should be budgeted for in all psychosocial responses.It is definitely a best practice to hire local external consultants – who have a solid understanding and knowledge of the culture and history of the affected population. If this is not possible, then an international external consultant should definitely have opportunities to consult closely with local representation to develop some of this understanding.Collecte de données
28Échantillonnage Population ciblée Large & varié Large & varié INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONÉchantillonnagePopulation cibléeLarge & variéCLICK: In the immediate aftermath of a disaster or before starting a psychosocial response to an ongoing crisis, it is important to obtain data from a sample that is as broad and varied as possible. This will enable a holistic assessment and understanding of how the different subpopulations have been impacted, allow identification of vulnerable groups; and help to identify resources outside the affected population that may be able to assist in the psychosocial response.CLICK: However, when the target groups for the psychosocial intervention have been chosen, the sampling for the population who provide data for monitoring especially, becomes more narrow. Focus is now on collecting data from the targeted population and from those who work directly with themThe exception may be for the mid-term evaluation, where, like in the final evaluation (CLICK) – the sampling again INCLUDES information from a broad and varied group – AS WELL as from the targeted population who provided data throughout the program.Large & varié
29Quand réaliser un suivi et quand évaluer INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONQuand réaliser un suivi et quand évaluerMonitoring schedules will vary according to whether a program is short term or long term, and also according to the program design. In a short term immediate response, monitoring may be done on a daily basis, and then later on a weekly basis.In a long term programme, monitoring may be done weekly in the beginning, and then later be extended to monthly, or even three-monthly. It may be that different aspects of the psychosocial response will need monitoring at different intervals or times. For example, logistic inputs and outputs may need monthly monitoring, whilst it may be decided that impact on psychosocial wellbeing should only be assessed bi-monthly.What is important is that monitoring is carefully planned as a regular, ongoing and structured series of activities. Monitoring activities should be budgeted for, both in terms of the time the activities take, and the resources – human and financial.Evaluations are not on-going activities like monitoring, but are usually scheduled to take place around the mid-term mark of the program life, and at the end. As mentioned earlier, real time evaluations take place in the very early stages of a response, and are usually scheduled very quickly in response to a need for them. They are typically multi-sectoral and do not focus, for example, only on psychosocial support or any other sector.
30Deux catégories principales de méthodes de collecte de données INTERVENTIONS PSYCHOSOCIALES ÉVALUATIONComment réaliser une évaluation ?Deux catégories principales de méthodes de collecte de donnéesQuantitatives QualitativesEnquêtes, Questionnaires Outils psychométriques(mesurer réactions, comportements, sentiments, à l’aide d’une échelle graduée1,2, =, %, a : bEntretiens de témoins privilégiésDiscussions dirigées de groupeObservationsDescription terminologiqueThere are two main types of data collection methods, (also explained in chapter and presentation on assessments). The methods used will depend on the nature of the data you want to collect, and what you are going to do with it.Quantitative data is collected through surveys or questionnaires or other tools, that have scaled measures. Such data is usually reported in numeric terms – as counts, ratios or percentages. Sometimes psychometric tools are also used in psychosocial interventions. These are tools that measure people’s reactions, behaviour and feelings, using a scaled measure. They are often self-reporting measures. For example, a psychometric tool may measure signs of depression, by asking a person to indicate from a scale of 1 – 5 how sad they feel, with 1 being ‘not sad at all’ and 5 being ‘so sad I cry all the time’.Whilst quantitative data is usually COUNTED, qualitative data is usually DESCRIBED WITH WORDS, and is collected through TALKING and LOOKING. Examples are focus group discussions, key informant interviews or behavioral observations.It is common to use both quantitative and qualitative data collection methods, because most monitoring and evaluation activities need both kind of data to develop the needed information.
31Faut-il des données pour ces indicateurs quantitatifs ou qualitatifs ? INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONFaut-il des données pour ces indicateurs quantitatifs ou qualitatifs ?Approche de cadre logiqueIndicateursBut généralDiminuer la souffrance et le risque de développement de traumatismes sévères dans la Population AAdultes et enfants montrent des signes sains d'adaptation à l'impact de la criseActivité 1Formation en PSPContributionsRessources permettant la formationQuantité d'argent ; personnel ; manuels de formation ; matérielsProductionsLa formation a eu lieuNombre de personnes forméesRésultatsLes bénévoles sont capables d'administrer les PSPLe niveau de compétences en PSP a augmentéActivité 2 :Ateliers avec les enfantsRessources pour les ateliersRéalisation des ateliersNombre d'ateliers réalisés ; nombre d'enfants concernés ; nombre de formateursLes enfants s'adaptent mieuxEnfants plus enjoués, plus grande confiance en soi et en les autresAsk the participants to go through each of the indicators on this LFA example, and to tell you whether the data needed for these indicators is quantitative or qualitative.See below for the correct answers:Amount of money – quantitativeNumber of personnel – quantitativeNumber of training manuals and materials – quantitativeNumber of people trained: - quantitativeSkill level in PFA – could be quantitative or qualitative – discuss why? Skill level could be measured on a self-reporting scale, or it could be observed through, for example, role plays.Number of workshops, children attending, trainers – quantitativeIncrease in children’s playfulness, self-confidence and trusting of others – again could be quantitative or qualitative – as above, it could be measured on a measured scaled or through observation. These are more likely to be qualitative
32Principes éthiques en matière de collecte de données INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONLa collecte de donnéesPrincipes éthiques en matière de collecte de donnéesJustifiable et correctement planifiéeCoordinationExpliquer les visées et les procéduresParticipative et collaborativePLEASE NOTE THE FOLLOWING TWO SLIDES ARE ALSO INCLUDED IN THE PRESENTATION ON ASSESSMENTS. IF THE PARTICIPANTS HAVE ALSO RECEIVED THIS PRESENTATION YOU DO NOT NEED TO INCLUDE THESE TWO SLIDES IN THIS PRESENTATION, UNLESS FOR REMINDER PURPOSES.There are a number of things one has to be aware of when planning to collect data in assessments.The first is that the data collection is well planned and justifiable. This means there has to be a good reason to collect the data – which there usually is if a disaster has taken place and a population has been hurt or displaced. The data collection activity has to be well planned – the data collectors have to be well trained, and adequate time and resources should be budgeted for the data collection activities.Secondly, it is important to coordinate data collection with others, both in your own organization, and with other organizations and government departments – for example, at coordination meetings. This will prevent the same people from being interviewed over and over again, which can be traumatic for them. It is also less confusing for the affected population.Third, during training of data collectors, it is important that the aims and the ethical and expected procedures are carefully explained and then followed. All data collectors should be able to explain the aims of the data collection they are doing clearly to the people they talk to.Explaining the aims of data collection is connected to this next point, since those who are interviewed should be given the opportunity to understand completely what the data is going to be used for – and then the option of whether they want to contribute with data. This reduces any unrealistic expectations.
33Principes éthiques en matière de collecte de données INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONLa collecte de donnéesPrincipes éthiques en matière de collecte de donnéesGroupes de comparaisonConduite et consentementRespect de l'intimité et confidentialitéAnticiper les conséquences négativesIn scientific experiments it is usual that one has a control group, or a comparison group, that was not included in the scientific experiment. This is done to show if the experiment worked on the group that was included. This is NOT ethical and should be avoided when planning psychosocial interventions. It is not ethical to deprive some people of psychosocial support purely for comparison purposes. The only way this sometimes works, is if a psychosocial response is implemented with a time delay in some groups – then one could make comparisons with groups that were waiting to receive support.It is very important that all people who collect data do so in a culturally appropriate and respectful manner. All persons who provide data should also give their informed consent at the beginning of the data collection activity.Whilst giving informers the opportunity to give consent, it should also be explained to them that all the data will be private and confidential, and their names will not be made public or shared with others, except for the purposes of assisting them with psychosocial support.When people have gone through a difficult experience, it may be painful and disturbing for them to talk about it again to others. Data collectors should be trained to deal with this, especially when they are collecting information on psychosocial needs. It is best data collectors expect strong reactions, so they know what to do.
34Planifier le suivi et l'évaluation INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONPlanifier le suivi et l'évaluationTravail de groupe : Que faut-il planifier pour garantir la qualité du suivi et de l'évaluation ?Explain: Now that we have talked about what monitoring and evaluation are, how they are done and why they are so important, I want you to consider what needs to be done to plan good monitoring and evaluation.Ask the participants to sit in groups and discuss the given question.Give them flip charts to record their findings and when they have had adequate time ask them to present their group work to plenary and move on to the next slides as a summary.
35Planifier le suivi et l'évaluation INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONPlanifier le suivi et l'évaluationCalendrierDurée de la réponse (court terme contre long terme)Moment opportun pour la pop. touchée (ex. ne pas prévoir d'activités de suivi auprès d'enfants scolarisés pendant la période des examens, etc.)Durée de planification, de mise en œuvre, d'analyse, de suivi (allouer temps et ressources)FormationGestion du SPSCollecteurs de donnéesÉquipe du programme – analyse et rapportTwo important aspects of planning for good M and E are timing and training.Regarding timing, it is first and foremost important to know and plan according to whether you are going to implement a short term or long term response. This will determine the appropriate intervals for monitoring activities, and planning of evaluations.Secondly, when planning data collection activities, it is of course important to plan these activities when it is convenient for the targeted population. For example, it would not be a good idea to plan data collection with in-school children during exam time, or during their holidays.Third, both monitoring and evaluation take TIME. The amount of time taken to conduct these activities well should NOT be underestimated. Time should be allocated and budgeted for, in terms of financial and human resources.Regarding training:It may be necessary for top management staff to receive some training, or at least, orientation on what psychosocial support is, how it will be provided, and what needs to be done in terms of M and E. The reason for this is that PSS is not a cross-cutting field YET, at some management staff may not yet understand exactly what it is. To enable budgeting for M and E, they need to understand what it is first.Second, data collectors need careful training in data collection methods and ethics.Third, the program team who have responsibility for the M and E activities also need to be well-trained and skilled in data collection, analysis and appropriate reporting.
36Planifier le suivi et l'évaluation INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONPlanifier le suivi et l'évaluationRessourcesPersonnel et bénévolesFinancesTempsLogistiquePartage de l'informationInterne - normes et formatsExterne – différents auditoires(bénéficiaires/donateurs/publics)As mentioned on the previous slide, good planning for M and E requires adequate resource allocation.The final important part of planning, is making sure that monitoring tools are well-prepared and carefully filled out – according to the required standards and formats. This will make reporting easier and more successfulAs results of monitoring and evaluation activities are achieved, decisions also have to be made on how to share the results. There may be a need to present the results in different ways, according to who is going to read them. Donors might, for example, want financial reporting, whereas the beneficiaries and the public may be more interested in results of the impacts of the interventions, and recommendations for the future.
37Rôle et importance du S&E dans le SPS INTERVENTIONS PSYCHOSOCIALESSUIVI ET ÉVALUATIONRôle et importance du S&E dans le SPSNous aide à comprendre la signification locale - et pertinente - du bien-être psychosocialNous maintient sur la bonne voieNous aide à faire de notre mieux, actuellement et à l'avenirIn summary, ask the participants why they think monitoring and evaluation is so important, particularly for psychosocial responses.When they have finished, present this simple summary with the three given points.M and E help to keep us on the right track throughout the program period.They also help us to understand what the local meaning of psychosocial wellbeing is, for the given population. This is very important as we increasingly recognize that psychosocial wellbeing is experienced differently by different people, in different situations.Successful data collection, reporting and information sharing in monitoring and evaluation are also invaluable tools for research in the field of psychosocial support – which is definitely still needed. The results of research, and of every monitoring and evaluation activity, help us to improve psychosocial responses, now and in the future.