Le Programme Statistique Accéléré PSA - ADP ATELIER REGIONAL FAO/PARIS21 SUR L’INTEGRATION ET L’ACCES AUX STATISTIQUES AGRICOLES POUR AMELIORER LA FORMULATION ET LE SUIVI DES POLITIQUES DE DEVELOPPEMENT RURAL EN AFRIQUE (En prélude à la 20ème Session de l’AFCAS/CASA) Alger, Algérie: 8-9 Décembre 2007 AW-07-02-1 Le Programme Statistique Accéléré PSA - ADP The pilot ADP was launched as a recommendation of the MAPS, to undertake urgent improvements needed for monitoring the MDGs. Since 2006, the pilot ADP is implemented as a partnership between PARIS21, the World Bank, and other international partners. François Fonteneau, PARIS21/OCDE

Le Programme Statistique Accéléré PSA - ADP The pilot ADP was launched as a recommendation of the MAPS, to undertake urgent improvements needed for monitoring the MDGs. Since 2006, the pilot ADP is implemented as a partnership between PARIS21, the World Bank, and other international partners.

Contexte Pourquoi ne réussissons nous pas à mesurer et suivre les résultats? 3 raisons (parmi d’autres): Données existantes pas totalement exploitées Méthodes et concepts non harmonisés Champ, ponctualité et fréquence non optimales 1 2 Measuring and monitoring development outcomes require timely, reliable, comparable, relevant, and accessible, survey data So much has been invested in conducting households surveys Why can’t we better measure and monitor results? ---------------------------------------------------------------------------- Measuring and monitoring development outcomes and assessing the effectiveness of development policies remains a major challenge. In most developing countries, statistics are not produced regularly enough and with sufficient quality. Building sustainable statistical capacity is a long process requiring institutional development and change. To satisfy short term data needs, the pilot Accelerated Data Program (ADP) has been launched in 2006. The ADP aims to improve national survey programs. In many countries, survey programs are still ad-hoc and donor-driven, characterized by their inefficient frequency and timing, a lack of alignment with national development priorities, a lack of coordination at national and international levels, and a low utilization of the data collected. Issues to be addressed in priority include: Data availability and timeliness. Few developing countries have the financial and technical resources to implement large-scale survey programs. Survey programs remain largely donor-driven, resulting in large data gaps in some countries, and duplication of efforts in others. The timeliness, frequency, and sequencing of surveys are far from optimal. Data reliability. In the last 15 years, national statistical capacity has increased in most developing countries. But many still lack the expertise needed to collect, process, and analyze complex multi-topic survey data in a satisfactory manner. Data comparability. Surveys become more valuable when they allow comparisons to be made with other surveys and data sets. Harmonization of survey methods and instruments would increase comparability of the results obtained. Data analysis and use. Data producers in developing countries have limited capacity, and sometimes limited interest or mandate, to analyze the data they collect. In too many cases, the output of surveys is limited to a descriptive report and a collection of tables of limited relevance to policy makers. As access to microdata by secondary users is often restricted for various technical, legal, and in a few cases political reasons, many surveys remain largely under-exploited. Fostering access to properly documented microdata by qualified researchers can be a cost-effective solution to rapidly increase the diversity and quantity of data analysis and use. 3

Objectifs Renforcer les capacités des pays à produire une information statistique pertinente pour l’appui aux politiques, en : Améliorant la documentation, l’archivage et la diffusion des micro-données existantes Exploitant davantage les données existantes (évaluation qualité, analyses complémentaires) Alignant les programmes d’enquêtes et les produits statistiques sur les priorités nationales, conformément aux NSDS 1 2 The ADP is focused on sample household surveys because they provide estimates of many key outcome indicators, as well as data needed for research and impact evaluation. 3

Mise en oeuvre: Pays pilotes

Mise en oeuvre: Pays pilotes Afrique Asie Amérique Latine Moyen-Orient Cameroun Congo (RDC) Ethiopie Gambie Kenya Liberia Mali Mozambique Niger Nigeria Senegal Ouganda Bangladesh Fiji Indonesiea Mongolie Philippines Sri Lanka Thailand Vietnam Guatemala Honduras Peru Yemen Additional countries are using the toolkit without ADP support (Tunisia, 12+10 MICS, 15 Latin America- Regional Award for Innovation in Statistics for Latin America and the Caribbean (DFID/WB) , regional trainings, etc.) Intérêt: Bolivie, Colombie, Guinée, Guyane, Mexique, Panama, Palestine

Mise en oeuvre: Partenaires Partenaires clés PARIS21 (secrétariat) Banque Mondiale Réseau International pour les Enquêtes auprès des Ménages (IHSN) Autres partenaires internationaux Economic and Social Commission for Asia and the Pacific (UNESCAP) Inter-American Development Bank (IDB)  United Nations Children Fund (UNICEF) Food and Agriculture Organization (FAO)  Economic and Social Commission for Western Asia (UN-ESCWA)* … Core partners PARIS21 Secretariat: Implementation of the pilot phase Management of the Development Grant Facility funds World Bank: DGF provides funding DDG provides global coordination Other departments contribute to implementing the activities International Household Survey Network (IHSN) Provision of tools and guidelines The ADP benefits from the contribution of other international partners. Economic and Social Commission for Asia and the Pacific (UN-ESCAP) Implementation in Asia Interamerican Development Bank (IADB)  Implementation in Latin America United Nations Children Fund (UNICEF) Documentation and dissemination of the MICS datasets. Food and Agriculture Organization (FAO) 

Mise en oeuvre Basée sur les besoins des pays Simplifiée et standardisée pour une mise en peuvre rapide : Initiation Requête du pays Préparation Mise en place d’une proposition d’appui spécifique Idéalement: liée à la SNDS De préférence: implique tous les producteurs sectoriels Mise en oeuvre Assistance technique, formation, soutien financier Suivi technique et supervision Modalités administratives spécifiques au pays Evaluation (plus tard) Initiation. The ADP is initiated in a country by an informal request for support formulated by an interested national agency. Priority is given to the poorest countries, who express a strong commitment in improving the management and use of their microdata. Preparation. After the PARIS21 Secretariat notifies the agency of its agreement to provide support, a country-specific ADP work program will be developed (tentative budget and calendar). This will typically involved a mission by an ADP staff to the country. Ideally, the ADP work program will be explicitly linked to the national statistical strategy. Also, the project should preferably involve all key microdata producers in the country, and not be limited to one producer. Implementation. Once a country-specific work program is formulated, the ADP will start providing the technical assistance, training, and financial support required for its implementation. The administrative modalities may vary from country to country. International consultants will usually be recruited directly by the PARIS21 Secretariat. For financing local activities (organization of workshops, procurement of equipment, hiring of local consultants), the preferred option is to establish a service contract between the executing agency and the PARIS21 Secretariat. The ADP team will provide regular technical follow-up and supervision. Revisions of the work programs are possible and require very little formalities. 

Activités: Tâche 1 Documentation, archivage et diffusion des micro-données Inventaire complet des micro-données disponibles Documentation des micro-données selon les standards internationaux Définition d’une politique de diffusion des micro-données, en cohérence avec la législation nationale Anonymisation des micro-données Mise en place d’une archive nationale Low capacity/interest from data producers Data not always accessible to secondary users (due to technical, financial, legal, political obstacles) Lack of metadata makes data difficult/risky to use

Activités: Tâche 1 Mise en oeuvre Outils Toolkit pour la gestion des micro-données (+ guides pratiques) Guide de référence pour une politique de diffusion des micro-données* Outils d’anonymization* Archive Nationale (NADA) Progrès Inventaires dans15 pays 250 staff formés au Toolkit (50 institutions dans15 pays) Enquêtes toolkitées, politiques de diffusion adoptées NADA installées

Activités: Tâche 1 Mise en oeuvre Progrès ‘Known surveys’ are Best Estimates Available ? ? ? ? ? ? Data inventories across the NSS are key to establishing and securing the treasury of information of a country. They also serve to harmonize data collection activities and survey instruments NIGERIA: An inventory is currently being conducted. Nigeria sent a Resource person to Uganda to provide assistance in the Ugandan national data inventory. MOZAMBIQUE:This activity is planned but put on hold due to Census 2007. This is particularly important in light of the National Emergency Response Plan that is being developed as a result of a fire at the Ministry of Agriculture

Activités: Tâche 2 Analyse des données d’enquêtes existantes et évaluations des programmes passés Focus sur les priorités identifiées dans les PRSP et les stratégies sectorielles Résultats attendus : Évaluations détaillées des forces et faiblesses des instruments d’enquêtes et amélioration Travaux d’analyses et notes pour les politiques Surveys are often ad-hoc; little attention paid to harmonize concepts and methods across surveys Resulting indicators not always relevant for policy needs Resulting indicators not fully comparable (over time, between countries); conflicting and confusing results are sometimes produced

Activités: Tâche 2 T1 montre le besoin d’améliorer la fiabilité Are you currently attending school or, if school is not in session, did you attend school in the session just completed and plan to attend next session? 2004 HIS, Malawi Is school in session? Are you currently attending school? Did you attend school last year? Do you intend to attend school next year? Yes No YES NO The EPDC selected 15 education indicators to calculate from the datasets - over-age children (%), under-age children (%), on-time children (%), over-age first grade students (%), gross intake rate, net intake rate, education attainment, literacy rate, gross attendance rate, net attendance rate, completion rate, dropout rate, promotion rate, repetition rate, and transition rate. This is a standard set of common education indicators that give information about how many children start school and remain in school, and how they flow through the education system. The indicators are extracted with STATA. The indicators are extracted from 24 of the 30 datasets at the time of writing. For the remaining six datasets there was information missing regarding the sample or coding, so the indicators could not be extracted.   For each indicator the EPDC calculated an average standard error (SE) to examine the reliability of data extracted at the national and subnational levels. For this analysis, any SE higher than 2.52 is considered statistically unreliable to use (a standard error of 2.52 provides a confidence interval of 95%). Table 10 and Figure 1 show the standard errors at the national level. Table 10 shows the actual numbers, and the standard errors that are higher than the acceptable level are marked in bold font. Figure 1 shows dots to give a general impression of which indicators have high, and which have low standard errors. In the figure, each dot represents the standard error for one country (as not all indicators can be extracted from all countries, there are usually fewer than 24 dots for each indicator). The highest range of standard errors is for the gross intake rate and the primary completion rates. National level indicators Percentage of over-age, under-age, on time students and over-age first grade students can be calculated for 16 out of 24 datasets. The standard errors for these indicators are generally within in the acceptable range. However, for the over-age first grade students (with smaller sample sizes because the selection is limited to one grade) standard errors were larger: only six of these 15 datasets had SE smaller than 2.52. Gross intake rates can be calculated for 15 of the 24 country datasets – the remaining nine data sets did not have sufficient information on single year grade levels to calculate the indicators. The average standard error for the extracted gross intake rates was higher than 2.52. The reason for such high standard errors may have to do with the fact that standard errors are calculated in such a way that leads them to almost always be high for indicator values greater than 100%. As Table 10 shows, gross intake rates have considerably high standard errors for all country dataset. Net intake rates can be calculated for the same 15 out of 24 countries as gross intake rates. Though standard errors for net intake rates are considerably lower than those for gross intake rates, many (10 out of 15 exceed 2.52). Gross attendance rates can be calculated for 19 out of 24 datasets. In five countries, attendance rates cannot be calculated because surveys do not include specific information about school levels attended. Of the 19 countries for which GAR was calculated, only seven provided indicators with standard errors below 2.52. The reason for this was probably related to the same cause as the high standard errors for GIR. Educational attainment and literacy rates are the most commonly extracted indicators. Educational attainment (by level) was extracted for 23 of the 24 datasets and the literacy rate for 20 out of the 24. The standard errors for both indicators and all countries are low and within the acceptable range. Efficiency indicators (completion, dropout, promotion, repetition, and transition) can be calculated for only three of 24 datasets. While 19 surveys asked questions related to school flows, only three included the full set of questions needed to calculate the efficiency indicators (for more on this, see Education Flows on page 15). The repetition rate for Cameroon is available because the survey specifically asks “Is [name] attending the same grade this year as they attended last year]. SE’s exceeded 2.52 for seven of the sixteen efficiency indicators . Across 24 datasets, Bangladesh HIES 2000, Madagascar 2001 and Sierra Leone HIS 2003 had over 50% of 15 education indicators with a high average standard error (SE>2.52). Subnational level indicators The 15 indicators were also extracted at the sub-national level. Because there are smaller sample sizes in the sub-national regions, it is possible that the standard errors are larger than at the national level, and that some regions have to be dropped. Table 11 shows the standard errors for the same indicators. As expected at the subnational level, even more of 15 education indicators has high standard errors, except for the education attainment data. All 24 datasets with education attainment data had an average SE smaller than 2.52. Out of 20 datasets that had sufficient information to extract literacy rates, only eight provided reliable data on average. Uganda NHS 2002, Pakistan HIS 2001, and Malawi HIS 2001 has more than 50% of their data reliable to use at the regional level whereas Tanzania HBS 2000 and Cameroon ECAM 2001 provide at least three indicators that are reliable to use on average. The rest 10 datasets had only one or two indicators with small SE for the use of the data at the regional level. SE ranges with minimum and maximum values of 15 education indicators for 24 IHSN country datasets can be found in Table 15 and Table 16 in Appendix B.

Activités: Tâche 2 T1 montre le besoin d’harmonisation Mesurer l’accès à une source d’eau potable au Ghana CWIQ 2003 CENSUS 2000 GLSS 1998 DHS 2003

Activités: Tâche 2 Mise en oeuvre Outils Base de Questions, en développement Plus tard, soutien aux Bases de Questions nationales: Cohérence entre sources nationales Réutilisation et harmonisation des questions litérales, instructions pour enquêteurs, contrôleurs, etc. Progrès Identification des activités: dépend de la situation des pays et des priorités L’appui sur T2 va augmenter en 2008

Activités: Tâche 3 Développement de programme d’enquêtes améliorés, et collecte de données. Restreint à quelques pays (budget) Définition de programmes d’enquêtes modulaires, alignés sur des priorités clairement définies Collecte de données: complète autres sources Mise en oeuvre: Outils Cadre d’Evaluation de la Qualité des Enquêtes * ; Base de Questions * Mise en oeuvre: Progrès Niger: soutien pour AT et suivi qualité Haiti: à venir Survey programs are often donor driven Timing not optimal Data gaps in some cases, duplications of activities in others The available funding does not allow us to play an important role here. The ADP will only provide support when there is an urgent need to complement other sources of funding to ensure quality (e.g., finance only the international TA in Niger ENBC; complement funding by IADB and French Cooperation in Haiti; financing completion of data editing and analysis in DRC 1-2-3 survey)

Mise en oeuvre: constats La production de micro-données est conséquente mais ‘cachée’ La diffusion est limitée et ad-hoc. Obstacles : Legaux (confidentialité) et politiques Budgétaires (rarement budgétisé par les bailleurs) Mais aussi techniques et “psychologiques” (peur de la contradiction, pas d’intérêt - feedback des utilisateurs Grosse demande des pays pour: Outils techniques (Toolkit, NADA, anonymisation) Recommandations politiques (confidentialité, diffusion) Formation Gros impact sur la qualité des futures enquêtes

Mise en oeuvre: constats Beaucoup d’externalités positives : Coopération sud-sud (CMR->DRC, UGA->KEN RWA, NIG->BKF, etc. ) SWAP Mise en oeuvre des SNDS Lien avec d’autres initiatives / programs : DevInfo MICS Peu cher Contraintes: capacités nationales (temps) De + en + de pays et de partenaires

Merci www.surveynetwork.org/adp