News from the Granting Agencies / Nouvelles des agences subventionnaires NSERC/CRSNG CAGS / ACES 51st Annual Meeting / 51 e Congrès annuel Montréal, November 3-6, 2013

Outline / Aperçu Program Evaluations – Canada Graduate Scholarships (tri-agency) – PGS/IPS/CREATE Outcome of consultation on the Postdoctoral Fellowships Program Évaluations de programme – Bourses détudes supérieures du Canada (tri-agence) – BES/BESII/FONCER Résultat de la consultation concernant le programme de bourses postdoctorales 2

Program Evaluations - Purpose To meet the requirements of the Federal Accountability Act and the Policy on Evaluation – Every program has to be evaluated every five years To provide insightful, actionable findings, about program performance and relevance To provide recommendations for future improvements 3

CGS Evaluation - Timelines Design report approved by relevant advisory committees Evaluation Divisions are proceeding with the implementation of the data collection phase Expected completion date is summer 2014 Evaluation will cover fiscal years to – Build on the findings from the last evaluation (2008) – Evaluation questions developed in consultation with program staff and external stakeholders – Multiple lines of evidence will be used to ensure robust and credible findings 4

Évaluation des BESC - Échéancier Rapport desquisse approuvé par les comités consultatifs Les divisions dévaluation des agences préparent les méthodes de collecte de données Date prévue pour compléter lévaluation: mai 2014 Lévaluation couvrira les années fiscales à – Considération des résultats de lévaluation précédente – Questions clés développées en consultation avec le personnel des agences et la collectivité – Méthodes dévaluation multiples afin dassurer des résultats crédibles et solides 5

CGS Evaluation - Objectives Assess the outcomes of CGS recipients in comparison to non-recipients Assess the longer term impact of the CGS program through a trajectory analysis – The extent of the programs contributions to a required supply of HQP Identify « unexpected » outcomes to help determine the programs suitability – Inform decisions about what the « expected » outcomes should be – Compare data to identify any differences across the three agencies 6

CGS Evaluation – Objectives (cont) Determine if changes occured in CGS recipients experiences since that last program evaluation Identify lessons learned from agency-specific operations of the program to inform harmonization project Coordinate the survey data collection with the on- going evaluations of other agency training programs – To ease the load of potential key informants 7

Évaluation BESC – Objectifs (cont) Déterminer si des changements ont eu lieu dans lexpérience des récipiendaires depuis la dernière évaluation Apprendre des différents mécanismes de livraison du programme des agences subventionnaires afin dinformer le processus dharmonization Coordonner la collection de données avec les autres évaluations présentement en cours – Afin de diminuer la charge des répondants 8

CGS Evaluation – Key Questions To what extent is the program achieving its expected immediate outcomes? – Incentives for enrolment in graduate studies – Incentives for students to complete studies in a timely fashion To what extent is the program achieving its expected intermediate outcomes? – Increased numbers of students completing studies – High quality research training; attract and retain What are the outcomes achieved by recipients? – Research output, career progression, etc. 9

CGS Evaluation – Key Questions (cont) Is the program delivered in a cost-effective manner? – Are there any best practices that could inform the harmonization project? Does the program remain relevant? Is there a continued need? Does the program align with roles, responsibilities and priorities of government? 10

CGS Evaluation – Discussion/Feedback How has the graduate education landscape evolved in the last five years? Are there emerging issues in graduate education in Canada that have bearing on the current evaluation? Comment lenvironnement pertinent aux études supérieurs a-t-il évolué dans les cinq dernières années? Quels sont les enjeux qui pourraient être pertinents pour lévaluation en cours? 11

CREATE/PGS/IPS Evaluation At a much earlier stage than the CGS evaluation Evaluation will be combined and expected completion is for spring 2015 Evaluation objectives are similar to that of the CGS evaluation – Especially for PGS and IPS At the stage of determining what the evaluation questions will be – A stronger focus on CREATE As with CGS, a mix of quantitative and qualitative data gathering methods will be used 12

CREATE/PGS/IPS – Questions Draft questions are similar to the CGS evaluation questions The CREATE program does trigger different questions – To what extent are high caliber HQP participating in the CREATE, PGS and IPS programs? – To what extent does CREATE provide a viable option for providing both research and professional skills training? – To what extent does CREATE complement existing university initiatives? 13

Po stdoctoral Fellowships Issues Following a positive evaluation of the PDF program, recommendations and focus on three issues – Value of the fellowship (currently at $40,000) – Timing: application deadline and announcement of results – Application pressure Input was requested from stakeholders – CAGS and CAGS Board of Directors (C-TACC) – CAPS and CAPA – Engagement and Regional Visits Upcoming discussion at NSERCs Committee on Grants and Scholarships (COGS) 14

PDF – Deadline/Announcement of Results NSERC has committed to announce results of its PDF competition earlier, i.e. mid-January Most respondents agreed that October 1st was the most appropriate deadline – Earlier than October was deemed problematic given the workload associated with the start of the fall semester NSERC will have to become more efficient in processing applications and getting them to its review committees 15

PDF – Date limite et annonce des résultats Le CRSNG sest engagé à annoncer les résultats du concours des postdoc plus tôt La plupart des répondants préfèrent une date limite du 1 er octobre – Plus tôt quoctobre comporte des difficultés liées au début du semestre dautome Le CRSNG devra être plus efficace avec le traitement des demandes ainsi quavec leur transmission aux comités de sélection 16

PDF – Value of the Fellowship PDF value is currently set at $40,000/yr NSERC was considering asking the university or supervisor to top up the award Majority opinion was that a top-up should not be mandatory – Average discovery grant not being high enough – Limiting financial capabilities of researchers and institutions – A top-up would undermine the purpose of the program, a prestigious award that offers independent training NSERC to work towards increasing the value? 17

PDF – Application Pressure NSERC reduced, from 2 to 1, the number of applications an individual could submit Rationale was to ensure a more efficient use of time for applicants and reviewers Following unfavourable feed-back, NSERC agreed to re-consider its decision There was no emerging trend in feedback received – Providing quotas to universities – Pre-screening of applications – Allow highly rated unsuccessful applicants to re-apply 18

Questions/Comments Questions/commentaires 19

Distribution des BESC par lorganisme Bourses détudes supérieures du Canada Bourses de maîtrise : $ (1 an) Bourses de doctorat : $ (jusquà 3 ans) Répartition des bourses par organisme CRSH : Maîtrise : 1 300, doctorat : CRSNG : Maîtrise : 800, doctorat : 800 IRSC : Maîtrise : 400, doctorat : 400 Suppléments pour études à létranger Michael Smith – Jusquà $ pour étudier 6 mois à létranger. Doit bénéficier dune BESC. 20 CRSH $CRSH $ CRSNG $CRSNG $ CRSNG : 80CRSH : 125 IRSC : 45 Nombre de bourses disponibles par année par organisme subventionnaire Maîtrise * Doctorat * IRSC $ IRSC $ *Dépenses annuelles totales liées aux BESC par organisme

Financement direct global pour la formation 21 IRSC Total du financement direct pour la formation (tous les niveaux : du 1 er cycle au postdoctorat) $ CRSNG Total du financement direct pour la formation (tous les niveaux : du 1 er cycle au postdoctorat) $ CRSH Total du financement direct pour la formation (tous les niveaux : du 1 er cycle au postdoctorat) $

Survol des budgets visant la formation * Ne comprend pas les bourses stratégiques des IRSC 22 CRSHCRSNGIRSC BESC M $BESC M $BESC M $ BESC D $BESC D $BESC D $ BESC – Total $BESC – Total $BESC – Total $ Suppléments Michael Smith $ Suppléments Michael Smith $ Suppléments Michael Smith $ CRSH – Doctorat $ B. détudes sup. – Maîtrise $BEDE $ B. détudes sup. – Doctorat $ B. détudes sup. – Total $ Vanier $Vanier $Vanier $ Banting $Banting $Banting $ BRPC $ ESII – Maîtrise $ ESII – Doctorat $ ESII – Total $ BP $BP $BP $ Total – fonds directs $Total – fonds directs $Total – fonds directs $ FONCER (indirect) $ ISFRS (indirect) $ Étudiants recevant indirectement de laide par des subventions (2 500) $ Étudiants recevant indirectement de laide par des subventions (18 500) $ Étudiants recevant indirectement de laide par des subventions () $ Total global $Total global $Total global $ * Ne comprend pas les bourses stratégiques des IRSC