Introduction to Program Evaluation
Dates:
February 20, 2012 - February 24, 2012
Instructor:
Dr Delwyn Goodrick This five day course is designed for public sector workers and academics who are interested in commissioning, managing or conducting evaluations of public policy or programs.
The course is structured to progress participants through the stages of an evaluation. Participants are exposed to key theories and approaches to evaluation and then apply these to real examples of evaluations, from negotiation and planning through to data collection and final reporting and dissemination of findings. Examples from published studies and the facilitator’s applied practice in evaluation will be drawn upon to illustrate and reinforce key concepts.
Prerequisites:
This course assumes a basic understanding of methods of data collection, including surveys, focus groups and other group methods, interviews, document analysis, and data retrieval for secondary data.
Learning Objectives:
This five day short course is interactive and participatory. Participants are encouraged to bring their own examples and work through issues throughout the week. A set of handout notes will be provided to participants and case exemplars reviewed in class sessions.
Monday
Program Evaluation: What is it? How does it differ from social research? The role of valuing in evaluation. Key Evaluation types and approaches. The roles of internal and external evaluation. Key evaluation design considerations. Fostering the use and influence of evaluations. Using evaluation models to inform practice – social programming, valuing, knowledge construction, use and influence, and practice.
Tuesday
Planning Evaluation: Working with Key Stakeholders. The role of program logic and program theory in evaluation: What is program logic and when is it helpful? How to facilitate a program logic mapping session. Program theory for complicated & complex social programs. From initiation to final evaluation brief (or plan).
Wednesday
Designing an evaluation plan: Key components of an evaluation plan. The importance of identifying key audience information needs, and evaluation scope. Budgeting for an evaluation. Defining a limited set of key evaluation questions (KEQs). Managing an evaluation: some principles and tips for practice. Designs relevant for evaluation of emergent and/or multifaceted public sector programs (complex and complicated programs). Types of Evaluation: Process and Outcome evaluation approaches. The roles of experimental, quasi-experimental, non-experimental and qualitative designs, including case studies.
Thursday
Data Collection and Analysis/Major methods of data collection. Topics to be reviewed will include interviews, Most Significant Change technique (MSC), photographs and other visual methods, focus groups, nominal group technique and Delphi technique, Q sort, surveys, and goal attainment scaling. The role of mixed methods in evaluation – Strategies to integrate qual and quant methods.
Analysis principles and approaches will be emphasised in Days 4 and 5 in relation to major methods addressed.
Friday
Reporting an evaluation and supporting use and influence. The 1, 5, 25 rule. Considerations in generating valid claims. Issues of rigour and relevance. The role of program evaluation standards and core competencies in evaluation. Disseminating evaluation findings and influencing the use of evaluation. Practical tips to enhance the value of evaluation. Building a culture of evaluative thinking into public sector agencies – the role of monitoring and evaluation frameworks.
References:
Patton, M.Q. (2008). Utilization-focused evaluation (4th ed). Thousand Oaks, CA: Sage.
Torres, R.T., Preskil, H., and Piontek, M.E. (2005). Evaluation strategies for communicating and reporting: Enhancing learning in organizations (2nd ed). Thousand Oaks, CA: Sage.
