Measuring Program Impact
The evaluation and analysis of program quality, impact and outcomes is an area that many organisations recognise is important but few are able to put the resources into doing it thoroughly. We are committed to demonstrating the impact our programs on participant behaviour and organisational outcomes. We have designed a comprehensive, customisable, evaluation methodology that we recommend for this program.
Our evaluation process is supported by our own behaviour change technology platform (Able) and is semi-automated to ensure consistent measurement and continuous improvement of program outcomes. Our evaluation methodology and platform is based on best-practice research to ensure it captures information on the most impactful aspects of a program, and is aligned with Kirkpatrick's learning evaluation model.
Measuring Behaviour Change
We recommend including pre- and post-program measures to demonstrate how your program has improved the leadership & management skills of participants, based both on participant self-assessment and feedback from others (eg. managers and/or peers). The inclusion of pre and post-program feedback is a particularly good way to measure tangible behaviour change.
On a recent program participants developed their leadership skills from just above the average leader to rank in the top 21% of leaders & managers, based on our benchmark of over 10,000 ratings. A 22% improvement, from ranking in the 57th percentile to the 79th.
Program Evaluation Process
Our evaluation process includes the following components:
Pre-Program Participant Questionnaire: prior to the first session we onboard participants with a pre-program questionnaire to help assess participant needs, skills and priorities.
Post-Session Participant Evaluation: our system will email participants a short survey after each session to collect feedback about the session quality, impact, and facilitator. We also use this information to continuously improve the program in an iterative way.
Post-Program Participant Evaluation: our system emails every participant a short survey in the month after the program has been completed. The survey collects participant feedback on quality scores for programs components; program strengths & areas for improvement; and self-assessed impact.
Pre- & Post-Program Integral Leadership 90º Profile (option): we have included a pre- and post-program feedback process to measure & benchmark participant capability development. The ILP collects feedback from participants; their manager (90º); and has an option for peers & direct-reports (360º).
Customised ILP - Aligned to Cert-4 Qualification (option): we have included an option to align the ILP to the Certificate-IV Leadership & Management qualification competency framework. This is an efficient way of gathering practical evidence on participant leadership & management skills for assessment.
Post-Program Summary Report (option): we have included an option to provide a summary program report. The report summarises data from the Program & Session Evaluations and the Leadership Profile. The report shows participant feedback on: quality scores for program components; program strengths & areas for improvement; self-assessed program outcomes and impact of the program on participant behaviour (along with comparison to our norms). It also helps identify areas for further development and we include recommendations on specific next steps.
Post-Program Review Meeting (option): Include as part of our standard program delivery and evaluation we incorporate a post program debrief meeting with the Program Lead to discuss the Post-Program Summary Report, program or participant issues and recommendations