Impact Evaluation of Grassroots Leadership Development

View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by ScholarSpace at University of Hawai'i at Manoa Hawaiʻi-Pacific Evaluation Association 2015 Annual conference (9/11/15) Impact Evaluation of Grass-Root Leadership Development (Roundtable) Yao Hill, UH Manoa yao.hill@hawaii.edu Impact Evaluation of Grassroots Leadership Development Assessment Leadership Building Project (ALBP) at UH Mānoa Context: - 230+ academic programs need to assess whether their graduating students have achieved program student learning outcomes (SLOs). In 2012, 73% of the programs reported assessment took place and 48% used the results. UH Mānoa Assessment Office’s charge: 100% of programs to conduct systematic, meaningful, and management program assessment of SLOs. UH Mānoa Assessment Office capacity: 2 full time faculty + 1 GA ALBP Goals and Outcomes Goal Cultivate a cadre of faculty to lead collaborative and meaningful program assessment and build a culture of assessment for improvement on campus. Learning outcomes 1. Assessment knowledge 2. Assessment skills 3. Assessment scholarship disposition 4. Facilitative leadership Skills Output Assessment Plans Impact Leadership Activities Scholarship Activities Change in Program 1. Engaged colleague partners 2. Lead collaborative projects 3. Mentored colleagues 1. Presentations 2. Publications Meaningful program assessment activities ALBP Model: Intensive multi-day AL Institute (ALI) Semester-Long follow-up support Features of ALI  Active learning and simulation of facilitating real program assessment activities  Guest speakers/presenters with real life examples  Individualized consultation + peer feedback  Networking opportunities with colleagues Status: 3 Cohorts since 2013, 34 participants from 30 programs and 13 colleges Poster presentation of project Hawaiʻi-Pacific Evaluation Association 2015 Annual conference (9/11/15) Impact Evaluation of Grass-Root Leadership Development (Roundtable) Yao Hill, UH Manoa yao.hill@hawaii.edu ALBP Evaluation Evaluation Target 1. Assessment Knowledge 2. Assessment Skills 3. Scholarship Disposition 4. Facilitation Skills Assessment Plans Leadership Activities Scholarship Activities Changes in Program Assessment Status Methods Results ALI Learning Outcomes Institute Evaluation 94% reported being able to Questionnaire distinguish good assessment practices from poor ones. [n = 34] Institute Evaluation 97% reported being able to utilize at Questionnaire least 3 assessment strategies/tools [n = 33] Assessment plan 100% planned to employ at least 4 content analysis assessment strategies/tools [n = 34] Institute Evaluation (1) 97% of the participants reported Questionnaire being able to locate resources on learning assessment? [n = 34] (2) 53% plan to conduct research or other forms of scholarly activity on learning assessment. [n = 34] Institute Evaluation 94% reported being able to utilize at Questionnaire least 3 facilitation techniques. [n = 34] Assessment plan 94% planned collaborative activities content analysis to engage faculty [n = 34] 65% explicitly planned to utilize facilitation techniques [n = 34] Output Expert judgement 100% meaningful and implementable assessment plans [n = 34] Impact Self-report and 91% engaged collaborative activities observation in their programs [n = 22] Documentation - 95% presented posters at UHM Assessment Poster Exhibit [n = 22] - 1 peer-reviewed article published Benchmark Self-report and observation 50 26 programs advanced their assessment activities 90% 90% 100% 3 strategies 90% 40% 90% 90% 75% 100% 80% 90% 10 articles Further Impact Evaluation Possibilities - Develop mentorship: mentor at least one colleague to be the core faculty in program assessment. Develop network: rely on and help fellow ALBP participants in program assessment activities and scholarship endeavors Move the assessment needle and lead the completion of an assessment cycle Exercise greater influence: Impact assessment activities beyond one’s program/department Hawaiʻi-Pacific Evaluation Association 2015 Annual conference (9/11/15) Impact Evaluation of Grass-Root Leadership Development (Roundtable) Yao Hill, UH Manoa yao.hill@hawaii.edu Useful articles: Black, A. M. & Earnest, G. W. (2009). Measuring the outcomes of leadership development programs. Journal of Leadership & Organizational Studies, 16(2), 184-196. Hannum, K. M., Martineau, J. W., & Reinelt, C. (Ed.) (2007) The Handbook of Leadership Development Evaluation. Jossey-Bass.