Leadership Education and Assessment

Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 Leadership Education and Assessment: A Developmental Approach Douglas R. Lindsay, Ph.D. Lieutenant Colonel and Assistant Professor Craig A. Foster, Ph.D. Associate Professor Robert J. Jackson, Ph.D. Associate Professor Anthony M. Hassan, Ed.D. Major and Assistant Professor Department of Behavioral Sciences and Leadership United States Air Force Academy, CO douglas.lindsay@usafa.edu craig.foster@usafa.edu robert.jackson@usafa.edu anthony.hassan@usafa.edu Abstract The number of leadership education and development programs has increased substantially over the past few decades. However, deliberate assessment strategies aimed at understanding actual student development have not kept pace. The primary reason for this limitation likely involves the challenges that are associated with this type of assessment. When examining leadership one is not only interested in the mere acquisition and retention of knowledge, but the actual application and practice of such knowledge. There are a host of challenges that stand in the way of such assessment. In the present paper we call attention to several of these challenges in an effort to understand what effective leader education assessment could look like. Additionally, we offer two examples of how intentional assessment strategies can be implemented to evaluate the effectiveness of leader education and development. Introduction Leadership education and development in academia is a challenging pursuit. Unlike many traditional disciplines, leadership educators are faced with a personal development process that has both individual and organizational level implications. This dynamic creates several unique challenges. First, leadership education is analogous to hitting a moving target. Not only are we trying to educate and develop individuals in the area of leadership, but students are 163 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 developing due to the normal experiences they face on a day to day basis. Because students are changing during our course based on expected transitions and what is happening outside of the classroom, the classroom experience may actually influence very little of that change process. In addition, due to varied experiences prior to attending the educational course (e.g., personal experience with good or bad leaders), students may receive the content in fundamentally different ways. For example, one’s experience with a bad leader (especially a toxic one) can affect their perceptions of effective leadership and thereby influence their attitude toward the course material (Reed, 2004). This construal process at the student level has implications for the second challenge in leadership education. Individuals do not come into a leadership course with clean slates regarding their perceptions about the topic of leadership (DiPaolo, 2008). In fact, when students enter leadership education, they have already started to develop what have been termed implicit leadership theories (Lord, Foti, & DeVader, 1984). These theories are implicit because they “are constructions by people that reside in the minds of these individuals. Such theories need to be discovered rather than invented because they already exist, in some form, in people’s heads” (Sternberg, 1985, p. 608). In a leadership context, implict theories are the assumptions or beliefs that an individual holds regarding the characteristics of effective leaders (e.g., Eden & Leviatan, 1975; Offerman, Kennedy, & Wirtz, 1994; Yukl, 2002). This represents one lens through which students will view leadership education. Accordingly, different students will have different interpretions of any given leadership eductation program. These first two challenges combine to form a final overarching challenge. Leadership education is clearly only one part of leader development. The academic study of leadership is crucial for students to communicate articulately about leadership and related constructs. However, rarely is leadership education targeted at the right time and the right place for the individual student. Often, individuals take a leadership course as part of a larger academic or individual development program. Even within leadership degree programs there is considerable variance as to when individuals will take different leadership courses (Brungardt, Greenleaf, Brungardt, & Arensdorf, 2006). Consequently, an individual is somewhat limited to when he or she can actually take the course and it may not align developmentally at the most appropriate time for that particular individual. As a result, leadership courses often have a one size or time fits all mentality. These challenges are relevant because they influence every aspect of leadership education from curriculum development to experiential activities. However, they become most critical in the area of assessment. Given the myriad of developmental starting points, ILTs, and experiences that students bring into the educational setting, in addition to the finite period of time that leadership courses 164 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 operate within (e.g., semesters), how does one go about measuring the effectiveness of leadership education? This is a primary question we address in the following pages. First, we examine several barriers to leadership assessment. Second, the issue of what should actually be assessed is considered. Finally, two examples of programs that have addressed these barriers will be presented as illustrative of ways to effectively integrate assessment into leadership education. The Challenge of Assessment Individual Considerations As alluded to, individuals approach leadership education with idiographic perspectives. Many factors at the individual level will influence the ultimate impact and subsequent assessment of such an educational program. Therefore, it is critical to consider an individual’s self-evaluation strategies as they relate to their leadership education because it is through this self-assessment that they will interpret and apply the educational material. Fundamental to this discussion are student perceptions regarding the “learnability” of leadership. In other words, do students believe that leadership is something that can be learned or improved? Students typically possess a range of views regarding this flexibility or malleability of leadership. On the one hand, some students endorse the view that leadership ability is fixed (e.g., you either have it or you don’t). Under this perspective there would be little that one could do to improve one’s ability regarding leadership. On the other hand, some students feel that leadership can be developed (e.g., it is malleable) through education and experience. It is not difficult to surmise that students that endorse fixed views toward leadership development may have different reactions to leadership education than do students who maintain more malleable views. Foster and Lindsay (2008) showed that these different views play an essential role in leader development. In three studies they found that fixed views regarding leadership were negatively associated with mastery goals, effort toward individual leadership development, interest in their own leader development, and motivation toward leadership-based programs. They concluded that even the most potent leader development effort might stand little chance against participants who believe that true leader development is not feasible. While these fixed views may be held by some, there is extensive evidence that individuals can, in fact, improve their leadership ability (e.g., Avolio, et al., 2005; Burke & Day, 1986; Collins & Holton III, 2004). The major implication here is that a one-size fits all approach to leadership education and development might not be the most effective strategy and those who are “forced” to participate in such leadership opportunities could create negative outcomes for the broader organization by elevating cynicism, turnover, and other counterproductive behaviors. Thus, assessments considering 165 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 fixed versus malleable views could help explain varying results about the effectiveness of a leader education course (Foster & Lindsay, 2008). In addition to perceptions about the fixedness of leadership there are the different individual perspectives that people have about themselves. For example, individuals are motivated to perceive themselves positively. One way this is done is through a process of self-enhancement, which is the desire to seek information and interpret information in a way that is favorable to the self (Sedikides, 1993). Another way is through a process related to self-enhancement that is referred to as the self-serving bias. This is the tendency to attribute favorable information about the self to internal reasons, but to attribute unfavorable information to external reasons (Mezulis, Abramson, Hyde, & Hankin, 2004). Instructors sometimes witness this process in students who view “A” work as a reflection of personal intellect, but view “D” work as a reflection of a bad situation. In addition, selfenhancement is closely linked to positive illusions which include the tendency to see oneself in an unrealistically positive manner. For example, when asked to compare personal levels of attractiveness to other group members, a strong majority of individuals will say they are at or above average (Taylor, 1989) despite the logical impossibility of this outcome. To summarize, individuals often view themselves more favorably than other people view them, and individuals will interpret information to protect these positive self-perceptions. In fact, individuals might be particularly prone to positive illusions in the leadership domain for a few reasons. First, leadership is difficult to measure. Information about one’s leadership ability is often ambiguous and developing leaders can interpret such ambiguous information in a self-enhancing manner. Second, leadership is often important to one’s selfconcept. Many people want to be great or at least decent leaders. Thus, when individuals receive negative information pertaining to their leadership, it can be more threatening than it would be for characteristics that are more peripheral to the self-concept. Third, leadership ability is a relatively global characteristic. Leadership essentially entails understanding and motivating other people. Because this is a relatively omnipresent endeavor, individuals might be particularly defensive about their abilities, as opposed to a more narrow skill. Fourth, by virtue of their leadership positions, leaders may receive consistently inaccurate feedback about their effectiveness (Church, 2000). Together, these issues clarify why many developing and existing leaders may have limited true awareness of their own leadership abilities. What Should Be Assessed? To remedy these developmental roadblocks, leadership educators must first address which attributes should be assessed. There are several factors to consider. The first involves the outcome to be measured. Is the focus of interest student 166 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 learning, individual development, individual performance, or organizational impact? Each of these outcomes will influence the assessment strategy and the strategy must align itself with outcomes. For example, if one is solely interested in knowledge retention of the individual, then one could use a test of some sort to examine the knowledge that was learned and subsequently retained (e.g., quiz or exam). If one is looking at examining actual behavioral change, then a different assessment strategy is necessary. Each leadership education intervention should have intended outcomes and these outcomes will help shape this assessment process. One useful categorization relevant to leadership education that assists in aligning assessment and outcomes is Bloom’s (1956) taxonomy. This framework separates the learning process into six primary categories: (a) remembering, (b) understanding, (c) applying, (d) analyzing, (e) evaluating, and (f) creating (Anderson & Krathwohl, 2001). Through the use of these categories an assessment strategy can be developed that is aligned with a particular outcome. For example, if leadership education is part of an overall leader development program, diligence should be paid to how this education is assessed so that it fits into the broader organizational plan as opposed to an independent activity with independent outcomes. Therefore, leadership education could be used to provide the requisite knowledge that could then be coordinated with an experiential setting where the individual is required to apply that knowledge in particular scenarios. The combined influence of leadership education and leadership application would be identified through a unified assessment goal of individual development. Knowledge for knowledge sake does not necessarily benefit the individual or the organization, but knowledge and application could be enhanced when personal leader development is part of a broader developmental program that extends well beyond the classroom. Using Bloom’s (1956) taxonomy, one can develop assessment methods that link multiple learning outcomes. Aligning the education and application pieces provides the basis for creating a transformational effect on the individual. This approach calls attention to what is often referred to as the ScientistPractioner divide (Murphy & Saal, 1990). Many educational programs (industrial/organizational psychology, clinical psychology, etc.) have attempted to bridge this divide by taking the approach that through the integration of education and application, students develop competency in both (Lindsay, Tate, & Jacobs, 2008). Of relevance here is that that both the educators of leadership (science) and the recipients of those being educated (practioners) have valuable input into determining the assessment strategy for leadership education programs. While education is not equivalent to training, the two are clearly interrelated), therefore one way to address this question is to apply Kirkpatrick’s (1994) model that examines four levels of outcome evaluation for training programs. Kirkpatrick’s model has been applied in educational contexts due to its straightforward 167 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 approach to evaluating learning. The four levels are (a) reaction, (b) learning, (c) behavior, and (d) results. Not surprsingly, the most basic outcome measure is reaction which is typically assessed by asking individuals of their opinions of the experience right after completion. This is a simplistic process and does not really examine how much the individual has learned more fundamentally. To do so, one must progress to higher levels of the model. Again, it is important to determine the level at which the outcomes are to be measured for the leadership education program. Is mere knowledge retention (Level 2) of interest or is the purpose to see if the education actually resulted in behavioral change (Level 3) and subsequent results (Level 4)? The answer to this question will drive not only the assessment strategy, but also the development of the entire educational program. The Synergy Between Academics and Experience It follows then that the challenge of assessing leadership education is multifaceted. In fact, the presence of this special issue of the Journal of Leadership Education is a testament to this challenge. Due to the complexity of leadership, it necessitates that one consider the design of the educational program, the timing in which the education takes place, the individual in that program and their individual differences, and the outcomes that are imperative to this process. While there are many different ways to approach this dilemma, an integrative approach is offered as one method. This approach not only incorporates the issues suggested above, but also considers how the education is incorporated into a personalized leader development program. The following provides two different examples of how such an approach has been used in order to examine the assessment of leader education. Both examples involved separate but interreleated leadership development efforts at the United States Air Force Academy in Colorado Springs, Colorado. United States Air Force Academy (USAFA) USAFA was established in 1954 as an avenue by which the Air Force could develop members using an academic experience that was tailored to their chosen profession. Indeed, to this day, graduating cadets overwhelmingly enter the United States Air Force as active duty members and serve a minimum of five years or more depending upon their respective career fields. The vision of USAFA is to be the Air Force’s premier institution for developing leaders of character. In fact, the entire USAFA experience is designed to facilitate growth in this area. This is conducted through both an academic and experiential process that increases in responsibility and scope as cadets progress through USAFA. The academic component is conducted by the Dean of the Faculty and relevant academic departments. The experiential component is provided primarily 168 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 by military experience through specific jobs that have been developed in each of the cadet squadrons (organizations). Through this process a combination of academic and a military experience, cadets are exposed to a four-year leader education and development program that is focused at the personal, interpersonal, team, and organizational levels. Core Leadership Course On the academic side, the Department of Behavioral Sciences and Leadership (DFBL) is responsible for providing the relevant leadership theory and concepts to all cadets. This is achieved through a required academic course completed by cadets during their Junior year. This specific time period is targeted intentionally because cadets are transitioning into leadership roles in their respective organizations. The course goals include (a) understanding conceptually the behavioral science and leadership concepts that are fundamental to leader development, (b) improving cadet’s interpretation and analysis of various leadership situations, and (c) improving cadet’s ability to facilitate their own leader development. Like most academic leadership courses, these goals are partially accomplished through providing relevant academic theory in a classroom setting. However, the institution is designed to address the science-practice approach by creating ample opportunities to apply this leadership education in all aspects of life. At the heart of the course is the foundational belief that leadership can be developed and is a process that takes place over time. Therefore, a cadet’s journey toward lifelong development in leadership is facilitated through this course. The education that is provided is not a means to an end, but a component of a larger and intentional development process designed to create synergy between education and experience. This approach is consistent with Lord and Hall’s (2005) progression of leadership skill from novice to expert. Our goal is to take novice leaders (cadets), provide them information and the opportunity to apply the information to help them on their leadership skill progression. Within the course there are two primary methods of assessment. The first of these is through various tests that are designed to examine learning along all six levels of Bloom’s taxonomy. These tests account for 20% of the course grade. These are used to assess the degree to which academic material has been learned and retained. The second and more significant part is experiential, accounting for 80% of the course grade. This occurs through numerous activities (e.g., leadership application exercises, role-play, case studies, scenarios, etc.). In this portion of the course, the assessment strategy extends beyond knowledge and application, but into actual behavioral change at Kirkpatrick’s (1994) Level 3. This is achieved through a series of leadership application exercises that allow students to see themselves from various perspectives and how that information fits into their overall leader development. This is a process that takes place over the semester; cadets start with a thorough self-assessment examining individual factors (e.g., 169 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 personality) and then reflect on the assessment in order to understand such concepts as positive illusions (Roberts, 2008). Next, they solicit 360 degree feedback on their leadership ability from their superiors, peers, and subordinates (Foster & Law, 2006). They take this information and compare it to their own self-assessment to discover the similarities and differences that exist. Through this process they become aware of their own limitations in self-assessment and how these limitations impact those around them. Finally, they use their selfassessments, 360 degree assessments, and the course material to synthesize their own individual leader development plan for their remaining time at USAFA. This not only takes advantage of the formal leadership education they receive, but makes it real by evaluating how they are going to use the material on a day-to-day basis. In addition, this process positions cadets developmentally to take advantage of other leadership opportunities (e.g., squadron positions, clubs, community events). An important component is that this is all conducted under the guidance of the instructor who serves not only as a teacher of the course material, but in a mentor or coach role for the cadets by providing them developmental feedback throughout the process. The feedback to date on this approach has been positive on the experiential assessments. As we typically find, our students were not excited about the objective assessments. However, they enjoyed the applied assessments where they were able to apply the material to their own lives. This information has been used to adjust the objective (20%) versus the experiential (80%) weighting in the course. One challenge is to continue to ensure that the experiential portions of the course correspond to their current life experience. For example, we often have them project themselves several years into the future. These scenarios are being restructured to apply to their current leadership challenges. Master’s Program In addition to individual cadet education and development, USAFA is also concerned about further developing the officers (commanders) who lead USAFA cadets. This objective is addressed primarily through the Air Officer Commanding (AOC) Master’s Degree Program. This program also recognizes that a paradox of leader development for the novice and expert leader is that self-awareness is a critical component for change, yet, as addressed, there are numerous limitations to accurate self-understanding. To support leaders in overcoming these limitations, we provide structure to these data gathering and data interpretation shortcomings. First, we orient leaders to the framework proposed by Hogan and Kaiser (2005) including (a) who you are is how you lead, (b) how you lead drives team functioning, (c) employee attitudes, and ultimately (d) organizational performance. If who you are is how you lead, then it is important to expand self-awareness about basic personal attributes, to 170 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 include personality and competencies and styles related to leadership. Thus, phase one of our leadership assessment involves measures of individual differences. The phase one assessment supports self-awareness by having leaders complete multiple inventories. Beginning with personality, commanders complete a measure based on the “big five” taxonomy (Smith & Canger, 2004). This is followed by discussions of the relevant literature as personality has been found to be related to leadership effectiveness and emergence (Judge, Bono, Ilies, & Gerhardt, 2002) and transformational leadership (Bono & Judge, 2004). Further, supervisor personality has been linked to various dimensions of subordinate behavior (Smith & Canger, 2004). This integration offers a useful examination of how a leader’s tendencies drive their leadership style and it is augmented by examining another dimension of personality often described as the “dark side” (Hogan & Hogan, 2001; Hogan & Kaiser, 2005). The emphasis on dark side assessment is to reveal probable vulnerabilities while under stress, in situations where the person is not self-monitoring, and when the leader is in familiar circumstances. This evaluation is also important, since these dark side features have been addressed in the context of derailment and supervisor-generated stress on subordinates. Whereas good measures of specific leadership competencies are somewhat difficult to find, we use a measure of emotional intelligence (EI) to address many of the interpersonal competencies that are often categorized as “soft skills” and present interpretive information in conjunction with appropriate articles. The topic of EI strongly relates to leadership style, so it serves as a nice pivot point between “who you are is how you lead” and the impact of leadership, which is how leadership style influences various results. This influence is the focus of phase two where we examine perceived leader behavior and impact on the organization. The assessment tool for this phase is a multifaceted inventory that includes questions oriented to specific unit functioning at the USAFA which includes leadership style (e.g., Bass & Avolio, 1997), multiple dimensions of organizational citizenship behavior, and organizational commitment. This instrument is administered to students and subordinates (e.g., cadets) so it provides multi-source feedback. Leaders receive feedback from this assessment in two of the program evaluation themes addressed by Kirkpatrick (1994) – feedback on their leadership behaviors (Level 3) and on their results (Level 4) as leaders. The first theme includes feedback around leadership style and the components of transformational leadership. In addition, leaders receive feedback from their subordinates in an aggregated fashion that not only reflects the general climate within the unit, but also provides information about specific organizational practices (e.g., communication, feedback and mentoring), organizational citizenship behavior in the form of supporting others, demonstrating loyalty, and how subordinates feel in terms of turnover intentions and general morale. Further, the commanders are given information about the relationship between leadership style and organizational impacts. Commanders 171 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 are also supported in a coaching context to use the results to adapt their leadership style and make appropriate adjustments to enhance their unit climate, and ultimately, their level of self-awareness and leadership effectiveness. A current challenge to this program is ensuring that many of the follow up assessments that are completed are valued by the organization versus just at the individual level. There are several positive leader development indicators based on data from the masters program. First, nearly all students rate the academic material in the classroom as highly related to the specific roles and responsibilities they will have as leaders. This finding addresses the science-practice divide. Second, the vast majority of students rate themselves as highly motivated and confident to begin their leadership duties. Thus, they have demonstrated success in the cognitive domain by mastering their coursework and they are appropriately oriented in the affective domain. Third, there is cross-sectional evidence of progression in leadership style because commanders in their second year are rated by subordinates as significantly more transformational than first year commanders. Conclusion Leadership education and development is multifaceted and the responsibility for developing leadership does not clearly fall within one program or one academic department. Accordingly, program assessment must reflect this complex dynamic. In fact, leadership education should be assessed deliberately in the context of the broader leader development program. While the USAFA may be at an advantage in that the institution is set up to not only educate, but also develop leaders, this does not mean it cannot take place in more traditional academic settings. The key is to find a way to make the material relevant to the current context in which the student is involved. This will likely take a little more time for the instructor, but the payoff in terms of leadership development are well worth the effort. While several challenges to this endeavor have been previously discussed, it is possible that a carefully constructed process can leverage all of the strengths and resources that are present at a given institution. This process follows individuals true challenges in future leadership situations where individual success in a leadership position is not just dependent upon what they bring to the situation, but also by how the situation influences the enactment of leadership. This sentiment was reinforced by Stech (2008) when he suggested that “it would seem that the ideal way to create good leaders would be to devise a program in which education, training, and development processes take place.” (2008, p. 45). Several examples of such intentional processes were offered as a starting point of where one could begin to tackle this challenging enterprise. In conclusion, while different parts of the organization may have the responsibility for managing the specific pieces of leadership education and development, ultimately, integrated 172 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 and aligned organizational processes are crucial for the deliberate development and assessment of leadership education. 173 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 References Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom's Taxonomy of educational objectives: Complete edition. New York: Longman. Avolio, B., et al. (2005). Executive summary: 100 year review of leadership intervention research: Briefings report 2004-01, Gallup Leadership Institute. Kravis Leadership Institute, Leadership Review, 5(Winter), 7-13. Bass, B. M., & Avolio, B. J. (1997). Full range leadership development: Manual for the Multifactor Leadership Questionnaire. Palo Alto, CA: Mind Garden. Bloom, B. S. (1956). Taxonomy of educational objectives. The classification of educational goals. Handbook I: Cognitive domain. New York: McKay. Bono, J. E., & Judge, T. A. (2004). Personality and transformational and transactional leadership: A meta-analysis. Journal of Applied Psychology, 89, 901-910. Brungardt, C., Greenleaf, J., Brungardt, C., & Arensdorf, J. (2006). Majoring in leadership: A review of undergraduate leadership degree programs. Journal of Leadership Education, 5, 4-25. Burke, M. J., & Day, R. R. (1986). A cumulative study of the effectiveness of managerial training. Journal of Applied Psychology, 71, 232-245. Church, A. H. (2000). Do higher performing managers actually receive better ratings? Consulting Psychology Journal: Practice and Research, 52, 99116. Collins, D. B., & Holton III, E. F. (2004). The effectiveness of managerial leadership development programs: A meta-analysis of studies from 1982 to 2001. Human Resource Development Quarterly, 15, 217-248. DiPaolo, D. G. (2008). Echoes of leadership education: Reflections on failure, forgetting, and our future. Journal of Leadership Education, 7, 77-90. Eden, D., & Leviatan, U. (1975). Implicit leadership theory as a determinant of the factor structure underlying supervisory behavior scales. Journal of Applied Psychology, 60, 736-741. 174 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 Foster, C. A., & Law, M. R. F. (2006). How many perspectives provide a compass? Differentiating 360-degree and multi-source feedback. International Journal of Selection and Assessment, 14, 288-291. Foster, C., & Lindsay, D. (2008, September). Lay theories of leadership ability and leader development. Paper presented at the 50th Annual International Military Testing Association Conference, Amsterdam, NE. Hogan, R., & Hogan, J. (2001). Assessing leadership: A view from the dark side. International Journal of Selection and Assessment, 9, 40-51. Hogan, R., & Kaiser, R. (2005). What we know about leadership. Review of General Psychology, 9, 169-180. Judge, T. A., Bono, J. E., Ilies, R., & Gerhardt, M. W. (2002). Personality and leadership: A qualitative and quantitative review. Journal of Applied Psychology, 87, 765-780. Kirkpatrick, D. L. (1994). Evaluating Training Programs: The Four Levels. San Francisco: Berrett-Koehler. Lindsay, D. R., Tate, B. W., & Jacobs, R. R. (2008). Practicum: A teaching tool to highlight the scientist-practitioner model. The Industrial-Organizational Psychologist, 45, 39-47. Lord, R. G., & Hall, R. J. (2005). Identity, deep structure and the development of leadership skill. Leadership Quarterly, 16, 591-615. Lord, R. G., Foti, R. J., & De Vader, C. L. (1984). A test of leadership categorization theory: Internal structure, information processing, and leadership perceptions. Organizational Behavior and Human Performance, 34, 343-378. Mezulis, A. M., Abramson, L. Y., Hyde, J. S., & Hankin, B. L. (2004). Is there a universal positivity bias in attributions? A meta-analytic review of individual, developmental, and cultural differences in the self-serving attributional bias. Psychological Bulletin, 130, 711-747. Murphy, K., & Saal, F. E. (1990). Psychology in organizations: Integrating science and practice. In K. Murphy & F. Saal (Eds.). Series In Applied Psychology (pp. 49-66), Hillsdale, NJ: Erlbaum and Associates. 175 Journal of Leadership Education Volume 8, Issue 1 – Summer 2009 Offerman, L. R., Kennedy, J. K., & Wirtz, P. W. (1994). Implicit leadership theories: Content, structure, and generalizability. Leadership Quarterly, 5, 43-58. Reed, G. E. (2004). Toxic Leadership. Military Review, 67-71. Roberts, C. (2008). Developing future leaders: The role of reflection in the classroom. Journal of Leadership Education, 7, 116-129. Sedikides, C. (1993). Assessment, enhancement, and verification determinants of the self-evaluation process. Journal of Personality and Social Psychology, 65, 317-338. Smith, M. A., & Canger, J. M. (2004). Effects of supervisor “big five” personality on subordinate attitudes. Journal of Business & Psychology, 18, 465-481. Sternberg, R. J. (1985). Implicit theories of intelligence, creativity, and wisdom. Journal of Personality and Social Psychology, 49, 607-627. Stech, E. (2008). Leadership education, training, and development: What should we be doing and what can we be doing? Journal of Leadership Education, 7, 43-46. Taylor, S. E. (1989). Positive illusions: Creative self-deception and the healthy mind. New York: Basic Books. Yukl, G. (2002). Leadership in organizations. Upper Saddle River, New Jersey: Prentice Hall. 176