Despite the increasing attention to social-emotional skills' development, few attempts have been ... more Despite the increasing attention to social-emotional skills' development, few attempts have been made to develop rubrics to formatively assess these skills in classrooms and evaluate their psychometric properties. This research aimed to develop rubrics for the social-emotional skill domains of Self-management and Openmindedness and assess their psychometric properties using Item Response Theory modeling. Two samples of students (Pilot study: N = 947 middle-school students; 54 % female; mean age = 13.8 years; Main study: N = 7404 middle-school students; 48 % female; mean age = 13.0 years) provided self-descriptions on socialemotional rubrics. Results showed that changes in rubrics' instructions, rating scale characteristics, and descriptors' content improved validity and reliability properties from the pilot to the main study, though there is still room for improvement. Overall, our research supports the use of rubrics for the assessment of Selfmanagement and Open-mindedness skills and provides recommendations for designing social-emotional rubrics in the school context.
Doren, and Carylanne Wolfington for providing access to some of the college and university sample... more Doren, and Carylanne Wolfington for providing access to some of the college and university sample data. For downloadable versions of the BFI-2 and its short forms, visit the Colby
People’s interest in personality is so pervasive that one encounters assessment instruments in th... more People’s interest in personality is so pervasive that one encounters assessment instruments in the most unlikely places. Recently, I happened to wander across a Spring Fair, an informal gathering of artists, craftspeople, cooks, and entertainers. One booth that piqued my interest was devoted to the “analysis of personality through handwriting.” Interested in the categories this handwriting analyst would find most important for personality description, I perused his hand-made “personality profile” which was on display at the booth. It consisted of a collection of about 60 trait-descriptive terms, such as assertive, energetic, generous, cooperative, perseverant, organizational ability, balanced, mature, imagination, creative, and analytical, which were grouped loosely under headings like Leadership, Economics, Behavior, Character, Emotions, and Thinking.
Journal of Personality and Social Psychology, May 1, 2022
People form relationships with people from their own racial groups, a phenomenon called racial ho... more People form relationships with people from their own racial groups, a phenomenon called racial homophily, which reduces interracial contact and exacerbates inequality and prejudice. Although viewed as arising from environmental factors, we argue that racial homophily also involves individual choice and, thus, personality factors. We address three major issues. First, are interpersonal concerns (Agreeableness) and intergroup concerns (Openness) differentially relevant to cross-race friendships? Second, are current conceptions of Openness sufficient, or do we need lower-level facets more attuned to intergroup concerns? Third, can we specify the interplay between personality and contextual factors in different settings? Across four studies (total N = 1,820), Agreeableness failed to predict more cross-race friendships, in both self- and peer reports, suggesting that interpersonal kindness was not sufficient to overcome racial homophily. In contrast, Openness and Openness to Other (O2, a new social facet of Openness) consistently predicted cross-race friendship. However, the O2 facet had the stronger and only unique effect, suggesting it is the "active ingredient." High-O2 individuals had an almost equal 1:1 ratio of same-to-different-race network members, whereas low-O2 individuals had 4:1 same-race. These results held for both college students and middle-aged adults, both friends and new acquaintances in the network, and both networks established before and at a diverse university. Finally, when moving to a more diverse environment, high-O2 individuals seemed to take advantage of the new environmental affordances, adding more different-race members to their networks. Overall, these studies advance understanding of person-environment transactions, showing how personality traits matter to the structure of people's social networks. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Educational practitioners have been increasingly interested in the use of formative assessment an... more Educational practitioners have been increasingly interested in the use of formative assessment and rubrics to develop social-emotional skills in children and adolescents. Although social-emotional rubrics are nowadays commonly used, a thorough evaluation of their psychometric properties has not been conducted. In this scoping review, we examine the knowns and unknowns of the use of formative assessment approaches and rubrics in social-emotional learning. We first review initiatives of formative assessment and development of rubrics to assess social-emotional skills. Then, we discuss challenges associated with the development and use of rubrics to evaluate social-emotional skills in youth focusing on 1) assessment of single skills versus assessment of a comprehensive taxonomy of skills; 2) developing rubrics' performance level descriptions that accurately describe increasing mastery of skills; 3) obtaining adequate internal consistency and discriminant validity evidence; 4) selfreports versus observer reports of skills; and finally 5) how to account for adolescents' development in the construction of rubrics. This review outlines a research agenda for the psychometric study of rubrics to be used in socialemotional skill assessment.
Journal of Personality and Social Psychology, Jul 1, 2020
Daily life is full of emotional ups and downs. In contrast, the objective conditions of our lives... more Daily life is full of emotional ups and downs. In contrast, the objective conditions of our lives usually remain relatively stable from day to day. The degree to which emotional ups and downs influence life satisfactionwhich prima facie should be relatively stableremains a puzzle. In the present paper, we propose the Individual Differences in Evaluating Life Satisfaction (IDELS) model to address this puzzle. The IDELS model posits that people differ in the processes by which they evaluate their life satisfaction: Some people's life satisfaction is more strongly associated with their current emotions (i.e., "emotion globalizing") whereas other people maintain a filter between their life satisfaction and current emotions. These individual differences should have important implications for the degree of short-term variability in life satisfaction and, in turn, for psychological health. In three diverse samples of women (total N = 536), we assessed life satisfaction and emotions daily or multiple times per day for two weeks. We tested four hypotheses derived from the IDELS model. First, participants differed substantially in the degree of short-term variability in life satisfaction, and these individual differences were moderately stable. Second, participants differed substantially in emotion globalizing, and these individual differences were moderately stable. Third, higher emotion globalizing predicted greater short-term variability in life satisfaction. Fourth, greater short-term variability in life satisfaction was associated with a maladaptive profile of greater neuroticism and worse psychological health. We discuss implications for life satisfaction theory and measurement.
This series of studies investigated whether the good psychometric properties of the English versi... more This series of studies investigated whether the good psychometric properties of the English version of the Big Five Inventory-2 (BFI-2) could be replicated using its Dutch adaptation. Second, it aimed to further examine the predictive validity of both the Big Five domain and the more specific facet scales of the BFI-2 in a large and representative sample. Results indicated that the structure found in the English version was replicated in the Dutch adaptation. The 60-item BFI-2 was reliable at the level of both domains and facets, as were the abbreviated versions. In terms of validity, the domain scales predicted a broad range of criteria. Examination of preregistered hypotheses regarding the discriminant validity of the facets indicated that experts were able to predict which facets would be most strongly associated with specific criteria. Overall, results confirm the strong psychometric properties of the BFI-2 Big Five domain scales and indicate that theoretically identified facets can be more valid predictors of criteria than other facets of the same domain.
Children's educational outcomes are strongly correlated with their parents' educational a... more Children's educational outcomes are strongly correlated with their parents' educational attainment. This finding is often attributed to the family environment-assuming, for instance, that parents' behavior and resources affect their children's educational outcomes. However, such inferences of a causal role of the family environment depend on the largely untested assumption that such relationships do not simply reflect genes shared between parent and child. We examine this assumption with an adoptee design in full-population cohorts from Danish administrative data. We test whether parental education predicts children's educational outcomes in both biological and adopted children, looking at four components of the child's educational development: (I) the child's conscientiousness during compulsory schooling, (II) academic performance in those same years, (III) enrollment in academically challenging high schools, and (IV) graduation success. Parental education was a substantial predictor of each of these child outcomes in the full population. However, little intergenerational correlation in education was observed in the absence of genetic similarity between parent and child-that is, among adoptees. Further analysis showed that what links adoptive parents' education did have with later-occurring components such as educational attainment (IV) and enrollment (III) appeared to be largely attributable to effects identifiable earlier in development, namely early academic performance (II). The primary nongenetic mechanisms by which education is transmitted across generations may thus have their effects on children early in their educational development, even as the consequences of those early effects persist throughout the child's educational development. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Proceedings of the National Academy of Sciences, 2020
SignificanceAre disagreeable individuals more likely to attain power than agreeable individuals? ... more SignificanceAre disagreeable individuals more likely to attain power than agreeable individuals? This question is important because highly disagreeable individuals in positions of power can do a lot of damage. For example, CEOs who are nasty and bullying create cultures of abuse and tend to lead their organizations to fail. In two longitudinal prospective studies, we found that disagreeableness did not predict the attainment of power. Selfish, deceitful, and aggressive individuals were no more likely to attain power than were generous, trustworthy, and nice individuals. Why not? Disagreeable individuals were intimidating, which would have elevated their power, but they also had poorer interpersonal relationships at work, which offset any possible power advantage their behavior might have provided.
Zusammenfassung. Die deutsche Version des Big Five Inventars 2 (BFI-2) erfasst die 5 Persönlichke... more Zusammenfassung. Die deutsche Version des Big Five Inventars 2 (BFI-2) erfasst die 5 Persönlichkeitsdomänen Extraversion, Verträglichkeit, Gewissenhaftigkeit, Negative Emotionalität (Neurotizismus) und Offenheit sowie insgesamt 15 Persönlichkeitsfacetten mit 60 Items. Das Inventar wurde im Rahmen eines mehrstufigen Übersetzungsprozesses vom Englischen ins Deutsche übertragen. Anhand einer nach Alter, Geschlecht und Bildungsabschluss quotieren Bevölkerungsstichprobe ( N = 1 224) wurden Reliabilität, Validität und Messinvarianz über Geschlechter und Altersgruppen untersucht. Die Ergebnisse der Studien deuten auf gute Reliabilitäten der Domänenwerte und ausreichende Reliabilitäten der Facettenwerte hin. Einzelitems und die Facettenwerte zeigen erwartungskonforme Ladungsmuster. Die Domänen- und Facettenwerte korrelieren erwartungsgemäß mit anderen Persönlichkeitsinventaren und sind mit Kriterien wie Bildungsabschluss, Einkommen, Gesundheit und Lebenszufriedenheit assoziiert. Strukturgle...
Journal of Personality and Social Psychology, 1996
Three replicable personality types were identified in a sample of 300 adolescent boys and shown t... more Three replicable personality types were identified in a sample of 300 adolescent boys and shown to generalize across African Americans and Caucasians. The types had conceptually coherent relations with the Big Five dimensions, ego resiliency, and ego control, and converged with three of the types identified by J. Block (1971). The behavioral implications of the types were explored using several independent data sources. Resilients were intelligent, successful in school, unlikely to be delinquents, and relatively free of psychopathology; Overcontrollers shared some of these characteristics but were also prone to internalizing problems; and Undercontrollers showed a general pattern of academic, behavioral, and emotional problems. This research demonstrates that replicable and generalizable personality types can be identified empirically, and that the unique constellation of traits defining an individual has important consequences for a wide range of outcomes.
A person's behavior and experiences can be described at different levels of abstraction. For exam... more A person's behavior and experiences can be described at different levels of abstraction. For example, a person might be described as charitable, as generous, as kind, or as good. Is there a level in such
Whereas the structure of individual differences in personal, social, and emotional attributes is ... more Whereas the structure of individual differences in personal, social, and emotional attributes is well understood in adults, much less work has been done in children and adolescents. On the assessment side, numerous instruments are in use for children but they measure discordant attributes, ranging from one single factor (self-esteem; grit) to three factors (social, emotional, and academic self-efficacy) to five factors (strength and difficulties; Big Five traits). To construct a comprehensive measure for large-scale studies in Brazilian schools, Ayrton Senna Institute is developing assessments mapping socio emotional skills on the conceptual framework of the five-factor model of personality. Here we present a study of internal structure of a 162-item inventory measuring 18 facet scales, representing two conceptual levels of measurement: one assesses the child’s current identity or self-concept and the other assesses self-efficacy beliefs. These 18 facets can be organized into five broad domains, similar to the Big Five domains: E: Engaging with others, A: Amity, WO Work orientation, ER: Emotional resilience, and O: Open-mindedness. The sample consisted of more than 30,000 participants, who were between 11 and 18 years old and attended grades 6 to 12. We split the sample in two random halves. We performed exploratory factor analysis of the 18 facet scales per layer with the first half and a confirmatory factor analysis with the second half of the sample. Internal consistencies were in the range of .70 to .80 for most scales. All facet scales had their highest loading on the expected domain. A bi-factor model best represented the self-efficacy level with a general factor plus five specific factors related with each domain. A confirmatory factor analysis showed a robust and replicable structure. Discussion focuses on the contribution to socio-emotional research in education and its measurement.
Despite the increasing attention to social-emotional skills' development, few attempts have been ... more Despite the increasing attention to social-emotional skills' development, few attempts have been made to develop rubrics to formatively assess these skills in classrooms and evaluate their psychometric properties. This research aimed to develop rubrics for the social-emotional skill domains of Self-management and Openmindedness and assess their psychometric properties using Item Response Theory modeling. Two samples of students (Pilot study: N = 947 middle-school students; 54 % female; mean age = 13.8 years; Main study: N = 7404 middle-school students; 48 % female; mean age = 13.0 years) provided self-descriptions on socialemotional rubrics. Results showed that changes in rubrics' instructions, rating scale characteristics, and descriptors' content improved validity and reliability properties from the pilot to the main study, though there is still room for improvement. Overall, our research supports the use of rubrics for the assessment of Selfmanagement and Open-mindedness skills and provides recommendations for designing social-emotional rubrics in the school context.
Doren, and Carylanne Wolfington for providing access to some of the college and university sample... more Doren, and Carylanne Wolfington for providing access to some of the college and university sample data. For downloadable versions of the BFI-2 and its short forms, visit the Colby
People’s interest in personality is so pervasive that one encounters assessment instruments in th... more People’s interest in personality is so pervasive that one encounters assessment instruments in the most unlikely places. Recently, I happened to wander across a Spring Fair, an informal gathering of artists, craftspeople, cooks, and entertainers. One booth that piqued my interest was devoted to the “analysis of personality through handwriting.” Interested in the categories this handwriting analyst would find most important for personality description, I perused his hand-made “personality profile” which was on display at the booth. It consisted of a collection of about 60 trait-descriptive terms, such as assertive, energetic, generous, cooperative, perseverant, organizational ability, balanced, mature, imagination, creative, and analytical, which were grouped loosely under headings like Leadership, Economics, Behavior, Character, Emotions, and Thinking.
Journal of Personality and Social Psychology, May 1, 2022
People form relationships with people from their own racial groups, a phenomenon called racial ho... more People form relationships with people from their own racial groups, a phenomenon called racial homophily, which reduces interracial contact and exacerbates inequality and prejudice. Although viewed as arising from environmental factors, we argue that racial homophily also involves individual choice and, thus, personality factors. We address three major issues. First, are interpersonal concerns (Agreeableness) and intergroup concerns (Openness) differentially relevant to cross-race friendships? Second, are current conceptions of Openness sufficient, or do we need lower-level facets more attuned to intergroup concerns? Third, can we specify the interplay between personality and contextual factors in different settings? Across four studies (total N = 1,820), Agreeableness failed to predict more cross-race friendships, in both self- and peer reports, suggesting that interpersonal kindness was not sufficient to overcome racial homophily. In contrast, Openness and Openness to Other (O2, a new social facet of Openness) consistently predicted cross-race friendship. However, the O2 facet had the stronger and only unique effect, suggesting it is the "active ingredient." High-O2 individuals had an almost equal 1:1 ratio of same-to-different-race network members, whereas low-O2 individuals had 4:1 same-race. These results held for both college students and middle-aged adults, both friends and new acquaintances in the network, and both networks established before and at a diverse university. Finally, when moving to a more diverse environment, high-O2 individuals seemed to take advantage of the new environmental affordances, adding more different-race members to their networks. Overall, these studies advance understanding of person-environment transactions, showing how personality traits matter to the structure of people's social networks. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Educational practitioners have been increasingly interested in the use of formative assessment an... more Educational practitioners have been increasingly interested in the use of formative assessment and rubrics to develop social-emotional skills in children and adolescents. Although social-emotional rubrics are nowadays commonly used, a thorough evaluation of their psychometric properties has not been conducted. In this scoping review, we examine the knowns and unknowns of the use of formative assessment approaches and rubrics in social-emotional learning. We first review initiatives of formative assessment and development of rubrics to assess social-emotional skills. Then, we discuss challenges associated with the development and use of rubrics to evaluate social-emotional skills in youth focusing on 1) assessment of single skills versus assessment of a comprehensive taxonomy of skills; 2) developing rubrics' performance level descriptions that accurately describe increasing mastery of skills; 3) obtaining adequate internal consistency and discriminant validity evidence; 4) selfreports versus observer reports of skills; and finally 5) how to account for adolescents' development in the construction of rubrics. This review outlines a research agenda for the psychometric study of rubrics to be used in socialemotional skill assessment.
Journal of Personality and Social Psychology, Jul 1, 2020
Daily life is full of emotional ups and downs. In contrast, the objective conditions of our lives... more Daily life is full of emotional ups and downs. In contrast, the objective conditions of our lives usually remain relatively stable from day to day. The degree to which emotional ups and downs influence life satisfactionwhich prima facie should be relatively stableremains a puzzle. In the present paper, we propose the Individual Differences in Evaluating Life Satisfaction (IDELS) model to address this puzzle. The IDELS model posits that people differ in the processes by which they evaluate their life satisfaction: Some people's life satisfaction is more strongly associated with their current emotions (i.e., "emotion globalizing") whereas other people maintain a filter between their life satisfaction and current emotions. These individual differences should have important implications for the degree of short-term variability in life satisfaction and, in turn, for psychological health. In three diverse samples of women (total N = 536), we assessed life satisfaction and emotions daily or multiple times per day for two weeks. We tested four hypotheses derived from the IDELS model. First, participants differed substantially in the degree of short-term variability in life satisfaction, and these individual differences were moderately stable. Second, participants differed substantially in emotion globalizing, and these individual differences were moderately stable. Third, higher emotion globalizing predicted greater short-term variability in life satisfaction. Fourth, greater short-term variability in life satisfaction was associated with a maladaptive profile of greater neuroticism and worse psychological health. We discuss implications for life satisfaction theory and measurement.
This series of studies investigated whether the good psychometric properties of the English versi... more This series of studies investigated whether the good psychometric properties of the English version of the Big Five Inventory-2 (BFI-2) could be replicated using its Dutch adaptation. Second, it aimed to further examine the predictive validity of both the Big Five domain and the more specific facet scales of the BFI-2 in a large and representative sample. Results indicated that the structure found in the English version was replicated in the Dutch adaptation. The 60-item BFI-2 was reliable at the level of both domains and facets, as were the abbreviated versions. In terms of validity, the domain scales predicted a broad range of criteria. Examination of preregistered hypotheses regarding the discriminant validity of the facets indicated that experts were able to predict which facets would be most strongly associated with specific criteria. Overall, results confirm the strong psychometric properties of the BFI-2 Big Five domain scales and indicate that theoretically identified facets can be more valid predictors of criteria than other facets of the same domain.
Children's educational outcomes are strongly correlated with their parents' educational a... more Children's educational outcomes are strongly correlated with their parents' educational attainment. This finding is often attributed to the family environment-assuming, for instance, that parents' behavior and resources affect their children's educational outcomes. However, such inferences of a causal role of the family environment depend on the largely untested assumption that such relationships do not simply reflect genes shared between parent and child. We examine this assumption with an adoptee design in full-population cohorts from Danish administrative data. We test whether parental education predicts children's educational outcomes in both biological and adopted children, looking at four components of the child's educational development: (I) the child's conscientiousness during compulsory schooling, (II) academic performance in those same years, (III) enrollment in academically challenging high schools, and (IV) graduation success. Parental education was a substantial predictor of each of these child outcomes in the full population. However, little intergenerational correlation in education was observed in the absence of genetic similarity between parent and child-that is, among adoptees. Further analysis showed that what links adoptive parents' education did have with later-occurring components such as educational attainment (IV) and enrollment (III) appeared to be largely attributable to effects identifiable earlier in development, namely early academic performance (II). The primary nongenetic mechanisms by which education is transmitted across generations may thus have their effects on children early in their educational development, even as the consequences of those early effects persist throughout the child's educational development. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Proceedings of the National Academy of Sciences, 2020
SignificanceAre disagreeable individuals more likely to attain power than agreeable individuals? ... more SignificanceAre disagreeable individuals more likely to attain power than agreeable individuals? This question is important because highly disagreeable individuals in positions of power can do a lot of damage. For example, CEOs who are nasty and bullying create cultures of abuse and tend to lead their organizations to fail. In two longitudinal prospective studies, we found that disagreeableness did not predict the attainment of power. Selfish, deceitful, and aggressive individuals were no more likely to attain power than were generous, trustworthy, and nice individuals. Why not? Disagreeable individuals were intimidating, which would have elevated their power, but they also had poorer interpersonal relationships at work, which offset any possible power advantage their behavior might have provided.
Zusammenfassung. Die deutsche Version des Big Five Inventars 2 (BFI-2) erfasst die 5 Persönlichke... more Zusammenfassung. Die deutsche Version des Big Five Inventars 2 (BFI-2) erfasst die 5 Persönlichkeitsdomänen Extraversion, Verträglichkeit, Gewissenhaftigkeit, Negative Emotionalität (Neurotizismus) und Offenheit sowie insgesamt 15 Persönlichkeitsfacetten mit 60 Items. Das Inventar wurde im Rahmen eines mehrstufigen Übersetzungsprozesses vom Englischen ins Deutsche übertragen. Anhand einer nach Alter, Geschlecht und Bildungsabschluss quotieren Bevölkerungsstichprobe ( N = 1 224) wurden Reliabilität, Validität und Messinvarianz über Geschlechter und Altersgruppen untersucht. Die Ergebnisse der Studien deuten auf gute Reliabilitäten der Domänenwerte und ausreichende Reliabilitäten der Facettenwerte hin. Einzelitems und die Facettenwerte zeigen erwartungskonforme Ladungsmuster. Die Domänen- und Facettenwerte korrelieren erwartungsgemäß mit anderen Persönlichkeitsinventaren und sind mit Kriterien wie Bildungsabschluss, Einkommen, Gesundheit und Lebenszufriedenheit assoziiert. Strukturgle...
Journal of Personality and Social Psychology, 1996
Three replicable personality types were identified in a sample of 300 adolescent boys and shown t... more Three replicable personality types were identified in a sample of 300 adolescent boys and shown to generalize across African Americans and Caucasians. The types had conceptually coherent relations with the Big Five dimensions, ego resiliency, and ego control, and converged with three of the types identified by J. Block (1971). The behavioral implications of the types were explored using several independent data sources. Resilients were intelligent, successful in school, unlikely to be delinquents, and relatively free of psychopathology; Overcontrollers shared some of these characteristics but were also prone to internalizing problems; and Undercontrollers showed a general pattern of academic, behavioral, and emotional problems. This research demonstrates that replicable and generalizable personality types can be identified empirically, and that the unique constellation of traits defining an individual has important consequences for a wide range of outcomes.
A person's behavior and experiences can be described at different levels of abstraction. For exam... more A person's behavior and experiences can be described at different levels of abstraction. For example, a person might be described as charitable, as generous, as kind, or as good. Is there a level in such
Whereas the structure of individual differences in personal, social, and emotional attributes is ... more Whereas the structure of individual differences in personal, social, and emotional attributes is well understood in adults, much less work has been done in children and adolescents. On the assessment side, numerous instruments are in use for children but they measure discordant attributes, ranging from one single factor (self-esteem; grit) to three factors (social, emotional, and academic self-efficacy) to five factors (strength and difficulties; Big Five traits). To construct a comprehensive measure for large-scale studies in Brazilian schools, Ayrton Senna Institute is developing assessments mapping socio emotional skills on the conceptual framework of the five-factor model of personality. Here we present a study of internal structure of a 162-item inventory measuring 18 facet scales, representing two conceptual levels of measurement: one assesses the child’s current identity or self-concept and the other assesses self-efficacy beliefs. These 18 facets can be organized into five broad domains, similar to the Big Five domains: E: Engaging with others, A: Amity, WO Work orientation, ER: Emotional resilience, and O: Open-mindedness. The sample consisted of more than 30,000 participants, who were between 11 and 18 years old and attended grades 6 to 12. We split the sample in two random halves. We performed exploratory factor analysis of the 18 facet scales per layer with the first half and a confirmatory factor analysis with the second half of the sample. Internal consistencies were in the range of .70 to .80 for most scales. All facet scales had their highest loading on the expected domain. A bi-factor model best represented the self-efficacy level with a general factor plus five specific factors related with each domain. A confirmatory factor analysis showed a robust and replicable structure. Discussion focuses on the contribution to socio-emotional research in education and its measurement.
Uploads
Papers by Oliver John