Mental Health Coping Stories on Social Media: A

Causal-Inference Study of Papageno Efect

Yunhao Yuan Koustuv Saha Barbara Keller
yunhao.yuan@aalto.f koustuv.saha@gmail.com barbara.keller@aalto.f
Aalto University Microsoft Research Aalto University
Espoo, Finland Montreal, Canada Espoo, Finland

Erkki Tapio Isometsä Talayeh Aledavood

erkki.isometsa@hus.f talayeh.aledavood@aalto.f
University of Helsinki Aalto University
Espoo, Finland Espoo, Finland

ABSTRACT Conference 2023 (WWW ’23), April 30–May 04, 2023, Austin, TX, USA. ACM,
The Papageno efect concerns how media can play a positive role New York, NY, USA, 9 pages. https://doi.org/10.1145/3543507.3583350
in preventing and mitigating suicidal ideation and behaviors. With
the increasing ubiquity and widespread use of social media, indi- 1 INTRODUCTION
viduals often express and share lived experiences and struggles According to a report from the World Health Organization [58],
with mental health. However, there is a gap in our understanding globally, approximately 700,000 people fall victim to suicide each
about the existence and efectiveness of the Papageno efect in so- year. Suicide attempts and particularly committed suicides cause
cial media, which we study in this paper. In particular, we adopt a severe and tragic consequences among relatives and friends of
causal-inference framework to examine the impact of exposure to the victims, as well as signifcant economic problems for society.
mental health coping stories on individuals on Twitter. We obtain Consequently, suicide has become a crucial global public health
a Twitter dataset with ∼2M posts by ∼10K individuals. We con- problem, and the World Health Organization has called for urgent
sider engaging with coping stories as the Treatment intervention, action to reduce the suicide mortality rate.
and adopt a stratifed propensity score approach to fnd matched While suicide is a combined outcome of multiple, interrelated
cohorts of Treatment and Control individuals. We measure the psy- factors, ranging from mental health issues to social factors, media
chosocial shifts in afective, behavioral, and cognitive outcomes in can play an important role either in a harmful or benefcial direc-
longitudinal Twitter data before and after engaging with the coping tion. A considerable amount of literature [16, 25, 49] has studied
stories. Our fndings reveal that, engaging with coping stories leads and re-confrmed the harmful efect of media, dubbed the “Werther
to decreased stress and depression, and improved expressive writ- efect” [38], describing a spike in suicides after a heavily publicized
ing, diversity, and interactivity. Our work discusses the practical suicide. However, there is much less research about the benefcial
and platform design implications in supporting mental wellbeing. efects of media, referred to as the “Papageno efect”, describing a de-
crease in suicides after reporting alternatives to suicide. Niederkro-
CCS CONCEPTS tenthaler et al. explored the possible protective efect of media
• Human-centered computing → Empirical studies in collabora- reporting about suicide [34]. This study fnds a decrease in suicides,
tive and social computing; Social media; • Applied computing → if reports of suicide related content portray ways of overcoming
Psychology. suicidal ideation without narrating suicidal behaviors. This work
provides important insights into the potential benefts of media that
KEYWORDS reports suicide related content with a focus on hope and recovery.
Following this work, other studies provide evidence of the Papageno
social media, mental health, suicidal ideation, natural language,
efect from fctional flms [53], suicide-educational websites [54],
causal inference, Papageno efect
and newspaper articles [1]. Given the prevalence and importance
ACM Reference Format: of social media, understanding more about the Papageno efect on
Yunhao Yuan, Koustuv Saha, Barbara Keller, Erkki Tapio Isometsä, and Ta- social media can play a crucial role in decreasing suicide rates.
layeh Aledavood. 2023. Mental Health Coping Stories on Social Media: A Studies of the Papageno efect commonly rely on self-reports,
Causal-Inference Study of Papageno Efect. In Proceedings of the ACM Web surveys, and publicly reported suicide statistics and only cover a
small, selected group of people. People with suicidal ideation can
face negative attitudes and stigmatization, which prevents many of
them from seeking help [40]. Additionally, the sensitive nature of
This work is licensed under a Creative Commons Attribution International
4.0 License. suicide makes it challenging to collect data at scale on individuals
who have suicidal ideations and conduct continuous long-term
WWW ’23, April 30–May 04, 2023, Austin, TX, USA follow-up studies.
© 2023 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-9416-1/23/04. The emergence of social media platforms, such as Twitter, Red-
https://doi.org/10.1145/3543507.3583350 dit, and TikTok, provides venues for people to not only connect

2677
WWW ’23, April 30–May 04, 2023, Austin, TX, USA Yunhao Yuan, Koustuv Saha, Barbara Keller, Erkki Tapio Isometsä, and Talayeh Aledavood

with others, but also to express and share diferent aspects and life While one body of research highlights that increasing public
events of their personal lives [43]. Social media platforms provide understanding of mental health therapy may prevent suicide at-
a non-intrusive means to collect people’s naturalistic data at scale. tempts [5], others suggest that negative media coverage of sui-
Research has leveraged social media data from diferent social me- cides, such as the suicide victim’s non-attractive features or the
dia platforms to explore psychological and health issues using data circumstances of the suicidal act, prevents imitative suicidal at-
from various domains such as drug misuse [18], minority stress [59], tempts [39]. Niederkrotenthaler et al. discover that reports of peo-
and mental health [9, 48]. Social media platforms provide timely and ple who considered suicide but afterward dealt with their problems
relevant information on examining risk attributes longitudinally. constructively are linked to a short-term reduction in suicide rates
The anonymous features of social media may reduce the biases in 2010. This preventive efect is coined the “Papageno efect”. In-
found in research based on surveys and self-reported data. Conse- spired by the Papageno efect, Till et al. [52] conduct a randomized
quently, social media data provide an unparalleled opportunity to controlled trial to explore the benefcial impact of educative news-
study the Papageno efect in a broader population and its evolution papers featuring suicide researchers in 2018. They observe similar
over time. suicide-protective efects on both readers with, as well as readers
In this paper, we leverage public data from Twitter, a popular without personal experience of suicide ideation. To further test the
social media site. We analyze longitudinal posts from Twitter users Papageno efect, Niederkrotenthaler et al. conduct meta-analysis re-
who reply to posts containing stories about coping with suicidal search and provide new evidence supporting the benefcial efect of
ideation. We examine psychosocial changes in afective, behavioral, media on individuals with suicidal ideation if the media narratives
and cognitive outcomes related to suicidal ideation. Specifcally, we focus on hope and recovery from suicidal crises [33].
target the research questions of, whether the Papageno efect exists So far, however, the Papageno efect on social media is under-
on Twitter and how we can quantify psychosocial changes of Twitter studied. Our work attempts to examine the psychosocial impacts of
users before and after engaging with Twitter posts containing mental the Papageno efect on Twitter. We gather longitudinal social media
health coping stories. data and compare multiple psychosocial outcomes of individuals
To achieve the research goals, we collect 13,022 Twitter posts engaging with coping story posts with a matched Control group.
containing keyword phrases, which might indicate coping stories.
We utilize a machine learning classifer from a previous study [29]
to automatically annotate the dataset and 3,077 Twitter posts are
labeled as coping stories, which might result in the Papageno efect. 2.2 Mental Health and Psycholinguistics
Among them, we manually verify the accuracy of the classifer on Although suicide is not an inevitable consequence of any psychi-
a sampled coping story dataset. We collect data from two popula- atric condition, research suggests a link between mental health and
tions on Twitter: 787K posts from 2,468 individuals who reply to suicidal behaviors. According to a psychiatric autopsy study [6],
Twitter posts containing coping stories and 1.4M posts from 8,465 more than 90% of people who die by suicide sufer a mental disorder
individuals in a control group. After applying stratifed propensity previous to their death. Patients who report anhedonia and sleep-
score matching, we aggregate psychosocial outcomes as afective, lessness with major anxiety symptoms, alcohol abuse, or emotional
behavioral, and cognitive outcomes and identify these with high sig- problems have the highest short-term risk for suicide [24].
nifcant efects. We observe that engaging with coping story posts Studying suicidal ideation attracts researchers from diferent
on Twitter is linked to lower stress and depression, and higher felds. Research suggests that psychological linguistic metrics may
expressive writing, diversity, and interactivity. be used to characterize people with suicidal ideation [2]. Stirman
and Pennebaker compare the linguistic expression of poets between
Privacy and Ethics. Although we use public accessible data from a small sample of suicidal and non-suicidal individuals, using a com-
Twitter, we are committed to protect the privacy of the data owners. puterized text-analysis program called Linguistic Inquiry and Word
We remove any information related to personal identity and para- Count (LIWC) [36]. The authors observed more self-references
phrase all quotations in this paper to avoid traceability. Given the through frst-person singular pronouns, more words related to
sensitive nature of the topic of suicide and to avoid potential misuse, death, and fewer social references in the poets written by those
we adhere to Twitter data sharing standards and will only share who did die by suicide [50]. Following that, other studies have used
the Twitter post IDs to other researchers. One of the authors is a LIWC and similar language analysis techniques to analyze lexical
certifed psychiatrist. This helps us better understand our fndings. and linguistic features in the text of suicidal individuals from dif-
ferent cultural backgrounds [17, 20, 27]. For example, Litvinova
et al. [28] use the Russian edition of the LIWC lexicon to analyze
2 RELATED WORK the text from blog posts and fnd that texts written by confrmed
suicidal individuals, containing more negation words, fewer social
2.1 Media Efects on Suicidal Ideation and perception-related words, fewer positive emotion words than
The question of how media reports about suicide infuence subse- texts from a control group.
quent suicides has received considerable attention [31]. For quite All together, these studies provide a core understanding of lever-
some time, studies focus on the negative impact of media portrayals aging mental and psycholinguistic cues for understanding the Pa-
on suicide and fnd a positive correlation between media coverage of pageno efect on social media. Based on the public content shared
suicidal behavior and suicidality [14]. These studies spark a debate on the social media platform, we focus on inferring psychosocial
about the possible preventive impact of media on suicide rates. outcomes from the perspectives of afect, behavior, and cognition.

2678
Social Media and Papageno Efect WWW ’23, April 30–May 04, 2023, Austin, TX, USA

H  'LVWULEXWLRQRIWUHDWPHQWDQGFRQWURO Table 1: Paraphrased example Twitter posts labeled with cop-
FRQWURO
 WUHDWPHQW ing story or non-coping story and Twitter posts responding
to coping story posts.

'HQVLW\

Coping Story Posts


“I’m posting this because I’ve had suicide ideas passively for a long
time. I finally realized I was suicidal three years ago. I believed that the

desire to be beter of dead was common. It is NOT the norm. If you

have such ideas, you should seek professional assistance.”
“When I was a patient in the psychiatric hospital, they had to remove

















Q

U

Q

W

E

Y

E
\

U
S
2F
0D

$S
1R
0D
-D

-D

)H

)H
'DWH 6H my shoelaces to prevent me from self-injury. Today marks one month
without suicide ideation. My life has improved afer receiving therapy
Figure 1: Treatment and Control (placebo) dates distribution. from all of my physicians. Cheers to the continuation of living in the
present!”
Non-Coping Story Posts
2.3 Social Media and Mental Health Research “Even more terrible than my thoughts of death are my suicide ideas.
People told me when I was 10 that it would get beter, but it hasn’t,
The emergence of social media provides a new powerful “lens” to and I want to die yet nothing works. It’s so unfair that no mater how
give insights into mental health and suicidal ideation. Prior research many times I try, I always fail. I’m sorry if this is frustrating; I just feel
uses social media posts to understand more about major depressive so alone.”
disorders [10], risk suicide behavior [9], and drug use [45]. “But I wanted to kill myself again this weekend. I’ve never been happier.
Relatedly, Kumar et al. compare the posting activity and con- But every day is so full of grief for the body I don’t have and will never
tent following celebrity suicides to fnd a rise in posting frequency be capable of having.”
and increased suicidal ideation [25]. Another work reveals the im- Twiter Posts Responding to Coping Story Posts
portance of linguistic features to predict users who move from “Dear friend, I’m happy that things didn’t turn out the way you had
mental health discourse to suicidal ideation [12]. Based on Reddit hoped. I wouldn’t want the past few of years to have been any other
way because they have been such an adventure. To where we all go in
postings, the authors develop a propensity score matching to in-
the future is something I am looking forward to see.”
vestigate how individuals may discuss their suicidal ideation while
“I’m glad to hear that you’re doing well. It’s good to know that you
controlling for the previous use of linguistic features of mental have support from close friends and family because I am aware of how
health. Following this work, De Choudhury and Kıcıman [11] apply challenging it may be to handle some circumstances.”
a similar matching framework to study the efect of social support
on suicidal ideation risk. In another work, Saha and Sharma con-
duct a causal-inference examination of what factors contribute to
improved mental wellbeing in online mental health communities
(particularly TalkLife) [44]. and 1 March 2022. The collected Twitter posts contain at least one
Our work draws motivation from the above body of work in term related to suicide attempts, including “suicidal thoughts,” “kill
examining the prevalence of the Papageno efect following being myself,” “suicidal ideation,” and “end my life,” and contain at least
exposed to coping story posts on suicidal ideation on social media. one of the terms indicating successful coping, including “happier,”,
Our work adopts the natural language techniques and causal infer- “better,” and “recover”. After collecting 13,022 Twitter posts, we
ence analyses to provide a computational framework of measuring apply the coping story classifer to annotate each Twitter post as a
this efect and reveals important insights about how people show coping story or a non-coping story. We fnd 3,077 Twitter posts are
changes in social media behaviors after engaging with coping story annotated as coping story posts. Among them, 709 Twitter posts
posts. labeled as coping story posts have at least one reply below them.

3 DATA 3.1.1 Annotating Coping Story Posts. We use a multi-label clas-

sifer provided by Metzler et al. to annotate coping story posts
Due to the absence of publicly available datasets of coping story
in our dataset. It categorizes Twitter posts into the following six
posts in social media, we utilize Twitter timeline data of individuals
categories [29]:
who reply to coping story Twitter posts. The steps of data collection
include: 1) collecting Twitter posts, which might describe coping (1) Personal experiences of coping (coping story): Personal expe-
stories; 2) applying coping story classifer from [29] and manually riences about suicide that express a feeling of hope, healing,
verifying the results; 3) collecting timeline data of individuals com- methods of coping, or reference alternative options to sui-
menting on the so found coping story posts; 4) building a control cide. The tone could be positive or neutral. Previous studies
dataset from randomly sampled, comparable, individuals. suggest that such narratives may have a Papageno efect.
(2) Personal experiences of suicidal ideation and attempts: Per-
3.1 Compiling the Coping Story Posts Dataset sonal stories about suicide that lack a sense of coping or
The frst step is to retrieve Twitter posts that may contain coping hope.
stories. This work uses the Twitter Application Programming Inter- (3) Suicide cases: Reports of suicides that have been carried out
face (API) to collect Twitter posts posted between 1 January 2018 or prevented

2679
WWW ’23, April 30–May 04, 2023, Austin, TX, USA Yunhao Yuan, Koustuv Saha, Barbara Keller, Erkki Tapio Isometsä, and Talayeh Aledavood

Twitter Timeline Single post from one user’s timeline

…
…
pre-treatment
Filter by Pre-treatment T Data

Group similar users on propensity scores

keywords or

2 weeks

Matched T Users

Measure psychosocial changes

phrases and
classifiers
Covariates
Treatment Placebo
Recovery story

Propensity Scores
Social Media Features

post-treatment

User Strata
ML Model
LIWC

2 weeks

…
Treatment Users
N-gram

Matched C User
DASS Classifier
Twitter

Control Users pre-treatment C Data

…

Data Collection Covariates Extraction Propensity Score Analysis

Figure 2: Schematic diagram of propensity score matching between Treatment individuals and Control individuals.

(4) Awareness: Twitter posts raising awareness about suicide, coping story. We collect Twitter metadata, including the number of
emphasizing high rates or links to problems such as such as posts, likes, followers, followees, and the account creation time. To
bullying, racism, depression, or veteran status. mitigate the confounding efects of engaging with multiple coping
(5) Prevention: Twitter posts that provide information on so- story posts, we remove 2 individuals who reply to multiple coping
lutions or initiatives aimed at addressing the problem of story posts. For the remaining 2468 individuals, we collect their
suicide, including prevention at both individual and public timeline data two weeks before their reply on the coping story
health levels. and two weeks after. In the end, the target dataset contains 2468
(6) Twitter posts that do not fall under any of the above cate- individuals with a total of 787K Twitter posts.
gories.
As the aim of this step is to fnd individuals who comment on 3.3 Compiling the Control Dataset
coping story posts, we subsequently focus on Twitter posts labeled
As we seek to isolate the efect of the coping story on individuals, we
as a coping story post. In order to verify the ability of the classifer
build a Control dataset of individuals who do not reply to coping
to fnd Twitter posts that contain actual coping stories, we manually
story posts during the investigated period. To do so, a Control
check Twitter posts that are labeled as coping story posts by the
dataset is built with individuals who have similar attributes to the
classifer.
treatment individuals prior to their engagement with a coping story.
3.1.2 Annotation Task. We randomly sample 400 Twitter posts out To fnd such control individuals, we use keywords, including “life”,
of 709 Twitter posts labeled as coping stories that have at least “job”, “music”, and “movie” to search for individuals on Twitter. For
one reply below them. Using the codebook from [29], one author each keyword, we collect 4000 individuals and the timeline of their
independently annotates 400 Twitter posts. If there are any posts Twitter posts. For each individual in the Control dataset, We assign
that they are unsure about, the author discusses the posts with a placebo date from the non-parametric distribution of treatment
other authors and together they reach an agreement on how to date in the Treatment dataset to any day the Control individual
code them. After fnishing the annotation, another author randomly replies to other Twitter posts to reduce any temporal confounds.
selects 50 posts out of 400 posts to verify the annotation result. We We utilize Kolmogorov-Smirnov (KS) test to measure the similarity
use Cohen’s kappa to validate our annotation process. This results in the two distributions (Figure 1). The KS test yields a low statistic
in Cohen’s � of 0.81 with an agreement of 92.8%, which indicates of 0.05, suggesting that the probability distributions of treatment
substantial agreement [26]. Out of the 400 posts labeled coping and placebo dates are similar. In the end, we collect timeline data
story by the classifer, we fnd 347 posts are correct predictions, two weeks before the placebo date and two weeks after of 8465
indicating 86.7% accuracy of the classifer to identify copy story individuals to build the control dataset.
posts.

3.2 Compiling the Treatment Dataset 4 METHODS

For the Twitter posts annotated as coping story posts, we assume 4.1 Study Design and Rationale
that the individuals who reply to the Twitter posts might have We adopt a causal inference framework [21] to isolate the Papageno
been impacted by the coping story. For our coping story dataset, efect. The schematic diagram of our approach is shown in Figure 2.
we identify 2470 unique individuals who replied to at least one Our approach frst matches individuals of the Control group with

2680
Social Media and Papageno Efect WWW ’23, April 30–May 04, 2023, Austin, TX, USA

individuals of the Treatment group1 based on pre-Treatment behav- individual. A lower quantity of posts on mental health symptomatic
ioral attributes. For this, we train a machine learning classifer to expressions indicates better psychosocial wellbeing.
estimate the likelihood of an individual being assigned to either the Behavioral Outcomes. Literature in psychology defnes behav-
Treatment or Control group (i.e., propensity) based on covariates ioral psychological well-being as including three factors: individ-
and perform matching across groups using estimated propensity ual’s overt actions, behavioral intentions, and verbal statement
scores. Within matched groups of Control and Treatment groups, regarding behavior [4]. Previous studies quantify behavioral psy-
we analyze the following psychosocial outcomes between matched chological wellbeing by measuring the shifts in social functioning
Control groups and Treatment groups. In sum, our approach en- and interests [19, 47]. Our work operationalizes the following mea-
sures that members of the Treatment group and Control group sures to obtain behavioral outcomes.
who are being compared have similar behavior prior to replying to
Activity. We investigate if engaging with coping story posts pro-
coping story posts. This gives us the means to analyze the difer-
motes individuals to be more active on Twitter. Higher activity
ences inpsychosocial outcomes between the matched Treatment
likely indicates increased extroversion, and is associated with thera-
individuals and the members of the Control group.
peutic benefts [15, 47]. To quantify activity on Twitter, we calculate
the average number of Twitter posts per day for every individual.
4.2 Measuring Psychosocial Outcomes
Interactivity. Interactivity is another indicator of an individual show-
To examine the psychosocial efects of engaging with coping story ing therapeutic efects [44, 47]. We measure participation in discus-
posts on social media, we measure three psychosocial outcomes sions on Twitter as interactivity, indicating social engagement. The
drawing from psychiatry and psychology literature: afective, behav- metric used is the proportion of replies (to other individuals’ posts)
ioral, and cognitive outcomes [4]. We operationalize these measures per original Twitter post.
drawing on prior research in social media and mental health [44, 47].
Topic Diversity. To measure the diversity of topics discussed by an
Afective Outcomes Afect is defned as any experience of feeling individual, we apply a language model on the posted texts. We cap-
or emotion [55]. As individuals use emotive, relativistic language in ture language semantics by adopting a word embedding model [30],
their self-motivated online texts, language is an efective way to in- which represents words in vectors in latent semantic dimensions. In
fer afective psychosocial wellbeing. To estimate afective outcomes, particular, we use 300-dimensional word embeddings pre-trained on
we use the following metrics: Google News. Then, for each post from the Treatment and Control
Afective Words. We employ the well-validated psycholinguistic datasets, we calculate the average cosine distance from the centroid
lexicon, Linguistic Inquiry and Word Count (LIWC) [35] to ob- of the corresponding corpus. Higher the average the distance from
tain normalized occurrences of words in afective categories per the centroid, greater is the topical diversity [57].
individual. The selection of these measures is inspired by studies Cognitive Outcomes. The cognitive component of psychosocial
like [10, 15, 47], where therapeutic symptoms are associated with health encompasses beliefs, knowledge structures, perceptual re-
self-initiated and expressive writing [7, 8]. sponses, and thoughts [4]. We adopt the following measures to
Symptomatic Mental Health Expressions. Prior research notes the quantify an individual’s cognitive behaviors.
comorbidity of multiple mental health conditions [41], and we Readability. Readability measures the complexity of a given text. We
operationalize the language indicative of diferent mental health employ the Coleman-Liau Index (CLI) to assess the readability per
symptomatic expressions of depression, anxiety, stress, and suicidal individual. CLI is calculated as, ��� = 0.0588 ∗ � − 0.296 ∗ � − 15.8,
ideation [45]. To identify mental health symptomatic expressions in where L is the average number of letters per 100 words and S is
social media language, Saha et al. [45] develop multiple binary ma- the average number of sentences per 100 words. Previous research
chine learning classifers based on transfer learning methodologies. shows a link between measures of language complexity and long-
These classifers are �-gram–based (�=1,2,3) binary support vector term improvements in psychosocial wellbeing [15].
machine (SVM) models, and are trained using appropriate Red-
dit communities (r/depression for depression, r/anxiety for anxiety, Complexity and Repeatability. Complexity and Repeatability are
r/stress for stress, and r/SuicideWatch for suicidal ideation). People syntactic measurements that refect an individual’s cognitive state
in these communities post about mental health symptoms to receive in terms of planning, execution, and memory [15]. We measure
feedback and to support others. The posts in these subreddits are repeatability as the normalized count of non-unique words and
used as training data to identify language used in connection with complexity as the average number of words per sentence. Psy-
mental health. The training data for texts not related to mental chosocial wellbeing positively correlates with language complexity,
health originates from non-mental-health-related content on Red- and negatively with repeatability [15, 47].
dit. These classifers perform at a high accuracy of approximately Psycholinguistic Keywords. We adopt the LIWC lexicon to analyze
0.90 on average on held-out test data [45], and have also been used the proportion of keywords related to cognition, perception, social
in other research [44, 46]. We use these classifers to measure the context, and linguistic style categories. We consider the follow-
aggregated proportion of expressing mental health concerns per ing fve aggregated categories: (1) Cognition & Perception (cause,
certain, cognitive, inhibition, discrepancies, tentativeness, percep-
tion, see, hear, feel, insight) (2) Social Context (biological processes,
1 Treatment: Conventionally, “Treatment” refers to being given Treatment or interven- achievement, body, family, friends, health, home, humans, money,
tion. We use the term “Treatment” in accordance with the causal inference terminology
to diferentiate individuals replying to coping story posts and the individuals not re- religion, social, work) (3) Lexical Density & Awareness (adverbs,
plying to coping story posts.

2681
 WWW ’23, April 30–May 04, 2023, Austin, TX, USA Yunhao Yuan, Koustuv Saha, Barbara Keller, Erkki Tapio Isometsä, and Talayeh Aledavood

+/- 2 results in 14 strata, containing 1,245 Treatment and 1,087 Control

2.5 Treatment
Control individuals.
2.0
Density (Users)

4.3.3 Qality of Matching. To determine whether individuals in

1.5
the Treatment group and Control group are statistically compa-
1.0 rable, we measure the balance of the covariates. We conduct this
comparison by calculating the standardized mean diferences (SMD)
0.5
between the two groups in all 14 valid strata [22, 45]. SMD is the
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 diference in the mean covariate values between the two matched
Propensity Scores groups, divided by the pooled standard deviation. The two groups
can be assumed to be balanced if the SMD of all covariates is lower
Figure 3: (a) Propensity score distribution (shaded region are than 0.2 [22, 42, 51]. For the unmatched dataset, the maximum SMD
the dropped strata), (b) Quality of matching. is 0.85, and the mean SMD is 0.22, whereas in our matched dataset,
the maximum SMD is 0.19 and the mean SMD is 0.06 (Figure 3b).
article, verbs, auxiliary verbs, conjunctions, inclusive, exclusive, Therefore, this satisfes the threshold of SMD<0.2, suggesting that
preposition, negation, quantifer, relative) (4) Interpersonal Focus our matching yields balanced Treatment and Control datasets.
(1st personal pronouns, 2nd personal pronouns, Impersonal pro-
4.3.4 Estimating the Average Treatment Efects. To estimate the
nouns) (5) Temporal References (future, past, present). Prior re-
efect of engaging with coping story posts, we compute the relative
search highlight the association between these lexicons and cog-
Treatment efect (RTE) for each outcome. For this, we calculate
nition [37]. An increased use of these lexicons is related to better
the ratio of likelihood of an outcome measure in the Treatment
psychological conditions [7, 8].
group to that in the Control group per stratum. Using the number
4.3 Matching of Treatment individuals in each stratum as a weight, we obtain the
weighted average RTE per outcome. The outcome is interpreted as
4.3.1 Matching Covariates. Our goal is to measure the psychoso- an increase (greater than 1) or decrease (less than 1) of observable
cial outcomes of engaging with a coping story post. Matching is psychosocial outcomes after engaging with coping story posts com-
an efcient strategy in Case-Controlled studies to estimate causal pared to the Control group with similar pre-Treatment attributes.
efects and minimize the possible occurrence of selection biases. In
our case, we build several covariates from the Treatment and Con-
trol datasets to control for similar pre-Treatment behavior on social 5 RESULTS
media. The frst set of covariates comprises Twitter individuals’ so- In this section, we present the shifts for each psychosocial outcome
cial media features (the number of posts, likes, followers, followees, across the matched Treatment and Control individuals in the cor-
and posting frequency). The second set contains the distribution responding datasets. We calculate the efect size (Cohen’s �), and
of word usage in their Twitter timelines. We extract the top 100 measure statistical signifcance in diferences using an independent
unigrams as the second covariates set. The third set includes the sample �-test. Figure 4 shows the distribution of RTE per stratum
psycholinguistic features in their timeline data by measuring the across diferent psychosocial outcomes. Table 2 summarizes these
word distribution in the LIWC lexicon. The last set of covariates diferences.
controls for the average usage of posts related to symptomatic men- Afective Outcomes. Afect. In Table 2, we observe that individuals
tal health expressions, including depression, anxiety, stress, and in the Treatment group use more afective words than the matched
suicidal ideation. Control individuals after engaging with coping story posts. The
4.3.2 Propensity Score Analysis. To ensure to have comparable indi- average number of afective words used by Treatment individuals is
viduals in our Treatment dataset and the Control dataset, matching 11% higher than among Control individuals. The efect size (Cohen’s
is employed to pair Treatment individuals and Control individuals, �=0.20) suggests small diferences between the two distributions
whose covariates are similar to each other. A logistic regression and the t-test indicates statistical signifcance (�=2.19, �<0.05). This
classifer is implement to predict the likelihood of an individual observation afrms that individuals use more afective words after
belonging to either the Treatment group or Control group based on engaging with coping story posts.
their covariates. We divide the propensity score distribution into Symptomatic Mental Health Expressions. We fnd that engaging with
50 strata with equal width. The individuals with similar propensity coping story posts is associated with decreases in the use of symp-
scores are grouped into the same stratum [22]. This helps us to eval- tomatic stress and depression expressions. This is revealed by lower
uate possible psychosocial outcomes within each stratum, where average percentages of symptomatic stress and depression Twitter
the Control group individuals are matched to the Treatment indi- posts from individuals in the Treatment group refecting stress (�=-
viduals based on the pre-Treatment behavioral traits. We remove 3.96, �<0.001) and depression (�=-2.84, �<0.01). In contrast, we fnd
the individuals with propensity scores falling outside two standard no signifcant diferences in our measures of anxiety and suicidal
deviations from the mean (Figure 3a ). We drop the strata failing ideation after individuals engage with coping story posts between
to satisfy the minimum sample size within each stratum based on the two corresponding groups. This illustrates that engaging with
previous causal inference research [13]. By ensuring that there are coping story posts does not increase the use of symptomatic anxiety
at least 50 individuals per group in each stratum, this approach and suicidal ideation expressions. These observations are consistent

2682
Social Media and Papageno Efect WWW ’23, April 30–May 04, 2023, Austin, TX, USA

Affect independent t-test indicate statistical signifcance (�=4.17, �<0.01).

Anxiety This might suggest that engaging with coping story posts likely
Depression
promotes an individual’s participation in online discussions. For
topical diversity, we measure the diversity of expressed topics in
Stress
posts after engaging with coping story posts; the efect size informs
Suicidal Ideation
small diferences between Treatment and Control distributions of
Activity
topical diversity, and the �-test afrms statistical signifcance (�=2.51,
Interactivity �<0.05). This indicates individuals in Treatment group discuss a
Diversity broader range of topics after engaging with coping story posts,
Readability suggesting higher psychological wellbeing.
Complexity Cognitive outcomes. To examine if engaging with coping story
Repeatability posts leads to shifts in cognition, we measure the diferences in
Cognition and Perception readability, complexity, repeatability, and psycholinguistic features.
Social Context Among these, we fnd no signifcant diferences in complexity, cog-
Lexical Density & Awareness nition & perception, lexical density & awareness, and temporal
Interpersonal Focus reference. However, we observe a signifcant diference in interper-
Temporal References sonal focus (�=2.77, �<0.05). The changes in the usage of pronouns
strata might suggest a shift in how individuals see themselves in relation
to others. Although independent sample � indicates a signifcant
diference in readability (�=-2.17, �<0.05), the result of cohen’s �
Figure 4: RTE per propensity stratum per outcome. The pink
shows no diference between the two distributions. One unantici-
color indicates RTE greater than 1, while the green color
pated fnding is the Treatment individuals show higher repeatabil-
suggests RTE less than 1.
ity compared to Controlindividual (�=5.09, �<0.001), which suggest
lower psychosocial health.
Table 2: Summary of psychosocial diferences across all
the outcomes between Treatment and Control individuals.
We report mean psychosocial outcomes across all matched 6 CONCLUSION AND DISCUSSION
individuals, efect size (Cohen’s �), independent sample t- 6.1 Summary of Findings
statistic. The p-values from LIWC categories are adjusted us- In this work, we develop a novel causal inference framework to ver-
ing non-negative two stage FDR correction (∗� < 0.05, ∗ ∗ � < ify and study the Papageno efect on social media. Using a Twitter
0.01, ∗ ∗ ∗� < 0.001). dataset with ∼2M posts by ∼10K individuals, we observe statisti-
cally signifcant psychosocial (afective, behavioral, cognitive) shifts
Categories Tr. Ct. RTE d t-test in individuals after engaging with coping story posts. In assessing
Afective Outcomes these psychosocial efects, our causal framework controls behav-
LIWC: Afect 0.09 0.08 1.05 0.20 2.19* ioral and linguistic covariates across the Treatment and Control
Anxiety 0.05 0.04 1.08 0.17 -0.33 groups. We verify that engaging with coping story posts positively
Depression 0.15 0.17 0.93 0.28 -2.84**
Stress 0.35 0.38 0.94 0.28 -3.96*** impacts individuals’ stress and depression, and improves expressive
Suicidal Ideation 0.07 0.06 1.04 0.15 -0.41 writing, topics diversity, and interactivity. Our results indicate that
Behavioral Outcomes
Activity 4.33 4.19 1.10 0.16 0.40
engaging with coping story posts on social media is associated with
Interactivity 8.89 2.78 3.34 0.34 4.17** positive benefts for one’s psychosocial wellbeing.
Topics Diversity 0.37 0.36 1.02 0.25 2.51*
Cognitive Outcomes
Readability 12.33 11.52 0.95 0.18 -2.16* 6.2 Implications
Complexity 9.26 9.52 0.99 0.19 -1.42
Repeatability 0.51 0.45 1.11 0.30 5.09*** While we do not directly measure shifts in suicidal ideation itself,
LIWC: Cognition & Perception 0.27 0.27 1.02 0.17 1.12 our fndings still provide evidence for the Papageno efect on social
LIWC: Social Context 0.18 0.17 1.01 0.08 0.55
LIWC: Lexical Density & Awareness 0.60 0.61 0.99 0.15 -1.23 media. Our results suggest that engaging with posts describing
LIWC: Interpersonal Focus 0.12 0.11 1.08 0.26 2.77* personal stories featuring coping with suicidal ideation can bring
LIWC: Temporal Reference 0.10 0.10 1.02 0.17 1.45 positive impacts on psychosocial wellbeing. Our work provides a
methodology to help measure the psychosocial outcomes of the
Papageno efect on social media. By focusing on the psychosocial
with prior research which indicates that media featuring individu- shifts of a large sample of individuals who engage with coping story
als coping with depression and suicidal ideation reduces depression posts, our work suggests a role for utilizing social media to access
and shows no efect on suicidal ideation [32]. the prospective psychosocial outcomes of the Papageno efect.
Behavioral Outcomes. For the second set of outcomes, we fnd no Our work bears practical implications for preventing suicide. On-
signifcant diference in activity after engaging with coping story line communities may develop strategies for the narratives about
posts. We fnd that the average interactivity of the Treatment users sharing suicidal behaviors and ideation, allowing vulnerable mem-
is higher than the Control. Both efect size (Cohen’s �=0.34) and bers in the community more protected. Our work bears design

2683
WWW ’23, April 30–May 04, 2023, Austin, TX, USA Yunhao Yuan, Koustuv Saha, Barbara Keller, Erkki Tapio Isometsä, and Talayeh Aledavood

implications for social media platforms in terms of how these plat- [3] Mehrab Bin Morshed, Koustuv Saha, Richard Li, Sidney K. D’Mello, Munmun
forms can encourage positive and thriving behavior, especially for De Choudhury, Gregory D. Abowd, and Thomas Plötz. 2019. Prediction of Mood
Instability with Passive Sensing. PACM IMWUT (2019).
those struggling with mental health concerns. Platforms such as [4] Steven J Breckler. 1984. Empirical validation of afect, behavior, and cognition as
Reddit, TalkLife, and 7Cups follow community-driven moderation distinct components of attitude. Journal of personality and social psychology 47,
6 (1984), 1191.
strategies, and these platforms can include in their community [5] Victoria Carmichael and Rob Whitley. 2019. Media coverage of Robin Williams’
norms what kinds of postings and support can help people draw suicide in the United States: A contributor to contagion? PloS one 14, 5 (2019).
therapeutic benefts. Social media platforms can show more posts [6] Jonathan TO Cavanagh, Alan J Carson, Michael Sharpe, and Stephen M Lawrie.
2003. Psychological autopsy studies of suicide: a systematic review. Psychological
with the Papageno efect when individuals search for suicidal in- medicine 33, 3 (2003), 395–405.
formation. These may be benefcial for individuals engaging with [7] Cindy Chung and James W Pennebaker. 2007. The psychological functions of
coping story posts and helpful for individuals with suicidal ideation function words. Social communication (2007), 343–359.
[8] Michael A Cohn, Matthias R Mehl, and James W Pennebaker. 2004. Linguistic
to seek support and prevent tragic outcomes. markers of psychological change surrounding September 11, 2001. Psychological
science (2004).
6.3 Limitations and Future Work [9] Glen Coppersmith, Ryan Leary, Patrick Crutchley, and Alex Fine. 2018. Natural
language processing of social media as screening for suicide risk. Biomedical
We acknowledge our study has several limitations, some of which informatics insights 10 (2018), 1178222618792860.
[10] Munmun De Choudhury, Michael Gamon, Scott Counts, and Eric Horvitz. 2013.
point to intriguing future research directions. We do not account Predicting depression via social media. (2013).
for passive engagement behaviors. For example, we do not know [11] Munmun De Choudhury and Emre Kıcıman. 2017. The language of social support
if an individual read a coping story and is afected by it if this in social media and its efect on suicidal ideation risk. In ICWSM.
[12] Munmun De Choudhury, Emre Kiciman, Mark Dredze, Glen Coppersmith, and
individual does not reply to it. Another limitation is the time of Mrinal Kumar. 2016. Discovering shifts to suicidal ideation from mental health
our measurement. We only measure the averaged psychosocial content in social media. (2016), 2098–2110.
outcomes within two weeks after an individual engaged with a [13] Munmun De Choudhury, Sanket Sharma, and Emre Kiciman. 2016. Characterizing
Dietary Choices, Nutrition, and Language in Food Deserts via Social Media. In
coping story post. However, the psychosocial outcomes may vary CSCW.
over time and show fuctuating results [3]. Future work can explore [14] Jan Domaradzki. 2021. The Werther efect, the Papageno efect or no efect?
A literature review. International journal of environmental research and public
psychosocial shifts in a fne-grained temporal manner. health 18, 5 (2021), 2396.
Our study sufers from selection biases. We gather data from [15] Sindhu Kiranmai Ernala, Asra F Rizvi, Michael L Birnbaum, John M Kane, and
those who publicly reply to coping story posts on Twitter, which is Munmun De Choudhury. 2017. Linguistic markers indicating therapeutic out-
comes of social media disclosures of schizophrenia. PACM Human-Computer
likely to be infuenced by self-selection bias. We can only collect Interaction CSCW (2017).
data from individuals who are active on social media. This is espe- [16] Robert A Fahey, Tetsuya Matsubayashi, and Michiko Ueda. 2018. Tracking the
cially true considering the stigma associated with people having Werther Efect on social media: Emotional responses to prominent suicide deaths
on twitter and subsequent increases in suicide. Social Science & Medicine 219
suicidal ideation. In a similar manner, the scope of this study solely (2018), 19–29.
focuses on Twitter, which might result in incomplete or inaccurate [17] Mercedes Fernández-Cabana, Julio Jiménez-Féliz, María Teresa Alves-Pérez,
Raimundo Mateos, Ignacio Gómez-Reino Rodríguez, and Alejandro García-
perspectives. Therefore, our observations might not be generalized Caballero. 2015. Linguistic analysis of suicide notes in Spain. The European
to other online communities on Twitter or beyond. Journal of Psychiatry 29, 2 (2015), 145–155.
We acknowledge the concerns raised by King and Nielsen [23], [18] Sanjana Garg, Jordan Taylor, Mai El Sherief, Erin Kasson, Talayeh Aledavood,
Raven Riordan, Nina Kaiser, Patricia Cavazos-Rehg, and Munmun De Choudhury.
and are aware of the limitations of the propensity score methods. 2021. Detecting risk level in individuals misusing fentanyl utilizing posts from
We chose to use the well-established stratifed propensity score an online community on Reddit. Internet Interventions 26 (2021), 100467.
method in our study [22, 44, 47, 56]—it ofers a more robust ap- [19] Sharath Chandra Guntuku, J Russell Ramsay, Raina M Merchant, and Lyle H
Ungar. 2019. Language of ADHD in adults on social media. Journal of attention
proach to mitigate the confounds, by essentially balancing for the disorders (2019).
bias-variance trade-of. This method, therefore, addresses some of [20] Lori D Handelman and David Lester. 2007. The content of suicide notes from
attempters and completers. Crisis: The Journal of Crisis Intervention and Suicide
the biases and limitations inherent in propensity score methods [23]. Prevention 28, 2 (2007), 102.
It is important to note that, despite our best eforts to control for [21] Guido W Imbens and Donald B Rubin. 2015. Causal inference in statistics, social,
confounding variables, we cannot infer “true causality” in our study. and biomedical sciences. Cambridge University Press.
[22] Emre Kiciman, Scott Counts, and Melissa Gasser. 2018. Using longitudinal social
As the outcomes might have be been infuenced by other online media analysis to understand the efects of early college alcohol use. (2018).
and ofine factors, e.g., the individual’s suicidal behavior history, [23] Gary King and Richard Nielsen. 2019. Why propensity scores should not be used
the length of a coping story post, and the length of the replies. De- for matching. Political analysis 27, 4 (2019), 435–454.
[24] Phillip M Kleespies and Elizabeth L Dettmer. 2000. An evidence-based approach
spite corroboration by a psychiatrist, we cannot be certain based on to evaluating and managing suicidal emergencies. Journal of Clinical Psychology
twitter posts that the individual is personally seriously considering 56, 9 (2000), 1109–1130.
[25] Mrinal Kumar, Mark Dredze, Glen Coppersmith, and Munmun De Choudhury.
suicide. Therefore the symptomatic variables and outcomes need 2015. Detecting changes in suicide content manifested in social media following
additional clinical validation. Future work could combine social celebrity suicides. In Proceedings of the 26th ACM conference on Hypertext & Social
media data together with clinically validated data which could lead Media. 85–94.
[26] J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement
to more generalizable results on the Papageno efect and the role for categorical data. biometrics (1977), 159–174.
of social media. [27] David Lester, Janet Haines, and Christopher L Williams. 2010. Content diferences
in suicide notes by sex, age, and method: A study of Australian suicide notes.
Psychological Reports 106, 2 (2010), 475–476.
REFERENCES [28] Tatiana A Litvinova, Pavel V Seredin, Olga A Litvinova, and Olga V Romanchenko.
[1] Florian Arendt, Benedikt Till, and Thomas Niederkrotenthaler. 2016. Efects of 2017. Identifcation of suicidal tendencies of individuals based on the quantitative
suicide awareness material on implicit suicide cognition: A laboratory experiment. analysis of their internet texts. Computación y Sistemas 21, 2 (2017), 243–252.
Health Communication 31, 6 (2016), 718–726. [29] Hannah Metzler, Hubert Baginski, Thomas Niederkrotenthaler, and David Garcia.
[2] Alan L Berman. 2005. FORENSIC PSYCHIATRY AND FORENSIC PSYCHOLOGY| 2021. Detecting Potentially Harmful and Protective Suicide-related Content on
Psychological Autopsy. (2005). Twitter: A Machine Learning Approach. arXiv preprint arXiv:2112.04796 (2021).

2684
Social Media and Papageno Efect WWW ’23, April 30–May 04, 2023, Austin, TX, USA

[30] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jef Dean. 2013. computing systems. 1–22.
Distributed representations of words and phrases and their compositionality. In [44] Koustuv Saha and Amit Sharma. 2020. Causal Factors of Efective Psychosocial
Advances in neural information processing systems. 3111–3119. Outcomes in Online Mental Health Communities. In ICWSM.
[31] Thomas Niederkrotenthaler and Benedikt Till. 2019. Suicide and the media: [45] Koustuv Saha, Benjamin Sugar, John Torous, Bruno Abrahao, Emre Kıcıman, and
From Werther to Papageno efects–A selective literature review. Suicidologi 24, 2 Munmun De Choudhury. 2019. A social media study on the efects of psychiatric
(2019). medication use. In Proceedings of the International AAAI Conference on Web and
[32] Thomas Niederkrotenthaler and Benedikt Till. 2020. Efects of awareness material Social Media, Vol. 13. 440–451.
featuring individuals with experience of depression and suicidal thoughts on [46] Koustuv Saha, John Torous, Eric D Caine, Munmun De Choudhury, et al. 2020.
an audience with depressive symptoms: randomized controlled trial. Journal of Psychosocial efects of the COVID-19 pandemic: large-scale quasi-experimental
behavior therapy and experimental psychiatry 66 (2020), 101515. study on social media. Journal of medical internet research 22, 11 (2020), e22600.
[33] Thomas Niederkrotenthaler, Benedikt Till, Stefanie Kirchner, Mark Sinyor, Mar- [47] Koustuv Saha, Ingmar Weber, and Munmun De Choudhury. 2018. A Social Media
lies Braun, Jane Pirkis, Ulrich S Tran, Martin Voracek, Florian Arendt, Maria Based Examination of the Efects of Counseling Recommendations After Student
Ftanou, et al. 2022. Efects of media stories of hope and recovery on suicidal Deaths on College Campuses. In Twelfth International AAAI Conference on Web
ideation and help-seeking attitudes and intentions: systematic review and meta- and Social Media.
analysis. The Lancet Public Health 7, 2 (2022), e156–e168. [48] Koustuv Saha, Asra Yousuf, Ryan L Boyd, James W Pennebaker, and Munmun
[34] Thomas Niederkrotenthaler, Martin Voracek, Arno Herberth, Benedikt Till, De Choudhury. 2022. Social media discussions predict mental health consultations
Markus Strauss, Elmar Etzersdorfer, Brigitte Eisenwort, and Gernot Sonneck. on college campuses. Scientifc reports 12, 1 (2022), 123.
2010. Role of media reports in completed and prevented suicide: Werther v. [49] Steven Stack. 1987. Celebrities and suicide: A taxonomy and analysis, 1948-1983.
Papageno efects. The British Journal of Psychiatry 197, 3 (2010), 234–243. American sociological review (1987), 401–412.
[35] James W Pennebaker and Cindy K Chung. 2007. Expressive writing, emotional [50] Shannon Wiltsey Stirman and James W Pennebaker. 2001. Word use in the poetry
upheavals, and health. Handbook of health psychology (2007), 263–284. of suicidal and nonsuicidal poets. Psychosomatic medicine 63, 4 (2001), 517–522.
[36] James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic [51] Elizabeth A Stuart. 2010. Matching methods for causal inference: A review and a
inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates 71, look forward. Statistical science: a review journal of the Institute of Mathematical
2001 (2001), 2001. Statistics 25, 1 (2010), 1.
[37] James W Pennebaker, Matthias R Mehl, and Kate G Niederhofer. 2003. Psycho- [52] Benedikt Till, Florian Arendt, Sebastian Scherr, and Thomas Niederkrotenthaler.
logical aspects of natural language use: Our words, our selves. Annual review of 2018. Efect of educative suicide prevention news articles featuring experts with
psychology 54, 1 (2003), 547–577. vs without personal experience of suicidal ideation: a randomized controlled trial
[38] David P Phillips. 1974. The infuence of suggestion on suicide: Substantive and of the Papageno efect. The Journal of clinical psychiatry 80, 1 (2018), 13780.
theoretical implications of the Werther efect. American sociological review (1974). [53] Benedikt Till, Markus Strauss, Gernot Sonneck, and Thomas Niederkrotenthaler.
[39] David P Phillips. 1978. Airplane accident fatalities increase just after newspaper 2015. Determining the efects of flms with suicidal content: a laboratory experi-
stories about murder and suicide. Science 201, 4357 (1978), 748–750. ment. The British Journal of Psychiatry 207, 1 (2015), 72–78.
[40] Alexandre Reynders, Ad JFM Kerkhof, Geert Molenberghs, and Chantal Van Au- [54] Benedikt Till, Ulrich S Tran, Martin Voracek, and Thomas Niederkrotenthaler.
denhove. 2015. Help-seeking, stigma and attitudes of people with and without 2017. Benefcial and harmful efects of educative suicide prevention websites:
a suicidal past. A comparison between a low and a high suicide rate country. randomised controlled trial exploring Papageno v. Werther efects. The British
Journal of Afective Disorders 178 (2015), 5–11. Journal of Psychiatry 211, 2 (2017), 109–115.
[41] Joshua D Rosenblat, Ron Kakar, and Roger S McIntyre. 2016. The cognitive [55] Gary R VandenBos. 2007. APA dictionary of psychology. American Psychological
efects of antidepressants in major depressive disorder: a systematic review and Association.
meta-analysis of randomized clinical trials. International Journal of Neuropsy- [56] Gaurav Verma, Ankur Bhardwaj, Talayeh Aledavood, Munmun De Choudhury,
chopharmacology 19, 2 (2016). and Srijan Kumar. 2022. Examining the impact of sharing COVID-19 misinfor-
[42] Koustuv Saha, Yozen Liu, Nicholas Vincent, Farhan Asif Chowdhury, Leonardo mation online on mental health. Scientifc Reports 12, 1 (2022), 1–9.
Neves, Neil Shah, and Maarten W Bos. 2021. AdverTiming Matters: Examining [57] Qiaosi Wang, Koustuv Saha, Eric Gregori, David Joyner, and Ashok K Goel. 2021.
User Ad Consumption for Efective Ad Allocations on Social Media. In Proc. CHI. Mutual Theory of Mind in Human-AI Interaction: How Language Refects What
[43] Koustuv Saha, Jordyn Seybolt, Stephen M Mattingly, Talayeh Aledavood, Chai- Students Perceive About a Virtual Teaching Assistant. In Proc. CHI.
tanya Konjeti, Gonzalo J Martinez, Ted Grover, Gloria Mark, and Munmun [58] WHO. 2019. Suicide in the world: global health estimates.
De Choudhury. 2021. What life events are disclosed on social media, how, when, [59] Yunhao Yuan, Gaurav Verma, Barbara Keller, and Talayeh Aledavood. 2022. The
and by whom?. In Proceedings of the 2021 CHI conference on human factors in Impact of COVID-19 Pandemic on LGBTQ Online Communities. arXiv preprint
arXiv:2205.09511 (2022).

2685