Young adults and online political participation

INTRODUCTION

On his first day in office, January 21, 2009, President Barack Obama signed the Memorandum for the Heads of Executive Departments and Agencies on Transparency and Open Government [6]. One of the planks of this document is that the new Administration would be "empowering the public -through greater openness and new technologies -to influence the decisions that affect their lives [2]." As a result of this directive, United States government agencies were encouraged to adopt Web 2.0 technologies and to develop a presence on social media websites to further the core principles of transparency, participation, and collaboration.

However, technologists argue that the government is ill-equipped to develop the full potential of this approach for several reasons [4]. First, the multiplicity of incoming message streams (in addition to the constituents' letters to Congress members) further exacerbates the information overload and exceeds the attention limitations of staffers. This is partly because the current implementations of social media platforms are not necessarily developed with the government's needs in mind [4]. Second, a software solution customized for the purposes of congress (that could potentially tackle information overload issues) has to overcome countless regulatory hurdles and needs to address the realities of deeply entrenched legacy systems [7].

Our goal is to contribute to the understanding of the progress of these government initiatives. In particular, we are interested in the constituent perspective of participation and collaboration with the government. As a result, we believe that a descriptive analysis of the websites of government agencies as well as their utilization of social networking sites could only partially account for the complex interaction processes that lead to effective absorption of citizens' concerns. Similarly, our work differs from content analysis studies of social media websites (e.g., [10]). Rather, we adopt an exploratory study approach to follow users through the process of identifying the appropriate communication channels with government agencies when they have specific scenario-based needs [8].

More specifically, we provide study participants with different scenarios that encourage them to identify the most suitable government agency and to locate a means to contact this party via the Web. Our analysis is focused on several objectives. First, we want to ascertain study participants' difficulties to accomplish the task goal. For this purpose we analyze the users' search behavior and formulation of mental models of search as evidenced by their query terms.

Second, we will determine the relative prominence and importance of Web 2.0 artifacts and social networking sites during the search and participation process.

Third, we comment on the participants' perception of the online participation process. For this purpose, we debrief participants after the experimental episode.

Academic studies predicted that commonplace access to the Internet would likely contribute to the inclusion of new types of Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, individuals in the political participation process [5]. Young adults are one of these constituencies who are traditionally underrepresented in the American electorate and other political activities. In contrast to offline political involvement where young adults (ages 18-24) are the demographic least likely to participate, when it comes to online political activity, the participatory deficit of young adults is less pronounced [9]. Recent events have shown that the youth can be mobilized with grassroots campaigns and the use of new communication paths to participate in elections. See, for example, the 2006 election cycle and the use of mobile phone text messaging [3].

We expect that the relative technology savviness of young adults also makes them particularly accessible to novel government participation tools and sites, and believe that they, therefore, represent a key population for our study goals. Further, our investigation helps to evaluate whether the new efforts by the government likely contribute only to the dissemination of its policy messages, rather than a meaningful bidirectional exchange of ideas between citizen and their directly elected representatives.

In the following, we describe in more detail the study setup and the recruitment particularities before presenting and discussing the study results.

STUDY DESIGN, RECRUITING AND SUBJECTS

We developed four user interaction scenarios geared towards different government agencies [8]. The Food and Drug Administration (FDA), Federal Aviation Administration (FAA), Environmental Protection Agency (EPA), and the White House (WH).

For example, the FDA scenario included the following key section: "Recently, the media has reported a number of diabetes prescription drug recalls. Since you are a diabetes patient, you are concerned about the recall issue and would like to let the government know that there should be more strict regulations for prescription drugs."

Participants were asked to identify an appropriate submission location on the Internet to deliver a personal comment to the government. Subjects were not explicitly given the names or acronyms of the target agencies responsible for handling the issues in each of the four scenarios. We provided all subjects with an identical experimental starting environment on the same personal computer, i.e., a blank Firefox browser page without additional search bar features. Subjects were sequentially participating in two scenarios. Therefore, our quasi-experimental setup is a combination of within (i.e., two scenarios per person) and between (i.e., four different scenarios in total) subjects factors. We conducted counterbalanced trials with the order systematically varied where every possible permutation of the experimental design was tested equally often.

Participants were recruited from the general student population present in a popular student lounge at an US-based Tier 1 public research university. Involvement in the study was voluntary and unpaid, and required agreement to a notice and consent form. In the present study, 12 undergraduate students from undefined majors participated. No students that we approached refused participation in the study, or had to be removed from the subject pool (e.g., for lack of basic online literacy). The limited number of participants allows for an initial appreciation of the search and interaction challenges for this particular task, and for necessary insights for a substantially larger and more focused study that is currently in progress. Our statistical analysis and findings need to be understood within the context of the limited population size for this preliminary study.

We recorded the entire sessions of participants' interaction behavior, and conducted a brief post-experimental debriefing section with each subject.

RESULTS

In the following, we present selected preliminary results derived from the experimental portion of our study. At first, we provide a more formal description of the individuals' search process and include flow diagrams of a successful and failed search ( Figure 1 and Figure 2, respectively) to provide the reader with the opportunity to get familiarized with the diversity of search patterns. Then, we delve into a more qualitative analysis of the interaction patterns.

Numerical observations about search behavior

We are considering a number of interaction variables that were observable in our recordings (R=24); with two sequential recordings for each of our subjects (N=12).

As basic variables we recorded the time spent during the search as well as the number of interaction steps. We counted as interaction steps each entry/modification of search terms or web addresses, as well as navigation moves that lead to a new website, external program (e.g., email) or document, but excluded scrolling, or other mouse movements.

Irrespective of success, participants completed each individual search task on average in 208 sec. (std. = 135 sec.) with about 13.75 interaction steps (std. = 9.4). The relatively high standard deviation is indicative of significant heterogeneity in the subject pool and/or difficulty of the different scenarios.

In fact, we find that the WH scenario (which involved the submission of comments about healthcare reform) required the shortest amount of search time (105 sec., p<0.015) and the fewest interaction steps (7 steps, p=0.02 for two-sided t-test between WH scenario compared to aggregated other observations). That is, the scenario with the most prominent government entity (i.e., the White House) and most pressing policy issue (i.e., healthcare reform) yielded, on the first glance, the most efficient search sessions. On average, participants undertook 5.8 interaction steps on search engine websites and 4.9 steps on federal government websites (ttest, p = 0.15) with the remainder being spent on non-profit, commercial, regional government websites etc.

Further, search patterns became slightly faster (difference=64 sec., p=0.1) and shorter (difference 4.3 steps, p=0.1) in the second session controlled for each individual. During their second search, participants utilized search engines fewer times (difference=1.5, p<0.08), but still spent about the same number of interaction steps on government sites (difference=0, p=0.5).

Search success and its determinants

Subjects were encouraged to decide themselves when they thought to have reached a suitable end point of their search and web navigation. In our analysis, a search was considered a failure if it concluded on a non-government website, non-government social networking page, at a government agency that was not federal or at a federal government agency unrelated to the task. Across all scenarios, the results were varied and can hardly be evaluated as overwhelmingly successful. We also did not identify a significant difference between the first and second session for each subject when comparing success rates.

Handling the above criteria leniently 50% of our subjects in each scenario managed to complete the search task in a satisfactory fashion. However, when evaluating more strictly, participants managed to navigate to the FDA (scenario 1) in only three sessions, only one individual navigated to the FAA (scenario 2), and one subject identified an EPA website (scenario 3). Finally, in scenario 4 one subject navigated to the White House website, and one individual ended the search at the respective Facebook site (see Figure 1). However, in four sessions subjects never managed to navigate to a government website at all during their entire search (12.5%).

From our analysis, we identified multiple factors impeding search success of which we present a sub-selection. Those causes are partly within the control of the individual (Factors 1 and 2), and otherwise more of a black-box structural nature (Factor 3).

Factor 1 Lack of knowledge of government structure:

In our debriefing, we learned that most participants did not know the departments they needed to reach at the beginning of the search. This knowledge, however, clearly matters during the search. For example, subject 3 rapidly concluded the search process (in 13 sec. and 3 interaction steps) with the query 'epa contact form' (scenario 3). See also Figure 1 for another positive example.

Nevertheless, we were surprised by some of the more egregious failures. For example, in scenario 2 one student concluded the search at the Transportation Security Administration (TSA) website (see Figure 2), while another participant wanted to deliver his over-the-counter drug complaint (scenario 3) to the Drug Enforcement Agency (DEA). Similarly, one participant planned to disseminate his message to change.gov even though the transition period between the old and new U.S. government has now clearly ended (scenario 4).

Factor 2 Imprecise and/or static mental models for search:

We observed that some individuals only varied their search terms slightly over time. See, for example, the overly generic search queries for the FAA scenario by subject 2 in the first session ( Figure 2): 'opinion to US government, questions to US government, how to get intouch with the government (sic.), and complaints to government'. The lack of impactful changes to the search queries is particularly problematic if the initial query yields little promising leads for further browsing behavior and creates negative path-dependence.

Similarly, we registered that search queries did not always change meaningfully with the scenario. For example, subject 12 used in the first scenario (WH) the query 'ways to submit Government opinions' and in the next session (EPA) the query 'contact government.'

Factor 3 Prominence of Government sites in search results:

Good queries led usually to rapid search progress, but not always to task success. For example, subject 4 in the FDA scenario searched for 'government regulations for prescription drugs comments' and eventually settled for a consulting firm with an economics focus.

One issue is the overwhelming reliance on search engine quality. Several subjects chose the very first search result irrespective of an obvious lack of relevance for the search task.

Moreover, due to the severe competition for search result placement with non-government organizations, for-profit companies, individual politicians, news organizations etc. the federal government sites frequently do not achieve top placement given the queries our subjects used. It appears obvious that the government is somewhat at a disadvantage in this competitive process due to more rigid rules in website promotion (e.g., we assume that the government is not proactively participating in search result manipulation, but is rather the target or victim).

IMPACT OF SOCIAL MEDIA

A noticeable impact of social media on the behavior of our participants was limited to two significant events. One subject selected a social media site to submit a message (see Figure 1). This individual followed the link 'Join us live at www.facebook.com/whitehouse…' while browsing at healthreform.gov. Another participant (subject 8, EPA scenario) selected the Facebook page of USA.gov from derived search results and navigated from there via 'contact us' to USA.gov to leave a commentary.

These limited interactions stand in contrast to a fairly consistent exposure to social media indicators on websites that participants navigated to. Consider that in 11 out of 24 sessions social media cues were present on the last page that subjects navigated to, but not used by most of these subjects.

For example, the last navigation step in Figure 2 includes a "The TSA Blog | Blog Now" button towards the top of the page. Participants 6 and 7 were presented with a "Stay Connected" box with the names and icons for Facebook, Twitter, Flickr, MySpace, YouTube, Vimeo, iTunes, LinkedIn at WhiteHouse.gov. Other participants browsed to non-governmental organizations' websites with links to their respective social network presence but did not follow up on those opportunities.

When asked about the relevance of social network cues to political participation we received nuanced results in the debriefing interviews. Most participants did not make the connection between government, political participation and social media. However, we also received responses encouraging its use, for example, stating that "social media is the way of the future -I'm learning about it in class." Subject 8 added that "Facebook is a lot more useful, I never would have guessed to access the government through Facebook."

Another somewhat differentiating opinion suggested that "the younger generations will definitely be using [social media], but it's not there yet. If I wanted to voice my opinions, I'd call, or email first -actually, I would join a group." This opinion was echoed by several subjects who emphasized the importance of social connections -however, in the offline world. E.g., one participant very firmly stated that "formal letters get no response, you need to KNOW someone (emphasis added)."

DISCUSSION AND FUTURE WORK

In our interviews, we noticed frustration related to political participation. For example, one subject interjected that "the government will only care about issues if you know the people personally. Unless you have an 'in' your opinions don't make a difference, [the politicians] just pretend to care." Participants mentioned "red tape" and other interaction hurdles.

However, government agencies are bound by suffocating restrictions themselves [5]. Therefore, we were surprised to find so many referrals to social network links mediated by agencies' and politicians' websites. Nonetheless, we found that those opportunities were rarely utilized by our study participants.

For the young adults in our study the dispersion of government activities across many websites posed a significant challenge. Several participants expressed sentiments similar to one individual who thought our task was "difficult because [he/she did] not know the main government page." Many sources indicate that young people today are not as knowledgeable and involved in politics as their parents where, and that this phenomenon is less related to their age and more so attributed to a social change in attitudes toward politics [1].

We are currently extending our pilot study on the relevance of online social networks for political participation to study the framing effects of different propaganda message delivery conduits, i.e., through social networks versus traditional communication channels. In this larger study, the scenario provided is better targeted to our university student demographic.

Through our work, we hope to deepen the understanding of the relevance of social media for the engagement of underserved groups, and interactions that reach beyond the traditional comfort zone of many social network users.

ACKNOWLEDGMENTS

We appreciate the detailed comments from the anonymous reviewers. We also thank James Katz for his support in defining and guiding our research, and even more so, for introducing us to one another.