ECI 510 FINAL PROJECT

ECI 510 Final Project

Serena Reavis
North Carolina State University
ECI 510-601
December 1, 2014
Dr. Jan Lucas

Table of Contents

ECI 510 FINAL PROJECT

IRB Proposal
WP #1
Title
Sampling
Survey Results
Personal Reflection on Survey Results
Final Write-Up of Interview
Overview
Interviewee
Consent
Interview Questions
Setting
Transcript of Interview
Conclusions
Research Implications
Personal Reflection on Qualitative Research
Focus Group Plan Activity
Field Research--Interview Activity
Research Questions
References

ECI 510 FINAL PROJECT

IRB Proposal
North Carolina State University
Institutional Review Board for the Use of Human Subjects in Research
Submission for New Studies

GENERAL INFORMATION

1. Date Submitted: December 3, 2014

2. Title of Project: Focus Group on Faculty Responsibilities in Accessibility
3. Principal Investigator: Serena Reavis
4. Principal Investigator Email: sreavis@ncsu.edu
5. Department: College of Education
6. Campus Box Number: 7113
7. Phone Number: 919-513-0113
8. Faculty Sponsor Name if Student Submission: Jan Lucas
9. Faculty Sponsor Email Address if Student Submission: jwlucas@ncsu.edu
10. Source of Funding (Sponsor, Federal, External, etc): College of Education, NCSU
If Externally funded, include sponsor name and university account number:
RANK:
Faculty: ; Student: Undergraduate Masters PhD; Other:
As the principal investigator, my signature testifies that I have read and understood the University Policy and
Procedures for the Use of Human Subjects in Research. I assure the Committee that all procedures performed under
this project will be conducted exactly as outlined in the Proposal Narrative and that any modification to this
protocol will be submitted to the Committee in the form of an amendment for its approval prior to implementation.
*Electronic submissions to the IRB are considered signed via an electronic signature*
Principal Investigator:
Serena Reavis
(typed/printed name)

Serena Reavis

December 3, 2014

(signature)

(date)

As the faculty sponsor, my signature (or electronic submission) testifies that I have reviewed this application
thoroughly and will oversee the research in its entirety. I hereby acknowledge my role as the principal investigator
of record.
Faculty Sponsor:

(typed/printed name)

(signature)

(date)

PLEASE COMPLETE AND E-MAIL TO: irb-coordinator@ncsu.edu

Please include consent forms and other study documents with your application and submit as one document.
*Electronic submissions to the IRB are considered signed via an electronic signature. For student submissions
this means that the faculty sponsor has reviewed the proposal prior to it being submitted and is copied on the
submission.

ECI 510 FINAL PROJECT

*********************************************************************************************
****
For SPARCS office use only

Reviewer Decision (Expedited or Exempt Review)

Exempt
Approved Approved pending modifications
Expedited Review Category: 1

2 3

Table

5 6 7 8a 8b 8c 9

___________________________________________________________________________________________
Reviewer Name
Signature
Date
North Carolina State University
Institutional Review Board for the Use of Human Subjects in Research
GUIDELINES FOR A PROPOSAL NARRATIVE
In your narrative, address each of the topics outlined below. Every application for IRB review must contain a
proposal narrative, and failure to follow these directions will result in delays in reviewing/processing the
protocol.
A.

INTRODUCTION

1. Briefly describe in lay language the purpose of the proposed research and why it
is important.

The purpose of this focus group is to provide feedback about the responsibilities of faculty
members when it comes to classroom accessibility.

2. If student research, indicate whether for a course, thesis, dissertation, or

independent research.

ECI 510-601
B.

SUBJECT POPULATION
1.
How many subjects will be involved in the research?

12
2.
Describe how subjects will be recruited. Please provide the IRB with any
recruitment materials that will be used.
Faculty members will be recruited across campus from a list of participants in an online
Introduction to Accessibility course (note: all faculty members are required to take this course
within one year of hire). Selection from this list will ensure that the participants have the
required knowledge to discuss accessibility in the focus group. To recruit faculty members, an
email will be sent to people who have completed the course asking for the time in the focus
group.

3. List specific eligibility requirements for subjects (or describe screening procedures),
including those criteria that would exclude otherwise acceptable subjects.

ECI 510 FINAL PROJECT

Participants must have successfully completed the online Introduction to Accessibility

course.
4.
Explain any sampling procedure that might exclude specific populations.
Because this is a focus group interview, some qualifying faculty members may be
unable to attend the meeting based on time or location.
5.
Disclose any relationship between researcher and subjects - such as,
teacher/student; employer/employee.
The researcher was a faculty member at the institution for five years.
6.

Check any vulnerable populations included in study:

minors (under age 18) - if so, have you included a line on the consent form for the
parent/guardian signature
fetuses
pregnant women
persons with mental, psychiatric or emotional disabilities
persons with physical disabilities
economically or educationally disadvantaged
prisoners
elderly
students from a class taught by principal investigator
other vulnerable population.

7.

If any of the above are used, state the necessity for doing so. Please indicate the approximate age
range of the minors to be involved.

Because the sample will include current faculty members at Wake Tech, it may include
women who are pregnant, persons with mental, psychiatric, or emotional disabilities,
or persons with physical disabilities. These statuses would not exclude someone from
participating because they would not affect their ability to speak to accessibility issues,
or be present in the focus group space. The space would be selected based on access.
C.

PROCEDURES TO BE FOLLOWED

1. In lay language, describe completely all procedures to be followed during the course of
the experimentation. Provide sufficient detail so that the Committee is able to assess potential risks to
human subjects. In order for the IRB to completely understand the experience of the subjects in your
project, please provide a detailed outline of everything subjects will experience as a result of
participating in your project. Please be specific and include information on all aspects of the research,
through subject recruitment and ending when the subject's role in the project is complete. All
descriptions should include the informed consent process, interactions between the subjects and the
researcher, and any tasks, tests, etc. that involve subjects. If the project involves more than one group
of subjects (e.g. teachers and students, employees and supervisors), please make sure to provide
descriptions for each subject group.
Faculty members will be recruited across campus from a list of participants in an
online Introduction to Accessibility course. To recruit faculty members, an email will

ECI 510 FINAL PROJECT

be sent to people who have completed the course asking for the time in the focus
group. Twelve participants will be recruited in attempt to have at least 6-10 follow
through on the day of the interview.
The focus group will be held in an available conference room, which includes sitting
arranged around a conference table, located in a central part of campus. The date and
time of the focus group will be selected based on common campus and faculty
schedules to ensure adequate participation, and will most likely be in the early
afternoon on a Friday.
A week before the focus group, participants will be sent a follow-up email reminding
of the location and time of the group.
On the day of the focus group, the conference room will be organized, providing a
demographic survey, consent form with incentive acknowledgement, and a bottle of
water for each participant at the seats. Candy and fruit will be provided in bowls on
the table to keep the atmosphere light. A recording device will be centrally located so
that it can pick up everyones contribution.
Once all participants have arrived, the moderator will greet the participants and thank
them for their time, explain how the focus group will work, and ask participants to
sign the consent form and fill out the demographic survey, ensuring them that all
information shared will be anonymous. Because the responsibility of ensuring
accessibility is charged and potentially a legal issue, the moderator will acknowledge
that this topic can be a controversial issue and ask for frank discussion based on
participants experience. It will be important to clearly indicate that goal of the focus
group and emphasize that participants experience will help with the research on this
topic, especially as it relates to faculty responsibilities at the college.
Participants will be asked to answer the following questions:
1. What kind of curriculum content materials do you create for your
class?
2. Since many of you create _____ (example from first question,
such as videos), can you explain how to ensure that it is accessible?
3. To what extent do you think it is your responsibility to make
_____ accessible? Why?
4. What other types of materials do you use that you do not create?
5. To what extent are those materials accessible when you find
them?
6. To what extent do you think it is your responsibility to make these
found materials accessible? Why?
7. What makes it challenging to ensure that content is accessible for

ECI 510 FINAL PROJECT

students in your class?

As participants answer questions, they will interact with one another and the
researcher, while the observer quietly takes notes.
At the end of the interview, participants will be thanked for their time and provided
compensation.

2.

How much time will be required of each subject?

1.5 hours
D.

POTENTIAL RISKS
1.
State the potential risks (physical, psychological, financial, social, legal or other)
connected with the proposed procedures and explain the steps taken to minimize these risks.

Participants will be asked to share their personal and professional experiences

learning and implementing accessibility in their course. Because accessibility can
coincide with legal issues, such as section 508 of the ADA, participants may feel at risk
sharing the experience. In addition, because the experience relates to their current
positions within the college, they may feel that the information may be shared with
supervisors, etc. To minimize their concern, an overview of the act, a statement about
confidentiality and anonymity, and a statement about the purpose of the information,
will be provided at the interview.
2. Will there be a request for information that subjects might consider to be personal or sensitive (e.g.
private behavior, economic status, sexual issues, religious beliefs, or other matters that if made public
might impair their self-esteem or reputation or could reasonably place the subjects at risk of criminal or
civil liability)?

No
a. If yes, please describe and explain the steps taken to
minimize these risks.

b. Could any of the study procedures produce stress or

anxiety, or be considered offensive, threatening, or degrading? If yes, please
describe why they are important and what arrangements have been made for
handling an emotional reaction from the subject.
No
3. How will data be recorded and stored?

ECI 510 FINAL PROJECT

The focus group interview will be recorded and then transcribed. An observer will
take additional notes that will be included in the transcription.
a. How will identifiers be used in study notes and other materials?
I will use anonymous identifiers to replace any names, courses, or assignments
referenced in the focus group interview.
b. How will reports will be written, in aggregate terms, or will individual
responses be described?
The whole interview will be transcribed and coded for use in data analysis.
4. If audio or videotaping is done how will the tapes be stored and
how/when will the tapes be destroyed at the conclusion of the study.
The audio recording will be stored on the researchers computer until it has been
described. The interview will not be transferred or uploaded to another device. The
audio recording will be deleted after the study has been published.
5. Is there any deception of the human subjects involved in this study? If
yes, please describe why it is necessary and describe the debriefing procedures that have been
arranged.
No
E.
POTENTIAL BENEFITS
This does not include any form of compensation for participation.
1.
What, if any, direct benefit is to be gained by the subject? If no direct
benefit is expected, but indirect benefit may be expected (knowledge may be gained that could help
others), please explain.

Each participant will receive a $25 honorarium for participating, and their input will
be considered for future implementation of accessiblity iniatives on campus.
F.

COMPENSATION
Please keep in mind that the logistics of providing compensation to your subjects (e.g., if your business
office requires names of subjects who received compensation) may compromise anonymity or complicate
confidentiality protections. If, while arranging for subject compensation, you must make changes to the
anonymity or confidentiality provisions for your research, you must contact the IRB office prior to
implementing those changes.

1.

Explain compensation provisions if the subject withdraws prior to completion of

the study.

Each participant will receive the honorarium after the completion of the interview.
Because there is only one interview, participants who do not attend will not receive
the compensation.

ECI 510 FINAL PROJECT

2.
If class credit will be given, list the amount and alternative ways to earn the
same amount of credit.
N/A
G.

COLLABORATORS
1. If you anticipate that additional investigators (other than those named on Cover Page)
may be involved in this research, list them here indicating their institution, department and phone
number.

Amy Smith, WTCC, eLearning Department, 919-513-1330

2. Will anyone besides the PI or the research team have access to the data (including
completed surveys) from the moment they are collected until they are destroyed.
Raw data will remain confidential between the PI and observer (named above).
Summative statistics and qualitative data will be shared with college administration,
published, and shared at conferences.
H.
CONFLICT OF INTEREST
1. Do you have a significant financial interest or other conflict of interest in the sponsor of this project? No
2. Does your current conflicts of interest management plan include this relationship and is it being properly
followed?
I.

ADDITIONAL INFORMATION
1.
If a questionnaire, survey or interview instrument is to be used, attach a copy to
this proposal.

2.

Attach a copy of the informed consent form to this proposal.

3.

Please provide any additional materials that may aid the IRB in making its

decision.
J.

HUMAN SUBJECT ETHICS TRAINING

*Please consider taking the Collaborative Institutional Training Initiative (CITI), a free, comprehensive
ethics training program for researchers conducting research with human subjects. Just click on the
underlined link.

WP #1

Name :

Reavis, Serena

ECI 510 FINAL PROJECT

10

Course :

ECI 510-601 Fall 2014

Title of Article :

FACTORS INFLUENCING THE USE OF CAPTIONS BY

FOREIGN LANUAGE LEARNERS: AN EYE-TRACKING

Bibliography
Citation :

Gass, S., Sydorenko, T., & Winke, P. (2013). Factors

influencing the use of captions by foreign language
learners: An eye-tracking survey. The Modern Language
Journal, 97(1), 254-275. doi: 10.1111/j.15404781.2012.01432.x

In the article Factors Influencing the Use of Captions by Foreign Language

Learners: An Eye-Tracking Study, Gass, Sydorenko, and Winke (2013) discuss their
research study that explores the underlying factors that affect foreign language students
use of captions when watching videos of classroom material. Past research studies have
shown a correlation between use of captions in foreign language study and improved
comprehension but provided few specifics on how often students used captions and
what other elements might influence students use of the captions. In particular, most
data for caption use was recorded in past studies based on participant interviews and
verbal reports of caption use. Therefore, the researchers conducted a study of foreign
language learners in Arabic, Chinese, Russian, and Spanish and used eye-tracking
software to record how often students read captions during a video so that they would
have specific data on how and when captions were used. They also explored whether
the language learned and content familiarity influenced the use of captions.
This review is based on the guiding questions from Pyraczaks Evaluating
Research in Academic Journals: A Practical Guide to Realistic Evaluation and will
examine the effectiveness of the studys title and sampling method.

ECI 510 FINAL PROJECT

11

Title
The article title, Factors Influencing the Use of Captions by Foreign Language
Learners: An Eye-Tracking Study, provides a succinct but descriptive overview of the
researchers study. The main element under study is clearly identified as the use of
captions, and the primary subjects are identified as foreign language learners. Since the
students were all in college, the title could include this information; however, the
distinguishing factor for this particular study in relation to past studies, eye-tracking, is
also provided in the subtitle. Once more studies have done in these populations with
eye-tracking software, this title may be less effective, but when published the
distinguishing factor is clear.
Though certain elements, such as the type of language under investigation and
the additional factors underlying the study are not provided in the title, the overall
guiding research question is apparent.
Overall, this title is effective because a researcher can understand the key
elements in the study based and distinguish it from similar past studies on this title.

Sampling
The participants in this study were recruited from classes at the home institution
of the researchers, Michigan State University, which suggests that the study used a
convenience sample. The specific method of recruitment and selection are not
discussed, but the sample size and distribution among the four languages are, making
the sample possible to replicate. Specifically, 33 participants were distributed into four
groups: seven in Arabic, seven in Spanish, eight in Russian, and ten in Chinese; eleven
of the participants were male, and 22 were female. Though a convenience sample is not

ECI 510 FINAL PROJECT

12

generalizable, this particular sample is sufficient for fulfilling the main goal of the study,
which was to record how often participants used captions by gathering eye-tracking
data. To be generalizable, this study would need to be repeated with a random sample.
By using a stratified sample across the four languages, the sample provides an
opportunity to compare caption use among the different languages under study. In
addition, researchers selected participants with controlling factors in mindeach
participant was a native English speaker in their fourth semester of foreign language
study who had never participated in study abroad. The researchers mention that they
would prefer to test participants on their level of foreign language knowledge to control
for other factors because the fourth semester status does not guarantee true similarity
between participants.
Researchers did obtain consent from the participants by using a consent form
and collected background data, such as demographics and prior foreign language study,
but no specific information is provided about the elements of consent involved with the
study.
Overall, the sample was adequate for this study but is not generalizable.

Survey Results
This survey was conducted to obtain an understanding of the types of students
who take online classes and why they take online classes. In addition, the survey
collected data about students perspectives on online classes and resources available to
online students.

ECI 510 FINAL PROJECT

13

The survey was opened on October 7, 2014 and closed on October 20, 2014.

Though 13 participants opened the survey, only ten participants completed the survey
questions, resulting in a completion rate of 77%. Some of the questions utilized display
logic to target participants based on previous answers, so those questions have a smaller
number of responses, but purposely to gather relevant data.
This was a convenience sample taken from students in an online ECI 510 course
offered in the College of Education at NC State University during Fall 2014. Of the ten
participants, 60% ranged in age from 25-34, and the other participants ranged from 16
to over 65. Eighty percent of participants were female, and 20% were male.
Overall, participants seemed satisfied with online classes and the supports
offered. However, one of the more interesting findings was that respondents were least
happy with the group work assigned in online classes with 43% of respondents replying
they felt neutral about group work. In comparison, participants were agreed or strongly
agreed that they were happy with other aspects of online learning, such as types of
assignments and timely feedback.
Based on the results, I would be interested in asking more targeted questions
about the types of assignments, including group work, and how the assignments affect
learning outcomes in the course. I would also like to get more data on how students
compare online and face-to-face environments.

ECI 510 FINAL PROJECT

14

Last Modified: 11/07/2014

1. Which of the following class types have you taken while

enrolled in your graduate program at NCSU?
#

Answer

Response

Face-to-Face

73%

Online

82%

Blended:
Classes with
face-to-face
and online
component

55%

Statistic
Min Value
Max Value
Total Responses

Value
1
3
11

2. Did you take fully online courses before enrolling in the

graduate program at NC State?

ECI 510 FINAL PROJECT

Answer
#

15
Response

Yes

45%

No

55%

Total

11

100%

Statistic
Min Value
Max Value
Mean
Variance
Standard Deviation
Total Responses

Value
1
2
1.55
0.27
0.52
11

3. Did your experience in the previous online classes

influence your decision to take online classes in your
graduate program at NC State?
* Only participants who answered Yes to Question 2 were asked to answer this
question.
#

Answer

Response

Yes

80%

No

20%

Total

100%

Statistic
Min Value
Max Value
Mean
Variance
Standard Deviation
Total Responses

Value
1
2
1.20
0.20
0.45
5

ECI 510 FINAL PROJECT

16

4. Briefly explain how your experience in previous online

classes influenced your decision to take online classes in
your graduate program at NC State.
* Only participants who answered Yes to Question 2 were asked to answer this
question.
Text Response
I previously had positive experiences with online courses, which encouraged me to enroll
in NCSU. I found these courses to be useful and appreciated the flexibility they provided
me in terms of time-management.
Good experience reduced hesitancy to enroll in online classes. Also familiarity and
comfort with technology.
It was much more convenient to take courses on my own time.
Statistic
Total Responses

Value
3

5. How many online classes have you taken in your graduate

program at NCSU?
* Only participants who answered Online to Question 1 were asked to answer
this question.
#

Answer

Response

1-2

22%

3-5

22%

6-9

33%

10-12

11%

13+

11%

Total

100%

Statistic
Min Value
Max Value
Mean
Variance
Standard Deviation
Total Responses

Value
1
5
2.67
1.75
1.32
9

ECI 510 FINAL PROJECT

17

6. Which of the following best describes your experience

while enrolled in the graduate program at NCSU?
#
1

Answer
taking
online
classes
only
taking
online and
face-to-face
classes
taking
online and
blended
classes
taking all
three types
of classes
Total

Statistic
Min Value
Max Value
Mean
Variance
Standard Deviation
Total Responses

Response

33%

11%

0%

56%

100%
Value
1
4
2.78
2.19
1.48
9

ECI 510 FINAL PROJECT

18

7. How would you rate your experience in online courses in

the graduate program at NCSU?
* Only participants who answered Online to Question 1 were asked to answer
this question.

Question

Very
Satisfie
d

Somewh
at
Satisfied

Neutr
al

1 Instructor
Involveme
nt
2 Interaction
with Peers
3 Group
Work
4 Types of
Assignmen
ts
5 Course
Organizati
on
6 Timely
Feedback

Very
Dissatisfi
ed

Total
Respons
es

Mea
n

Somewha
t
Dissatisfi
ed
0

1.75

1.88

2.43

1.63

1.75

1.63

ECI 510 FINAL PROJECT

Instructor
Statistic
Involvemen
t
Min Value
1
Max Value
2
Mean
1.75
Variance
0.21
Standard
0.46
Deviation
Total
8
Response
s

19
Interactio
n with
Peers
1
3
1.88
0.70
0.83

Grou
p
Work
2
3
2.43
0.29
0.53

Types of
Assignment
s
1
3
1.63
0.55
0.74

Course
Organizatio
n
1
2
1.75
0.21
0.46

Timely
Feedbac
k
1
2
1.63
0.27
0.52

8. What, if anything, would you change about the online

classes?
* Only participants who answered Online to Question 1 were asked to answer
this question.
Text Response
Some courses get caught in the "Read, forum post, three replies, then repeat" week after
week. They would be better served maximizing some of the online collaborative tools
that can provide students with a more meaningful, engaging learning experience.
Making instructors have a more clear Moodle and grade book layout
I would like to meet sometime for a beer with the class.
More meaningful projects
If we could meet with each other in a video such as group talking, that's will be better.
All large projects or papers need to be broken down into sections being due one at a
time.
I really enjoy online learning, there are some instances when it would be nice to have
more of a presence from the professor. I also don't enjoy long-distance group
work/projects because they can be so hard to organize; however, I don't love group
projects in general.
Statistic
Total Responses

Value
7

ECI 510 FINAL PROJECT

20

9. What are the main reasons you are taking classes only
online?
* Only participants who answered Online to Question 1 were asked to answer
this question.
#
1
2

Answer
commute to
campus is too
long
classes are
not offered at
times I can
come
I prefer the
online
learning
environment
other:

Response

0%

67%

67%

33%

other:
work and family obligations
Statistic
Min Value
Max Value
Total Responses

Value
2
4
3

ECI 510 FINAL PROJECT

21

10. How satisfied are you with the academic support offered
through NCSU?
Question

Very
Satisfi
ed

Somewh
at
Satisfied

Neutr
al

1 Advisor
2 DepartmentSpecific
Corresponde
nce
3 NCSU
Writing
Center
4 Thesis and
Dissertation
Support
Services
5 Department
Website
6 MyPack
Portal
7 Preparing
Future
Leaders
8 Student
Support
Groups
9 Other:

4
4

5
3

Other:

Very
Dissatisfi
ed

Total
Respons
es

Mea
n

0
3

Somewh
at
Dissatisfi
ed
1
0

0
0

10
10

1.80
1.90

2.00

2.00

10

2.10

10

1.90

2.25

1.75

0.00

ECI 510 FINAL PROJECT 22

Advi Departmen NCS
sor
t-Specific
U
Correspon Writi
Statisti
dence
ng
c
Cent
er
Min
Value
Max
Value
Mean
Varianc
e
Standa
rd
Deviati
on
Total
Respon
ses

Thesis
and
Dissert
ation
Support
Service
s
2

Depart
ment
Website

MyP
ack
Porta
l

Prepar
ing
Future
Leade
rs

Stud
ent
Supp
ort
Grou
ps

Oth
er:

1.80
0.84

1.90
0.77

2.00
2.00

2.00
0.00

2.10
0.54

1.90
0.54

2.25
0.25

1.75
0.25

0.00
0.00

0.92

0.88

1.41

0.00

0.74

0.74

0.50

0.50

0.00

10

10

10

10

11. How satisfied are you with the personal support offered
through NCSU?
#

Questio
n

1 Student
Health
Care
2 Legal
Service
s
3 Diversit
y
Support
Centers
4 Student
Life
Website
5 Other:

Very
Satisfie
d

Somewh
at
Satisfied

Neutr
al

Very
Dissatisfie
d

Total
Response
s

Mea
n

Somewha
t
Dissatisfie
d
0

1.25

0.00

0.00

2.00

1.00

Other:
Counseling Center

ECI 510 FINAL PROJECT 23

Student
Legal
Statistic
Health Care
Services
Min Value
Max Value
Mean
Variance
Standard
Deviation
Total
Responses

1
2
1.25
0.25
0.50

0.00
0.00
0.00

Diversity
Support
Centers
0.00
0.00
0.00

Student Life
Website

Other:

2
2
2.00
0.00
0.00

1
1
1.00
0.00
0.00

12. What is your current age? (U.S. Census)

Answer

Response

16 to 19

0%

20 to 24

10%

25 to 34

60%

35 to 44

10%

45 to 54

10%

55 to 64

10%

65 or over

0%

Total

10

100%

ECI 510 FINAL PROJECT

Statistic
Min Value
Max Value
Mean
Variance
Standard Deviation
Total Responses

24
Value
2
6
3.50
1.39
1.18
10

13. What is your gender?

Answer

Response

Female

80%

Male

20%

Other

0%

Total

10

100%

Other
Statistic
Min Value
Max Value
Mean
Variance
Standard Deviation
Total Responses

Value
1
2
1.20
0.18
0.42
10

ECI 510 FINAL PROJECT

25

14. What is your race?

#

Answer

Response

African American

10%

Asian

30%

Hispanic

0%

Native American

0%

Pacific Islander

0%

White/Caucasian

60%

Other

0%

Total

10

100%

Other
Statistic
Min Value
Max Value
Mean
Variance
Standard Deviation
Total Responses

Value
1
6
4.30
4.90
2.21
10

15. What is the highest level of education you have

completed?
#
1
2
3
4

Answer
4-year
College
Degree
Masters
Degree
Doctoral
Degree
Professional
Degree (JD,
MD)
Total

Response

80%

20%

0%

0%

10

100%

ECI 510 FINAL PROJECT

Statistic
Min Value
Max Value
Mean
Variance
Standard Deviation
Total Responses

26
Value
1
2
1.20
0.18
0.42
10

16. What is your current enrollment status at NC State?

#
1

Answer

Response

Full-time (at
least 3
graduate
classes a
semester)
Part-time

40%

60%

Total

10

100%

Statistic
Min Value
Max Value
Mean
Variance
Standard Deviation
Total Responses

Value
1
2
1.60
0.27
0.52
10

ECI 510 FINAL PROJECT

27

Personal Reflection on Survey Results

During the Web Survey Project, I designed a survey about students experiences
in online classes at NC State University. Because I am working in DELTA currently, this
information could be potentially useful in determining what aspects of online teaching
could be improved, and thus, what types of workshops would be most useful to online
faculty at the university. During the first iteration of the survey, I learned to use the
Display Logic in Qualtrics to control what participants answered certain questions based
on their previous answers (and experiences) and found that some of the questions
needed revising or more explanation to obtain clear information. After receiving
participant feedback and data from students in ECI 510, I was able to revise the survey
with more effective questions.
One of the most enjoyable aspects of this project was the practical activities,
including designing the survey, evaluating the data, and creating a report, which allowed
me to design elements that could potentially be used in actual research. Having the
opportunity to practice before I was conducting real research allowed me to find some of
the flaws in my design as well as learn how the software works, specially because we
used Qualtrics, which is the software offered to faculty and students at NC State
University, where I will most likely complete my academic research. In addition, the
participant data really allowed me to see what issues the survey had and have a chance
to redesign the survey so that I was collecting more relevant feedback.
At the start of the project, I found the most challenging aspect figuring out ways
to ask questions that would be clear to the participants but also obtain the information
that I was seeking. For instance, on the first draft of the survey, I had included two

ECI 510 FINAL PROJECT

28

questions that seemed to measure different elements, both what type of classes students
were taking and what percentage of the classes they were taking were online, but I
quickly realized in review that I could rewrite the first question slightly and obtain both
of those statistics from one question, thus reducing the amount of time that participants
had to fill out the survey and also potential confusion over the two questions.
However, as I worked on the final write-up, I found that I did not have the data to
create a report using my second survey, which I preferred because it had better
questions. Only four classmates completed the second survey, so I did not have enough
responses to substantiate a report. I also found that only ten people actually completed
the full survey in the first iteration. Because I completed everyones survey twice, I was
expecting to have two full datasets to use to compare the survey results and was
disappointed that I was not able to do that analysis. I think having a way to tie the grade
to completion might be important, but given the restraints of the system, I am not sure
that is possible. Otherwise, I really enjoyed all aspects of the project and learned a lot
about designing surveys.
After completing the project, I think that I have come to appreciate the intricacy
of writing questions that pinpoint the elements I am truly looking for. For instance,
when I asked students to rate their experiences with certain elements of online classes,
such as feedback, group work, etc., I ultimately wanted to know what was working and
what wasnt working, and the qualitative question I asked next, allowed me to revise the
questions about certain elements in the second iteration to ensure I was asking about
the right type of elements. In addition, I saw how adding more text entry questions
helped explain some of the responses in the matrix tables. Overall, this project allowed
me to learn the effective elements of survey design.

ECI 510 FINAL PROJECT

29

Final Write-Up of Interview

Overview

The purpose of this interview is to gather information about types and amount of
smartphone use among current smartphone users.
Interviewee
The interviewee is a female-identified, 34-year old. She is currently a doctoral student
in a social work program, writing her thesis and researching part-time job options. She
has owned a smartphone for three years and currently owns a iPhone 4.

ECI 510 FINAL PROJECT

30

Consent

Before conducting the interview, I informed the interviewee of the nature and length of
the interview, the ability to stop participating at any time, and about confidentiality of
the data collected. She provided informed consent to the interview and recording.

Interview Questions
1. Tell me a little about your decision to buy a smartphone. (to establish
smartphone history--how long has she had a smartphone, why did she buy a
smartphone, etc.)
2. How do you use your smartphone? (to establish personal/professional use,
primary use, etc.)
3. How has your smartphone use affected your life? (to establish overall
perception of smartphones, potential for continued use in the future, etc.)

Setting

I meet the interviewee at 1:00 p.m. on Saturday, November 15, 2014, at a local
coffeeshop. We both got a drink before choosing a table near the back where fewer
people were sitting. Then, we began the interview, using the VoiceMemos app on my
smartphone, an iPhone 5c, to record.
The interviewee sat across the table from me with her iPhone on the table. She was
wearing business casual outfit and was friendly and relaxed throughout the interview.

Transcript of Interview
Note: In the transcription below, R indicates the interviewer, and E indicates the
interviewee.
R: Tell me a little bit about your decision to buy a smartphone.
E: I bought a smartphone because my other phone died, and I was ambivalent about a
smartphone because I was not a big user of technology prior to purchasing it, but it was
the right time to try a smartphone, so I bought one. Mostly because it was just easier.
R: It was easier than a regular cell phone? [wondering about the reference in that last
sentence]
E: It was...um...I had a good plan, and I was able to get a new one without...um...I was
able to get a smartphone inexpensively so that I could try it out without too much risk,
and I had a shared data plan so it was more affordable.
R: Okay. How long have you had the smartphone now?
E: Well, for a couple of years, I had a Android, and then more recently, I had to upgrade
to an iPhone, so I probably have had a smartphone for three years.
R: How do you use your smartphone?

ECI 510 FINAL PROJECT

31

E: I use it for phone functions. [matter-of-factly]

R: Like making phone calls?
E: Yes, primarily. I also use it to check email. I use it for texting although I am not a big
texter. Mostly phone and email.
R: So, is there anything on the smartphone that you wouldn't use on a regular cell
phone? [wanting to distinguish between smartphone and regular cell phone functions]
E: Well, I imagine the internet and the texting...all the email and other functions, I
wouldn't use on another phone.
R: So, email is the primary thing that you use? Do you use it for checking your email for
work, for school, and personal reasons?
E: Yes, for all of the above.
R: You're using it for all email applications?
E: Yes.
R: Do you use any other applications on the phone?
E: Sometimes, I look at Facebook but not really. Sometimes, I play a game. Sometimes,
I use the map application and the weather application. [thinking hard about this
question]
R: So, weather, maps, and games?
E: and the Flashlight?! [jokingly]
R: Which of those apps do you primarily use the most?
E: Maps [hesitatingly]. [E picks up the iPhone and looks at it]. Yeah....and the photos.
[very happily]. Actually, I take a lot of photos. I forgot that.
R: So, you use the camera as well?
E: Yeah, I use the camera a lot. [smiling] I actually probably use the phone for email
first, phone second, pictures third, texting fourth, and all other applications after that.
R: How would you say that having the smartphone has affected your life?
E: Well, I do think that it has improved it substantially. Like, I use the internet too. Can
I assume that is part...I use the internet to look up things. For instance, last night I was
in Chapel Hill doing something for my school and I was hungry and wanted to go to eat,
so I looked up a restaurant...or if I am trying to make decisions about something, I will
use my phone to look it up. So, I feel like that has freed me up to be able to be more
flexible in my life in some ways. Although, I don't really like being super accessible to
people, so I still prefer to have control over when and how I use the phone. I don't live
by this if someone calls I have to answer, if someone texts I have to respond right away
because I just think that we should not live in a society where we are expected to be on
24 hours a day, so I don't keep my phone on me all the time. But, I think it has created a
lot of convenience in other parts of my life.
R: Do you find yourself using the smartphone over using your laptop or tablet at times?
E: I don't like to compose emails on my smartphone, so I check my email on it, but I
typically reply using a laptop.

ECI 510 FINAL PROJECT

32

R: Is that the same thing for internet usage? You use it when it is convenient, but
otherwise, you would use the other devices? [trying to summarize the usage]
E: That is correct.
R: How much time do you think you use the smartphone in a week?
E: [takes some time to answer] Well, that is hard to say because in some ways I will use
Skype or Facetime on my computer as a way of communicating in addition to using my
phone, so that cuts into my phone usage time. I don't know. That's a hard question.
[pauses] How many hours a week? [pauses] 14?
R: 14 hours, so about two hours a day?
E: Yeah, I'm assuming. Is that not normal? [questioningly]
R: I think it is probably less than normal. So, most of that time is calling?
E: Yeah. Right, because again if I am in a place where I have my phone but I also have
access to my iPad or my laptop, I will use one of those over my phone. Because I find
them easier to use because the phone takes a long time to load or its funky. I am not a
good typist with my thumb.
R: When your next upgrade comes up, will you stick with the smartphone? Do you think
you'll ever go back to a regular cell phone?
E: I don't think so, but I think that I like the convenience of having a smartphone. Even
though I don't use it to its maximum capacity, I like the flexibility it offers.
R: Well, thanks for sharing your experience with the smartphone.
E: Youre welcome.

Conclusions
The interviewee primarily uses the phone (call and texting), internet (email and internet
browsers), and the photo features of her phone to gain access to email and other
information based on present need, rather than using the entertainment, health,
multimedia, or other applications available on the smartphone. She prefers using her
laptop or tablet for any non-phone related services when possible. Thus, I would
categorize her as a low-level smartphone user. Though she prefers the smartphone over
a regular cell phone, she primarily uses the smartphone for communication and photo
taking, much of which could be accomplished using a regular cell phone.

Research Implications
Given that the interviewee is a low-level smartphone user, she provides the groundwork
to think about types of smartphone users. I would like to conduct a number of other
interviews, and then surveys, to determine the type and percentage of type of

ECI 510 FINAL PROJECT

33

smartphone users. This information could be used for marketing or to educate certain
users about additional options that the smartphone provides.

Personal Reflection on Qualitative Research

Focus Group Plan Activity
Much of my work in Instructional Technology centers around accessibility, and I
worked extensively training faculty members to create accessible course materials in my
last job. However, the accessibility initiative was rolled out at my former institution
without much input from faculty members, so accessibility was the first topic that came
to mind when thinking of a focus group. In particular, I would want to recruit faculty
members from a range of disciplines and with varied experience incorporating
accessibility in their courses so that I could gather a more complete understanding of
the accessibility with in current courses and the feasibility of incorporating further
principles.

ECI 510 FINAL PROJECT

34

The focus group plan allowed me to really think about what goes into designing a

focus group, including the room location, incentives, and recruitment, which I imagine
is actually quite challenging and limits the generalizability of the study. Nonetheless,
within the context of an on-going campus-wide initiative, I imagine that it would
provide helpful information.
The most challenging element of this activity was the inability to apply the plan to
an actual setting, I think. For instance, with data and feedback from the web survey, I
was able to significantly change the survey for its second iteration, which was more
effective. It would have been nice to do the same with this activity.
Obviously, this activity would be hard to do in a real setting with limited notice,
but as I was conducting my interview, I realized that I have probably made my focus
group plan too broad. It would not be possible to answer the number of questions even
of a small group in a short time period. I would probably need to do a survey first and
then narrow down the topic based on what seemed like the greatest need.
Overall, it is clear that a lot of planning goes into setting up a focus group plan,
and in future research, I would need to leave plenty of time to arrange everything when
conducting a focus group.

Field Research--Interview Activity

For the interview activity, I designed a set of open-ended questions to guide a
discussion about smartphone usage. I met the interviewee at a local coffee shop and
conducted the interview while recording the conversation to transcribe later. My
interviewee used very little of the smartphone specific features but still preferred access
to the internet, making her a low-level user. Based on the information obtained, I would

ECI 510 FINAL PROJECT

35

be interested in finding a way to categorize cell phone users, especially those involved in
the educational sector.
Unlike the focus group, the interview required the practical application by
conducting an interview. Though obviously the interview with someone I didnt know
would go differently because I would establish rapport, it was interesting to have to
think through the interview process live--to remember the guiding questions but also
think on my feet and consider important follow-up questions. The experience was
useful in thinking about what I would need to do to be prepared for more formal
interviews.
I enjoyed each step in the interview process, and though transcribing the
interview took some time, I am accustomed to transcribing when I create captions, so
the process was not overly time consuming for me. I could not imagine, though,
transcribing a set of interviews or a focus group.
Overall, I dont think this activity needs much improvement because it allowed
for feedback from peers about the questions before the interview and a reasonable
amount of time to conduct and transcribe the interview. The only challenging element
for me was my particular schedule. I was actually away at conference during the week
that the interview time was set and would have trouble conducting if I hadnt been able
to plan ahead and conduct the interview the Sunday before I left.
For future interviews, I think it would be really important to conduct them in a
quiet setting with little distractions. The particular coffee shop I chose was a little too
loud though I was still able to understand everything. I wouldnt want to loose the
welcoming setting, so I would need to find a spot that offered both.

ECI 510 FINAL PROJECT

36

Research Questions
Problem Sheet: Purpose, Research Question & Justification
1. My working Title is:
How Captions Support Student Learning in Online Environments
2. My research problem stated as purpose is:
To understand what role closed captions play in the learning process and student
success in online classes
3. My research question(s) is/are:
a. Do students use closed captions, when available, in their
learning process when watching content-based videos?
b. Do closed captions increase student success, when used, on
evaluation procedures in online classes?
4. Following are key terms in the problem or question that are not clear and
thus need to be defined:
a. learning process
b. student success
c. captions
d. online classes
5. Here are my definitions of these terms:
a. The learning process is the step-by-step process of learning
that students use independently to gain proficiency in the learning
objectives in the class. For instance, if captions are available, students
may use the visual representation of speech to reinforce their
understanding, which may increase their learning, or shorten the process
of learning the material.
b. Student success is ability to demonstrate learning on course
evaluation procedures, such as quizzes, final exams, and final course
grades.

ECI 510 FINAL PROJECT

37

c. Captions, in this case, mean closed captions that duplicate

the spoken narration in a video.
d. Online classes are course offered in a fully online
environment.

6. Here are my variables of interest:

Independent variable: closed captions in online videos used
in the classroom
Dependent variables: Quiz scores throughout the semester,
final grades, and individual students learning processes reported through
surveys

7. My justification for investigating this question/problem (why I would

argue that it is an important question to investigate) is as follows:
Previous research has shown a slight increase in student success when captions
are used with videos but with limited populations, including foreign language learners
and people with hearing impairments (Gass, Sydorenko, & Winke, 2013). I would like to
understand if captions have an effect on a larger population of students and the extent
to which they affect students self-regulatory processes as they design their learning
process.
In addition, if captions improve student success, there would be another reason
to argue that funding for captions should be increased (currently, funding is allocated
only in the case that a student with complete hearing loss is attending the course).
Of course, a negative impact on student learning process and success would
suggest that captions should be omitted.

ECI 510 FINAL PROJECT

38

References
Centre for Evaluation & Monitoring (2014). Effect size resources. Retrieved from
http://www.cem.org/effect-size-resources
Gass, S., Sydorenko, T., & Winke, P. (2013). Factors influencing the use of captions by
foreign language learners: An eye-tracking survey. The Modern Language
Journal, 97(1), 254-275. doi: 10.1111/j.1540-4781.2012.01432.x
Human Subjects (IRB) at NC State University (2014). Retrieved from
http://research.ncsu.edu/sparcs/compliance/irb/
Lietz, P. (2010). Research into questionnaire design: A summary of the literature.
International Journal of Market Research, 52(2), 249-272.
Nestor, P.G. & Schutt, R. K. (2015). Research methods in psychology: Investigating
human behavior. (5th ed.) Los Angeles, CA: Sage.
Pyrczak, F. (2008). Evaluating research in academic journals: A practical guide to
realistic evaluation. Glendale, CA: Pyrczak Pub.