ICT Literacy and School Performance: Gina M. M. C. Santos Eleusina M. C. P. S. L. RAMOS Joaquim ESCOLA
ICT Literacy and School Performance: Gina M. M. C. Santos Eleusina M. C. P. S. L. RAMOS Joaquim ESCOLA
ICT Literacy and School Performance: Gina M. M. C. Santos Eleusina M. C. P. S. L. RAMOS Joaquim ESCOLA
Eleusina M. C. P. S. L. RAMOS
Universidade de Trás-os-Montes e Alto Douro, 5000-801 Vila Real, Portugal
Joaquim ESCOLA
Dept. Educação e Psicologia, Escola de Ciências Humanas e Sociais, Universidade de Trás-os-Montes e Alto Douro, 5000-801 Vila Real,
Portugal.
Manuel J. C. S. REIS
Dept. Engenharias / IEETA, Escola de Ciências e Tecnologia, Universidade de Trás-os-Montes e Alto Douro, 5000-801 Vila Real, Portugal.
mcabral@utad.pt
ABSTRACT
There is no doubt that Information and Communication Technologies (ICT) are now an integral part of the life of
children and young people. Some of the recent literature studies on literacy in ICT show that this should
incorporate Internet literacy, Computer literacy and Information literacy, and being these three forms of literacy
indispensable in many aspects of human life in the 21st century. To determine the relationships between these
three dimensions of literacy in ICT, the importance of parental and teachers support and use of the Internet as a
pedagogical tool, as well as the influence of these factors on school performance and assess how they relate to
explain, in general, why ICT literacy influences school performance, structural equation models have been used.
The results show that ICT literacy is undoubtedly a factor in school performance of students of the 3rd cycle of
basic education and secondary education (7th to 12th grades) in the district of Vila Real, Portugal. It was further
found that the use of the Internet as a pedagogical tool is a major factor in school performance, and that parental
and teacher support has a positive influence on ICT literacy.
INTRODUCTION
Learning with the support of Information and Communication Technologies (ICT) has been constantly evolving,
having come to generalize the idea that the generation of today’s young people is surrounded and immersed in
technology, such as smartphones, computers, tablets, game consoles, among other (Prensky, 2010). As a
consequence, this generation has proven to be more experienced in the use of technology (Palfrey and Gasser,
2013; Prensky, 2010). It is common sense that the students of the Portuguese 3rd cycle of basic education and
secondary education fall in this classification, and as such they are comfortable in using ICT, even when
compared with their parents and teachers.
Some studies on ICT literacy (see, for example, Lau and Yuen (2014); Aesaert et al. (2014)) show that this
should include Internet literacy, Computer literacy and Information literacy, and being these three forms of
literacy indispensable in many aspects of human life in the 21st century.
To determine the relationships between the various dimensions of literacy in ICT (Internet literacy, Computer
literacy and Information literacy), the importance of parental and teachers support in using the Internet as a
pedagogical tool, as well as the influence of these factors on school performance and assess how they relate to
explain, in general, why ICT literacy influences school performance, structural equation models have been used.
It is important to know not only new technologies, but also to master them and to realize that they are constantly
changing. To understand the full meaning of this area, it means having the notion that ICT are not only able to
perform skillfully the most diverse software, be proficient in the Web domain, etc., but also be able to infer all
this knowledge and know how to use it critical- and competently.
In short, as sooner a child acquires ICT literacy skills the better. Better results are achieved when there is a
parental and teacher support as a way to encourage the use of technologies to achieve better school performance.
Note that students of the new millennium, not only have more aptitude and skills in the use of ICT, when
compared to their parents and teachers, but also were shaped by them, in respect to their standard thoughts and
communication, notions of learning, and even in their personal and social values, such as multitasking that is
taken as a current social practice (Pedró, 2007).
Here, it is aimed to examine how ICT literacy influences the academic performance of students of the 3rd cycle
of basic education and secondary education (7th to 12th grades) of the Vila Real district, in Portugal. It is
intended to make a contribution in this research area through the development of a model that allows us to assess
the causal relations between the variables under study: ICT literacy (Internet literacy, Computer and Information
literacy), Internet to support study and leisure, parental support, teacher support and school performance.
So, to achieve these aims, it was asked the following questions, given raise to the corresponding hypotheses:
• H1: Does the use of Internet to study influence scholar performance?
• H2: Does ICT literacy foster school performance?
• H3: Does ICT literacy influence the use of the Internet to study?
• H4: Does ICT literacy encourage school performance mediated by the use of the Internet to study?
• H5: Do teachers influence the ICT literacy of students from the 3rd cycle of basic education and
secondary education in the municipality of Vila Real?
• H6: Do parents encourage their children to use ICT literacy to support their studies?
The answers to these question and the test of the corresponding hypotheses will be given in the remaining of the
paper. Section 2 is dedicated to the literature review; section 3 is used to present the methodology used; sections
4 to 6 will be used to characterize and analyze the data obtained, validate the measuring instrument and the
proposed model; and in section 7 the results will be discussed and the main conclusions will be presented.
RELATED WORK
ICT literacy
There are several terms used to define the range of human attributes associated with the use of ICT. The most
common names in the latest reports and review articles include the terms competencies, skills and literacy.
Despite their distinct and specific meanings, they are often used interchangeably in similar contexts
(Markauskaite, 2006). The concept of literacy has been defined and studied in depth in the area of education
(Hannon, 2000). Its definition varies from being seen as a set of skills, a process, a line of thought or practice
(Herring, 2009). Different researchers tend to work separately addressing different aspects of the concept (Lau
and Yuen, 2014).
According to Lau and Yuen (2014) digital literacy, as a concept, integrates Internet literacy and Computer
literacy. ICT literacy, as a concept, integrates Internet literacy, Computer literacy and Information literacy. Here,
this concept of ICT literacy will be adopted.
The European Commission included in 2007 digital skills as one of the eight types of key competencies for
lifelong learning, also known as the 21st century skills. In this context, digital skills are associated with critical
thinking, problem solving, as well as the creative and innovative use of a computer, besides simply mastering
technical skills in ICT. According to Aesaert et al. (2014), digital competence is defined as the proficient and
critical use of ICT for work, leisure and communication. According to the European Commission it underpins
the basic skills in ICT, the use of computers to retrieve, access, store, produce, present and exchange
information, and to communicate and participate in collaborative networks via the Internet (Figel, 2007).
Similarly in the United States of America, the National Educational Technology Standards for Students of the
International Society for Technology in Education (ISTE) were organized into 6 categories (“creativity and
innovation”, “communication and cooperation”, “research and fluency in information”, “critical thinking,
problem solving and decision making”, “digital citizenship” and “technological operations and concepts”), that
demarcate from the technical skills in ICT stooping to the creative use of ICT, problem solving and information
literacy (ISTE, 2007).
Concerning ICT literacy there has been a number of different defended settings, such as the OECD (Organisation
for Economic Co-operation and Development) that has defined it as the interest, attitude and ability of
individuals to properly use digital technology and communication tools to access, manage, integrate and evaluate
information, construct new knowledge, and communicate with others in order to effectively participate in
society. OECD has also suggested five critical components of ICT literacy: access, manage, integrate, evaluate
and create (Lennon et al., 2003). Baek et al. (2008) defined ICT literacy as the ability to recognize problems
related to ICT and the ability to search, analyze, evaluate, organize, create, use, manage and communicate
information in order to solve the problems related to ICT. In short, it can be inferred that this concept includes
cognitive and technical aspects.
Regarding Internet literacy (Internet literacy, or Network literacy, or Hyperliteracy), van Deursen (2010) used
the term “Internet skills” referring to these as skills related to the environment (e.g., knowing how to use
browsers, search engines, filling out forms) as well as skills related to content (e.g., find, select and evaluate
information). In this work, Internet literacy is the ability to access, analyze, evaluate and create online content
(Lee and Chae, 2012). Internet literacy covers all types of actions related to the use of the internet, such as
information retrieval, downloads, online shopping and online interpersonal communication (Savolainen, 2002).
These skills are seen as an important component of human capital, because highly skilled users are best
positioned to benefit from the Internet (Hargittai and Hinnant, 2008), enhancing communication and informal
learning (Lee and Chae, 2012). The concept of Internet literacy has been emphasized on the issue of online
security, and the European Commission created the Safer Internet action plan, whose aim is at the Internet
literacy as a means for children and young people to protect themselves from harmful content (pedophilia,
pornography, bullying, etc.). Lee and Chae (2012) observed that this kind of literacy helps not only to protect
against online negative influences, but also reduces the inequality of information and involves the children in
creative and social activities. However, Livingstone and Helsper (2007) reported that, due to the strong
correlation between online participation and online risks, the most qualified users are more likely to engage in
online activities, so they are more likely to be exposed to risks of breach of privacy, unwanted exposure, among
others. Buckingham (2007) emphasizes that education must go beyond the ability to access and locate
information and that it is equally important to know how to evaluate and use information critically. Thus, it is
important to know the interests of the authors, to question the sources of information, and how that information
is represented in reality.
The Computer literacy (Computer literacy or Information Technology (IT) or Electronic literacy) is a widely
discussed concept. Horton Jr (1983) defines computer literacy as the understanding of what the machine can do,
through the knowledge of hardware and software. The concept of Computer literacy, according to Bawden
(2001, 2008), can be understood as a set of necessary operational powers for handling a wide range of software
applications, including word processors, spreadsheets, databases, etc., as well as the knowledge of some generic
skills such as copying files or configuring a printer driver. Tsai (2002) defines Computer literacy as the
knowledge, skills and attitudes necessary for all citizens to be able to deal with computer technology in their
daily life. Poynton (2005) states that the computer assumes the role of disseminating information, such as the
press did for many years, increasing the ease with which information can be reproduced and disclosed. Thus, it is
essential to hold the skills in Computer literacy to benefit from the information provided by the press as well as
to benefit from the information provided by the personal computer. Poynton also grouped the lifespan (children,
youth and adults) effects of computer literacy. He found that the more time children spend using the computer
and software applications the better performance, in their future, they will have, when faced with new skills in
emerging literacies. For young people, Poynton has measured that Computer literacy is positively associated
with the scores obtained in practical measuring skills tests. Concerning adults who did not obtained specific
knowledge in the area of technology, they need specific training to develop their skills and maintain equitable
access to information. Selber (2004) claims to be essential to incorporate more opportunities to allow students
and teachers to recognize Computer literacy as a social practice and raising the topic beyond the why and how to
use technology.
Regarding Information literacy, Koltay (2011) argued that information plays a vital role in the development of
democracy, cultural participation and active civic participation, having this literacy an increasingly important
role, particularly for knowledge workers who make intensive use of Internet and computer tools. Also,
Information literacy emphasizes the need for recovery and careful selection of the information available in the
workplace, at school, and in all aspects of personal decision-making, especially in the areas of citizenship and
health. Education in Information literacy emphasizes critical thinking, metacognitive and procedural knowledge
used to locate information in specific domains, areas and contexts. The main emphasis is placed on the
recognition of the message of quality, authenticity and credibility (Hobbs, 2006). Lloyd and Williamson (2008)
in a study on how Information literacy is understood in educational contexts, in the workplace and in the
community, concluded that the context is an influential factor in determining the phenomenon. Catts and Lau
(2008) reported that Information literacy is appropriate in all areas of human development, defining it as the
ability to identify information needs, assess their quality, manage this information, use it effectively and in an
ethical way, besides being able to create and communicate the knowledge gained through the application of
information. Among the referred definitions there are common aspects, being perhaps the most important one the
awareness that information skills cannot be seen in separately, as they are interrelated processes that involve the
way people think about information and make use of it (Eisenberg et al., 2004).
Currently, all aspects that relate to critical thinking and communication skill are emphasized, as new generations
are increasingly characterized by an immersion in the world of the Internet, which is an integral part in their
daily life. This critical thinking is important, if not crucial, to be able to filter all the unregulated information and
communication existing in the universe of the Internet (McPherson, 2008). It is then necessary to develop habits
of reflection that make us clearly question what it is known about each subject, and then build a new knowledge.
This critical literacy will enable us to participate and communicate comfortably and safely in these contexts, it
offers us the ability to find the searched information, observe this information in various formats, solve problems
and infer about the ideas that arise. This capacity is referred to as “multi-literacy” (McPherson, 2008).
It is also important to note not only how teachers use technology in the classroom, but, more than that, the way
they help promoting the use of technology by students as a complementary support to their study outside the
classroom (Greenhow et al., 2009). According to Lai (2015) and McLoughlin and Lee (2010), teachers may also
influence the behavior of students concerning the use of technology outside the classroom, through other means,
such as encouragement and emotional support, resource recommendations, homework involving the use of
technological resources and guidance on how to use technological resources for learning. This is evidenced by
Luckin et al. (2009) which state that the educational resources used by students in their study are suggested
mostly by their teachers. This pedagogical or meta-cognitive practice is critical to facilitate the transition from
“living tools” to “learning tools” (McLoughlin and Lee, 2010). However, according to Toffoli and Sockett
(2015), it is being found that most teachers do not exercise this influence on students regarding the autonomous
learning outside the classroom.
Teachers need the knowledge and skills to play this role of advice on their students, particularly in what concerns
to the selection of technological resources that meet the individual needs of their students, the ability to create
educational activities that make a bridge between learning inside the classroom and outside of the classroom,
always through the use of technology, in order to promote the use at home (Kop and Fournier, 2011). Thus, Lai
(2015) argues that it is important that professional development programs for teachers emphasize their
responsibility, concerning autonomous education of students, as well as the various ways that may influence
students in the use of technology outside the classroom.
According to Yu et al. (2012), the family environment, in particular the influence of parents, has an impact on
the use of ICT by their children, which in turn has an impact on their studies.
It should be noted that parents tend to assume that their children learn how to use a computer at school, but in
fact the quality and quantity of available education may not necessarily be satisfactory (Oksman, 2003). Thus,
parents may not have the perception of how important is the key role they play in influencing the digital skills of
their children (Zhong, 2011).
For Wilson (2009) families are an immeasurable resource that should be used by teachers to improve the
academic performance of their students. However, according to Plevyak (2003), not always there is a parental
involvement with the school, being very often due to negative reasons (Friedel, 1999) (e.g., poor school
performance of their children).
For Yu et al. (2012), one of the barriers to parental involvement in computer use at home is related to the lack of
knowledge on how to engage properly with their children. According to Yu et al. (2011) when parents no longer
have control or do not have a guiding role on how to use the computer at home and children refuse to
communicate with their parents about this issue, parents become what these authors call “concerned outsiders”.
Here, it is adopted a classification that takes into account the typology of use, starting from the resources used. In
this chain, several authors identify the use of the Internet for several purposes, adopting different classifications.
For example, Large (2005) adopted a classification of Internet use in four groups: 1) education, where the
Internet is a source of information to support school work; 2) leisure, where the Internet is a place to find songs,
images, videos, movies, and information essential to support hobbies, as well as pertinent information to
facilitate the daily life of the young; 3) cultural enrichment, when using the Internet to know other cultures,
access virtual museums, among others; and 4) social interaction with friends, relatives and sometimes strangers,
via email, or online chat in real time. However, Kalmus et al. (2011) classified the use of the Internet into two
main groups: a first group consisting of entertainment and social networks, and a second group consisting of
work and information. Other kind of classifications are the ones proposed by Hamburger and Ben-Artzi (2000)
that differentiate the use of the Internet in social networks, leisure and information services, or Landers and
Lounsbury (2006) who also adopted three types of classification: leisure, socialization, and academic use.
It is a common fact that both teachers and students use the Internet to obtain information to help in doing school
work. There is a huge number of sites that exist solely and exclusively as educational resources.
Becker (2000), after analyzing the US Census data, concluded that the main reason for Internet access was a way
of supporting the study, but grouping all types of leisure use, such as playing online, listening to music,
consulting/ sending e-mails, etc., counted for more time than the study support.
Nowadays, with the emergence of social networks, the type of Internet use for leisure by young people has
undergone significant changes. Thus, according to Karal and Kokoc (2013), Internet users have given preference
to the use of social and communicational networks, which, as sharing environments, offer opportunities in any
age group and at any level of schooling, being, currently, the most popular online communication method
(Glusac et al., 2015).
The themes explored by adolescents in social networks are varied, and some lines of research are associated with
this phenomenon, such as focusing on the impact of social networks on cognitive abilities. For example, Alloway
et al. (2013) indicated that in their study, youths who routinely used Facebook (but not YouTube) for more than
a year had better oral, reading, and writing skills, as well as better storage capacity compared to colleagues with
shorter usage times.
Jenkins (2010) reported that young people participating in large participatory culture communities around online
games, online knowledge libraries such as Wikipedia, fan groups, etc., work together to solve problems and
complete tasks, developing a new knowledge in a collaborative way.
Similarly, school performance has been an influential factor in ICT literacy, as stated by Baek et al. (2010), who,
when doing a study on ICT literacy with three groups of students with different school performance (good,
sufficient and insufficient), verified that the group with a good school performance showed a level of ICT
literacy higher than the remaining groups.
Barber (1997) found that the vast majority of teachers believed that the use of the Internet did not improve
student performance because its use was disorganized and unrelated to the school curriculum.
Kubey et al. (2001) have found that the intensive use of the Internet for leisure is highly correlated with poor
academic performance, especially by those using the online communication applications. These researchers also
noted that there is a significant number of individuals that when able to be with their friends at any time of the
day, their time of rest will become disturbed, and thus reducing their school performance.
On the other hand, Jackson et al. (2011) found that a greater use of the Internet is associated with better reading
abilities, especially in young people with difficulties in this area. Chen and Peng (2008) note that casual users
have better interpersonal relationships, better academic performance, and greater personal satisfaction, in parallel
with those who use the Internet on an ongoing basis, who are prone to becoming depressed, physically ill, lonely
and introverts.
Huang and Leung (2009) believe that being proficient in ICT has a positive impact on academic performance,
because, by definition, it means having competence to easily solve academic problems.
METHODOLOGY
Our sample considers the students of the 3rd cycle of elementary education and secondary education (7th to 12th
grades) of the public and private schools of the district of Vila Real, in Portugal. It should be noted that the
district of Vila Real is made up of 20 parishes, whose population is around 52,000 people. According to the data
provided by the “General Directorate of School Facilities” (Direção-Geral dos Estabelecimentos Escolares), in
the district of Vila Real there are 5 schools with an educational offer for the 3rd cycle of basic education and 4
for the secondary education.
In a first moment of our research the Administrative Councils of the schools of the district of Vila Real were
contacted. Next, primary data were collected by filling out a questionnaire with personal descriptive data,
Internet use for leisure, Internet use to support students’ study, Internet literacy, Computer literacy, Information
literacy, the role of teachers concerning the students’ work and the degree of encouragement of parents towards
the use of the Internet to support their children’s (students) study.
As stated above, this research covers Information literacy, Internet literacy (communication) and Computer
literacy (technology), which form the conceptual framework of the ICT literacy scale to be implemented and
validated. In terms of measurement, the scale was considered to be a unitary construction with three correlated
sub-scales representing the three literacies mentioned above. The questionnaire was applied between May and
June of 2015, and 1100 surveys were distributed, with 808 valid answers (74% of response rate). A copy of the
complete questionnaire can be downloaded from http://http://www.mcabral.utad.pt/inq.pdf.
Students
Regarding gender, it can be seen from table 1 that 53.1% of the students in the sample are of the masculine
gender, being 46.9% of the female gender. The most represented age is 13 years old (23.4%), followed by 14
years old (21.7%), 15 years old (18.3%) and 16 years old (16.8%). The less represented ages are 12 years old
(12.3%) and students aged 17 or above (7.5%). The most represented years of schooling in the sample are the 7th
grade (27.4%), the 8th grade (26.2%) and the 10th grade (26.1%), the least represented being the 9th grade
(12.4%) and the 11th and 12th grades with 4.3% and 3.6%, respectively (table 1).
Students were asked to answer how often they accessed the Internet to support their study by assigning a score of
‘1’—never accessed, ‘2’—accessing less than 1 hour a week, ‘3’—accessing between 2 to 3 hours a week, ‘4’—
accessing 4 to 5 hours a week and ‘5’—accessing more than 5 hours a week.
Internet access to support students’ study is less frequent than for leisure, table 2, almost reaching the average of
2 to 3 hours a week at home (2.87 ± 1.093), but being lower in school (1.87 ± 0.879) and in other spaces (1.70 ±
0.876).
Regarding school performance, students were challenged to fit into a ranking of their class with five ranks,
ranging from the best 20% of the class to the worst 20%. About 34.7% said they were in the best 20% of the
class, with 29.6% falling in the middle of the class (between 40% and 60%). The second (20% to 40%) and the
fourth (60% to 80%) groups were equitable, representing 15.8% and 15.7%, respectively. Lastly, the least
representative group was the group of students in the worst 20% of the class, with only 4.2% of the students
stating they belong to this group.
Table 2. Descriptive statistics associated with the frequency of access to the Internet for leisure and to support
the study.
Internet access Mean Std. Dev.
Leisure
At home 4.25 1.022
At school 2.17 1.134
Other spaces (e.g., cafe, bar, restaurant, ...) 2.59 1.191
Support the study
At home 2.87 1.093
At school 1.87 0.879
Other spaces (e.g., cafe, bar, restaurant, ...) 1.70 0.876
Parents
Regarding the school qualifications level, it can be observed that parents generally have a higher level, being
slightly higher than that of mothers, which is verified for those who have MSc or PhD degrees (17.9% vs.
12.5%), (21.6% vs. 18.0%) and 12th grade or equivalent (30.2% vs. 30.0%), which is the most represented level
of schooling, table 3. In the lower school qualifications level, parents score slightly lower than mothers, as is the
case of parents who only have the 4th grade of schooling (10.5% vs. 8.1%), the 6th grade of schooling (10.6%
vs. 4.7%) and the 9th grade of schooling (17.3% vs. 16.4%).
When asked about the situation of the parents in their job, the majority of students indicated that parents are
employed and there is a slightly higher value of employed parents than mothers. The prevailing situation is that
of employees (60.5% vs. 59.4%), followed by self-employed (20.1% vs. 12.3%) and workers on their own (8.6%
vs. 3.8%). Regarding retirement, it was also found that parents scored the largest number (2.9% vs. 1.2%). The
number of parents unemployed reported by the students is similar among parents and mothers, being slightly
higher in mothers (6.9% vs. 6.7%). As expected, it was reported a number of domestic mothers that is much
higher than that of the parents (16.0% vs. 0.5%).
forestry
Skilled workers in industry, construction and craftsmen 88 13.8% 24 3.9%
Operators of plant and machinery and assembly workers 59 9.2% 8 1.3%
Non-qualified workers 50 7.8% 126 20.4%
This section aims at the empirical validation of the four constructs (factors or latent variables) used in this
research (ICT literacy); a second order construct composed of three constructs of first order (Internet literacy,
Computer literacy and Information literacy), Internet use to study, support from teachers, and support from
parents.
Once determined the Confirmatory Factorial Analysis related to the construct with the three types of Literacy
(1st order factors), then it will be verified if the construct ICT literacy can be considered as a 2nd order construct
that includes the three mentioned 1st order factors. The factorial validity of the model to describe the levels of
ICT literacy was evaluated by means of a confirmatory factorial analysis with the AMOS software (please refer
to SPSS Inc., Chicago, IL) as described in Marôco (2007). The composite reliability and mean extracted variance
for each factor were evaluated as described in Fornell and Larcker (1981). The existence of outliers was
evaluated by the square distance of Mahalanobis (DM2) and the normality of the variables was evaluated by uni-
and multivariate coefficients of asymmetry (Sk) and kurtosis (Ku).
The overall adjustment quality of the factorial model was made according to the indexes and respective values of
χ2/df, CFI, GFI, PCFI, PGFI, RMSEA, P[rmsea ≤ 0.05] and MECVI. The quality of the local adjustment was
evaluated by the factorial weights and the individual reliability of the items. The refinement of the model was
performed from the values of the modification indexes by the Lagrange multipliers (LM) produced by AMOS,
considering that trajectories and/or correlations with LM > 11 (p < 0.001) were indicators of significant variation
in the quality of the model.
information in digital environments”, ‘9c’ “I am able to use ICT to properly process the obtained
information”, ‘9d’ “I am able to interpret and represent information, such as the use of ICT to synthesize,
summarize, compare and contrast information from different sources”, ‘9e’ “I am able to use ICT to
design or create new information from information already obtained”, and ‘9f’ “I am able to use ICT to
transmit the correct information to suitable targets”.
Four observations showed DM2 values that suggested to be outliers; however, the improvement in the overall fit
quality of the model without these observations was not significant, so the confirmatory factorial analysis was
done with the totality of the observations. No variable showed Sk and Ku values indicating severe violations to
the Normal distribution (|Sk| < 3 and |Ku| < 10; see Kline (2005) and Marôco (2007)).
The initial model that aims at describing the levels of ICT literacy, adjusted to a sample of 808 students from the
3rd cycle of basic education and secondary education (7th to 12th grades), showed a low quality of adjustment
(χ2/df = 5.925, CFI = 0.928, GFI = 0.911; PCFI = 0.781, PGFI = 0.676, RMSEA = 0.078, P[rmsea = 0.05] <
0.001, MECVI = 0.830). It was possible to improve the quality of the adjustment (χ2/df = 2.788, CFI = 0.979,
GFI = 0.965, PCFI = 0.727, PGFI = 0.627, RMSEA = 0.047; P[rmsea ≤ 0.05] = 0.732; MECVI = 0.376) after a
correlation of some of the errors present in the three factors, as can be seen in figure 1, and once removed the
item ‘8e’ “I am able to configure a printer (e.g., installing the printer drivers)” from the Computer literacy list of
factors whose modification indexes suggested the saturation of this item in factors different from those proposed
in the original version. This new model presented a quality of adjustment higher than the original model ( =
380.919), as well as a lower MECVI (0.830 vs. 0.376).
Fig. 1. Standardized factorial weights and individual reliability of each of the items of the “ICT Literacy”
(χ2(78) = 217.460, χ2/df = 2.788, p = 0.000, CFI = 0.979, PCFI = 0.727, GFI = 0.965, PGFI = 0.627,
MECVI = 0.376, RMSEA = 0.047, P(rmsea ≤ 0.05) = 0.732, and I.C. 90%]0.040, 0.055[).
Table 4 shows the standardized factorial weights, the composite reliability (HR) and the mean extracted variance
(MEV) of the “ICT literacy” measurement instrument. HR of the factors was high, being 0.835 for Internet
literacy, 0.803 for Computer literacy, and 0.919 for Information literacy, so that the reliability of the construct is
adequate (HR ≥ 0.7). The MEV, an indicator of the convergent validity of the factors, was also adequate (MEV ≥
0.5), being 0.503 for Internet literacy, 0.508 for Computer literacy, and 0.654 for Information literacy. It was
also calculated the Cronbach’s α, being it 0.840 for Internet literacy, 0.784 for Computer literacy, and 0.913 for
Information literacy.
Table 4. Standardized factorial weights of the items α, λ, composite reliability (HR), and mean extracted
variance (MEV) of the measurement instrument “ICT literacy”.
Literacy factor Item λ HR MEV α
7a 0.763
7c 0.737
7b 0.679
7d 0.662
8c 0.780
8a 0.771
Computer 0.803 0.508 0.784
8b 0.696
8d 0.588
9d 0.886
9c 0.828
9e 0.801
Information 0.919 0.654 0.913
9f 0.788
9a 0.774
9b 0.771
The discriminant validity of the factors was evaluated by comparing the MEV with the squares of the correlation
between the factors. Table 5 shows the square of the correlations between the various factors and, at the main
diagonal, in bold, the respective values of MEV.
Table 5. Square of the correlations between the various factors and their respective values of MEV (diagonal) of
the measurement instrument “ICT literacy”.
Internet Computer Information
The square of the correlation between the factors is lower than the corresponding value of MEV, for which it is
possible to confirm the existence of discriminant validity between these factors.
Fig. 2. Standardized factorial weights, individual reliability of each of the items in each factor in the 2nd order
model of “ICT literacy” ( χ2(91) = 283.178, χ2/df = 3.112, p = 0.000, CFI = 0.972, PCFI = 0.737, GFI = 0.957,
PGFI = 0.640, MECVI = 0.465, RMSEA = 0.051, P(rmsea ≤ 0.05) = 0.378, and I.C. 90%]0.044, 0.058[).
In order to test whether the factors related to the “ICT literacy” measure (Internet literacy, Information literacy,
and Computer literacy) are the only second-order factors (“ICT literacy”), the 2nd order model shown in figure 2
was tested.
The standardized regression coefficients are indicated in table 6, sorted in descending order.
Table 6. Standardized regression coefficients of the 2nd order model “ICT literacy”.
Literacy factor Coefficient p
Two observations showed DM2 values that suggested to be outliers; however, the improvement of the overall fit
quality of the model without these observations was not significant, so the confirmatory factorial analysis was
done with the totality of the observations. No variable showed Sk and Ku values indicating severe violations to
the Normal distribution (|Sk| < 3 and |Ku| < 10).
The initial model aims at representing the use of the Internet to support the study and it showed a quality of
adjustment that was inadequate (χ2/df = 15.421, CFI = 0.772, GFI = 0.896, PCFI = 0.552, PGFI = 0.498,
RMSEA = 0.134, P[rmsea = 0.05] < 0.001, MECVI = 0.422).
Fig. 3. Standardized factorial weights and individual reliability of each of the items of the “Internet to study”
(χ2(3) = 1.401, χ2/df = 0.467, p = 0.705, CFI = 1.000, PCFI = 0.300, GFI = 0.999, PGFI = 0.200, MECVI =
0.032, RMSEA = 0.000, P(rmsea ≤ 0.05) = 0.971, and I.C. 90%]0.000, 0.044[).
It was possible to improve the quality of the adjustment (χ2/df = 0.467, CFI = 0.772, GFI = 0.96, PCFI = 0.552,
PGFI = 0.498, RMSEA = 0.044, P[rmsea ≤ 0.05] = 0.971,MECVI = 0.032) after a correlation of some of the
errors present in the three factors, presented in figure 3, and once removed the items ‘3c’ “Participate in online
courses”, ‘3d’ “Learn online how to create web sites”, and ‘3f’ “Participate in online teaching discussion
groups”, whose factorial weights were below 0.5. As can be seen in table 7, the model presented a quality of
adjustment higher than the original model ( (17) = 307.015), as well as a lower MECVI (0.422 vs. 0.032).
Table 8. Standardized factorial weights of items λ, α, composite reliability (HR) and mean extracted variance
(MEV) of the “Internet to Study”.
Factor Item λ HR MEV Α
3h 0.801
3a 0.731
Internet to
3b 0.569 0.759 0.396 0.714
study
3g 0.495
3e 0.486
Table 8 shows the standardized factorial weights, the composite reliability and the mean extracted variance of
the “Internet to study” measurement instrument. HR was high (0.759), so the reliability of the construct was
adequate (HR ≥ 0.7). MEV was 0.396 ≈ 0.4, a little lower than the reference 0.5. However, the Cronbach’s α was
acceptable, being 0.714.
Two observations showed DM2 values suggesting to be outliers; however, the improvement of the overall fit
quality of the model without these observations was not significant, so the confirmatory factorial analysis was
done with the totality of the observations. No variable showed Sk and Ku values indicating severe violations to
the Normal distribution (|Sk| < 3 and |Ku| < 10).
The initial model showed a poor quality of adjustment (χ2/df = 11.063, CFI = 0.920, GFI = 0.931, PCFI = 0.657,
PGFI = 0.517, RMSEA = 0.112, P[rmsea ≤ 0.05] < 0.001, MECVI = 0.314). After some of the errors were
correlated, as shown in figure 4, and item ‘10f’ removed (the factorial weight was less than 0.5), it was possible
to improve the quality of adjustment (χ2/df = 1.978, CFI = 0.996, GFI = 0.993, PCFI = 0.474, PGFI = 0.355,
RMSEA = 0.035, P[rmsea ≤ 0.05] = 0.855, MECVI = 0.070). As can be seen in table 9, the model presented a
higher adjustment quality than the original model ( (10) = 201.47), as well as a lower MECVI (0.314 vs.
0.070).
Fig. 4. Standardized factorial weights and individual reliability of each of the items of the “Support from
teachers” (χ2(10) = 19.780, χ2/df = 1.978, p = 0.031, CFI = 0.996, PCFI = 0.474, GFI = 0.993, PGFI = 0.355,
MECVI = 0.070, RMSEA = 0.035, P(rmsea ≤ 0.05) = 0.855, and I.C. 90%]0.010, 0.057[).
Table 10. Standardized factorial weights of the items (λ), composite reliability (HR) and mean extracted variance
(MEV) of the “Support from teachers”.
Item Question λ HR MEV α
10d Suggest the structure of the work 0.716 0.868 0.488 0.865
Table 10 shows the standardized factorial weights, the composite reliability and the mean extracted variance of
the “Support from teachers” measurement instrument. HR was high (0.868), so the reliability of the construct
was adequate (HR ≥ 0.7). MEV was 0.488, which is very close to the reference value of 0.5, and the Cronbach’s
α was very good, being 0.865.
One observation presented a value of DM2 suggesting to be an outlier; however, the improvement of the overall
adjustment quality of the model without this observation was not significant, so that the confirmatory factorial
analysis was done with the totality of the observations. No variables showed Sk and Ku values indicating severe
violations to the Normal distribution (|Sk| < 3 and |Ku| < 10).
The initial model showed a poor adjustment quality (χ2/df = 11.059; CFI = 0.974, GFI = 0.986, PCFI = 0.325,
PGFI = 0.197, RMSEA = 0.112, P[rmsea ≤ 0.05] = 0.006, MECVI = 0.047). After removing item ‘11d’ (whose
factorial weight was less than 0.5), and although the unifactorial construct is less than 4 items, thus preventing
the calculation of a set of indexes associated to the model, it was possible to arrive at the final representative
model of this measurement instrument, as can be seen in figure 5 that presents the standardized factorial weights
and the individual reliability of each of the items in the simplified final model.
Fig. 5. Standardized factorial weights and individual reliability of each of the items of the “Support from
parents” (CFI = 1.000, GFI = 1.000).
Table 11. Standardized factorial weights of the items (λ), composite reliability (HR) and mean extracted variance
(MEV) of the “Support from parents”.
Item Question Λ HR MEV α
I am always supported and encouraged by my parents to use
11c 0.823
the Internet
My parents always encourage me to use the Internet to
11b 0.814 0.793 0.566 0.786
finish homework
My parents recommend me to use the Internet to do the
11a 0.598
schoolwork
Table 11 shows the standardized factorial weights, the composite reliability and the mean extracted variance of
the “Support from parents” measurement instrument. HR was high (0.793), so the reliability of the construct was
adequate (HR ≥ 0.7). MEV was 0.566, which is above the reference value of 0.5, and Cronbach’s α was good,
being 0.786.
As before, the proposed structural model was evaluated in two steps using the AMOS software (v. 20, SPSS Inc.,
Chicago, IL), using the maximum likelihood estimation method, as described in Marôco (2007).
In a first step the adjustment quality of the sub-model of measurement (corresponding to the confirmatory
factorial analysis) was measured and in a second stage the adjustment quality and plausibility of the structural
model. As in the previous section, these were performed according to the adjustment quality indexes and
respective reference values, described in Marôco (2007), namely: χ2/df, CFI, GFI, PCFI, PGFI, RMSEA,
P[rmsea ≤ 0.05] and MECVI. The quality of the local adjustment was evaluated by the factorial weights and the
individual reliability of the items and the significance of the causal trajectories were evaluated with a Z test at the
critical ratios. Trajectories with p < 0.05 were considered significant.
Eight observations showed DM2 values suggesting to be outliers; however, the adjustment difference was not
significant, so the confirmatory factorial analysis was done with all the observations. After analyzing the
factorial weights, individual item reliability and modifying indexes, the model was kept, because the model
presented a good adjustment (χ2/df = 2.257; CFI = 0.952; GFI = 0.925; PCFI = 0.836, PGFI = 0.764, RMSEA =
0.039, P[rmsea ≤ 0.05] = 1.000, MECVI = 1.457).
Figure 6 shows the values of the standardized factor weights and the individual reliability of each of the items in
the measurement sub-model of “Model 1”, associated to ICT literacy to support the study.
Fig. 6. Measure submodel of “Model 1” (χ2(436) = 983.929, χ2/df = 2.257, p = 0.000, CFI = 0.952, PCFI =
0.836, GFI = 0.925, PGFI = 0.764, MECVI = 1.457, RMSEA = 0.039, P(rmsea ≤ 0.05) = 1.000, and I.C.
90%]0.036, 0.043[).
CONCLUSIONS
Taking into account the statistical results obtained and presented in table 12, it can be stated that ICT literacy is a
determining factor in school performance of students of the 3rd cycle of basic education and secondary
education in the district of Vila Real, Portugal. It is also verified that the use of Internet to study is a
preponderant factor for school performance. It should be stressed that parental support and teachers support have
a positive influence on ICT literacy.
Our first research question “Does the use of Internet to study influence scholar performance?” (hypothesis H1) is
supported by the structural model. It is inferred that the use of the Internet to study, namely through the use of
search engines, the use of libraries and/or digital databases and the completion of homework, through the use of
the Internet, significantly influences school performance (as expected; see, for example, Jackson et al. (2011)).
Fig. 7. Structural submodel of “Model 1” and standardized factorial weights, individual reliability of each of the
items in each factor in the 2nd order model of “ICT literacy” (χ2(440) = 1053.462, χ2/df = 2.394, p = 0.000,
CFI = 0.946, PCFI = 0.839, GFI = 0.921, PGFI = 0.768, MECVI = 1.533, RMSEA = 0.042,
P(rmsea ≤ 0.05) = 1.000, and I.C. 90%]0.038, 0.045[).
Regarding the research question “Does ICT literacy foster school performance?” (hypotheses H2 and H4), it was
found that it is supported by the structural model. It was verified that ICT literacy significantly influences school
performance in a direct way (H2) and mediated by the use of the Internet to support the study (H4) (Huang and
Leung, 2009). Computer literacy shows a greater convergence, namely through the use of software for
multimedia presentations, word processing and spreadsheeting (Poynton, 2005). This is followed by Information
literacy, where students report that they are able to interpret and represent information, such as the use of ICT to
synthesize, summarize, compare and contrast information from different sources, also reporting that they are able
to use ICT to process the information obtained in this way, and also they are able to use ICT to design or create
new information from information already obtained (Lloyd and Williamson, 2008; McPherson, 2008). Internet
literacy showed a lower convergence, where students mentioned that they are able to set the default home page
for a web browser (e.g., Internet Explorer), use e-mail to communicate and download files from the Internet (see,
for example, Lee and Chae (2012).
In the same way, our third question “Does ICT literacy influence the use of the Internet to study?” (hypothesis
H3) is supported by the structural model. It should be noted that there is a statistically significant relationship
between “ICT literacy” and “Internet to study”, through the points mentioned above.
Regarding the fifth research question “Do teachers influence the ICT literacy of students from the 3rd cycle of
basic education and secondary education in the municipality of Vila Real?” (hypothesis H5), it was verified that
this is supported by the structural model. It was found that the support from teachers is relevant to the acquisition
and development of ICT literacy skills (Internet literacy, Computer literacy and Information literacy),
corroborating, among other authors, Yu et al. (2012); Wilson (2009); Vekiri (2010); Zhong (2011) and Kim et al.
(2014). It is noted that the teachers indicate the sources to be used for the works to be done, giving directions for
information processing and supervising search.
Concerning the sixth research question “Do parents encourage their children to use ICT literacy to support their
studies?” (hypothesis H6), it was found that this is supported by the structural model. It can be verified that
parental support is important for the acquisition and development of the competencies inherent in ICT literacy
(Internet literacy, Computer literacy and Information literacy), corroborating, among others, the works of Lai
(2015); and Kop and Fournier (2011). It must be pointed out that parents encourage their children to use the
Internet to complete homework, as well as feel supported and encouraged by them for its use.
Table 12. Results for the supported research hypotheses for “Model 1’ (**p < 0.001; *0.010 ≤ p < 0.050).
Standardized
Hypothesis Trajectory Result
regression coefficient
H1: Does the use of Internet to study influence Internet to study → Scholar
0.089* Verified
scholar performance? performance
H4: Does ICT literacy encourage school ICT literacy → Scholar performance
performance mediated by the use of the (mediated by Internet to Study) 0.030* Verified
Internet to study?
H5: Do teachers influence the ICT literacy of
students from the 3rd cycle of basic education Support from teachers → ICT
0.138*** Verified
and secondary education in the municipality of literacy
Vila Real?
As for the frequency of Internet use to support the study, it is predominant in ICT literacy and in school
performance that the results are less significant than the previous ones, with a correlation between the frequency
of Internet use and computer literacy and Informational, since in terms of school support it is necessary to
develop the critical spirit inherent in information literacy in order to build new knowledge, as well as mastering
the working tools of any software.
Finally, it is important to mention that the results obtained about the influence of parents’ educational attainment
on ICT literacy and their children’s school performance show that a higher degree of parental schooling
corresponds to higher ICT literacy and school performance of the students. The same applies to the professional
activity of the parents, noting that the higher the professional qualification of the parents, the higher the ICT
literacy and the better the school performance (Vekiri, 2010; Kim et al., 2014). It was also verified that there are
differences in ICT literacy and student performance regarding the use of the Internet by parents, that is, children
whose parents use the Internet have a higher degree of ICT literacy and school performance.
In short, the ICT literacy of students of the 3rd cycle of basic teaching and secondary teaching (7th to 12th
grades) has a predominant influence on school performance, being also important the stimulus of parents and
teachers to use the Internet as a tool to support the study.
ACKNOWLEDGMENTS
This Research was partially funded by National Funds through the FCT – Foundation for Science and
Technology, in the context of the project UID/CEC/00127/2019.
REFERENCES
Aesaert, K., Van Nijlen, D., Vanderlinde, R., and van Braak, J. (2014). Direct measures of digital information
processing and communication skills in primary education: using item response theory for the
development and validation of an ict competence scale. Computers & Education, 76:168–181.
Alloway, T. P., Horton, J., Alloway, R. G., and Dawson, C. (2013). Social networking sites and cognitive
abilities: Do they make you smarter? Computers & Education, 63:10–16.
Baek, S. G., Kim, D. I., Kim, M. R., Kim, H. S., Yu, Y. L., Park, S. H., Kim, S. W., and Kim, M. R. (2008). A
research on the development of ICT literacy assessment tool: For secondary school student. Technical
report, Korea Education and Research Information Service.
Baek, S. G., Lim, C. I., Kim, H. S., Yu, Y. L., Kim, M. R., and Lee, S. R. (2010). Analysis of influential
variables on performance-based ICT literacy level for korean students. In World Conference on E-
Learning in Corporate, Government, Healthcare, and Higher Education, volume 2010, pages 1560–1565.
Barber, A. (1997). Net’s educational value questioned. USA Today. 4D.
Bawden, D. (2001). Information and digital literacies: a review of concepts. Journal of documentation,
57(2):218–259.
Bawden, D. (2008). Origins and concepts of digital literacy, pages 17–32. Peter Lang, New York.
Becker, H. J. (2000). Who’s wired and who’s not: Children’s access to and use of computer technology. The
future of children, 10(2):44–75.
Buckingham, D. (2007). Beyond technology: Children’s learning in the age of digital culture. Polity Press
(Wiley), Cambridge, UK.
Catts, R. and Lau, J. (2008). Towards information literacy indicators. Technical report, UNESCO.
Chen, Y. F. and Peng, S. S. (2008). University students’ internet use and its relationships with academic
performance, interpersonal relationships, psychosocial adjustment, and self-evaluation. CyberPsychology
& Behavior, 11:467–469.
Clark, W., Logan, K., Luckin, R., Mee, A., and Oliver, M. (2009). Beyond WLLA 2.0: mapping the technology
landscapes of young learners. Journal of Computer Assisted Learning, 25:56–69.
Darolia, R. (2014). Working (and studying) day and night: heterogeneous effects of working on the academic
performance of full-time and part-time students. Economics of Education Review, 38:38–50.
Eisenberg, M. B., Lowe, C. A., and Spitzer, K. L. (2004). Information literacy: Essential skills for the
information age. Greenwood Publishing Group, 88 Post Road West, Westport.
Erten, I. H. and Burden, R. L. (2014). The relationship between academic self-concept, attributions, and L2
achievement. System, 42:391–401.
Figel, J. (2007). Key competences for lifelong learning-european reference framework. Technical report,
Luxembourg: Office for Official Publications of the European Communities.
Fornell, C. and Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and
measurement error. Journal of Marketing Research, 18(1):39–50.
Friedel, T. L. (1999). The role of aboriginal parents in public education: Barriers to change in an urban setting.
Canadian Journal of Native Education, 23(2):139–158.
Glusac, D., Makitan, V., Karuovic, D., Radosav, D., and Milanov, D. (2015). Adolescents’ informal computer
usage and their expectations of ICT in teaching-case study: Serbia. Computers & Education, 81:133–142.
Greenhow, C., Robelia, B., and Hughes, J. E. (2009). Learning, teaching, and scholarship in a digital age web 2.0
and classroom research: What path should we take now? Educational researcher, 38(4):246–259.
Hamburger, Y. A. and Ben-Artzi, E. (2000). The relationship between extraversion and neuroticism and the
different uses of the Internet. Computers in Human Behavior, 16(4):441–449.
Hannon, P. (2000). Reflecting on literacy in education. Routledge Falmer, London.
Hargittai, E. and Hinnant, A. (2008). Digital inequality: Differences in young adults’ use of the internet.
Communication Research, 35(5):602–621.
Hargittai, E. and Shafer, S. (2006). Differences in actual and perceived online skills: The role of gender. Social
Science Quarterly, 87(2):432–448.
Hobbs, R. (2006). Multiple visions of multimedia literacy: Emerging areas of synthesis, chapter 2, pages 15–28.
Routledge, New York, USA.
Horton Jr, F. W. (1983). Information literacy vs. computer literacy. Bulletin of the American Society for
teknologiadiskursseissa. [even three-year olds can use computers: Children in mundane technology
discourses]. Tietotekniikkasuhteet: Kulttuurinen näkökulma [Relationships with computers: A cultural
viewpoint], pages 56–72.
Palfrey, J. and Gasser, U. (2013). Born digital: Understanding the first generation of digital natives. Basic Books,
USA.
Pedró, F. (2007). The new millennium learners: challenging our views on digital technologies and learning.
Nordic Journal of Digital Literacy, 2(4):244–264.
Plevyak, L. H. (2003). Parent involvement in education: Who decides? The Education Digest, 69(2):32–38.
Poynton, T. A. (2005). Computer literacy across the lifespan: a review with implications for educators.
Computers in Human Behavior, 21(6):861–872.
Prensky, M. R. (2010). Teaching digital natives: Partnering for real learning. Corwin Press, USA.
Savolainen, R. (2002). Network competence and information seeking on the internet: from definitions toward a
social cognitive model. Journal of Documentation, 58(2):211–226.
Selber, S. A. (2004). Multiliteracies for a digital age. Southern Illinois University Press, Carbondale, IL.
Taylor, S. and Todd, P. A. (1995). Understanding information technology usage: a test of competing models.
Information Systems Research, 6(2):144176.
Toffoli, D. and Sockett, G. (2015). University teachers’ perceptions of online informal learning of English
(OILE). Computer Assisted Language Learning, 28(1):7–21.
Tsai, M. J. (2002). Do male students often perform better than female students when learning computers?: A
study of Taiwanese eighth graders’ computer education through strategic and cooperative learning.
Journal of Educational Computing Research, 26(1).
van Braak, J. P. and Kavadias, D. (2005). The influence of social-demographic determinants on secondary
school children’s computer use, experience, beliefs and competence. Technology, Pedagogy and
Education, 14(1):43–46.
van Deursen, A. J. (2010). Internet Skills: vital assets in an information society. PhD thesis, University of
Twente.
Vekiri, I. (2010). Socioeconomic differences in elementary students’ ICT beliefs and out-of-school experiences.
Computer & Education, 54(4):941–950.
Wilson, B. R. (2009). A correlational study: Parental involvement to student achievement in public education.
PhD thesis, Lindenwood University, Missouri.
Yu, M., Yuen, A. H. K., and Park, J. (2012). Students’ computer use at home: a study on family environment and
parental influence. Research and Practice in Technology Enhanced Learning, 7(a):3–23.
Yu, M., Yuen, A. H. K., Park, J., Lam, H. C., Lau, K. K., and Lau, W. (2011). Web 2.0 Divide among Naughty
Insiders, Worried Outsiders, and Invisible Monitors: A Case Study, chapter -, pages 51–62. Springer
Berlin Heidelberg, Berlin, Heidelberg.
Zhao, L., Lu, Y., Huang, W., and Wang, Q. (2010). Internet inequality: The relationship between high school
students internet use in different locations and their internet self-efficacy. Computers & Education,
55(4):1405 – 1423.
Zhong, Z. J. (2011). From access to usage: The divide of self-reported digital skills among adolescents.
Computers & Education, 56(3):736–746.