Design and Student Learning

Journal of Applied Linguistics and Language Research Volume 2, Issue 5, 2015, pp. 138-151 Available online at www.jallr.ir ISSN: 2376-760X ‘HOTS’ in Iran's Official Textbooks: Implications for Material Design and Student Learning Gerannaz Zamani PhD candidate of TEFL, Razi University, Kermanshah, Iran Reza Rezvani Assistant professor of TEFL, Yasouj University, Iran Abstract Higher-order thinking skills or ‘HOTS’ for short are essential outcome criteria of higher education in any discipline. It is thinking that is characterized in terms of ‘analysis’, ‘evaluation’ and ‘creation’ levels of Anderson and Krathwohl's taxonomy. This paper attempts to evaluate higher order thinking skills represented in Iran's Organization for Researching and Composing University Textbooks in the Humanities (known as SAMT) official English textbooks of TEFL (Teaching English as Foreign Language) using Anderson and Krathwohl's taxonomy of the cognitive domain. Three University English textbooks pertaining to the specialized courses; that is, Methodology, Language testing, and Linguistics were included in the analysis. To codify the cognitive processes involved in these materials, a coding scheme was developed by the researchers based on Anderson and Krathwohl's taxonomy of the cognitive domain. The exercises and activities of the textbooks were codified and the frequencies and percentages of occurrence of different thinking processes were calculated. The most important finding emerging from this study is that in all the textbooks lower-order thinking skills were more frequently targeted and represented than higher-order ones. Moreover, the difference between the language testing, methodology and linguistics textbooks in terms of the degrees of the engagement of higher-order thinking was notable, as in the language testing textbook, from among these three, attention has been paid most to critical thinking. Results of this study have implications both for specialized language materials development and evaluation. Keywords: higher-order thinking, critical thinking, creative thinking, Anderson and Krathwohl's taxonomy of the cognitive domain INTRODUCTION Some researchers and scholars use the terms ‘higher-order thinking’, ‘critical thinking’ and ‘scientific thinking’ interchangeably. Higher order thinking, or ‘HOT’ for short, takes thinking to higher levels than memorizing and recalling information. Others define ‘critical thinking’ as a part of the process of evaluating the evidence collected in problem solving or the results produced by thinking creatively (Crowl, 1997; Lewis & Smith, * Correspondence: Gerannaz Zamani, Email: geranzamani@gmail.com © 2015 Journal of Applied Linguistics and Language Research Journal of Applied Linguistics and Language Research, 2015, 2(5) 139 1993). Critical thinking also has been described as: reasonable thinking, reflective and goal-directed in evaluating the evidence for an argument for which all the relevant information may not be available (Cotton, 1997; Crowl, 1997; Facione, 1998; Lewis & Smith, 1993). Crowl et al., (1997) regard critical thinking as vital component in metacognitive processes. Interpretation, explanation, analysis, inference, and selfregulation require analytical, systematic, inquisitive, judicious, open-minded, truthseeking, and confident dispositions toward critical-thinking processes (Facione, 1998). Definitions of higher-order thinking fall into three categories: (1) those that define higher-order thinking in terms of transfer, (2) it in terms of critical thinking, and (3) in terms of problem solving. Anderson and Krathwohl (2001) division of learning into learning for retention and learning for transfer is the most general of the approaches to higher-order thinking. Learning for retention surely requires a type of thinking, but it is learning for transfer that they consider as ‘meaningful learning’. This approach leads to their construction of the Cognitive dimension in Bloom’s revised taxonomy. For many teachers, operating with their state standards and curriculum documents, higher-order thinking is approached as the ‘top end’ of Bloom’s revised taxonomy: Analyze, Evaluate, and Create, or, in the older language, Analysis, Synthesis, and Evaluation (Anderson & Krathwohl, 2001). The teaching goal behind the cognitive taxonomy is equipping students to be able to do transfer. ‘Being able to think’ means students can apply the skills and knowledge they developed during their learning to new situations and contexts. ‘New’ here implies applications that the student has not thought of or encountered before, not necessarily something new. Higher-order thinking is considered as students being able to relate their learning to other elements further than those they were taught to associate with it. The following definition refers to the transfer category: "Two of the most important educational goals are to promote retention and to promote transfer. Retention requires that students remember what they have learned, whereas transfer requires students not only to remember but also to make sense of and be able to use what they have learned" (Anderson & Krathwohl, 2001, p. 63). In the critical thinking category, higher-order thinking is, ‘being able to think’ means students can apply wise judgment or produce a reasoned critique. The critical thinking category includes this definition: "Critical thinking is reasonable, reflective thinking that is focused on deciding what to believe or do" (Norris & Ennis, 1989, p. 3). In this case the goal of teaching is helping students to be able to reflect, reason, and make sound decisions. Barahal (2008), also defines critical thinking as ‘artful thinking’ (p. 299), which includes reasoning, observing and describing, comparing and connecting, finding complexity, and exploring viewpoints. ‘HOTS’ in Iran's Official Textbooks: Implications for Material Design and Student Learning 140 Nitko & Brookhart (2007), define problem solving as the non-automatic strategizing required for reaching a goal. A problem is a goal that cannot be solved with a memorized solution. Bransford and Stein also point out that problem solving is the general mechanism behind all thinking, even recall. To recall something, students have to identify it as a problem. The goal of teaching then is to equip students to be able to identify and solve problems in their academic work and in life. In the problem solving category higher-order thinking is when: "A student incurs a problem when the student wants to reach a specific outcome or goal but does not automatically recognize the proper path or solution to use to reach it. The problem to solve is how to reach the desired goal. Because a student cannot automatically recognize the proper way to reach the desired goal, she must use one or more higher-order thinking processes. These thinking processes are called problem solving" (Nitko & Brookhart, 2007, p. 215). Despite the challenges related to defining higher order thinking, administrators, educators, and evaluators have expressed agreement about the value of teaching it (Patrick, 1986; Marzano, Hughes, 1988; Carrol, 1989; Cotton, 1997; Young, 1997). Although information and memorization provide “a refrigerator in which to store a stock of meanings for future use,” it is the judgment that “selects and adopts the one to be used in an emergency . . .” (Dewey, 1933, p. 125). Thus teaching and assessing higher-order thinking provides students with relevant skills and helps them improve their content knowledge, lower order thinking, and self-esteem (DeVries & Kohlberg, 1987; McDavitt, 1993; Son & VanSickle, 1993). A study conducted by Zamani and Rezvani (2014) investigated critical thinking skills in MA high-stakes tests of TEFL and English Translation through the use of Anderson and Krathwohl's (2001) taxonomy. The findings showed that the degree of critical thinking skills in TEFL and English translation UEEs were not so pleasing and the majority of the test items revealed lowerorder thinking skills. Recent literature on critical thinking explicitly has addressed critical thinking as the ability to evaluate evidence and arguments and to construct rationales for beliefs plus examination of one’s own reasoning (Bruning, 1994). Two perspectives have been employed in the investigation of critical thinking (King & Kitchener, 1994). One perspective centers on the development of logical reasoning and the ability to form logical arguments and draw logically inductive and deductive inferences (Salmon, 1989). The second and main perspective in the literature regards critical thinking as reflective judgment and is considered as a problem-solving process where a demonstrably correct solution cannot be identified (Beachboard, 2010). All education involves transmitting to student's two different things: (1) the discipline content of the course ‘what to think’, and (2) ‘how to think’, the correct way to understand and evaluate this subject matter content (Schafersman, 1991). The first goal of education, ‘what to think’, is so traditionally apparent that instructors and textbooks Journal of Applied Linguistics and Language Research, 2015, 2(5) 141 may focus all their efforts on transmitting basic knowledge. On the other hand, the second goal of education, ‘how to think’ or critical thinking, is often so subtle that textbooks and instructors fail to recognize it and students fail to realize its absence (Schafersman, 1991). Hence this elusive issue is of critical importance to be realized in educational settings. Textbook evaluation Torres (1994 cited in Litz, 2005) suggests that the textbook is an almost universal element of English language teaching and no teaching-learning situation, is complete until it has its relevant textbook. There are different attitudes towards textbooks. Tomlinson (2001) divides the attitudes into two groups: supporters and opponents. Supporters argue that textbooks are the most convenient form of presenting materials because they give systematicity, consistency, cohesion, and progression. The second group contends that textbooks are superficial and reductionist in their coverage and are not able to satisfy the diverse and broad needs of all their users. It is implied that the first group In spite of this disagreement it is widely agreed that textbooks are of great value in the process of teaching and learning (e.g. Cunningsworth, 1995; Haycroft, 1998; O’Neil, 1982; Sheldon, 1988). Continuous evaluation of textbooks to see if they are appropriate is of great importance. Genesee (2001) stated evaluation in TESOL settings is a process of collecting, analyzing and interpreting information. This process allows us to make decisions through which student achievement will increase and educational programs will be more successful. Chadran (2001) conducted a study about English textbooks used in Malaysian schools. English teachers of over thirty schools participated in informal interviews with the researcher about their perceptions, attitudes and beliefs about the textbooks prescribed to them by the Malaysian Ministry of Education. Results revealed that, teachers preferred commercially produced materials available in the market over the prescribed textbooks, and they considered the textbooks dull and outdated and that they were not properly graded in terms of difficulty. Morgan (2003) evaluated IELTS materials and showed that there is a need for more materials that are beyond test-taking practice and aim at developing the language competencies that the candidates need for their work or study destinations. Weiten, Deguara, Rehmke, and Sewell (1999) focused on textbook pedagogical aids while they focused on students as the main users of textbooks. They examined students’ evaluation of textbook pedagogical aids and found that chapter glossaries, boldface technical terms, chapter summaries and self-tests received the highest marks in their evaluation. Vellenga (2004) ran a study on how pragmatics was presented in EFL/ESL textbooks. She studied eight English as a Second Language (ESL) and English as Foreign Language (EFL) textbooks to determine the quality and amount of pragmatic information included. She believes that textbooks seldom provide enough information for learners to effectively acquire pragmatic competence. The results indicated that the textbooks include a scarcity of meta-linguistic and meta-pragmatic information, and the ‘HOTS’ in Iran's Official Textbooks: Implications for Material Design and Student Learning 142 comparison of EFL and ESL textbooks shows that while there is small amount of pragmatic information across all texts, a greater percentage of pages of EFL texts comprise pragmatic information. While different approaches and criteria have been presented to evaluate textbooks (e.g. Cunningworth, 1995; Harmer, 1996; Williams, 1983), taxonomies like of educational objectives also prove useful in textbook evaluation studies. Sultana (2001) used the taxonomy to examine 67 teacher interns' lesson plans in Kentucky to determine the extent to which their lesson objectives develop higher-order thinking skills in their students. This analysis revealed that 41.3% of the new teachers’ lesson objectives were at the ‘knowledge’ level, the lowest cognitive category. Only 3.2% of the teachers’ lesson objectives were found to be at the highest level of ‘evaluation’ in Bloom’s taxonomy. Aviles (2000) believes that Bloom’s taxonomy of educational objectives can be used in the wider context of education to help educators to think more precisely about what it means to teach and test for critical thinking. Most of the English textbook evaluation studies conducted in Iran focus on three main goals. The first group has tried to develop some criteria to contribute to more successful textbook evaluation studies (e.g., Ansary & Babaii, 2002). The second group has evaluated textbooks for their strength and weaknesses to find their advantages and flaws (e.g., Jahangard, 2007; Riazi & Aryashokouh, 2007), and the third group has studied discourse features and the representation of discourse elements in the textbooks (e.g., Darali, 2007; Tavakoli, 1995). Jahangard (2007) evaluated four EFL textbooks used in the Iranian high schools by the Ministry of Education. He discussed the advantages and weaknesses of the textbooks with reference to 13 common criteria extracted from different materials evaluation checklists. Riazi and Aryashokouh (2007) also studied four high school and pre-university English textbooks centering on the consciousness-raising aspect of vocabulary exercises. They realized that from among all activities in the four books, only one percent of them could be classified as consciousness-raising. In the area of pragmatics a study has been conducted by Darali (2007). She made a careful analysis on Spectrum series with the application of six models proposed by Searle (1976), Leech (1983), Matreyek (1990), Holms (1990), Leech (1983) and Thomas (1983), and Halliday (1978). The results of the study showed that the series have provided a variety of language functions, but some important language functions that are used in daily speech more frequently, e.g. promising, vowing, and threatening, not only were in the form of unintended function, but also they were not as frequent as others. Razmjoo (2007) used Hymes’ (1972) scheme to investigate the extent to which the Iranian high school and private institute textbooks represent the CLT principles. The analysis of the data indicated that while high school textbooks are not conductive to CLT implementation, private institute textbooks represent the CLT principles to a great extent. Gordani (2010) explored different types of learning objectives inherent in Iranian guidance school English textbooks from the viewpoint of Bloom's taxonomy. The study Journal of Applied Linguistics and Language Research, 2015, 2(5) 143 used Bloom's taxonomy of educational objectives (1956) in analyzing the material found in Iranian guidance school English textbooks. The results showed that all of the items were concentrated in the first three levels of Bloom's taxonomy which are referred to as the lower levels of cognitive skills. In addition, a significant difference was found between the textbooks in their inclusion of different levels of cognitive skills. Riazi and Mosallanejad (2010) investigated the types of learning objectives represented in Iranian senior high school and pre-university English textbooks using Bloom’s taxonomy of learning objectives. To codify the learning objectives, a coding scheme was developed based on Bloom’s (1956) taxonomy of learning objectives. The results of the study indicate that in all grades lower-order cognitive skills were more prevalent than higher-order ones. Furthermore, the difference between the senior high school and the pre-university textbooks in terms of the levels of the taxonomy were significant insofar as the pre-university textbook used some degrees of higher-order learning objectives. As this review of literature indicates, there is little research on the cognitive domains in textbook evaluation studies. No comprehensive study has yet been carried out to evaluate higher-order thinking using Andersons and Krathwohl's taxonomy. Therefore, in the current study an attempt has been made to focus on the components of the cognitive domain of TEFL English textbooks currently in use in Iranian universities. OBJECTIVES OF THIS STUDY The purpose of this study was to evaluate Iranian SAMT English textbooks of TEFL. The evaluation took place with regard to the six levels of Anderson and Krathwohl's taxonomy of the cognitive domain. The study intended to investigate how the content of textbooks represents higher-order thinking skills in terms of Anderson and Krathwohl's taxonomy of the cognitive domain. The study, therefore, attempted to find answers to the following questions: 1. How are the TEFL books evaluated in terms of higher-order cognitive skills? 2. How could higher-order thinking skills in SAMT TEFL books be compared? 3. Which levels of the cognitive process dimension are most frequently represented in TEFL textbooks currently in use in Iranian universities? METHOD This study was a textbook evaluation study. Using a coding scheme, the exercises and activities of SAMT English textbooks pertaining to the specialized courses of TEFL, including methodology, language testing, and linguistics were content analyzed and coded in terms of the cognitive processes and the frequency and percentage of each cognitive process were calculated for each textbook. A Chi-square test was performed In order to determine if there was a significant pattern in the occurrence of higher and lower-order thinking skills in the textbooks. ‘HOTS’ in Iran's Official Textbooks: Implications for Material Design and Student Learning 144 Coding Scheme The content of the textbooks were evaluated by making use of a coded scheme based on the respective classification of the cognitive processes in Anderson and Krathwohl's Taxonomy. Their definitions of different levels of the cognitive process dimension were carefully studied and the key word examples were used. The coding scheme represented the six cognitive processes from the simple recognition and recall of facts, as the lowest level, through progressively more complex and abstract mental levels of evaluation and creation. The coding categories were labeled as: 1) to remember 2) understand 3) apply 4) analyze 5) evaluate 6) create. Each category comprised examples for each level, key words that signified intellectual activity on each level and task samples. Since the verbs describe the intended cognitive process, we examined the verb in each question or exercise in relation to the cognitive process categories. For example a question from the methodology textbook states: ‘Compare GTM with the cognitive method and explain why the former is not scientific’. In this example, based on the coding scheme the two verbs, ‘compare and explain’ are associated with the cognitive category ‘understand’, which is a lower-order cognitive skill. (See appendix 1 for the coding scheme). Materials SAMT (The Organization for Researching and Composing University Textbooks in the Humanities) was established by order of the Supreme Council of Cultural Revolution to render scientific services to universities, and publish research-based and university books. SAMT provides basic and reference books as well as textbooks and various other educational sources in the humanities for Iranian universities and higher education institutes. In this paper the English SAMT textbooks of TEFL pertaining to the specialized courses including methodology, language testing, and linguistics were used as the materials to be evaluated using Anderson and Krathwohl's cognitive process dimension. The list of the books published by SAMT for B.A. TEFL program is as follows:    Farhady, H., Delshad, S. (2007). An Introduction to Methodology for TEFL/TESL. Tehran: The organization for researching and Composing University textbooks in the Humanities (SAMT). Farhady, H., Ja'farpur, A., & Birjandi, p. (2007). Testing Language Skills from Theory to Practice. Tehran: The organization for researching and Composing University textbooks in the Humanities (SAMT). Farrokhpey, M. (2006). Linguistics & language. Tehran: The organization for researching and Composing University textbooks in the Humanities (SAMT). Journal of Applied Linguistics and Language Research, 2015, 2(5) 145 RESULTS AND DISCUSSION Table 1 includes the number of chapters in each of the textbooks; the numbers of questions which are included in the textbook and the types of knowledge textbooks have focused on. Table 1. Overall features of TEFL English textbooks Textbooks Lang teaching Methodology Number of chapters Number of of questions 13 172 16 111 21 28 Lang testing Linguistics Types of knowledge Factual, conceptual, procedural Factual, conceptual, procedural Factual, conceptual The frequency and percentage of the thinking processes in TEFL English textbooks are presented in Table 2. The results were obtained through the codification of the activities of all three textbooks. The most frequent thinking processes were understand and remember in the methodology textbook (34.9%, 34.3% ; respectively), while the least frequent cognitive process was create, with the frequency of 2.9%. No attention has been paid to ’apply’. In language testing, the most frequent cognitive processes were remember and evaluate (28.8%, 26.1% respectively) while analyze was totally absent in the coded data. Understand, create, and apply came in between in this range (28.8%13%). With regard to linguistics, the two higher-order cognitive skills, evaluation and creation were totally ignored and notoriously absent. And apply with the frequency of 50% received the highest attention. Table 2. Frequency and percentage of thinking processes pertaining to TEFL English textbooks Textbooks Remember Language teaching methodology 59 (34.3%) Understand 60 (34.9%) Apply Analyze Evaluate Create 0 0 13 (7.6%) 35 (20.3%) 5 (2.9%) Language testing 32 (28.8%) 20 (18%) 15 (13.5%) 0 0 29 (26.1%) 15 (13.6%) Linguistics 6 (21.4%) 4 (14.3%) 14 (50%) 4 (14.3%) 0 0 0 0 Average 32.3 (28.2%) 38 (22.4%) 9.7 (21.2%) 5.6 (7.3%) 21.3 (15.4%) 6.6 (5.5%) If we categorize the six levels of Anderson and Krathwohl's taxonomy into “lower” order thinking skills including ‘remember, understand, and apply’ and ‘higher’ order thinking ‘HOTS’ in Iran's Official Textbooks: Implications for Material Design and Student Learning 146 skills comprising ‘analyze, evaluate and create’, then we can restate the information in Table 2 as demonstrated in Table 3 below. Table 3. Lower and higher-order thinking skills in TEFL English textbooks Textbooks Lower-order thinking skills Higher-order thinking LTM 119 (69.2%) 53 (30.8%) LT 67 (60.3%) 44 (39.6%) L 24 (86.7%) 4 (14.3%) Average 80 (71.8%) 33.5 (27.6%) As indicated in Table 3, lower-order thinking skills are the most frequent thinking skills according to the classification of learning objectives of Anderson and Krathwohl's taxonomy. It is important to note the frequency of occurrence of evaluation and creation, which are central to critical thinking, in the language testing textbook with the frequency of (26.1%, 13.5%). While it was absent in the linguistic textbook, it occurred even more than the lower-order thinking processes including understand and apply. This shows that attention to higher-order cognitive skills is considerable in the language testing textbook by allocating 39.6% of its activities to higher-order cognitive skills. Although lower-order cognitive skills are still more frequent in this textbook, the frequency of occurrences of the skills at the highest levels of the taxonomy is noteworthy. In order to observe how SAMT TEFL textbooks could be compared in terms of higher and lower levels of Anderson and Krathwohl’s taxonomy, a Chi-square was carried out which gave a significant difference (X2 = 7.49, df= 2, p=.02) between higher and lower order thinking skill in the three textbooks. The activities in each chapter of the methodology textbook consist of three parts. The first part deals with information type questions which mainly fall in the cognitive categories ‘Remember and understand’, since these questions require recall and explanation of information presented earlier in the text. The next part are discussion questions which entails students to construct and explain the causes and effects of a model and evaluate concepts, therefore they mostly deal with ‘understand and evaluate’. The last part is a scenario followed by troubleshooting questions. These questions mainly comprise higher-order thinking skills including ‘evaluate and create’. Because these questions require students to transfer what they have learned to a new context. They are engaged in problem solving which entails systematic and reflective thinking. The two main categories of evaluation including, checking and critiquing were prevalent in language testing book activities. Most exercises involve students to make judgments Journal of Applied Linguistics and Language Research, 2015, 2(5) 147 based on predetermined criteria or standards, especially when it comes to chapter 5 which concerned test construction. The activities require students to detect the problem in each defective test item and find a way to solve the problem. The linguistics book overlooked higher-order thinking skills specially ‘evaluate and create’. It is however surprising a book with 21 chapters only includes 28 questions half of which are associated with the cognitive process ‘apply’. The exercises required students to execute certain linguistic rules particularly the chapter pertaining to phonology. After language testing, the methodology textbook received the highest frequency in giving preeminence to higher-order thinking. But mainly the exercises focused on the two first cognitive levels of remember and understand, requiring students to recall and explain factual knowledge. The overall finding to emerge from this study is that lower-order cognitive skills were more frequent than higher-order cognitive skills. This could be a result of the fact that there is much more emphasis on acquiring knowledge in the form of rote learning and memorization, rather than constructing it through higher-order thinking skills such as evaluation and creation. CONCLUSION The conclusion that can be drawn from the present study is that the most frequent learning objectives pursued in the Iranian SAMT English textbooks of TEFL were lowerorder cognitive skills, that is, ‘remember’, ‘understand’ and ‘analyze’. Among the TEFL textbooks, language testing addressed the significance of critical thinking skills to the largest extent. Despite the fact that textbook developers sporadically have tried to change the activities to address higher-order thinking skills, preoccupation with the growth of learners’ comprehension is vivid. The result of this study indicated that, scant attention has been paid to higher-order thinking skills and the main objectives of the textbooks were the development of lower-order cognitive skills. In order to foster the content of the textbooks, textbook writers should try to devise activities and exercises that go beyond lower-order thinking skills and require critical thinking; furthermore, in the process of textbook revision, good qualities of the textbooks should be preserved and the flaws should be excluded. It should be mentioned that further studies or the same can be replicated on other SAMT textbooks in order to improve the quality of the current textbooks. Based on the results of the study, some pedagogical implications can be stated with the hope that the present study would be a useful source to solve many problems in the area of language learning and teaching, material production, textbook design, discourse analysis, conversation analysis, and even test construction. First of all, this study can mainly be beneficial for teachers since they will have an idea about the degree of the higher-order thinking skills in TEFL SAMT textbooks, so that they can better compensate for the shortcomings. This study can also provide material developers and textbook writers with the necessary information regarding the higher-order thinking skills in TEFL SAMT textbooks. And finally textbook developers can take the pitfalls of ‘HOTS’ in Iran's Official Textbooks: Implications for Material Design and Student Learning 148 the textbooks into consideration as a useful source to modify and revise other developing textbooks. REFERENCES Anderson, L.W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman. Ansary, H., & Babaii, E. (2002). Universal characteristics of EFL/ESL textbook: A step towards systematic textbook evaluation. The Internet TESL Journal, 2, 1-8. Aviles, C. B. (2000). Teaching and testing for critical thinking with Bloom’s taxonomy of educational objectives. Eric Document ED446023. Available: http://www.eric. ed.gov Barahal, S. L. (2008). Thinking about thinking. Phi Delta Kappan, 90(4), 298–302. Beachboard, M. R., Beachboard, J. C. (2010). Critical-Thinking Pedagogy and Student Perceptions of University Contributions to Their Academic Development. Informing Science: the International Journal of an Emerging Transdiscipline, 13. Bransford, J. D., & Stein, B. S. (1984). The IDEAL problem solver. New York: W. H. Freeman. Brookhart, S. M. (2010). How to Assess Higher-Order Thinking Skills in Your Classroom. Alexandria, Virginia USA. Bruning, R. (1994). The college classroom from the perspective of cognitive psychology. In K. Prichard & R. Sawyer (Eds.), Handbook of college teaching. Westport, CT: Greenwood Press. Cotton, K. (1997). Teaching thinking skills. School Improvement Research Series [Online]. Available: http://www.nwrel.org/scpd/sirs/6/cu11.html Crowl, T. K., Kaminsky, S., & Podell, D. M. (1997). Educational psychology: Windows on teaching. Madison, WI: Brown and Benchmark. Cunningsworth, A. (1995). Choosing your coursebook. Oxford, UK: Heinemann. Darali, G. (2007). Pragmatics dimension in Spectrum textbooks. Unpublished master’s thesis, Shiraz University, Iran. DeVries, R., & Kohlberg, L. (1987). Programs of early education: The constructivist view. New York: Longman. Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process. Boston: D. C. Heath and Company. Facione, P. A. (1998). Critical thinking: What it is and why it counts. Millbrae, CA: The California Academic Press. Salmon, M. (1989). Logic and critical thinking. Orlando, FL: Harcourt Brace Jovanovich. Genesee, F. (2001). Evaluation. In R. Carter, & D. Nunan (Eds.), The Cambridge guide to teaching English to speakers of other languages, pp. 144-150. Cambridge: Cambridge University Press. Glaser, R., & Resnick, L. (1991). National research center on student learning. (ERIC Document Reproduction Service No. ED 338 704) Harmer, J. (1996). The practice of English language teaching. London: Longman. Haycroft, J. (1998). An introduction To English language teaching. London: Longman. Krathwohl, D. (2002). A revision of Bloom's taxonomy: An overview. Theory In to Practice, 41(4), pp. 212-264. Journal of Applied Linguistics and Language Research, 2015, 2(5) 149 Lewis, A., & Smith, D. (1993). Defining higher order thinking. Theory into Practice, 32(3), 131−137. Litz, D. R. A. (2005). Textbook evaluation and ELT management: a South Korean case study. Asian EFL Journal. Retrieved from: http://www.asian-efl-journal.com /Litz_thesis.pdf. Manafi, A. (2005). Evaluation of the place of peace and war in Iranian guidance school textbooks content in 2005-2006. Unpublished master’s thesis. Shahid Beheshti University, Tehran, Iran. Marzano, R. J., Brandt, R. S., Hughes, C. S., Jones, B. F., Presseisen, C. S., Rankin, S. C., & Suhor, C. (1988). Dimensions of thinking: A framework for curriculum and instruction. Marzban, P. (2005). Evaluation of primary school textbooks content with regard to the views and perspectives in Islamic Republic of Iran cultural developmental programs. Unpublished master’s thesis. Alzahra University, Tehran, Iran. McDavitt, D. S. (1993). Teaching for understanding: Attaining higher order learning and increased achievement through experiential instruction. (ERIC Document Reproduction Service No. ED 374 093) Morgan, T. (2003). IELTS preparation materials. ELT Journal, 57(1), 66-76. Nitko, A. J., & Brookhart, S. M. (2007). Educational assessment of students (5th ed.). Upper Saddle River, NJ: Pearson Education. Norris, S. P., & Ennis, R. H. (1989). Evaluating critical thinking. Pacific Grove, CA: Critical Thinking Press & Software. O’Neill, R. (1982). Why use textbooks? ELT Journal, 36(2), 104-111. Patrick, J. H. (1986). Critical thinking in the social studies. ERIC Digest, 30. Available: http://ericae.net/db/digs/ed272432.htm Riazi, A.M., & Aryashokouh, A. (2007). Lexis in English textbooks in Iran: Analysis of exercises and proposals for consciousness-raising activities. Pacific Association of Applied Linguists, 11, 17-34. Riazi, A.M., & Mosalanejad, N. (2010). Evaluation of Learning Objectives in Iranian HighSchool and Pre-University English Textbooks Using Bloom’s Taxonomy. TESL-EJ: The Electronic Journal for English as a Second Language. 13(4) Schafersman S. D. (1991). An Introduction to Critical Thinking. Sheldon, L. (1988). Evaluating ELT textbooks and materials. ELT Journal, 42(2), 237246. Son, B., & VanSickle, R. L. (1993). Problem-solving instruction and students’ acquisition, retention and structuring of economics knowledge. (ERIC Document Reproduction Service No. ED 366 627) Sultana, Q. (2001). Scholarly teaching – Application of Bloom’s taxonomy in Kentucky’s Classrooms. Paper presented at the Third Annual Conference on Scholarship and Teaching, Bowling Green, KY. (ERIC Document Reproduction Service No. ED 471982.) Tavakoli, F. (1995). Functional analysis of the dialogues in the Iranian senior high school English textbooks. Unpublished master’s thesis. Allameh University, Tehran. Tomlinson, B. (2001). Materials development. In R. Carter, & D. Nunan (Eds.), The Cambridge guide to teaching English to speakers of other languages, pp. 66-71. Cambridge, UK: Cambridge University Press. ‘HOTS’ in Iran's Official Textbooks: Implications for Material Design and Student Learning 150 Vellenga, H. (2004). Learning pragmatics from ESL and EFL textbooks: How likely? TESL-EJ,8(2). Weiten, W., Deguara, D., Rehmke, E., & Sewell, L. (1999). University, community college, and high school students’ evaluations of textbook pedagogical aids. Teaching of psychology, 26(1), 19-21. Williams, D. (1983). Developing criteria for textbook evaluation. ELT Journal, 37(3), 251-261. Young, A. C. (1997). Higher-order learning and thinking: What is it and how is it taught? Educational Technology, 37, 38−41. Zamani, G. & Rezvani, R. (2014). A Comparative Study of Iran’s TEFL and English Translation U.E.Es: Do High-stakes Tests Asses Critical Thinking? Theory and Practice in Language Studies (TPLS), 4(2), 379-386. APPENDIX Coding scheme based on Anderson and Krathwohl's Taxonomy of the cognitive process dimension. Levels Definition Keywords and examples Task Samples Retrieve relevant information from long term memory Remember recognize identify recall retrieve match define select name -Verification tasks (true/false) Example: Students recognize/recall information from long term memory. Construct meaning from messages, including oral, written or graphic material Understand interpret clarify paraphrase translate exemplify illustrate classify subsume summarize generalize inferring conclude predict compare contrast explain -Oddity tasks In each group of sounds identify the member which does not belong there Example: The student interprets, infers, explains the information they receive. Carry out or use a procedure in a given situation execute carry out Compute the standard deviation Journal of Applied Linguistics and Language Research, 2015, 2(5) 151 implement use modify manipulate Apply of the scores in the following distribution. Example: Students carry out a Procedure when confronted with a familiar task. -Students solve a problem using a selected procedure. Break the materials in to parts And determine how they relate To one another and the overall structure Analyze differentiate discriminate distinguish select organize find structure coherence attribute deconstruct -What does this statement mean in the passage? Example: students distinguish relevant from extraneous material. Make judgments based on criteria And standards Evaluate check detect Monitor Test critique judge -Point out the major weaknesses in each item. - Which procedure best suits this item? Example: the student is able to Find the best solution to a problem And can justify it. Create Put elements together to form a coherent whole, reorganize elements into a new pattern or structure generate hypothesize plan design produce construct Think of other other ways of preparing multiplechoice items. Example: students integrate information For devising a solution method that meets A problem's criteria.