Textbook Evaluation : A Case Study of Iranian Teacher and Student Perspectives

Textbooks are fundamental mediums to guide the process of teaching and learning. There is a significant attention toward aspects of a textbook to qualify for these ends. This study aims at investigating a descriptive survey to evaluate teachers’ and students’ attitudes on the PreUniversity English textbook in Iranian High Schools. It determines the pedagogical value of the textbook according to student’s needs. To this end,30 teachers and 156 students, randomly selected, participated in the research. The textbook evaluation was conducted through the questionnaire developed by Litz (2005). The checklist considered seven main criteria: practical consideration, layout and design, activities, skills, language type, subject and content and overall consensus. The results revealed that both teachers’ and students’ perceptions about the textbook were almost positive in general. However, teachers stated that the textbook lacksskill appropriacy. Students believed that the textbook was not suitable regarding activities, skills, and the overall consensus. Keywords—Evaluation, textbook, textbook evaluation


I. INTRODUCTION
Textbooks are packages with different but interrelated parts.They are the main sources that could convey the knowledge and information to the learners in an easy and organized way (Ahour & Ahmadi, 2012).English is one of the basic courses in the educational setting of Iran and its instruction starts in junior secondary school.It should be mentioned that, in Iran, English textbooks are prepared regarding the policies of the Ministry of Education and also all major educational policies concerning the school systems, the curriculum standards, the compilation of textbooks, and the examination system are under the control of the Ministry of Education.In Iran, students usually study the English language for seven years during their school period.To Prabhu (1987) textbooks are fully specified and preconstructed materials that provide a certain amount of uniformity in what occurs in many different classes with different teachers and students, which serves the interests of accountability.Despite the emergence of recent technologies in the field of education, copies of textbooks are still the most commonly used source material for most instructional situations as it is for language teaching contexts.On the significance of course books, Hutchinson and Torres (1994, as cited in Litz, 2005) suggest that "the textbook is an almost universal element of [English language] teaching.Millions of copies are sold every year, and numerous aid projects have been set up to produce those in [various] countries.No teaching-learning situation, it seems, is complete until it has its relevant textbook" (p.315).There are different views about the textbook, Cunningsworth (1995), for example, identifies a textbook as a source for presenting the materials and learners to practice and do the activities.Hutchinson and Torres (1994) argue that the textbooks have a very important and assertive role to play in teaching and learning.Sheldon (1988) suggests that "textbooks do not only represent the visible heart of any English Language Teaching program but also offer considerable advantages" (p.237).The most essential function of a textbook is to motivate students to learn (Mikk, 2000).For Dubin and Olshtain (1986) "the tangible element that gives language course face validity to many teachers and learners is the textbook" (p.167).Ur (1996) believes that a textbook provides a clear framework.It makes clear what is coming next and learners know where they are going.With regard to the essential aim of the ELT textbooks, Byrd (2001) argues that ELT textbooks include two kinds of information which are topic content (e.g., family, school, etc.) and linguistic content (e.g., grammar, vocabulary, and skills) and that they help learners to learn the latter from the former.Considering the importance of textbooks, evaluating a book needs a significant interest because it not only provides useful information for teachers but also leads to helping students in learning settings.The present study aims at evaluating Pre-University English textbook used in Iranian High Schools from teachers' and students' perspectives.The result of this study could be used for both English teachers and curriculum designers.It would help teachers, ELT administrators and experts, and textbook authors and material developers.This study tried to highlight the new ways of reforming Pre-University English textbook quality.Hutchinson (1987) defines evaluation as a "matter of judging the fitness of something for a particular purpose" (p.41).While considering the evaluation process in language teaching, mostly, the initial analyses are conducted to understand whether the textbook fits into the official curriculum or the intended language program.Moreover, the textbook should meet the needs of the students by addressing their interests and abilities and it should also be consistent with the teaching style of a teacher (Grant, 1987).Evaluating the design of a textbook covers a number of issues such as the physical appearance, layout, and format and other features such as the presentation of illustrations, the tables and figures in a textbook.On the other hand, some scholars relate the design features with the organization of the content of the textbook and also focus on the internal features of the content.Furthermore, for narrowing the concept of design, a number of authors focus on the design of the units in a textbook (Aytug, 2007).As leading considerations regarding the use of exercises, Richards (1990) focuses on the importance of the selection procedures and criteria considering the principal types of activities and tasks consistent with a specific language program.He further states that it is necessary to determine how much time will be devoted to different types of activities.According to Daoud and Celce-Murcia (1979), the language teaching materials should improve students' comprehension abilities and achieve to test main ideas and the detailed aspects of the relevant material.Competency over certain structures and vocabulary is another aim of the textbook exercises.Regarding the writing exercises, the presentation should be varied and the assessment of new issues needs to be done after the recycling process of those issues.In addition to the use of exercises in meaningful contexts to provide authentic communication, the exercises should also be consistent with the students' background knowledge, experience and current situation (Daoud & Celce-Murcia, 1979;Graves, 2000).Three prominent authors in ELT, Swan (1985), Harmer (1996) and McDonough and Shaw (1998) advocate an integrated, multi-skills syllabus because it considers and incorporates several categories of both meaning and form.So they considered receptive and productive skills together.Receptive skills involve reading and listening, on the other hand, productive skills include speaking and writing.Receptive skills are those that involve active participation on the part of the reader or listener.They could be taught/ practiced by using both 'top-down' and 'bottom-up' processing skills and learning strategies.Top-down processing skills require learners to use schematic and contextual knowledge as well as specific topics to arrive at comprehension.Conversely, bottom-up processing skills require learners to derive meaning through the decoding of single words in a message or passage (Nunan, 1998).In general, the combination of these processes requires the reader or listener to "decipher the meaning of individual lexical items [and] have clear ideas about the overall rhetorical organization of the text" (McDonough and Shaw, 1998, p. 109).

III. METHODOLOGY 3.1 Participants
The present study is an investigation of the ideas of two different groups of those who take part in the direct interaction with the textbook and each other, teachers and students.This study was carried out at high schools in Piran Shahr and Khoy.The total participant of teachers who took part in this study was 30 EFL teachers with the teaching experience varied between five to twenty years.Out of the total number of teachers participate in this study, 14 teachers were selected from high schools in Khoy and 16 from high schools in Piran Shahr.In addition, 156 pre-university students randomly selected from different high schools in Piran Shahr and Khoy, were other participants of this study.

Instrument and Materials
The studyapplied an evaluation questionnaire to collect data from the teachers' and students' attitudes about the Pre-University English textbook.Litz's (2005) checklist was used in this study which consists of four parts: student textbook evaluation form, teacher textbook evaluation, student profile, and student needs analysis.The first two parts of the checklist are primary in evaluating the textbook while the other forms are used as supplementary ones.They do not directly contribute to the evaluation of the textbook.Thus, this study included only two parts of Litz (2005) checklist, student textbook evaluation form and teacher textbook evaluation form.Litz (2005) checklist is a valid measure since it is a standardized checklist used worldwide for book evaluation.Based on Litz's (2005) checklist, textbook evaluation form focuses on seven criteria namely, practical consideration, layout and design, activities, skills, language type, subject and content, and overall consensus.The items of checklists were in the form of 10-point scale which proceeds from 1(Highly Disagree) to 10 (Highly Agree).In the current study, for the convenience of statistically defining and coding, these 10 Scales were reduced to 5-Point Likert Scales (e.g., Completely Disagree = 1, Disagree = 2, Moderately Agree = 3, Agree = 4, Completely Agree = 5).The teacher questionnaire and student questionnaire consisted of 40 and 25 items respectively.The questionnaires for student application were translated into Persian to be understandable.

Data Collection and Analysis
The data were collected through both teacher and student responses to the checklists adapted from Litz (2005).With the permission of the officials in the Education Administration, data collection procedure was carried out in Piran Shahr and Khoy among English teachers and students who were randomly selected from different high schools.Prior to the implementation of the instruments, teachers and students were informed about the textbook evaluation study and data collection procedure.Considering the teachers' weekly teaching timetables, it was planned to distribute both teacher and student questionnaires at the beginning of class time.Then the questionnaires were distributed among teachers and students to complete the questionnaires in about 30 minutes.After collecting the study instruments data analysis was done.The mean of teachers and students in every single item was calculated to summarize the teachers' and students' perspective about the textbook.

IV. FINDINGS 4.1 Practical Consideration
According to Cunningsworth (1995), it is necessary to consider a number of practical considerations for textbook evaluation such as the durability and attractiveness of the physical appearance of the textbook, the easiness in obtaining the textbook package in a short period of time, and reasonability of the textbook package price.The results of the study showed that the majority of teachers stated their agreement with the reasonableness of the price and easy accessibility of the textbook.However, more than half of the teachers stated that the textbook was not a recent publication.The book has been revised in 2002 which is rather a long time ago.Moreover, all of the teachers indicated that a teacher's guide, workbook, and audio-tapes did not accompany the textbook.The study also showed that twothirds of the teachers' views on language and methodology were different from those of textbook authors.The teachers were moderately satisfied with the section of practical consideration.Based on the results, it can also be concluded that students were satisfied with this section.The results in term of "Practical Consideration" are in agreement with the findings of the study (ELT teachers' perspectives about preuniversity English textbook) done by Rezaeian and Zamanian (2014).They indicated that the book is cheap and easily accessible and that it is not a recent publication.Moreover, they stated that there are no supplementary materials like teacher's guide, workbook, and audio-tapes accompanying the book.

Layout and Design
As the results revealed, a majority of the teachers stated that the textbook included a detailed overview of the functions, structures and vocabulary that will be taught in each unit.Furthermore, more than half of the teachers indicated that the layout and design were appropriate and clear.The study also showed that more than half of the teachers believed that the textbook was organized effectively and that an adequate vocabulary list or glossary was included.However, a majority of the teachers stated that there was not an adequate set of evaluation quizzes or testing suggestions in the preuniversity English textbook.All of the teachers indicated their disagreement with the item saying "the teacher's book contains guidance about how the textbook can be used to the utmost advantage".More than half of the teachers stated that the materials objectives were apparent to both the teacher and the student.Overall, most of the teachers were moderately satisfied with the layout and design of the textbook.In this regard, a majority of the students stated that the layout and design were appropriate and clear.The study also showed that more than half of the students believed that the textbook was organized effectively.Students, as well as teachers, had a moderate satisfaction regarding the layout and design.

Activities
In this part, the pre-university English textbook provided a balance of activities; that is, there was an even distribution of free versus controlled exercises and tasks that focus on both fluent and accurate production.Moreover, more than half of the teachers stated that the activities did encourage sufficient communicative and meaningful practice.One of the activities in Pre-University English textbook which encourages meaningful and communicative practice is the discussion questions after the reading where the students are asked to discuss their personal ideas about the topics raised in the reading passage.Likewise, half of the teachers indicated that the activities incorporated individual, pair and group work.
The other half of the participants were of the same belief that the activities did not incorporate individual, pair and group work.
In conclusion, most of the teachers stated that the grammar points and vocabulary items were introduced in motivating and realistic contexts.To the teachers, the activities did not promote creative, original and independent responses.Since more than half of the participants indicated their disagreement with this part, it might be assumed that preuniversity textbook activities mostly promote independent responses not creative and original ones.In the same vein, more than half of the teachers believed that the tasks do not result in the internalization of newly introduced language because they do not have any relevance to the real-life situation and students' needs.Moreover, most of the teachers stated that the textbook's activities could not be modified or supplemented easily.The textbook's activities might be modified in the grammar section for practice exercises, but it seems to be difficult to modify or supplement them in the reading section.The teachers were moderately satisfied with activities presented in the pre-university English textbook.It is while the students were dissatisfied with the activities of the textbook.

Skills
This section of the evaluation form deals with such issues as whether the materials focus on the skills that the students need, whether there is a balance among the four language skills, and finally, whether the textbook pays attention to sub-skills, natural pronunciation, and the integration of individual skills into each other.In this regard, it was found that most of the teachers believed that there is a balance among all four skills of reading, writing, listening, and speaking.They also believed the material did not provide an appropriate balance of the four language skills.Likewise, a majority of the teachers were of the same opinion that the textbook did not pay attention to the sub-skills, that is, listening for gist, note-taking, skimming for information, etc.As for the sub-skills, there are some activities devised to improve reading skills.There are no activities teaching the sub-skills of listening comprehension.In addition, most of the teachers believed that there are no activities highlighting and practicing natural pronunciation, stress, and intonation.They argued that practice of individual skills was not integrated into the practice of other skills.With reference to the textbook, it was found that the activities in comprehension check might integrate speaking and writing skills to some extent.Other than that, no traces were found for the integration of different language skills.The textbook mostly focuses on reading comprehension, grammar, and vocabulary.It does not focus on the skills and their integration.In general, the results of this study for this part showed teachers' dissatisfaction with the section of skills.The findings revealed that most of the students indicated that the materials include and focus on the skills that they need to practice.However, about seventy percent of the students believed that the materials did not provide an appropriate balance of the four language skills.It might be concluded that the skills which students care more about are reading and writing skills because the pre-university textbook does not cover listening and speaking skills.In addition, a majority of the students stated that the textbook did not pay attention to the sub-skills, that is, listening for gist, note-taking, skimming for information, etc.The mean score for this section was 2.53, which shows that students were not satisfied with the section of skills.

Language Type
The Language Type component consists of six subsections tapping areas like the authenticity of the language, the appropriateness of the language for the level, the suitable progression of vocabulary and grammar, exemplification of the grammar points, use of language functions in real life, and finally the incorporation of different registers and accents.As the results revealed, most of the teachers believed that the language used in the textbook is like real-life English.As students are studying at the Pre-University level, authentic language is recommended in order to motivate the students' interest.Unlike the findings of this study in term of "Language Type", the results of the study (a content evaluation of Iranian pre-university ELT textbook) have done by Maleki, Mollaee, and Khosravi (2014) showed that the language used in pre-university English textbook is not authentic.Regarding the authenticity of the used language used in the textbook, the results of this study are in line with the results of the study (ELT teachers' perspectives about pre-university English textbook) conducted by Rezaeian and Zamanian (2014).They found out, the same as this study, that the language used in pre-university English textbook is authentic.In addition, most of the teachers stated that the language used was not at the right level for their (students') current English ability.It was found that most of the indicated that the progression of grammar points and vocabulary items was appropriate.In the same vein, more than seventy per cent of the participants stated that the grammar points are presented with the brief and easy examples and explanations.As for the presentation of grammar points, more personalized, real-life examples would facilitate the internalization of the new structure.Similarly, most of the teachers stated that the language functions exemplified English that they/their students would be likely to use.A majority of the teachers unanimously indicated that the language did not represent a variety of registers and accents.The only register used in English for Pre-University Students was Standard English.It would be much more interesting if a diverse range of registers and accents were represented.The mean score of 3 showed the moderate satisfaction of the teachers for the section of language type in the textbook.In term of "Language Type", contrary to the results of the study which have done by Rezaeian and Zamanian (2014), the findings of this study revealed that the language used in pre-university English textbook is rather not at the right level for the students'.The results also revealed that the progression of grammar points and vocabulary items is not appropriate.As for the presentation of grammar points, more personalized, real-life examples would facilitate the internalization of the new structure.Similarly, the findings of this study are in agreement with the findings of the study performed by Rezaeian and Zamanian (2014).The results of the study indicated that there is not a variety of registers and accents in the textbook.The only register used in Pre-University English textbook is 'Standard English'.It would be much more interesting if a variety of registers and accents were represented.As the results revealed, a majority of the students indicated their agreement with the authenticity of the language used in the textbook.In addition, most of the students stated that the language used was at the right level for their current English ability.Moreover, a majority of the students were of the same belief that the progression of grammar points and vocabulary items was appropriate.According to the results, about two-thirds of the participants stated that the grammar points were presented with the brief and easy examples and explanations.About half of the students stated that the language functions exemplified English that they would be likely to use in the future.As it can be concluded from the results, a majority of the participants stated that the language did not represent a diverse range of registers and accents.As it is mentioned previously, it would be much more interesting if a diverse range of registers and accents were represented.The mean score of 3.01 revealed that students were moderately satisfied with the language type of the textbook.

Subject and Content
A high number of the teachers stated that the subject and content of the textbook was relevant to their students' needs as English language learners.Findings show that a majority of the teachers stated that the subject and content of the textbook was generally realistic and more than half of the teachers were unanimous with the opinion that the subject and content of the textbook was interesting, challenging and motivating.Similarly, most of the teachers declared that there was sufficient variety in the subject and content of the textbook.Likewise, a vast majority of the teachers stated that the materials were not culturally biased and they did not portray any negative stereotypes.The related mean score (M= 3.33) revealed that teachers moderately contend with the Pre-University English textbook regarding subject and content of the textbook.The majority of students believe that the subject and content of the textbook was relevant to their needs as English language learners.Likewise, a high number of the students indicated their agreement regarding the second item of this section and most of them stated that the subject and content of the materials was interesting, challenging and motivating.They also believed that there was sufficient variety in the subject and content of the textbook.Most of them were of the same opinion that the materials were not culturally biased and they did not portray any negative stereotypes.The mean score for this section was (M= 3.44) which showed the moderate satisfaction of the students in relation to subject and content of the textbook.

Overall Consensus
This section taps areas like whether the textbook is appropriate for the language learning aims of the institute, whether the textbook is suitable for small-medium, homogeneous classes of university students, whether the textbook raises students' interest in further English language study, and finally whether the teachers would choose to teach this textbook again.The results revealed that a high number of the teachers stated that the textbook was appropriate for the language-learning aims of their institution.According to the results of the survey, more than half of the teachers believed that the textbook was not suitable for smallmedium, homogeneous, co-ed.classes of university students because it is designed for Pre-University students.About ninety per cent of the teachers were of the same belief that the textbook raised their (students') interest in further English language study.Moreover, they preferred to choose to study or teach this textbook again.The mean score of 3.42 showed the moderate satisfaction of the teachers within the section of overall consensus of the textbook.It was also found that the students were overall dissatisfied (M= 2.59) with the textbook.The results of this study in term of "Overall Consensus" are in agreement with the findings of the study (ELT teachers' perspectives about pre-university English textbook) carried out by Rezaeian and Zamanian (2014).As the results of their study indicated, the aim of pre-university English textbook is improving students' reading comprehension skill.Given such an aim, we might judge that the textbook is appropriate for the language learning aims set by the Ministry of Education.But on a larger scale, we see that the book does not develop other skills of language like listening, speaking, and writing, which is a deficiency of the textbook.The textbook is not at the suitable level for small-medium, homogeneous classes of university students.It is designed for Pre-University students.Most of the raters agreed that the textbook raises the students' interest in further English language study.By applying a more communicative approach and covering all the skills and components of language the textbook would become much more interesting.Almost most of the teachers chose to use this book again.In the present study, unlike teachers, most of the students refused to use this book again.

Difference Between Teachers' and Students'
Perspectives The results revealed that, although there were some matches and mismatches between the teachers and student's perspectives regarding different parts of the textbook based on the Litz (2005) checklist, there was not any significant difference between their attitudes.The overall mean score for both teachers and students showed their moderate satisfaction with the textbook.

V. CONCLUSION
The current study was an attempt to evaluate the appropriateness and effectiveness of the Pre-University English textbook from the teachers' and the students' perspectives.To do this, two forms of the checklist (teachers' textbook evaluation form and students' textbook evaluation form) were used to collect the data quantitatively.Thirty English teachers, randomly selected from different high schools in Piran Shahr and Khoy, participated in this study.In addition, 156 students based on random selection from different high schools in Piran Shahr and Khoy, Iran, took part in the current study.All of the students were studying in Pre-University level and they had English as an essential course which the students require to take and pass it.In summary, the main conclusion that can be drawn from the results of this study is that both teachers' and students' perceptions about the Pre-University English textbook were almost positive in general.The results showed that both teachers and students were moderately satisfied with the Pre-University English textbook.Based on the teachers' perceptions in this study, the Pre-University English textbook is not appropriate for Iranian high school students in term of skills.They believed that the Pre-University English textbook, in terms of practical consideration, layout and design, activities, language type, subject and content and overall consensus meets the appropriateness to some extent.In general, teachers were moderately satisfied with the Pre-University English textbook except for the skills.Regarding students' perspectives toward the Pre-University English textbook, it can be concluded that students did not rate to the appropriateness of the textbook in terms of activities, skills, and overall consensus.They believed that the textbook was appropriate in terms of practical consideration, layout and design, language type, and subject and content.It can be concluded that the Pre-University textbook focuses just on reading and somehow on writing.It can be seen that the textbook does not improve other skills of language like listening and speaking which is a deficiency of the textbook.To make the textbook much more interesting, materials developers and syllabus and curriculum designers in Ministry of Education should apply a more communicative approach in the textbook and also they should consider all the four language skills in the textbook.As Riasati and Zare (2010) express, "no single textbook can meet the needs of a large and different group of language learners" (p.60).The burden, as Riasati and Zare (2010) conclude, lies on the teacher to use the appropriate teaching methods and techniques to compensate for the deficiencies of a certain textbook in order to bring about effective learning outcomes.Additionally, it would be the duty of the materials developers and syllabus and curriculum designers in Ministry of Education and other pedagogical experts to revise the current textbook and remove the weak points of the textbook especially in term of skills.