Indonesian Teachers’ Beliefs and Experiences of Computer-Based English Summative Tests

Computer-Based English Summative Test (CBEST) has gained popularity in many educational institutions recently. In this testing mode, students do the exam with the help of computer in language laboratory. The result of the test is usually gained immediately after the test is done. Though CBEST has been popular, the implementation of CBEST in Indonesian educational context has not been widely published. This paper specifically presents teachers' beliefs and experiences in doing CBEST testing mode. The data were collected by distributing questionnaire to 9 (nine) English teachers from 3 (three) different vocational schools. The result of the study showed that teachers believe that the implementation of CBEST give them benefits in several aspects, such as aspect of economy, aspect of system implementation and aspect of test administration and design..


I. INTRODUCTION
In this 21 st century, computer has been one of tools which is usually used in education. It is not only used for the purposes of learning activities, but also used in testing purposes. In the context of English language teaching (ELT), the use of computer in language testing is popular with the term: Computer Assisted Language Testing (CALT). Specifically, when CALT is done during summative tests, this terminology is also known as Computer-Based Summative Test (CBST). In other words, CALT is a type of testing procedure in which language performance is assessed using computer. Three processed were involved during the implementation of CALT: generating the test, interaction with the candidate, and evaluation of responses (Noijon, 2012). In the first process, computer selects a number of items from an item bank, randomly or following some selection procedure until a number of items have been collected and for use in the construction of a test. In the second procedure, test candidate do the test by using certain instructions in the computer. In the third procedure, all candidates' data which have been taken during the previous process are called up for a final evaluation of the complete response.
In Indonesia, especially in vocational school context, CBEST is considered as a new testing mode. Based on the data from Indonesian Ministry of Education and Culture (Kemendikbud), in 2017, there were 9.829 schools which has already implemented CBEST, while in 2016, there were 2100 schools and in 2015, there were 379 schools. Since the number of the school with CBEST testing mode is increasing each year, it is then important to indentify how teachers believe the new testing mode, and how do they prepare their students for facing the CBEST. Therefore, this study focuses on the implementation of CBEST in vocational school context by investigating teachers' beliefs and experiences in the testing mode.

II. METHODOLOGY Research Site
This research was conducted in three different vocational schools in Bandung. They are: SMK Assalam, SMK Negeri 9 Bandung and SMK Sandy Putra Telkom. SMK Assalam is located on Jalan Situ Tarate, Cibaduyut, Bandung. SMK Negeri 9 Bandung is situated on Jalan Soekarno Hatta, Bandung. While SMK Sandy Putra Telkom is located on Jalan Bojong Soang Bandung.
From each school, three English teachers were involved. Since 2016 the schools have become one of adopter schools for Cambridge. In the process of teaching and learning English, the schools combine the 2013 curriculum in combination with several Cambridge books, like Interchange, etc. The schools have already implemented Computer Based English Summative Test (CBEST). This summative test for English is delivered via computers conducted in language laboratory.

Participant
There are 9 participant teachers involved in this study. Three participants are English teachers from SMK Assalam Bandung. Three other participants come from SMK Negeri 9 Badung. And the other three participants are from SMK Sandy Putra Telkom. The more detailed data about the participants are displayed in the table 1 below:

Instrument
In this research project, the researcher used questionnaire to gain the data. Questionnaires were distributed to 9 participant teachers. "Questionnaires are any written instruments that present respondents with a series of questions or statements to which they are to react either by writing out their answers or selecting from among existing answers" (Brown, 2001). The researcher considers this data collection technique is the most suitable technique to elicit teachers' beliefs and experiences in preparing and taking Computer-Based Summative English Tests (CBSET). This is because questionnaire can measure attitudinal information by asking attitudinal questions. Dornyei (2009) elaborates that attitudinal questions are used to find out what people think. This is a broad category that concern attitudes, opinions, beliefs, interests and values.
The use of questionnaire in this research project has several advantages. The first issue is cost-effectiveness. Questionnaire can save researcher time, effort and financial resources. By administering questionnaire to the a class of students which consists of this research, a huge amount of information about teachers' beliefs and experiences about CBEST in vocational school levels can be collected in a relatively short time. The second issue is versatility. Dornyei (2009) explains that questionnaire can be used successfully with a variety of people in a variety of situations targeting a variety of topics. This is also in accordance to what Bryman (2008) says that a wellconstructed questionnaire can reduce the bias of interviewer effects and thus increase the consistency and reliability of the results. Due to these merits, it is no wonder that many research projects in social sciences employ questionnaire as one of the data collection techniques. The questionnaire consisted of 8 open-ended questions about teachers' beliefs and experiences of CBEST.

Procedure
The first step in the research procedure was collecting the data. The data were collected by distributing questionnaire to 9 participants from three different vocational schools. The next process after collecting the data is to analyze it. Burns (2000) says that the purpose of analyzing the data is to find meaning in the data and this is done by systematically arranging and presenting the information. It has to be organized so that comparisons, contrasts, and insights can be made and demonstrated. Specifically, thematic analysis was used in analyzing the data. Boyatzis (1998) elaborates thematic analysis as a strategy in qualitative research to analyze information in a systematic way in order to make the data understandable. It organizes and describes the data in detail according to emergent themes. In doing thematic analysis, the researcher used the phases by Braun and Clarke (2006) as the guidance. report of the analysis. The first step in the process of data analysis was organizing the data. This step involved transcribing the data gained from questionnaire. The data then were analyzed and interpreted to identify the links between the data in every questionnaire. After transcribing and translating the data, the next procedure was labeling the data based on the data sources. The next analytic procedure was repeatedly reading of the text of the transcription of the questionnaire. The next step was coding. Creswell (2009) states that coding process is to make sense out of data, divide it into text or image segment, label the segments with codes, examine codes for overlap and redundancy and collapse these codes into broad themes. In this study, coding was intended to identify certain ideas in the data that represented the same meanings. Finally, the data were categorized into the aspects related to teachers' beliefs and experiences in the implementation of Computer Based English Summative Tests (CBEST).

III. RESULT AND DISCUSSION
The elaboration about the result of this study is based on each question posed about teachers' beliefs and experiences in CBEST from three different aspects: economy, system implementation, administration and design.
Economy Aspect Of CBT Economy Aspect) As we can seen from table 3 above, the majority of participants (100%) state that CBT testing mode can save times. They said that CBT testing mode can be done is shorter duration than PBT. In PBT, one subject test usually take 90-120 minutes. However, in CBT test duration can be made shorter to approximately 60 minutes, because students just need to click the answers without blackening the answer sheets like in PBT. The second question is about whether CBT can promote ecofriendly environment. All participants agree that CBT can promote greener environment since it is paperless. No test booklet needed, no answer sheets needed. All scoring mechanism is done by computers. And the last question is about whether CBT is cheaper than PBT. The participants showed different beliefs. Three participants (33%) said that CBT is more expensive, since schools have to provide a number of computer units to facilitate students in doing CBT. Sometimes the number of the computer is not balanced with the number of students. And 6 other participants said that CBT is cheaper because schools can reduce the cost for printing test booklets, printing answer sheets, paying teachers for scoring students' works and so on.
System Implementati on Aspect Of CBT

Questions Answers Yes
No NA CBT has efficient administration 8 (89%) 1 (11%) 0 (0%) CBT save exam data accurately 9 (100%) 0 (0%) 0 (0%) In CBT, data can be retrieved easily Viewed from aspect of system implementation, participants showed different beliefs. The majority of participants (89%) said that CBT has efficient administration since schools do not need to print the test booklets, store it in one room before distributing them to every classroom. However, there is one participant (11%) believed that CBT is not efficient since the school becomes dependent on IT experts. Once there is no IT experts, CBT will not run effectively. The second question is about whether CBT save accurate data https://dx. doi.org/10.22161/ijels.3.6.36  ISSN: 2456-7620 www.ijels.com Page | 1201 collection. All participants (100%) believed that in CBT, exam data will be more likely accurately collected and easier to store. Also, results and other data can be stored in much less space and it is easier to retrieve. Moreover, responses generally are accurately captures and scored. In PBT however, students often make some marks on the answer sheets which can result in inaccurate scoring. .

Questions Answers Yes
No NA CBT gives immediate feedback 9 (100%) 0 (0%) 0 (0%) CBT is preferred by students Viewed from the aspect of test administration and design, all participants (100%) believed that CBT is effective because it can give immediate feedback. In PBT, the result of summative tests usually takes several days to finish. Teachers as test raters often need time to assess the test and to make decision about the tests. However, in CBT immediate viewing of scores on screen is provided in CBT to give test takers the instant feedback. This is in line with Mojarrad et al (2014) that immediate feedback, accurate test result reports and the possibility of printing the basic testing statistics are other advantages of using computer in assessment field that enable test takers take the test at any. The second question is about whether CBT is preferred by students. 5 participants (56%) said that students like CBT than PBT because many students are already familiar with the use of computer. Some students have used computers to play games and some of them might receive the instruction through computers. Some students might prefer CBT since they can customize the assessment based on their personal preference, like colors on the screen, font types, font sizes, and so on. Due to the possibility of customizing the assessment based on personal preferences, some people prefer to take CBT version of the test. However, 4 participants (44%) said that CBT is not always preferred by students. Some students are still confused how to use computers and they feel anxious when dealing with computers, so some students will perform better in PBT mode. This is in line with what Cater et al. (2010;Russell et al. (2010) that although some students may prefer CBT, others may prefer paper and pencil-based test. Some test takers prefer paper-based testing process because they are accustomed to taking notes and circling questions and answers for later review. Therefore, training for students before the CBT is needed to be conducted so students will be familiar with the computer and will feel comfortable in doing the test.

IV. CONCLUSION
The aim of this study is to investigate teacher's beliefs about CBT in English testing. Their beliefs are based on their experiences in preparing students to face CBT. Teachers' beliefs are based on the three aspects of CBT: economy, system implementation and test administration & design of CBT. The result of the study showed that teachers' have different beliefs about the implementation of CBT. However, CBT can be one of the solutions in preparing students in facing the 21 st century needs by having technology literacy.