Chapter one Introduction
1.1 Background
Language test is an important instrument to assess learners' language proficiency. With the revolution and widespread use of the computer technology, language testers have made every effort to utilize computer technology to measure language skills to satisfy the ever-growing need for maximal accuracy and efficiency. Among them are the TOEFL CBT by Educational Testing Service and College English Test, the largest language test in China, which had made its long-term plan on computer-based test reform. Language test witnesses an increasing change of test media with the tendency of CBT to replace PBT. CBT is increasingly being viewed as a practical alternative to PBT. This influence also has been seen in the field of achievement test. In teaching activities, achievement test is a very important procedure of teaching during four-year undergraduate education. It can be a valuable tool for providing feedbacks on the effectiveness of the teaching program itself. In the coming decades, CBT will continue to be more frequently used in assessment circles. The English majors' Achievement test of Sichuan International Studies computer-based University has been conducted under the above situation.
New test method facets made possible by computer-based test administration, including modes of presentation, raise issues pertaining to construct validity (Green, 1990). According to Sawaki (2001)，in order to support construct validity of computerized tests, the equivalence of paper-based and computer-based test modes must be established from various direction. For example, to find out what extent are the results of a PBLT comparable to those of its CBLT version and whether CBLT version can be considered to have equal reliability or accuracy of measurement. These views merit a more careful reconsidering of the issue of validity with respect to the mode of administration, which is considered to reflect or simulate real world settings much better than PBLT (Brown, 1997; Alderson, 1999; Dunkel, 1999). Thus, a multitude of comparability studies have been conducted on a variety of types of tests, typically to compare scores across computer and paper administrations of the same fixed-form multiple-choice tests. Many researchers subsequently devoted to the study of equivalence or comparability of scores from CBLT and PBLT, and it has been found that measures obtained from tests delivered in the computer-based mode are similar to those obtained from the paper-based mode (Bergstrom, 1992; Boo and Vispoel, 1998; Wang et al., 2001).A review of previous literature shows inconsistent conclusions of the comparability studies. In China, there are few studies on equivalence of CBLT and PBLT, such as, Li (2006) discussed the concept, significance and principles of equivalence study, making a summarization of previous foreign studies on the comparability between the paper-based test and the computer-based test. Chen (2009) claimed that the validity of CBLT is possibly affected by three factors: computer familiarity, audio-visual cognitive competence, and other discrepancies in construct. Che, Wang (2010) explore whether the change of test medium would affect the variation of students' whole test scores.
It is even more important to explore the equivalence of the two modes of test since the results of these studies are inconsistent and the widespread use of computer technology. Therefore, sufficient empirical data, which would allow researchers to look into the CBT with regard to validity, reliability and fairness, are yet to be available. Moreover, many research studies related to CBT were conducted in othercountries, but such studies in China are mostly concentrated on College English Test and Oral test. Studies on achievement test are relatively lack. Therefore, it is significant that some comparative studies related to CBT and PBT be conducted to verify those research reports in China. In a continuation of the discussion concerning present and future research on comparative study of computer-based and paper-based tests in language test, the author of this paper conduct the empirical research in order to collect fresh evidence for the equivalence of computer-based and paper-based test.
2.3 Relationship between...............27
2.4 Previous Studies on the............... 27-35
2.4.1 Overseas Studies ............... 27-31
2.4.2 Previous Studies............... 31-34
2.4.3 Limitations ...............34-35
Chapter Three Theoretical ...............35-40
3.1 General Procedures ............... 35-36
3.1.1 Test Preparation............... 35
3.1.2 Administration...............35
3.1.3 Scoring of ............... 35-36
3.1.4 Score Recording............... 36
3.1.5 Test Paper ...............36
3.2 Points of Comparison ............... 36-38
3.2.1 Test Paper ............... 36-37
3.2.2 Cost of Test ............... 37
3.2.3 Effectiveness of ...............37
3.2.4 Convenience of ............... 37
3.2.5 Rater ............... 37-38
3.2.6 Response Validity............... 38
3.3 The General Framework ............... 38-40
Chapter Four Methodology............... 40-45
4.1 Research Questions............... 40
4.2 Research Design............... 40-41
4.3 Method............... 41-43
4.3.1 Subjects............... 41
4.3.2 Instruments............... 41-43
4.3.3 Procedure............... 43
4.4 Data Collection............... 43-45
Major Findings
The present study conducted comparatives study on CBEMAT and PBEMAT. More specifically, it testified whether CBEMAT could replace the PBEMAT. With such attempts, the present study analysis the two forms of test from five aspects: test paper preparation, cost of test administration, effectiveness of administration of test, convenience of scoring and scoring record, rater reliability and response validity. Through the analyses of the questionnaire survey and correlation study of rater reliability, six research questions have been answered in Chapter four and the major findings in response answering five research questions can be summarized as follows.
(1)Computer-based testing is a computing strategy used to reduce paper consumption. By using test software platform to prepare test on the computer, the use of paper printing is minimized. This could reduce paper preparation costs as well as environmental impact, and it saves time and energy.