A NEW APPROACH TO THE ACHIEVEMENT TEST ITEMS EVALUATION: THE CORRECTNESS COEFFICIENT
Keywords:
evaluation methodologies, intelligent tutoring systems, media in educationAbstract
This paper deals with the problem set of an achievement test item scoring. The scoring process is generalized with the help of correctness coefficient – the new concept set up by the author. The paper describes complexly formalization of the scoring process, contextualizes the contemporary used methods to the general context and brings new methods as well. The scoring methods of sorting items and guessing penalty are described in detail. Observations described in this paper can help examiners with more accurate assessment of achievement test results. In the first part, the theoretical basics of the test item scoring are given. We are going to find out that whole scoring process depends on the teaching objectives, test item types, curriculum taxonomy and achievement test objectives. Then the theory of the test item types is described. After this theoretical introduction the concepts of the total achievement test score and correctness coefficient are set up. Let’s emphasize that using of the correctness coefficient is the new contribution of the author. Than the correctness coefficient is used to express the measure of examinee’s answer accuracy within the different test item types. Using the correctness coefficient for evaluation of closed multiple-choice items, injective and general relational items, narrow open items, joining items and sorting items are deeply examined and described. Various scoring method for these item types are discussed, especially for the sorting and joining items. Afterwards the theory of penalty guessing is expressed with the help of the correctness co efficient, which strengthens the ability and universality of theory being described. The main goal of this research paper is to provide the complex theoretical overview of the test item scoring problems, which can be useful for pedagogues, examiners and testing application (or e-learning system) developers to provide more accurate and clear evaluation process of the achievement test.
References
Bloom, B. S. (ed.) (1956) Taxonomy of Educational Objectives. Vol. 1: Cognitive Domain. New York: McKay, 1956.
Booth, R.; Clayton, B.; Hartcher, R.; Hungar, S.; Hyde, P.; Wilson, P. (2003) The development of quality online assessment in vocational education and training. Leabrrook: Australian National Training Authority.
Brusilovsky, P. (1999) ‘Adaptive and intelligent technologies for web-based education’. In C. Rollinger, & C. Peylo (Eds.). Intelligent systems and teleteaching [Special issue]. Künstliche Intelligenz, vol. 4, pp. 19–25.
Davis, B.G. (1993) Tools for Teaching. San Francisco: Jossey-Bass.
Foltýnek, T. (2005) The electronic test results as a feedback for teachers. In Trends in e-learning – Belcom’05 proceedings of papers. Prague: CVUT.
Foltýnek, T.; Motyčka, A. (2006) On the achievement test items scoring. In Efficiency and responsibility in education – proceedings of papers 2006. Prague: ČZU.
Hobgood, B.; Thibault, M.; Walbert, D. (2004) Kinetic connections: Bloom’s taxonomy in action, [online], Available: http://www.learnnc.org/articles/bloom0405-1 [31 Aug 2006].
Linrace, J.M.; Wright, B.D. (1995) How to Assign Item Weights. In Rasch Measurement Transaction, Vol. 8 no. 4.
Mužić, V. (1993) How to Outwit a Test. Zagreb: Školske novine.
Payne, D.A. (1968) The specification and measurement of learning outcomes. Waltham: Blaisdell Publishning Company.
Segall, N.; Doolen, T.L.; Porter J.D. (2005) ‘A usability comparison of PDA-based quizzes and paper-and-pencil quizzes’, Computers & Education, vol. 45, no. 4, pp. 417 – 432.
Stalker, M.J. (1968) ‘The Penalty for Not Guessing’, Journal of Educational Measurement, Vol. 5, No. 2 , pp. 141-144.
Additional Files
Published
How to Cite
Issue
Section
License
Authors declare with this manuscript intended for publication to ERIES Journal that:
- all co-authors agree with the publication of the manuscript even after amendments arising from peer review;
- all co-authors agree with the posting of the full text of this work on the web page of ERIES Journal and to the inclusion of references in databases accessible on the internet;
- no results of other researchers were used in the submitted manuscript without their consent, proper citation, or acknowledgement of their cooperation or material provided;
- the results (or any part of them) used in the manuscript have not been sent for publication to any other journal nor have they already been published (or if so, that the relevant works are cited in this manuscript);
- submission of the manuscript for publication was completed in accordance with the publishing regulations pertaining to place of work;
- experiments performed comply with current laws and written consent of the Scientific Ethics Committee / National Animal Care Authority (as is mentioned in the manuscript submitted);
- grant holders confirm that they have been informed of the submitted manuscript and they agree to its publication.
Authors retain copyright and grant ERIES Journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the published work with an acknowledgement of its initial publication in ERIES Journal. Moreover, authors are able to post the published work in an institutional repository with an acknowledgement of its initial publication in ERIES Journal. In addition, authors are permitted and encouraged to post the published work online (e.g. institutional repositories or on their website) as it can lead to productive exchanges, as well as earlier and greater citation of published work.
