TESTING MEMO 1: GUESSING ON MULTIPLE-CHOICE TESTS
by
Robert B. Frary
Virginia Polytechnic Institute and State 
University

     Ever since multiple-choice tests became widely used, 
in the 1920s, there has been concern over the fact that 
guessing affects the scores on these tests.  At first the
phenomenon was not well understood, and score increases due 
to guessing were uncritically viewed as ill-gotten gains 
even though these score components usually reflect partial
knowledge--the ability to eliminate some wrong choices before
guessing.
   The reaction of some educators was to admonish students
against all guessing, directly or indirectly condemning it 
as dishonest.  Of course, admonishing students against guessing
was widely ineffective as well as unfair to those who refrained,
so long as the tests were scored on the basis of the number of
right answers.
   As a result, many educators avoided use of multiple-choice
tests. However, multiple-choice tests became indispensable for
mass testing and were found to have other virtues which argued
for their inclusion in the educational process, such as broader
coverage of instructional topics, accuracy of scoring, and
provision of statistical feedback at the item level.  Hence,
neither admonishment against guessing nor avoidance of
multiple-choice tests was a satisfactory approach to resolving
what was perceived as a problem.
   One approach which gained wide acceptance was the use of a
scoring formula which "corrects" for purely random guessing.  
The conventional correction formula subtracts a fraction of the
wrong answers from the number-right score.  A mathematically
equivalent procedure is to award partial score credit for 
omitted questions rather than deduct score credit for wrong
answers.  The later approach has a psychological advantage
over the former method because it rewards the desired behavior,
not guessing completely at random, rather than penalizing
undesired behavior.
   Regardless of which correction formula is employed, it is 
an ethical requirement that students be encouraged to answer 
all questions on which one or more choices can be eliminated 
as incorrect.  Only if the answer would represent a sheer guess
among all choices should the examinee be advised to omit the
question when formula scoring is to be used.
   Either of the procedures just described may be desirable when
many examinees are expected to be unable to finish or to be
completely ignorant on large proportions of test questions. 
However, there are several reasons why either method of
"correcting for guessing" is likely to be undesirable in a
typical academic setting:
    1. Very few examinees will be so ignorant or so slow that
they will fail to attempt or be completely unable to eliminate 
a single wrong choice on any substantial proportion of questions. 
Hence the effort of "correcting for guessing" is largely wasted. 
The few who legitimately should omit substantial proportions of
questions under formula scoring will be so low in achievement
that very low scores will result regardless of whether completely
random guessing is suppressed.
    2. The admonishment not to guess in the absence of
information may be interpreted differently by each examinee and
thus may introduce score variance associated with personality or
background factors. This phenomenon has been confirmed in
numerous published studies. Other published studies have shown
that when students do omit questions under conventional
"correction for guessing" instructions, they are (on
the average) able to choose significantly more correct answers 
to these questions than under chance expectation.
    3. Individuals may choose to disregard the instructions
because, on the average, "correcting for guessing" does not
penalize for random guessing but only removes the score gain
expected from completely random guessing. In fact, if a 
student's knowledge is inadequate for obtaining a needed score,
the best strategy for that student is to guess on all questions,
hoping that luck in the short term will be favorable.  Because
this action is contrary to the instructions not to guess randomly
among all choices, the instructor is placed in the questionable
position of giving directions which some students may ignore to
their benefit.
   In balance, then, it is difficult to recommend any scoring
procedure to control guessing for multiple-choice testing in a
typical academic setting. In the absence of this practice, the
only fair procedure is to advise all students that it is to 
their advantage to answer every question regardless of perceived
knowledge.

For more information, contact Bob Frary at

Robert B. Frary, Director of Measurement 
and Research Services
Office of Measurement and Research Services
2096 Derring Hall
Virginia Polytechnic Institute and State 
University
Blacksburg, VA 24060
703/231-5413 (voice)
frary#064;vtvm1.cc.vt.edu
###


.