Linguistic Simplification: A Promising Test
Accommodation for LEP Students?
Charles W. Stansfield
Second Language Testing
In recent years, there has been much discussion about the role of language
minority students in state assessments. The vast majority of states surveyed
have dealt with the issue by exempting language minority students, with
forty-four of forty-eight states exempting limited English proficient (LEP)
students from one or more assessments, and more than half (27 of 44) routinely
exempting LEP students from state assessments altogether (Rivera et al., 1997).
Rivera and Vincent (1997) have questioned the wisdom of this policy. They argue
that if LEP students are meant to attain the same high performance standards as
their monolingual counterparts, they should be included in state assessments as
well. Instead of excluding LEP students from assessments, they argue that states
should make judicious use of accommodations that are specially designed with
these students' linguistic needs in mind.
There has been little experimental research conducted to investigate the overall
effects of accommodations such as those used for students with disabilities, let
alone research on accommodations that address the linguistic needs of LEP
students. Without empirical data, it is unclear what role a particular test
accommodation may play. One accommodation may give an unfair advantage to
examinees receiving it, whereas another may not improve the performance of even
those who have special needs and should benefit the most from it. Therefore, it
is essential that research be conducted to determine whether accommodations are
a threat to a test's reliability and validity, or to score comparability for
examinees who receive them and examinees who do not.
This article is a synopsis of an experimental study of the effects of linguistic
simplification, a test accommodation designed for LEP students. Conducted as
part of Delaware's statewide assessment program, this study examined the effects
of linguistic simplification of fourth- and sixth-grade science test items and
specifically looked at score comparability between LEP and non-LEP examinees.
Why Linguistic Simplification? A Review of the Literature on Simplified English
Although the concept of simplifying English has been around for more than
seventy years, it has received little attention in research. The first
"Basic English" system was designed in 1932 as an alternative, easy
means of cross-cultural communication (Ogden, 1932). It consisted of a core
vocabulary of 850 words and a few limited syntactic structures.
The concept lay dormant until the 1970s and 1980s, when it was picked up again
by multinational corporations looking to facilitate communication and training.
Among others, the Caterpillar Corporation (Association Européene de
Constructeurs de Matérial Aerospatiale, 1972) and Boeing, Inc. (Shubert et al.,
1995) used simplified English to prepare their training manuals for use around
the world. Despite its use in corporate settings, only two experimental studies
appear to have been conducted on linguistic simplification as an accommodation
for LEPs.
Abedi and others (1998) did a study of simplification using mathematics items
from the National Assessment of Educational Progress (NAEP). He administered
regular NAEP math assessment, a simplified English version, or a Spanish version
of the items to 1400 eighth-grade students in southern California middle
schools. Results indicated that both LEP and non-LEP students performed best on
the simplified version and worst on the Spanish version. However, his analyses
also suggested that linguistic simplification doesn't always work as intended,
as significant differences in item difficulty were obtained on only 34% of the
simplified items. Abedi concluded that linguistic simplification of math items
might be beneficial to all students, not just those with limited English
proficiency.
Kiplinger et al. (2000) conducted another study using mathematics items from
NAEP. This time, a simplified English version, a version with a glossary
containing definitions of non-technical terms, and an unsimplified version were
administered in Colorado. The instruments were randomly assigned to 1200 special
education, LEP, and regular fourth-grade students. Their results showed no
significant difference for the three versions across all three types of
students, and neither regular nor LEP students performed significantly better on
either version. They did find, however, that the students who performed best on
the test benefited most from the version that had a glossary, and somewhat from
the simplified version. On the basis of these findings, the researchers
concluded that glossaries and linguistic simplification might benefit all
students.
Experimental Study on the Effects of Linguistic Simplification on a Statewide
Science Assessment
Rivera and Stansfield (2001) used Abedi (1998) and Kiplinger et al. (2000) as an
impetus for further research on linguistic simplification. Both of these
previous studies seemed to provide evidence that linguistic simplification of
items might be a useful accommodation for LEPs in formal assessment settings.
However, Rivera and Stansfield highlighted the need for a formal experimental
study to determine the effect linguistic simplification might have on scores for
LEP and non-LEP students. Only once score comparability has been established can
an accommodation be rightfully endorsed.
The two researchers conducted a study to examine the effects of linguistic
simplification on fourth- and sixth-grade science test items used in the
Delaware Student Testing Program. At each grade level, four experimental 10-item
testlets were included on the operational forms of the science test. Two of the
testlets contained regular field test items that had been linguistically
simplified, and the other two contained the same field test items written in
regular (unsimplified) English. The testlets were randomly assigned to both LEP
and non-LEP students throughout the state.
A total of 11,306 non-LEP students and 109 LEP students took one of the forms of
the test. Because the number of LEP students was split among the eight forms,
the number of LEP students taking each test form was small, ranging from 6 to 23
students. While the researchers caution that due to the limited sample size,
nothing can be generalized about linguistic simplification as an aid to LEP
students, the findings for the large non-LEP sample are quite clear. Results of
t-tests performed on mean raw scores, analyses of variance (ANOVAs), and
post-hoc pairwise comparisons all indicated that overall, there was no
significant difference in scores of non-LEP students who took the simplified
version as opposed to the regular (unsimplified) one. This is an important
finding because it shows that linguistic simplification can be used without fear
of providing an unfair advantage to those who receive it, and thereby affecting
the comparability of scores across examinees in this condition. Since linguistic
simplification is able to reduce the level of English language proficiency
needed to comprehend a test item, it is likely that it can reduce the role of
language proficiency in achievement test scores in general.
Limitations and Suggestions for Further Research
Other studies should now address the issue of the usefulness of linguistic
simplification for LEP students taking formal and high-stakes assessments. If
experimental studies involving large samples of LEP students who are randomly
assigned to treatments show that those LEP students who receive simplified items
perform statistically and meaningfully better than those who receive the
unsimplified version of such items, then the utility of linguistic
simplification in meeting the needs of LEP test-takers will be established.
In this study, we chose to simplify items on a statewide science assessment.
Therefore, the preliminary results we obtained may not hold for other subject
areas, and further research is needed to determine the effects of linguistic
simplification in other areas such as math and social studies.
While the small sample size did not allow us to address the effectiveness of
linguistic simplification for LEPs, the study's results did show that tests and
items can be linguistically simplified without compromising score comparability.
However, test developers must exercise caution when carrying out the process of
linguistic simplification. The result of the process of linguistic
simplification must be to make items accessible to LEPs while not altering the
difficulty of the content being tested. And at times, in some items, language
and content interact to such an extent that simplification is not possible.
However, the results of this study suggest that if test developers and
researchers are careful in carrying out linguistic simplification, the resulting
assessment could address the linguistic needs of the LEP students without
compromising the comparability of the scores obtained on the assessment by
taking the standard English version.
References
Abedi, J., Lord, C., & Hofstetter, C. (1998). Impact of Selected Background
Variables on Students' NAEP Math Performance. Los Angeles: UCLA Center for the
Study of Evaluation/National Center for Research on Evaluation, Standards and
Student Testing.
Association Européene de Constructeurs de Matérial Aerospatiale (1972). AECMA
Simplified English Document: A Guide for the Preparation of Aircraft Maintenance
Procedures in the International Aerospace Maintenance Language. AIA Issue,
Change 4.
Kiplinger, V.L., Haug, C.A., & Abedi, J. (2000). A Math Assessment Should
Assess Math, Not Reading: One State's Approach to the Problem. Paper presented
at the 30th National Conference on Large Scale Assessment, Snowbird, UT, June
25-28.
Ogden, C.K. (1932). Basic English, A General Introduction with Rules and
Grammar. London: Paul Treber & Co.
Rivera, C., & Stansfield, C.W. (2001). The Effects of Linguistic
Simplification of Science Test Items on Performance of Limited English
Proficient and Monolingual English-Speaking Students. Paper presented at the
annual meeting of the American Educational Research Association, Seattle, WA.
Rivera, C., Vincent, C., Hafner, A., & LaCelle-Peterson, M. (1997).
Statewide Assessment Programs: Policies and Practices for the Inclusion of
Limited English Proficient Students. ERIC Digest. Washington D.C.: ERIC
Clearinghouse on Language and Linguistics. ED 362 073.
Rivera, C., & Vincent, C. (1997). High school graduation testing: Policies
and practices in the assessment of English language learners. Educational
Assessment, 4 (4): 335-55.
Shubert, J.K., et al. (1995). "The Comprehensibility of Simplified English
in Procedures." Technical Writing and Communication, 25 (4): 347-369.
