Volume: 8 7 6 5 4 3 2 1



A peer-reviewed electronic journal. ISSN 1531-7714 
Search:
Copyright 1992, EdResearch.org.

Permission is granted to distribute this article for nonprofit, educational purposes if it is copied in its entirety and the journal is credited. Please notify the editor if an article is to be used in a newsletter.


Find similar papers in
    ERICAE Full Text Library
Pract Assess, Res & Eval
ERIC RIE & CIJE 1990-
ERIC On-Demand Docs
 
Find articles in ERIC written by
     Rudner, Lawrence & Farris
 Farris, Michael P.
Rudner, Lawrence & Farris, Michael P. (1992). A precedent for test validation. Practical Assessment, Research & Evaluation, 3(6). Retrieved August 18, 2006 from http://edresearch.org/pare/getvn.asp?v=3&n=6 . This paper has been viewed 5,855 times since 11/13/99.

A Precedent for Test Validation

Lawrence Rudner, 
Clearinghouse on Assessment and Evaluation

Michael P. Farris, 
Home School Legal Defense Association

In December 1991, the South Carolina Supreme Court ordered the state Board of Education to stop using the Education Entrance Examination (EEE) for licensing parents who want to teach their children at home. That ruling struck down a statute requiring parents with only high school diplomas to pass the EEE before being approved as home schoolers.

In ruling that the test's validation process did not meet a standard of reasonableness, the Court established a significant precedent for test validation. This article discusses the South Carolina Supreme Court's decision and its implications for future validation efforts.

BACKGROUND

South Carolina's home-schooling statute required the Board of Education to evaluate the suitability of the EEE, a test designed as an admissions test for teacher education programs, as a test for licensing home schoolers. Through the state Department of Education, the Board of Education contracted a traditional content-based validation study. A panel of judges was asked to evaluate each item on the EEE for task-relatedness and bias.

Of the 33 panelists, 17 were home schoolers, and the remaining 16 were public school and college teachers. Panelists who were not home schoolers received no description of the requirements for successful home schooling.

For task-relatedness, the panelists had to decide whether the knowledge or skill needed to answer the particular item was "a necessary prerequisite" for home schoolers. To evaluate bias, the panelists considered whether the test items would offend or unfairly penalize any group of home schoolers because of gender, ethnicity, religion, or socio-economic status.

The contractor scored the entire panel's responses for the items on the EEE. For the entire panel, the ratings were:

task-relatedness

presence of bias

reading

84%

reading

10%

math

79%

math

4%

writing

75%

writing

39%

The contractor reported to the Department of Education that these scores were good enough to validate the EEE's use for home schoolers. The Department then accepted the EEE as provided in the statute.

THE CHALLENGE

The plaintiffs, the Home School Legal Defense Association, brought a class action suit challenging the validation process. They convinced the Court that the validation was defective for two main reasons:

  • panelists were not given a "job analysis" or description of successful home schooling, and
  • the panel was not qualified to make the judgments expected of them.

Since 16 of the 33 panelists knew nothing about home schooling and received no information about its prerequisites, the Court ruled that relying on their evaluations of task-relatedness in validating the EEE was unreasonable.

Reanalyzing the contractor's data, the plaintiffs found large differences in the task-relatedness ratings for the subgroup of home schoolers compared with the entire panel. The subgroup ratings were: 

task-relatedness

presence of bias

reading

74%

reading

12%

math

66%

math

8%

writing

56%

writing

50%

The Court noted that the task-relatedness scores of teacher panelists were 20% to 44% higher than the scores of home-schooling panelists. Panelists with no knowledge of home schooling found the EEE items more task-related to home schooling than did panelists familiar with home schooling.

THE STATE'S POSITION

The Board of Education relied heavily on two main arguments to justify using the EEE for home schoolers. First, the board argued that the EEE is designed to test basic literacy, an underlying qualification for teaching in either the public school or the home. This argument relied on the theory of validity generalization, which means that the validity of a test established for one task may be assumed or generalized for another task if the tasks are reasonably similar. The Court dismissed this argument because it overlooks a specific requirement of the statute--validation of the EEE for home schoolers. The state legislature considered the tasks of home schooling and public school teaching very different.

The Board of Education also argued that home schoolers who have taken the EEE have a high pass rate: 258 of 310 test takers have received a passing score. The Court found that just because home schoolers can pass the EEE does not mean it is reasonable to require them to do so. At best, the high pass rate indicates that the EEE requirement has infringed little on parents' entitlement to home schooling; it does not, however, justify imposing such a requirement.

IMPLICATIONS FOR TEST VALIDATION

The Court did not accept the argument that the validity of the EEE generalizes from its original purpose, as an admissions test for teacher-education programs, to its use as a licensure examination for home schoolers. Educators need to base arguments for validity generalization on the accumulated research.

If the EEE or any other test were valid as a licensure examination for home schoolers in 30 other states, then we could reasonably expect it to be valid in South Carolina as well. But here, the state's purpose differed from the original purpose of the test. The Board of Education presented no evidence that the skills needed for home schooling are similar to the skills needed to succeed in teacher-education programs.

Finally, we cannot generalize from an unvalidated test. The Board of Education failed to present any evidence that the EEE was valid for any purpose.

Passing rates alone do not prove validity. The Board of Education argued that the test was not an imposition for home schoolers because of high passing rates. But the Court found that argument irrelevant. People who can effectively teach their children should be permitted to do so, regardless of their numbers. The passing rate only indicates a test's difficulty; it has nothing to do with validity. A perfectly valid test can have a zero pass rate. And a perfectly invalid test can have a 100% pass rate.

To judge the validity of a test for a particular use, we need to know which skills we're testing. With no description of home schooling, the plaintiffs argued, the panelists who were not home schoolers were not necessarily qualified to rate the test items. If the contractor had given them a job analysis, however, the Board of Education could have argued that those panelists were familiar with home-schooling tasks and thus qualified to judge the relevance of the test items.

Further, we need to clearly establish the expertise of the judges. In this case, the panelists were chosen to be representative college staff, public school teachers, and home schoolers. Still, they were not necessarily qualified. No evidence indicated that any of the judges, including those representing home schoolers, were qualified to make the types of judgments expected of them.

REFERENCES

For more detail on this case, see Opinion No. 23526, by South Carolina Supreme Court Judge the Honorable C. J. Gregory, December 9, 1991. (ED 338 719)

American Psychological Association (1986). Standards for Educational and Psychological Tests and Manuals. Washington, DC: APA.

Equal Employment Opportunity Commission (1978). Uniform Guidelines on Employee Selection Procedures, Federal Register 43, 116, 38295-38309.

Lawrence v South Carolina State Board of Education, 412 s.e.2d 394, 71 Ed. Law Reptr. 1226 (S.C. 1991).

Rafilson, F. (1991). The Case for Validity Generalization. Practical Assessment, Research and Evaluation 2(14). [Available online http://ericae.net/pare/getvn.asp?v=2&n=14].

-----

Descriptors: *Certification; Civil Liberties; *College Entrance Examinations; *Court Litigation; Elementary Secondary Education; *Home Schooling; *Parents as Teachers; School Attendance Legislation; State Legislation; State Standards; Test Use; *Test Validity

Sitemap 1 - Sitemap 2 - Sitemap 3 - Sitemap 4 - Sitemape 5 - Sitemap 6