A precedent for test validation. Rudner, Lawrence & Farris, Michael P.

In December 1991, the South Carolina Supreme Court ordered the state Board of Education to stop using the Education Entrance Examination (EEE) for licensing parents who want to teach their children at home. That ruling struck down a statute requiring parents with only high school diplomas to pass the EEE before being approved as home schoolers.

In ruling that the test's validation process did not meet a standard of reasonableness, the Court established a significant precedent for test validation. This article discusses the South Carolina Supreme Court's decision and its implications for future validation efforts.

South Carolina's home-schooling statute required the Board of Education to evaluate the suitability of the EEE, a test designed as an admissions test for teacher education programs, as a test for licensing home schoolers. Through the state Department of Education, the Board of Education contracted a traditional content-based validation study. A panel of judges was asked to evaluate each item on the EEE for task-relatedness and bias.

Of the 33 panelists, 17 were home schoolers, and the remaining 16 were public school and college teachers. Panelists who were not home schoolers received no description of the requirements for successful home schooling.

For task-relatedness, the panelists had to decide whether the knowledge or skill needed to answer the particular item was "a necessary prerequisite" for home schoolers. To evaluate bias, the panelists considered whether the test items would offend or unfairly penalize any group of home schoolers because of gender, ethnicity, religion, or socio-economic status.

The contractor scored the entire panel's responses for the items on the EEE. For the entire panel, the ratings were:

task-relatedness		presence of bias
reading	84%	reading	10%
math	79%	math	4%
writing	75%	writing	39%

The contractor reported to the Department of Education that these scores were good enough to validate the EEE's use for home schoolers. The Department then accepted the EEE as provided in the statute.

The plaintiffs, the Home School Legal Defense Association, brought a class action suit challenging the validation process. They convinced the Court that the validation was defective for two main reasons:

Since 16 of the 33 panelists knew nothing about home schooling and received no information about its prerequisites, the Court ruled that relying on their evaluations of task-relatedness in validating the EEE was unreasonable.

Reanalyzing the contractor's data, the plaintiffs found large differences in the task-relatedness ratings for the subgroup of home schoolers compared with the entire panel. The subgroup ratings were:

task-relatedness		presence of bias
reading	74%	reading	12%
math	66%	math	8%
writing	56%	writing	50%

The Court noted that the task-relatedness scores of teacher panelists were 20% to 44% higher than the scores of home-schooling panelists. Panelists with no knowledge of home schooling found the EEE items more task-related to home schooling than did panelists familiar with home schooling.

The Board of Education relied heavily on two main arguments to justify using the EEE for home schoolers. First, the board argued that the EEE is designed to test basic literacy, an underlying qualification for teaching in either the public school or the home. This argument relied on the theory of validity generalization, which means that the validity of a test established for one task may be assumed or generalized for another task if the tasks are reasonably similar. The Court dismissed this argument because it overlooks a specific requirement of the statute--validation of the EEE for home schoolers. The state legislature considered the tasks of home schooling and public school teaching very different.

The Board of Education also argued that home schoolers who have taken the EEE have a high pass rate: 258 of 310 test takers have received a passing score. The Court found that just because home schoolers can pass the EEE does not mean it is reasonable to require them to do so. At best, the high pass rate indicates that the EEE requirement has infringed little on parents' entitlement to home schooling; it does not, however, justify imposing such a requirement.

The Court did not accept the argument that the validity of the EEE generalizes from its original purpose, as an admissions test for teacher-education programs, to its use as a licensure examination for home schoolers. Educators need to base arguments for validity generalization on the accumulated research.

If the EEE or any other test were valid as a licensure examination for home schoolers in 30 other states, then we could reasonably expect it to be valid in South Carolina as well. But here, the state's purpose differed from the original purpose of the test. The Board of Education presented no evidence that the skills needed for home schooling are similar to the skills needed to succeed in teacher-education programs.

Finally, we cannot generalize from an unvalidated test. The Board of Education failed to present any evidence that the EEE was valid for any purpose.

Passing rates alone do not prove validity. The Board of Education argued that the test was not an imposition for home schoolers because of high passing rates. But the Court found that argument irrelevant. People who can effectively teach their children should be permitted to do so, regardless of their numbers. The passing rate only indicates a test's difficulty; it has nothing to do with validity. A perfectly valid test can have a zero pass rate. And a perfectly invalid test can have a 100% pass rate.

To judge the validity of a test for a particular use, we need to know which skills we're testing. With no description of home schooling, the plaintiffs argued, the panelists who were not home schoolers were not necessarily qualified to rate the test items. If the contractor had given them a job analysis, however, the Board of Education could have argued that those panelists were familiar with home-schooling tasks and thus qualified to judge the relevance of the test items.

Further, we need to clearly establish the expertise of the judges. In this case, the panelists were chosen to be representative college staff, public school teachers, and home schoolers. Still, they were not necessarily qualified. No evidence indicated that any of the judges, including those representing home schoolers, were qualified to make the types of judgments expected of them.

For more detail on this case, see Opinion No. 23526, by South Carolina Supreme Court Judge the Honorable C. J. Gregory, December 9, 1991. (ED 338 719)

American Psychological Association (1986). Standards for Educational and Psychological Tests and Manuals. Washington, DC: APA.

Equal Employment Opportunity Commission (1978). Uniform Guidelines on Employee Selection Procedures, Federal Register 43, 116, 38295-38309.

Lawrence v South Carolina State Board of Education, 412 s.e.2d 394, 71 Ed. Law Reptr. 1226 (S.C. 1991).

Rafilson, F. (1991). The Case for Validity Generalization. Practical Assessment, Research and Evaluation 2(14). [Available online http://ericae.net/pare/getvn.asp?v=2&n=14].

A Precedent for Test Validation