Merging Discipline-Based Instruction and ParSYSTEM Technology in Assessment Courses for Preservice and Inservice Teachers By Lou M. Carey University of South Florida ABSTRACT Current movements in schools and universities include better linkages among curriculum, instruction, and classroom tests; improved authenticity for instruction and assessments; and the integration of instruction and testing technology in the classroom. These movements are all accommodated in the Second Edition of Measuring and Evaluating School Learning (Allyn & Bacon) and the ParSYSTEM demonstration programs included within the text. Learning to use the ParSYSTEM programs while students learn principles of measurement and testing in their university courses helps ensure more authentic instruction and better transfer of measurement and technology skills to the classroom. ARTICLE Schools are challenged today to provide computers for classroom teachers and help teachers acquire the skills necessary to use them effectively. Classroom applications of computers for instruction, reference, and telecommunications are emphasized, but computers are also a valuable productivity tool for helping teachers produce and score tests, monitor individual and group progress, maintain grade books, and generate individualized progress reports for students and parents. One long range strategy for helping all teachers acquire technology skills is to incorporate computer applications in university courses for preservice and inservice teachers; however, this strategy is sometimes difficult for university faculty who do not have the materials and equipment necessary for incorporating computer applications in the courses they teach. A solution for this dilemma is to provide textbooks, instructor's guides, and student workbooks in which computer applications are integrated with course content, thereby providing student and professor alike with the support needed for moving into technology. One current example of textbook and technology integration for university-based preservice and inservice teacher education is Allyn and Bacon's textbook, Measuring and Evaluating School Learning. [Carey, L. M. (1994). Measuring and evaluating school learning. Boston, MA: Allyn & Bacon.] The text is supplied with a demonstration version of Scantron's ParSYSTEM software. Using the software in combination with the textbook helps teacher education students learn the principles and procedures of assessment and acquire hands-on computer skills for classroom applications. Learning to use these programs while learning principles of measurement and testing helps ensure more authentic university instruction and better transfer of measurement principles and skills to the classroom. MAKING UNIVERSITY INSTRUCTION AUTHENTIC Using the ParSYSTEM programs with Measuring and Evaluating School Learning has revolutionized assessment courses for preservice and inservice teachers. Teachers previously lamented not having enough time to accomplish in the classroom all the tasks they were learning in the measurement course. While unfortunate, this was often true. Integrating the computer programs into the text and class instruction, however, helps teachers see how assessment tasks can be accomplished efficiently in their classrooms. The relevance of ParSYSTEM becomes quite clear after students have experienced, just once, the effort it takes to obtain indices such as item and test quality, group performance, and term composite scores using a calculator. MANAGING THE INTEGRATED COURSE From a course management perspective, the ParSYSTEM programs are easy for professors to integrate with the text, Measuring and Evaluating School Learning and its accompanying Student Manual, Instructor's Manual. Students study the text and learn the ParTEST and ParGRADE tasks using their own computers at home, those provided in a designated computer laboratory, or an open lab on campus. The Student Manual that accompanies Measuring and Evaluating School Learning contains step-by-step, task-oriented instructions that even computer novices are using successfully. They follow instructions in their Student Manual to open the program, examine a 93-item test item bank, produce a product test with a companion scoring rubric, and produce an objective test. They also follow step-by-step instructions in their Student's Manual to design and develop their own test item bank and to explore the ParGRADE program. By working independently, students develop a high level of confidence in using the programs that helps ensure they will transfer their new computer skills to the classroom. The following paragraphs describe skill areas where the text, Measuring and Evaluating School Learning and the ParSYSTEM programs are integrated. DESIGNING ITEM BANKS As students study the text to develop skills in linking classroom tests to curriculum and instruction, they use the ParTEST demonstration program to create user-descriptor and subcategory structures for their own ParTEST item bank . There are six main user descriptors in the program, each with multiple categories, and students use these descriptors to define the basic structure of their bank . The six main user-descriptors are definable in the program, and students typically choose the following item classifications: (1) instructional goal; (2) subject or topic; (3) test type (purpose) ; (4) test format ; (5) enabling skills or objectives; and (6) learning categories. With the six main descriptors defined, they choose logical categories within each. For the first descriptor, instructional goal, they usually enter the code for the instructional goals included in the state or district curriculum guide. For the second, subject or topic, they enter the main topic areas included within a particular goal. Within test type (purpose), they usually enter categories such as readiness tests, pretests, practice tests, unit posttests, term exams, and semester exams. The test format descriptor typically contains the categories of product development instructions, live performance instructions, scoring rubrics, essay tests, and objective tests. For enabling skills or objectives; students often include the enabling skill codes from their instructional goal frameworks. The last descriptor, learning categories, typically includes levels from Bloom's taxonomy or intellectual skills categories such as concrete concepts, defined concepts, classification, interpretation, evaluation, and production or performance. DEVELOPING ITEM BANKS As students learn to write performance objectives and format test items, performance directions, and scoring rubrics, they actually enter their own items, directions, and rubrics into the ParTEST bank. They use the performance objectives they write along with formatting rules from the text as the basis for developing their items. Each item is classified as it is created using the six main user descriptors and subcategories. Students then print their banks and use rubrics contained in the textbook to evaluate their items for item-objective congruence, learner appropriateness, bias, and clarity. Any problems they detect are corrected in the program. Students also produce a ParTEST Cross-Reference Report (all items in the bank cross-referenced by main user descriptor and subcategory) and use the report to evaluate the content-by-learning level balance they have in their bank. Where there are gaps, they write additional items. CREATING TESTS Students learn from the text how to create tables of test specifications for given test purposes, and consider item sampling strategies to help ensure reliable scores and valid inferences from scores. As a related activity, students use their six user descriptors and subcategories to retrieve from their ParTEST item banks the specific items, directions, and scoring rubrics prescribed in their tables of test specifications. They can then print the test they specify within a matter of minutes. Both preservice and inservice teachers are pleasantly surprised at how little time it takes them to assemble and print pretests, practice tests, posttests, and makeup tests once their test items, directions, and scoring forms are entered into the bank. For the tests they create, their alternative assessments are accompanied by matching scoring rubrics, and they can produce multiple forms of their objective tests by scrambling either item order or the order of answers within an item. INTERPRETING SCORE REPORTS The ParSCORE program is not included with Measuring and Evaluating School Learning; however, as students learn to calculate and interpret indices of item and test quality, group performance, and individual mastery, they learn to interpret the various reports provided for program users. Samples of ParSCORE reports are included in each of the data analysis chapters in the text. Students use the reports to locate potentially faulty items and tests, interpret group performance indices, and describe group and individual student's performance levels. DESIGNING AND USING GRADE BOOKS Students learn how to use the ParGRADE program as they study procedures for designing a grade book, weighting various scores, creating composite scores, and grading students' progress using both traditional and mastery procedures. They also practice generating interim and term progress reports for pupils and parents. REALIZING BENEFITS FOR PRESERVICE TEACHERS In addition to learning classroom assessment skills, another important outcome of the course is the assessment technology portfolio that students create during the term. Step-by-step instructions for creating the portfolio are contained in the Student Manual. The portfolio includes a student's original item bank; printed pretests, practice tests, and posttests for a state/district mandated instructional goal; and test design documentation ( instructional goal, description of pupils and learning context, performance objectives, tables of test specifications, and item bank Cross-Reference Report). This portfolio becomes a part of the professional portfolio that students submit with their application for a teaching position. The assessment technology portfolio is a powerful demonstration to district personnel that these new teachers and the teacher education institutions are actively supporting the school districts' efforts to infuse technology into classrooms.