DETERMINING WHO IS RESPONSIBLE FOR WHAT AND TO WHOM ^Accountability has always been a basic concept in public education, although ideas about how to accomplish it have changed. In recent years, the urgent need to improve big city schools has been a powerful incentive to the adoption of new accountability systems. This digest explores the strengths and weaknesses of various accountability tools, the use and misuse of indicators, and ways to create genuine accountability at the school level. ^TYPES OF EDUCATIONAL ACCOUNTABILITY ^^At least five types of accountability mechanisms may exist alongside each other (Darling-Hammond, 1989): ^Political. School board members and legislators must stand for election. ^Legal. Boards and legislatures enact policies, and courts can hear complaints about schools. ^Bureaucratic. State and district education departments set rules and regulations to ensure that schools meet standards and follow procedures. ^Professional. Teachers and other school staff must acquire specialized knowledge, pass certification exams, and uphold professional standards of practice. ^Market. Parents and students may choose the programs or schools they believe are most appropriate for their needs. ^Over the past few decades American schools have relied most heavily on bureaucratic mechanisms for achieving accountability. Because professional and market accountability tools are currently being proposed as strategies for school improvement, it is important to understand what each of these three offers and what its limits may be. ^BUREAUCRATIC ACCOUNTABILITY ^^Bureaucratic accountability is based on the hope of finding the "one best system" under which all students will be educated. Its strength rests in its possibility to ensure equal and standardized education. Policies are made at the top and handed down to administrators who translate them into procedures for teachers to follow in educating students. This system of accountability does not hold teachers accountable for meeting the individual needs of their students; they can be held accountable only for following standard procedures. ^PROFESSIONAL ACCOUNTABILITY ^^Unlike bureaucratic accountability, whose goal is uniformity and standardization, professional accountability allows practitioners to make their own decisions about how to meet the educational needs of individual students. This model seeks to ensure that teachers will be highly knowledgeable, competent, and committed to good teaching. Consequently, a professional accountability system must pay particular attention, not only to student progress, but also to policies governing the preparation, certification, selection, and evaluation of teachers and other staff. ^MARKET ACCOUNTABILITY ^^Magnet schools and other choice plans are based on quasi-market mechanisms. These are supposed to make schools more accountable in two ways. Because "customers" choose schools, (1) schools are expected to work harder to provide services that parents or students want; and (2) problems in undersubscribed schools are revealed, which policymakers can then address. ^Market accountability poses several difficult questions: How can all citizens be guaranteed access to quality schools? How do students and their parents make school choices? What information should schools provide so that students and their parents can be informed consumers of public education? Other than changing schools, what vehicles for communication and redress can parents and students use when the schools do not meet their needs? ^BALANCING DIFFERENT FORMS OF ACCOUNTABILITY ^^No form of accountability is sufficient by itself to ensure that all students are well served. Because each form of accountability has both strengths and weaknesses, a combination of tools is needed to make schools responsible and responsive. ^In school choice plans, for example, when the most desirable schools are filled, the remaining schools, desirable or not, must serve the rest of the students. If no other public policy mechanisms are in place to support improvements in all schools, choice will not improve education across the board. A different accountability mechanism is needed to regulate and ensure equity. Similarly, if bureaucratic mechanisms are de-emphasized, greater guarantees of staff competence and commitment would need to be provided through professional accountability mechanisms. ^Further, new approaches create new dilemmas that must be resolved. For example, in cases of interdistrict choice, parents lose electoral accountability at the district level when students leave the district in which they reside. Thus, mechanisms must be devised to give parents a voice in their new schools. ^Similarly, school-based management proposals call for local ownership of education and make professionals at the school site primarily accountable, based on the belief that better decisions will be made by those who are closest to the situation. But in a system where significant authority devolves to the school level, how much responsibility should state governments and local districts still carry for students' treatment and achievement? ^In deciding which aspects of education should be relegated to bureaucratic accountability and which should be left to professional accountability, it is useful to distinguish between equity and productivity concerns. Wise and Gendler (1989) argue that equity issues, such as the allocation of resources and guarantees of equal access, can and should be resolved by higher units of governance, accountable to a wider public, and should not be left to the decision of individual teachers or the parents of one school district. ^However, since teaching, at its best, is a highly individualized process, productivity questions cannot be solved well by bureaucratic regulation. Uniform state- or district-level policy decisions about teaching methods and school processes cannot meet the needs of varying school and student circumstances. Thus, these needs are better addressed using professional accountability mechanisms. At the same time, the broader goals of the public require that states have some way of evaluating schools. ^USING STATISTICAL INDICATORS ^^Given the complexities, it is not surprising that policymakers often try to achieve accountability by the apparently easiest strategy: monitoring students' test scores and sometimes linking teacher or school rewards and sanctions to such measures. But accountability also encompasses how a school or school system hires, evaluates, and supports its staff; how it relates to students and parents; how it ensures that the best available knowledge will be acquired and used; how it evaluates its own functioning; and how it corrects its problems and provides incentives for continual improvement. ^Thus, performance indicators do not themselves create accountability. At best, test scores may provide data for accountability systems that enable schools to improve and correct problems. At worst, they may deflect attention from needed school changes. In fact, improperly designed performance indicators can undermine accountability by creating incorrect assumptions. ^CRITERIA FOR SELECTING INDICATORS ^^Because statistical indicators are used to support broad inferences about schools and students, policymakers developing indicator systems should consider carefully what they want to measure, how they can best measure it, and what other information they need to interpret trends intelligently. ^Oakes (1986) and Koretz (1989) maintain that indicators should offer information that is: ^Problem-oriented. Indicators should be able to detect current problems or potential difficulties. ^Relevant to policy. Information should be described in a way that is amenable to change by policy decisions or that shows policymakers where to target their efforts. ^Reflective of educational outcomes. The data might include graduation or dropout rates; college attendance rates; voting rates; achievement test scores; writing samples; and quality of participation in science fairs, debates, or dramatic productions. ^Indicative of student backgrounds. Indicators cannot be interpreted meaningfully without corollary information on which students are involved. Furthermore, if a school's student population is highly mobile, test scores intended to measure student growth at different points in time may not even be measuring the same students. ^Illustrative of school context. Data on how schools are organized and what they provide are needed to offer clues about why schools achieve the outcomes that they do. ^Oakes (1989) suggests that three interrelated variables are important to school achievement, and that each can be tapped with a set of indicators. The first, access to knowledge, is the extent to which schools give students opportunities to learn various domains of knowledge and skills. The second, press for achievement, is the mix of incentives--graduation requirements, program rigor, recognition of accomplishments--that schools provide students to work and achieve. Third are the professional teaching conditions--salaries, work load, time for planning--that can empower or constrain teachers as they create and implement instruction. ^CRITERIA FOR INTERPRETING INDICATORS ^^The greatest danger of indicators is the ease with which they can give false impressions because they are misunderstood or interpreted in invalid ways. The most common misunderstandings derive from the use of average student test scores to indicate the quality of schools or school districts. ^The danger is especially relevant when the stakes are high. When rewards or sanctions are automatically triggered by test scores, incentives are created not to improve teaching and learning but to push low-scoring students out of the school, or at least out of the test score count (Darling-Hammond, 1990; Haney & Madaus, 1986). ^Validity. If averages hide information about individual students, interpretations drawn from average test scores will be invalid. Similarly, test items are meant to be samples of domains of learning, and the assumption that student performance on the test represents the broader curriculum goals is negated if teachers "teach to the test" (Koretz, 1989). ^Reliability. To be reliable, a test score must be free of random errors of measurement. That is, if statistics on a school's curricular offerings are likely to vary, depending on who is collecting the data or how they were collected, the scores are not a reliable indicator of student progress. ^Corruptibility of Indicators. Because the emphasis on basic skills test scores has prompted teaching to the test, many students spend less time on untested subjects, such as science, social studies, and the arts. The scores, therefore, no longer necessarily indicate students' general achievement. Emphasis on basic skills can also mean that the scores may no longer provide comprehensive assessments of student' ability, even in the tested subjects, because classwork narrowly oriented toward a test does not heighten students' proficiency in aspects of the subject not tested--analysis, complex problem-solving, and written or oral expression. ^Indeed, since about 1970, while test scores on basic skills have improved, scores on assessments of higher-order thinking skills have declined in virtually all subject areas. Many observers, including the National Research Council and the National Councils of Teachers of English and Mathematics, argue that the overuse of multiple choice basic skills tests has actually corrupted teaching practices. Aspects of the subjects that are not tested--especially higher-order thinking and performance skills--are left untaught. ^Test scores are not the only indicators that can be corrupted. Indicators of student course-taking, for example, may prove to mean something other than what policymakers intended. When pressure is great to meet new standards, schools with poor resources or shortages of qualified teachers may implement new science requirements quite differently from wealthy schools. In one school, three years of science may mean three years of general science lectures; in another school, it may mean a rigorous sequence of biology, chemistry, and physics classes replete with laboratory experiences. ^MAKING FAIR COMPARISONS ^^Accountability systems often compare schools and school districts and increasingly connect rewards and punishments to these comparisons. Thus, it is important to set up comparisons fairly. This means that school outcomes cannot be compared without also comparing populations, resources, and educational goals. For example, schools serving low-income students often have far fewer highly experienced and qualified teachers and offer many fewer advanced courses than wealthier schools do. Comparisons of test scores that ignore these factors hold little promise of directing policymakers' attention to real problems. ^One way to create fair comparisons is to develop measures of how much each individual student has learned over a period of time. Another approach, employed in California, is to compare schools with similar student populations and mobility patterns, and a wide range of other similar characteristics. ^CREATING ACCOUNTABILITY AT THE SCHOOL LEVEL ^^An accountability system is a set of commitments, policies, and practices designed to: ^increase the use of good educational practices; ^reduce the use of harmful or wasteful practices; and ^create internal mechanisms to identify, diagnose, and change courses of action that do not lead to learning. ^Accountability is achieved only if a school's policies and practices work to provide good education and correct problems as they occur. Performance indicators and ongoing diagnostic processes are needed to evaluate whether these conditions are being met. These processes must include methods for changing school practices if they are not working well on behalf of students. ^Until now, most indicator systems have been defined by states, and schools or

Title: Accountability Mechanisms in Big City School Systems. ERIC/CUE Digest No. 71.
Author: Darling-Hammond, Linda; Ascher, Carol
Publication Year: Apr 1991
Document Identifier: ERIC Document Reproduction Service No ED334311
Document Type: Eric Product (071); Eric Digests (selected) (073)
Target Audience: Practitioners

Descriptors: * Accountability; Elementary Secondary Education; Evaluation Criteria; Evaluation Methods; Evaluation Utilization; * Outcomes of Education; * School Effectiveness; * Systems Approach; * Urban Schools

Identifiers: *Educational Indicators; ERIC Digests