INTRODUCTION

Counselor educators and field supervisors often feel uncomfortable about assessing trainee skills and struggle to find an appropriate vehicle for delivering essential constructive feedback regarding performance. Most have received little or no training in evaluation or assessment practices. However, current and proposed accreditation, certification, and licensure regulations place an increasing emphasis on the evaluation and assessment of counselor performance. Clearly, evaluation practices will need to be augmented by theoretical and conceptual knowledge, as well as programmatic research.

The purpose of this digest is to suggest that there exist some fairly basic premises from educational psychology (Gage & Berliner, 1984), educational evaluation (Isaacs & Michaels, 1981), and counselor supervision literature (Bernard & Goodyear, 1992) that can improve supervision evaluation practices, and thus reduce the ambiguity and uncertainty about evaluation in supervision. Although this digest does not specifically address program evaluation, it should be clear that this is also an important component of any comprehensive evaluation endeavor.

EVALUATION DEFINED
Professional competence evaluation is made in a series of formal and informal measurements that result in a judgment that an "individual is fit to practice a profession autonomously" (McGaghie, 1991). Summative evaluation describes "how effective or ineffective, how adequate or inadequate, how good or bad, how valuable or invaluable, and how appropriate or inappropriate" the trainee is "in terms of the perceptions of the individual who makes use of the information provided by the evaluator" (Isaac & Mitchell, 1981, p. 2). Counselor supervisors are responsible for summative evaluations and assessments of supervisee competence to university departments, state licensing boards, and agency administrators. Summative evaluation is described by Bernard and Goodyear (1992) as "the moment of truth when the supervisor steps back, takes stock, and decides how the trainee measures up" (p. 105). Effective summative evaluation requires clearly delineated performance objectives that can be assessed in both quantitative and qualitative terms and that have been made explicit to the trainee during initial supervision contacts.

The heart of counselor evaluation, however, is an on-going formative process which uses feedback and leads to trainee skills improvement and positive client outcome. In this case the trainee is the person using the information. Bernard and Goodyear (1992) refer to this kind of evaluation as "a constant variable in supervision." As a result, every supervision session will contain either an overt or covert formative evaluation component.

EVALUATION PRACTICES AND PROCEDURES
When supervisors measure behavioral therapeutic skills they find several difficult areas. First, they find that measurement and subsequent evaluation of therapeutic skill is a complex process in a field where many skills inventories and behavioral checklists abound, and research findings suggest that these may lack adequate reliability and validity. Second, university supervisors recognize the tension between providing a supportive facilitative environment within which counselors-in-training can feel free to stretch and learn counseling skills and the anxiety that results from academic grades. Third, lacking a theory of supervision, supervisors are unable to articulate desired outcomes for their supervisees and may revert to the evaluation of administrative detail and case management. As a result of these difficulties, numerous areas of competency may be neglected, anxiety may persist, and supervisors may resort to summative evaluation practices in global and poorly measured terms.

There are resources which outline requisite skills and knowledge for effective evaluation practices. The Curriculum Guide for Training Counselor Supervisors (Borders et al., 1991) provides specific learning objectives for supervisors-in-training. Other current publications (Bernard & Goodyear, 1992; Borders & Leddick, 1987; McGaghie, 1991; Stoltenberg & Delworth, 1987) further develop the Guide's "three curriculum threads" (p.60) of self awareness, theoretical and conceptual knowledge, and skills and techniques. The guidelines and suggestions from these resources are summarized in the following list of effective evaluation practices:

1. Clearly communicate evaluation criteria to supervisees and develop a mutually agreed upon written contract reflecting these criteria.

2. Identify and communicate supervisee strengths and weaknesses. The Ethical Guidelines for Counselor Supervisors (ACES, 1993) recommend that supervisors "provide supervisees with ongoing feedback on their performance." This performance feedback establishes for supervisees a clear sense of what they do well and which skills need to be developed. Supervisee strengths and weaknesses can be evaluated in terms of process, conceptual, personal, and professional skills (Bernard & Goodyear, 1992, p. 42).

3. Use constructive feedback techniques during evaluations. Supervisees are more likely to "hear" corrective feedback messages when these are preceded by positive feedback, focused on observable behaviors, and are delayed until a positive relationship has been established.

4. Utilize specific, behavioral, observable feedback dealing with counseling skills and techniques; avoid terms such as "understanding," "knowing and appreciating," and "being aware of." Successful evaluation practices should include behaviorally-based learning objectives (Gage & Berliner, 1984).

5. Use Interpersonal Process Recall (IPR) to raise supervisees' awareness about their personal developmental issues. The unobtrusive and nonthreatening nature of IPR is particularly helpful as supervisees retrospectively explore their thoughts, feelings, and a variety of client stimuli during counseling sessions. This process can assist supervisees in contributing to, and benefiting from, formative evaluation.

6. Employ multiple measures of supervisee counseling skills. These can include a variety of standardized rating scales including measures completed by both supervisor and supervisee, client ratings, and behavioral scales (Stoltenberg & Delworth, 1987). Additional measures such as work samples from audio/videos, critiques of counseling sessions, and conceptual case studies (both brief and detailed) can provide a comprehensive picture of a supervisee's competency, expectations, needs and professional development, as well as an understanding of the context within which both the counseling and the supervision take place.

7. Maintain a series of work samples in a portfolio for summative evaluation. Since the evaluation of only one session provides an inadequate assessment of supervisee competency, and the selective nature of work samples may prove to be an overly negative reflection of current competency level, the portfolio provides both the supervisor and the supervisee with a more comprehensive and useful basis for a summative evaluation.

8. Use a developmental approach which emphasizes both progressive growth toward desired goals and the learning readiness of the trainee (Nance, 1990). The Nance model emphasizes a learning readiness based on the supervisee's ability, confidence, and willingness--the assessment of which directs the roles and practices of the supervisor. As a result, supervisors can "match" their supervisee's level and "move" them toward independent functioning one step at a time. Although Nance does not specify evaluation practices, he clearly describes effective supervisory styles, interventions, role, contracts, and agendas for each developmental stage. These variables can guide the evaluation process indirectly by enabling the supervisor to understand the characteristics and appropriate expectations for supervisees at each developmental level.

SUMMARY
A structured approach to supervisee assessment and evaluation produces several beneficial outcomes. First, supervisors can reduce their own, as well as their supervisee's, anxiety about the process. The meanings associated with assessment can be altered to suggest a positive experience from which both partners can grow and learn. Second, supervisors who articulate their adopted "supervision theory" to their supervisees will also clarify their evaluation criteria as well as their supervision practices. Third, when evaluation is viewed as a process of formative and summative "assessment" of the skills, techniques, and developmental stage of the supervisee, both supervisees and their clients benefit. Fourth, as supervisors deal successfully with the process of supervisee evaluation, they also bring similar skills to the evaluation of their training programs, an area in search of an appropriate evaluation paradigm. Finally, just as training is most successful when multiple methods (didactic, modeling, and experiential) of skills acquisition are employed, so too the use of multiple methods for evaluation contributes to the supervisee's sense of self-worth and success.

REFERENCES
Association for Counselor Education and Supervision. Ethical Guidelines for Counseling Supervisors (1993). ACES Spectrum, 53(4), 5-8.

Bernard, J. M., & Goodyear, R. K. (1992). Fundamentals of clinical supervision. Boston: Allyn & Bacon.

Borders, L. D., Bernard, J. M., Dye, H. A., Fong, M. L., Henderson, P., & Nance, D. (1991). Curriculum guide for training counselor supervisors: Rationale, development, and implementation. Counselor Education and Supervision, 31, 58-77.

Borders, L. D., & Leddick, G. R. (1987). Handbook of counseling supervision. Alexandria, VA: Association for Counselor Education and Supervision.

Gage, N. L., & Berliner, D. C. (1984). Educational psychology (3rd ed.). Boston: Houghton Mifflin.

Isaacs, S., & Michael, W. B. (1981). Handbook in research and evaluation (2nd ed.). San Diego: EdITS.

McGaghie, W. C. (1991). Professional competence evaluation. Educational Researcher, 20, 3-9.

Nance, D. W. (1990). ACES Workshop on Counselor Supervision. Workshop presented at the annual convention of American Association for Counseling and Development, Cincinnati, OH.

Stoltenberg, C. D., & Delworth, U. (1987). Supervising counselors and therapists: A developmental approach. San Francisco: Jossey-Bass.

-----

Morag B. Colvin Harris, Ph.D., is an Associate Professor of Counselor Education and Director of Master's Level Clinical Training at East Texas State University in Commerce, Texas.

-----

ERIC Digests are in the public domain and may be freely reproduced and disseminated. This publication was funded by the Office of Educational Research and Improvement. Opinions expressed in this report do not necessarily reflect the positions of OERI or ERIC/CASS

Title: Supervisory Evaluation and Feedback. ERIC Digest.
Author: Harris, Morag B. Colvin
Publication Year: 1994
Document Type: Eric Product (071); Eric Digests (selected) (073)
Target Audience: Teachers and Practitioners
ERIC Identifier: ED372348
This document is available from the ERIC Document Reproduction Service.

Descriptors: Counselor Educators; Counselor Training; * Counselors; * Evaluation Methods; * Feedback; * Supervision; Supervisors

Identifiers: ERIC Digests