|From the ERIC database
Locating and Evaluating Career Assessment Instruments. ERIC Digest.
For the purpose of both locating and evaluating career assessment instruments, there are three primary sources. Best known among these are the Buros Institute's publication "Tests in Print" and its comparison set of reviews in the "Mental Measurements Yearbooks (MMY)." A second source, which first became available in 1983-84, is "Tests and Test Critiques (TC)." Both include a listing and brief description of most tests commercially available in English speaking countries (i.e., "Tests in Print" and "Tests") as well as periodically published volumes of test reviews (i.e., "MMY" & "TC")
The third source, which is published by the National Career Development Association, is "A Counselor's Guide to Career Assessment Instruments." This book, first published in 1982 and every six years since, contains reviews of the most prominent career assessment instruments as well as brief descriptions of most others commercially available. In addition, this book also includes chapters on selecting, evaluating, using, and interpreting career relevant tests.
There are a number of other sources that focus on specialized aspects of career assessment. Also, certain journals publish test reviews or articles that provide evidence of the quality of specific career assessment instruments. These qualities typically are evaluated under the categories of norms, reliability and validity. The American Educational Research Association (AERA), American Psychological Association (APA) and National Council on Measurement in Education (NCME) jointly publish "Standards for Educational and Psychological Testing," which provides guidance for both test publishers and users of all types of tests (AERA, APA & NCME, 1985). In addition, the American Counseling Association (formerly the American Association for Counseling and Development, 1989) has produced its own guidelines, "Responsibilities of Users of Standardized Tests," which provides additional help for users in counseling situations.
Much information is available to help locate and evaluate career assessment instruments, but it is the users who must employ these resources to make their own judgments about the appropriateness of a particular instrument for a specific situation. The purpose of this digest is to help users locate and organize information that will improve their evaluations.
If the instruments selected in the initial search are among the 52 with a complete review, the user can consult the reviews to further narrow the choice. Each entry includes a section of publisher-provided information that includes target population, statement of purpose, titles of subtests, scales and scores, forms and levels, date of most recent edition, languages available, time, norm groups, results reported, format, scoring, computer software, costs, comments, and published reviews. The review section is divided into the following headings: Description, Use in Counseling, Technical Considerations, Computer-Based Version (if available), Overall Critique, and References. If the instruments on which further information is needed are not among the 52 reviewed, the Additional Instruments chapter can be consulted to obtain the publisher, date and intended population on any of the 245 additional instruments. In addition, citations for all reviews published in the "Mental Measurements Yearbooks," "Test Critiques," previous editions of "A Counselor's Guide" as well as several other sources are listed. A brief description for each test is also included.
For those instruments not reviewed in "A Counselor's Guide" (3rd edition) or for a second opinion, the reader can consult one or more reviews cited in either the Additional Instruments chapter or at the end of the publisher information. If a "Mental Measurements Yearbook (MMY)" is to be consulted, it is necessary to know both the volume (Sixth, 1965 through Eleventh, 1992) and test number. The "MMY" entries typically contain brief publisher information along with two independent reviews. The reviews themselves are not divided into sections and tend to focus primarily on psychometric characteristics. Additional references are also provided.
To access "Test Critiques (TC)," it is also necessary to know the volume (Volume 1, 1984 through Volume 10, 1994). Each of these entries are written by a single reviewer and are divided into five categories (Introduction, Practical Applications/Use, Technical Aspects, Critique, and References). In addition to "MMY" and "TC" it may also be useful to consult the first (1982) and second (1988) editions of "A Counselor's Guide" or any of the other sources listed in the chapter on Sources of Information about Testing and Career Assessment in the third edition, which is an annotated bibliography of sources.
In addition to the technical qualities of norms, reliability and validity, there are many other qualities of a career assessment instrument that need to be evaluated before selection for a particular use. In his chapter in "A Counselor's Guide" on "Selecting a Career Assessment Instrument," Mehrens covers many of these, including types of scores and interpretation materials, appropriateness for various groups and a host of practical issues (e.g., qualifications of users, time, costs, and publisher support). In addition, he provides a "Test Evaluation Outline" that is included here to assist the user to systematically identify and collect all information necessary to conduct an adequate evaluation (Mehrens, 1994, p. 29):
1. State your purpose for testing
2. Describe the group that will be tested (for example, age or grade)
3. Name of test
6. Copyright date(s)
7. Purpose and recommended use as stated in the manual
8. Grade/age levels for which the instrument was constructed
9. Forms: Are equivalent forms available? What evidence is presented on the equivalence of the forms?
10. Format: comment on legibility, attractiveness, and convenience
12. Content of test and types of items used
13. Administration and timing requirements
14. Scoring processes available (e.g., machine scoring)
15. Types of derived scores available
16. Types and quality of norms
17. Adequacy of reliability evidence presented in the manual
18. Validity evidence
19. General quality of administrative, interpretative and technical manuals
20. Comments about the instrument by outside reviewers
American Educational Research Association, American Psychological Association, National Council on Measurement in Education. (1985). Standards for educational and psychological testing. Washington, D.C.: American Psychological Association.
Kapes, J.T., Mastie, M.M., & Whitfield, E.A. (Eds.). (1994). A counselor's guide to career assessment instruments (3rd edition). Alexandria, VA: National Career Development Association.
Keyser, J.D., & Sweetland, R.C. (1984-94). Test critiques (Vols. 1-10). Austin, TX: PRO-ED. Kramer, J.J., & Conoley, J.C. (Eds.). (1992). Eleventh mental measurements yearbook. Lincoln, NE: Buros Institute of Mental Measurement.
Mehrens, W.A. (1994). Selecting a career assessment instrument. In J.T. Kapes, M.M. Mastie, & E.A. Whitfield (Eds.), A counselor's guide to career assessment instruments (3rd edition). Alexandria, VA: National Career Development Association.
Murphy, L.L., Conoley, J.C., Impara, J.C., & Mitchell, J.V. (Eds.). (1994). Tests in print IV. Lincoln, NE: Buros Institute of Mental Measurement.
Sweetland, R.C., & Keyser, D.J. (Eds.). (1991). Tests: A comprehensive reference for assessments in psychology, education, and business (3rd edition). Austin, TX: PRO-ED.
Jerome T. Kapes, Ph.D., NCC, NCCC, is a professor of Educational Psychology and a faculty member in the Counseling Psychology, Educational Psychology Foundations and Vocational Education programs at Texas A&M University in College Station, Texas.
ERIC Digests are in the public domain and may be freely reproduced and disseminated. This publication was funded by the U.S. Department of Education, Office of Educational Research and Improvement, Contract No. RR93002004. Opinions expressed in this report do not necessarily reflect the positions of the U.S. Department of Education, OERI, or ERIC/CASS
Title: Locating and Evaluating Career Assessment Instruments. ERIC Digest.
Descriptors: * Aptitude Tests; Career Choice; * Career Counseling; Career Exploration; Career Guidance; Career Planning; Careers; Evaluation Methods; Evaluation Research; * Interest Inventories; * Occupational Tests; Test Interpretation; * Test Reviews; * Test Selection; Testing
Identifiers: ERIC Digests
Full-text Library | Search ERIC | Test Locator | ERIC System | Assessment Resources | Calls for papers | About us | Site map | Search | Help
Sitemap 1 - Sitemap 2 - Sitemap 3 - Sitemap 4 - Sitemap 5 - Sitemap 6
©1999-2012 Clearinghouse on Assessment and Evaluation. All rights reserved. Your privacy is guaranteed at