Organizational issues related to portfolio assessment implementation in the classroom. Forgette-Giroux, Renée & Marielle Simon

This paper explores organizational issues that arose when implementing portfolio assessment in eleven classrooms during the field trial of a generic content selection framework². Some researchers have already examined, to various degrees, the organizational process teachers go through when implementing portfolios within their classrooms to assess learning as opposed to showcasing or reporting achievement. Their results point to four categories of factors that seemed to affect portfolio implementation process:

Other studies have suggested the existence of a possible portfolio assessment implementation process or continuum. Fingeret’s (1993) study, for example, led to the identification of a four-stage process revolving around specific tasks such as examining fit within assessment practices to revision of actual portfolio use. Calkins (1992), on the other hand, ranked teachers on a five-point continuum based on their level of acceptance of the portfolio and its integration within their teaching styles and approaches. The purpose of this paper is to highlight possible relationships between input and process variables and resulting organizational issues surrounding portfolio assessment implementation in the classroom when a generic content selection framework is provided.

Portfolio assessment is defined here as a cumulative and ongoing collection of entries that are selected and commented on by the student, the teacher and/or peers, to assess the student’s progress in the development of a competency (Simon, & Forgette-Giroux, 2000). The generic portfolio assessment content selection framework recommends the collection of entries (items or contents) along five learning dimensions of a competency:

The pieces of evidence are combined to provide an interrelated, complete, dynamic, and holistic picture of the students’ development toward mastery of a complex skill such as problem solving or oral communication. Whereas the five categories are considered fixed within the framework, organizational decisions regarding storing, scheduling, sharing of responsibilities, number and source of entry among others, within each category, "remain flexible for better integration and adaptation to the teachers' individual teaching and assessment styles and practices" (Simon & Forgette-Giroux, 2000, p.89).

Eleven volunteer teachers from five school boards in Eastern Ontario, Canada, agreed or asked to apply within their classes, the portfolio assessment content selection framework described above. Five of these teachers taught in two of the three boards that initiated a three-day workshop near the end of the school year to present the framework, while the other six teachers entered the study at various points during the following year. (See Table 1 for a description of the teacher variables). The latter received documentation and coaching on the framework upon request. All teachers were each visited twice from February to May in the year following the three-day workshop. The visits consisted of two in-class observations of portfolio use, followed by a 30 to 45-minute semi-structured interview with each teacher. Of the fifteen general questions, the following four were more or less related to organizational issues surrounding portfolio assessment implementation:

Observations within the classroom and content analysis of sample portfolios complemented the data obtained from the interview.

Table 1: Input variables for each teacher
	Variables
Teachers	school board	grade level	discipline	focus of portfolio assessment	year of portfolio use	attendance at a three-day workshop	attendance at group meetings (total of three)
Mark¹	1	1	Language	Writing	1	-	3
Corey	2	9	Geography	Communication	-	*²	-*
Gisele	2	7/8	Language	Communication	1	-*	1*
Sally	3	8	All	All	-	*	2
John	3	7/8	All	All	-	-*	-
Paul	3	7	Writing	All	-	-*	-
George	4	9	Math/Science	Problem solving	2	-	3*
Frances	4	8	Math	Problem solving	2	-	3*
Edith	5	6	Math	Problem solving	-	-	2*
Joanne	5	7/8	Math	Problem solving	-	-	3*
Carol	5	9	Geography/ Science	Problem solving Critical thinking Team work	-	-	2*
Fictional names have been given to protect the anonymity of the participants. An asterisk indicates that the respective school board liaison member also attended

A closer examination of the participants’ responses to the questions, their actions, and the content analysis lead to four categories of results. These are presented in the following sections.

The eleven teachers showed variability in planning and scheduling time around portfolio use. The three teachers from the school board #3 (See Table 1) allowed occasions for the students to select items from a file folder of their best work, to reflect briefly on each item selected, or to mark their own projects. This time was unscheduled and generally seen as extra to their teaching load. The three Grade Nine teachers, on the other hand, planned monthly or biweekly slots of time for their students to select and sort items. The five other teachers, all focusing either on problem solving or communication skills, reserved a full period each Friday or one every day or every other day for portfolio use.

Again, all three teachers from school board #3 used a brown accordion style folder with an elastic tie and five inside pockets provided by their board. These were stored in cardboard boxes or in a filing cabinet. The Grade Nine and Grade Eight teachers from school board #2 constructed their portfolios with two large cardboard sheets glued together on three sides. These were stacked haphazardly on a shelf at the back of the classroom. The six others provided their students with manila style folders with two inside pockets all stored in boxes or filing cabinet. All participants had their students keep a "working portfolio" in addition to the assessment portfolio and the students usually did not have free access to their portfolios.

Judging by their comments and actions, George, Paul, John, and Corey tended to underestimate their students’ ability to set up, maintain, and reflect on their portfolios. On the few occasions when their students were asked to reflect, they were only required to justify the selection of individual entries. The students’ involvement in organizing their portfolios was encouraged primarily to ease teaching tasks. On the other hand, Frances, Edith, and Joanne invited their students to reflect on and self-assess individual applications of problem solving skills. Gisele, Mark and Sally encouraged their students to organize their work themselves and to compare various items within their portfolios using rubrics, checklists, and award stickers.

The three teachers from school board #3 had their students select entries across subjects but with no clear focus. They had been initially instructed by school board officials to use the portfolio in support of the report card. The three Grade Nine teachers assessed communication or problem solving skills across one or two subjects but felt constrained by the school administrative structure and policies. The Grade One teacher assessed writing skills across Language Arts. In order to holistically assess communications skills across all Language Arts strands using the portfolio, the Grade Eight teacher from school board #2 had her students provide evidence of the framework’s five learning dimensions but the entries were not always clearly related to the targeted skill. In stressing problem solving skills, Joanne, Edith, and Frances extended the selection of entries to various disciplines.

The results suggest the formulation of three sets of research hypotheses. The first alludes to the portfolio assessment implementation process as involving four types of organizational issues: temporal, spatial, human and contextual. Temporal issues concern time spent on planning and scheduling portfolio assessment related activities and their fit within existing teaching and assessment practices. Spatial issues deal with organizing the portfolio’s format, physical characteristics, storage, and access. Human aspects include role-sharing such responsibilities as establishing and updating a table of contents, dating and sorting portfolio entries, reflection, and marking for formative or summative assessment purposes. Finally, contextual matters have to do with specifying the object of assessment, determining the scope of disciplines from which portfolio items are selected, and establishing their quantity and quality.

The level of variability among the participating teachers regarding organizational issues suggests a second hypothesis: In implementing portfolio assessment within their classroom, teachers fall along a three or four stage continuum. Novice teachers tend to loosely plan and schedule a rather unfocussed collection of best work across subjects. Storage, access and maintenance are controlled mainly by the teacher. Entries are collected and assessed separately. At the next stage of the continuum, the planned collection over time still remains largely under the responsibility of the teacher but now contains evidence related to the development of a few more or less specified skills or competencies. Students have input in deciding portfolio format, access and storage, and their reflections on and determination of their level of competency are based primarily on the comparison of first drafts to final products within individual assignments. In the final stages, portfolio assessment empowers students to select a minimum number of entries from a variety of contexts in order to provide evidence of the development of all five learning dimensions associated with one or a few clearly articulated competencies. Students regularly reflect on and judge their progress using structured prompts and rubrics that encourage the examination of links and relationships among the portfolio contents.

The data from this study also indicate that particular location and movement of the teachers on the implementation continuum may be a function of variables such as willingness to empower students, previous portfolio experience, school board expectations, training, support and guidance, grade level, and discipline being taught. These factors may be grouped under Myerson’s (1997) three generic categories of factors said to affect the implementation process of change within the classroom: teacher uniqueness, professional development, and teaching environment. They also relate to three of Stiggins, & Conklin’s (1992) eight assessment environment dimensions: teacher characteristics, teacher perception of students, and policy issues. Whereas the portfolio assessment item selection framework offers specific parameters around assessment purpose, focus, nature, and context, its successful implementation may depend particularly on the extent to which teachers a) accept that portfolio assessment integrates learning and assessment activities, b) obtain training and coaching specifically related to the framework, c) recognize that students are capable and responsible decision-makers with vested interest in self-assessing their own learning, d) learn to better manage the quarter of their professional time they tend to spend on assessment (Stiggins et al., 1992) by planning fewer but complete, sophisticated, and meaningful assessments of competencies involving their students throughout the assessment process, and e) contribute to the development of assessment policies at the school level that facilitate cooperation among teachers, particularly at the high school level. The third set of research hypotheses could focus on the exact nature of the relationships between each of these variables and portfolio assessment implementation in the classroom.

1. The research reported in this paper was partially supported by a transfer grant from the Ministry of Education and Training, Ontario, Canada to The Ontario Institute for Studies in Education.

2. Details regarding the initial validation study of the framework are reported in Simon, M., & Forgette-Giroux, R. (2000). Impact of a content selection framework on portfolio assessment at the classroom level. Assessment in Education, 7(1), 83-101.

Calkins, A. (1992). Juneau Portfolio Stories. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.

Fingeret, H.A. (1993). It Belongs to Me. A Guide to Portfolio Assessment in Adult Education Programs. (ERIC Document Reproduction Service No. ED 359 352)

Glazer, S. M. (1994). User-friendly portfolios: The Search Goes On. Teaching K-8, 25(3), 105-106.

Lescher, M. L. (1998). Portfolio Assessment: Aspects of Implementation and Perspective. An Investigation into the Implementation of Portfolio Assessment in Literacy. Unpublished Doctoral Dissertation, Boston College.

Sawyer, M. H. (1994). Professional Development and Educational Reform: A Study of Changes in Teachers and Classrooms During Literature Portfolio Implementation. Unpublished doctoral thesis, State University of New York at Albany.

Simon, M., & Forgette-Giroux, R. (2000). Impact of a content selection framework on portfolio assessment at the classroom level. Assessment in Education, 7(1), 83-101.

Smit, D., Kolonosky, P, & Seltzer, K. (1991). Implementing a portfolio system. In P. Belanoff, & M. Dickson (Eds.), Portfolios: Process and Product (pp. 46-56). Portsmouth, NH: Boynton/Cook Publishers.

Stiggins, R. J., & Conklin, N. F. (1992). In Teachers’s Hands. Albany: State University of New York Press.