[ Proceedings Contents 1996 ] [ Abstracts 1996 ] [ WAIER Home ]

Portfolio assessment in higher education

Trudi Cooper
Edith Cowan University

Background

It is widely accepted that the assessment procedures used in any course influence both how lecturers structure their teaching, and how students approach their learning. Staff and students of the youth work course had expressed some concerns about assessment procedures for practicum which were based on set criteria for each placement and which relied almost exclusively agency supervisors' recommendations.

The staff of the youth work programme made the decision to investigate the possibility of using a different approach to the assessment of the student practicum for a number of reasons:

These concerns emerged over a period of two or three years. The opportunity to change assessment procedures occurred when a number of other factors came together. Firstly, University policy demanded the restructuring all courses to get rid of non HECS bearing 'stand alone' practicum units. Secondly we were preparing for a program review, both of these events meant that we would be examining and changing the course structure. The decision to try portfolio based practicum assessment arose form my involvement in a research project on competency based training. As part of this project I had examined the use of portfolio assessment methods used in youth worker training in the UK.

Main features of portfolio assessment

The portfolio based assessment schemes were used in the UK for the professional training courses for part time youth workers. These schemes made use of other innovative features such as personalised training design, the use mentoring, on the job training, off the job training, personal reflection and peer learning. The starting point for this training process was to assist the student in identifying the knowledge, skills and experience which they brought with them. The next stage was the development of strategies for extending their existing skills and knowledge and for developing new areas of skill and understanding.

The UK schemes used a defined curriculum, usually written in competency terms with performance indicators. The curriculum defined core skills which were necessary in all settings. This could be supplemented by additional competencies specific to particular settings. The student had a personal training adviser with whom they negotiate both the order in which they developed their core skills and their selection of additional skills. The selection of additional skills was intended to take into consideration the student's current employment, their pre existing interests, skills and abilities, and their career aspirations. The key person was the personal training adviser who provided the support and supervision for the student, and in some cases provided witness testimony of their skill acquisition. The personal training adviser was the student's mentor, but was not generally responsible for the overall assessment of the student.

Benefits and liabilities of portfolio assessment

A preliminary literature search and discussion with people already using portfolio assessment indicated that there are a number of benefits and liabilities associated with the use of portfolio assessment. The four major 'stakeholders' in the professional preparation of students, namely the students, the professional field, the university teachers and the institution may expect different benefits and liabilities. Each group will be considered separately below.

Benefits and liabilities for students

The potential benefits for students include that

Potential benefits or liabilities, depending upon the outlook of the student include that

Benefits and liabilities for agency supervisors

The benefits for the field include that Potential liabilities of the system include that

Benefits and liabilities for teaching staff

For teaching staff some benefits include that

Liabilities for teaching staff include that

Benefits and liabilities for the institution

Suggested institutional benefits include that Potential liabilities of portfolio based assessment include

Implementation: Compromises and adaptations

Our plans for implementing portfolio assessment of the youth work placements were constrained by a number of factors. These constraints shaped the way in which we decided to implement portfolio assessment. We realised that the kind of individual program design and regular personal discussions of progress which was used in the part time youth worker training programmes was not possible within the University context because of constraints on staff time and student numbers. It was also probably less necessary because students had more regular contact with staff and their peers. We decided to make more use of peer support and less use of individual supervision. Thus individual supervision takes place once per semester, whilst during the first year placement there are opportunities for peer support meetings every week. Issues arising from these groups are then discussed with tutors. We decided that we would attempt a staged implementation of the portfolio assessment procedure. The new intake in 1996 would be portfolio assessed for all their placements, but for existing students their assessment would be mixed.

Initial suggestions for practicum arrangements, the proposed pattern of supporting units, the proposed structure of the course, including which units the practicum was attached to, and the way the curriculum was translated into units were modified because of institutional constraints.

Restructuring of the placement

Under the revised arrangements, each semester students are given a set of core skills relevant to the level of their placement. (See Appendix 1: Extract from Student Placement Guide.) These can be supplemented by additional skills of the students choice. The units to which the placements are attached, focus around the development of core skills on which the students will be tested when on placement. Core skills may also be linked to other academic units. Additional skills are those which the student chooses because of their interests or because of particular opportunities which are available to them in the placement. Assessment for the portfolio is on a pass/fail basis. Student must made reasonable process in placement in order to pass, but this does not require that they necessarily provide satisfactory evidence in all of the core skill areas. Any core skills on which they do not produce adequate evidence will be carried forward to future placements. By the end of their last placement, they must have demonstrated all of the core skills for all practicum units in order to pass the final unit and graduate from the course. All students are required to pass the placement component in order to pass the unit to which the practicum is attached. The grade which the student receives for the unit is based on the aggregate of their marks in tw o academic assignments, which may be structured to link the placement experience with academic study.

The standard placement pattern for first year students is that students undertake the first five weeks of the course at University and then do a two week block placement. After the two week block placement, they then attend the agency for half a day per week for the next eight weeks concurrently with their academic study. Students are required to work out placement objectives, strategies and performance indicators with respect to both core skills and additional skills before they start their placements. The performance indicators are intended to give students an indication of what evidence they need to collect.

Students are expected to record different kinds of information during their placement to assist their learning and to help make links with theory units. During their first placement, students are asked to keep a reflective diary, an appointments diary and to begin to collect information for a resource file. The appointments diary and the resource file are 'public property' and should be made available to their supervisors on request. The reflective learning diary is their own and not available for inspection by their supervisors. It is intended that they make use of the reflective diary as a source of data for their assignments and as a basis of discussion in their peer groups. However the prime function of the diary is as a learning tool to help the student make sense of their learning experience.

Agency supervisors are asked to:

Staff prepare students in the classroom setting and routinely visit each student once each semester during their placement. They spend about an hour at the agency, normally half that time is spent with the student and the supervisor, and half with the student alone. If there are difficulties then more frequent visits will be made.

Research questions

The research questions which we will be trying to answer revolve around an attempt to ascertain whether the ways in which we have implemented portfolio assessment has delivered the benefits expected and whether we have avoided the liabilities which were anticipated. We also wanted to find out whether there were any other unforseen benefits and liabilities. To discover this, interviews have been conducted with the student group, and are being conducted with the staff group and with the agency supervisors. We chose to survey the first year group at the end of their first semester, as they were the only group for whom the portfolio assessment system had been fully implemented.

Research methodology and sample

We engaged an independent researcher who is administering all surveys. The student sample consisted of all first year students enrolled in the first placement unit, Introduction to Youth Work Practice. This was a written structured questionnaire, administered at the end of semester one. Follow up telephone interviews were conducted with a purposive subsample of six students, three of whom were selected randomly from the group who were generally happy with the process, one from the group who were neutral, and two from the group who were initial dissatisfied. These proportions represented the proportion of students who fell into each category. The purpose of these interviews was to get greater clarification on their perceptions of portfolio assessment.

Written structured questionnaire have been sent to all agency supervisors and these are currently being processed. Follow up telephone interviews will be undertaken with a purposive subsample from this group. The questionnaire to staff involved in assessing the portfolios has yet to be analysed and the follow up discussion with staff on the findings from placement supervisors and students will take place when the results are available.

Preliminary findings

Students

There was a 73% response rate for the first student questionnaire, which was administered at the end of the first semester. The questionnaire investigated the general question of how well students felt that the unit had prepared them for their placement experience and then specifically, how they felt about portfolio assessment. The responses showed that when students were asked to indicate whether they felt that overall, they had been adequately prepared for the placement, 54% responded positively, 32% gave a neutral response and 14% replied negatively. When asked a similar question about portfolios 48% responded positively, 28% responded neutrally and 24% responded negatively.

The follow up survey took place about six weeks after the initial survey and after the students had received feedback on their portfolios. This survey indicated that all six of the students still felt that the placement a useful learning experience, however all except one had some reservations about the preparation process. Two of the students felt that they needed more time in class to discuss the placement and the portfolio use; one felt there were too many objectives (each student should have had six objectives relating to core skills); one student stated that they felt that the objectives were not clearly defined before going on placement; one student felt that they needed more support from the Youth Work staff.

The findings from agency supervisors and from youth work staff have not yet been processed. However, initial staff discussions have raised the following points.

Discussions of preliminary findings

The students go on placement after only five weeks on the course and are expected to become actively involved with the work of the agency. It has been a major challenge in the first placement unit to prepare the students sufficiently in this time to enable them to survive the experience and benefit from it. This is compounded by the fact that increasing numbers of the student group are school leavers without previous experience in youth work. The most positive aspect of the findings was that for almost all students the placement was perceived as a positive learning experience. Student difficulty with objective setting was expected, as we feel that this is a skill which needs to be introduced early in the course, but one that for some students may take time to develop. The first assignment for this unit focussed on the development of placement objectives, strategies and performance indicators and indicated that there was great diversity in the extent to which these skills had been absorbed by students. The survey indicates that this is something which we could try to follow up more effectively.

It is interesting to note that despite the fact that half the students in the follow up survey were feeling that they were not well enough prepared in terms of identifying their learning objectives the survey found that all the students felt that the objectives were relevant to the placement. This finding is consistent with the experience of staff that, especially for the first placement, some students will never feel well enough prepared before they begin their placement. It is also interesting to note that of the students interviewed more fully, all considered that their portfolio would be either useful or very useful to them in seeking future employment and as a record of their personal learning. None of the students mentioned that they felt that producing a portfolio had been an unduly time consuming exercise,(it replaced an academic assignment). Most felt that the portfolio reflected their learning, (67%) though most also felt that some aspects of their learning were not reflected in the portfolio. (67%).

The findings of the follow up survey and the assessment of the portfolios indicate that it might be useful to spend more time in class in discussion of portfolio building, on the differences between resource files and portfolios and on how to write commentaries on the evidence presented in the portfolios.

Conclusions

At this stage it is too soon to draw firm conclusions about the use of portfolio assessment, however it seems that so far, we have avoided most of the potential disadvantages and seem to have realised some of its potential advantages. The survey results will lead to changes being made to the detail of how portfolio building skills are developed in the classroom setting. The major effect of the change process has been that it provided the impetus to redesign youth work course structure in its entirety and this has led to close attention being given to the process of developing professional skills within the degree program.



Appendix 1: Extract from Student Placement Guide

The initial structure of placements

Youth Work units studied Practice assessment: Core skills Practice assessment: Additional skills
Semester 1

1.Intro to YW

2. Ideology & YW Practice

3. Interpersonal skills

Self awareness

Applied interpersonal skills

Information gathering skills.

Team work: Attendance at a youth camp or residential course as part of the staff team.
Semester 2

1. Helping skills in YW

2. Principles & Practices of YW

3. Intro to Social Analysis

Applied helping skills

Applied interpersonal skills

Information gathering skills

Knowledge of agencies in youth field who provide counselling and individual support to young people.

Team work: Attendance at a youth camp or residential course as part of the staff team.
Semester 3

1. Health & YW

2. Groups in YW

3. Youth Theory

Risk management plans

Health & safety Audit

Self management

Designing appropriate programs

Working with groups in a structured setting

Working with groups in an unstructured setting

Working with groups or individual young people

Group work skills

First Aid certificate

Intervention skills: preventing crises and responding to crises.

Team work: Attendance at a youth camp or residential course as part of the staff team.

Semester 4

1. YW & the Law

2. Youth in Community settings

3. Option: Race & ethnicity or Gender & Sexuality

Professional judgement

Handling difficult situations

Intervention skills: preventing crises and responding to crises.

Group work skills

Working effectively with community members, working with special populations

Understanding different communities: profile

Project planning and evaluation

Team work: Attendance at a youth camp or residential course as part of the staff team.

Semester 5

1. Admin & Management in YW

2. Research in Communities

Skills in administration and management

At least one skill area from the optional list

Applied research project

Working effectively with community members

Working with special populations

Understanding different communities: profile

Project planning and evaluation

Semester 6

1. Ethics in YW

2. Option: Social change or Policy

Skills in being able to explain and justify the philosophical basis of their work and demonstrate how this can be implemented in practice

At least one skill area from the optional list

Strategic planning

Policy development

Social change project.

Applied research project.

Changing levels of responsibility

As you progress through the course you will gradually be expected to take on increasing levels of responsibility.

In your first year placements you will be expected to assist an experienced youth worker in working directly with young people. You should not normally be left by yourself with a group of young people or be expected to take responsibility for locking or unlocking the premises.

In the second year you will be expected to be able to take responsibility for planning and implementing programmes with groups of young people. In this case you may sometimes work by yourself with individuals or groups, but a more experienced youth worker should be available to support you at your request and to help you to develop your skills, by working with you for at least part of the time.

In the final year of the course you will be moving towards being able to take on all aspects of the role of a youth worker. In your third year placements it is quite likely that you will agree tasks with your supervisor and then be expected to organise your own plan of work and to complete this without daily supervision.

Author: Trudi Cooper, Edith Cowan University, t.cooper@cowan.edu.au

Supported by the Edith Cowan Innovations in Teaching and Learning Project

Please cite as: Cooper, T. (1996). Portfolio assessment in higher education. Proceedings Western Australian Institute for Educational Research Forum 1996. http://www.waier.org.au/forums/1996/cooper.html


[ Proceedings Contents 1996 ] [ Abstracts 1996 ] [ WAIER Home ]
Last revision: 2 May 2006. This URL: http://www.waier.org.au/forums/1996/cooper.html
Previous URL 30 July 2001 to 16 May 2006: http://education.curtin.edu.au/waier/forums/1996/cooper.html
Previous URL from 16 Nov 1999 to 30 July 2001: http://cleo.murdoch.edu.au/waier/forums/1996/cooper.html
Original created 1996 by M. Wild at liswww.fste.ac.cowan.edu.au/wild/waier/
HTML: Roger Atkinson [rjatkinson@bigpond.com] and Clare McBeath [c.mcbeath@curtin.edu.au>]