[ Proceedings Contents 1996 ] [ Abstracts 1996 ] [ WAIER Home ] Portfolio assessment in higher education Trudi Cooper Edith Cowan University

Background

The staff of the youth work programme made the decision to investigate the possibility of using a different approach to the assessment of the student practicum for a number of reasons:

• There was a feeling among some of the staff that whilst the academic components of the degree course had been strengthened, the placement component of the degree program had become increasingly marginalised. This had occurred gradually, and had been hastened by the abolition of the Associate Diploma qualification. After a number of course restructures, there remained little effective link between most units and content of the practical placements.

• Because of the original concept of process based practicum units, the units appeared to have less defined content, and units had been used for other ancillary purposes which did not relate to placement tasks or performance, but related in a generic way to academic performance, for example a study skills module in the first placement, and the use of practicum units for student consultation meetings with their representatives.

• Some students were expressing the view that the course work components of the practicum units were less important than the academic units as sometimes they appeared to have no immediate relevance to placement tasks and because they had no points of assessment other than performance on placements.

• There was concern among some of the staff about placement assessment procedures which were based on agency supervisors recommendations in relation to a set of fairly broad statements about expectation. These had been developed broadly to accommodate the wide variation in agency context. However, they were unsatisfactory firstly because the imprecision about performance levels and standards gave little indication to students about what was expected of them, secondly because the assessment criteria gave little indication to agencies about how they should assess students and thirdly this structure made it very difficult to moderate assessment standards between students and between agencies. Thus poor a student might pass in one agency whilst an average student might fail at an agency where the agency supervisor's interpretation of standards was higher.

• There was no satisfactory way to handle a situation where a student, through no fault of their own did not have opportunities to demonstrate particular skills. In practice, this could either lead to a fail through lack of evidence or a pass through lack of sufficient evidence, at the discretion of the agency supervisor. Both of these outcomes are unsatisfactory.

Main features of portfolio assessment

The UK schemes used a defined curriculum, usually written in competency terms with performance indicators. The curriculum defined core skills which were necessary in all settings. This could be supplemented by additional competencies specific to particular settings. The student had a personal training adviser with whom they negotiate both the order in which they developed their core skills and their selection of additional skills. The selection of additional skills was intended to take into consideration the student's current employment, their pre existing interests, skills and abilities, and their career aspirations. The key person was the personal training adviser who provided the support and supervision for the student, and in some cases provided witness testimony of their skill acquisition. The personal training adviser was the student's mentor, but was not generally responsible for the overall assessment of the student.

Benefits and liabilities of portfolio assessment

Benefits and liabilities for students

• the portfolio approach ensures that practical experience is student focused and hence individually relevant.
• the student receives clear expectations about what is required and clear feedback about their progress and performance.
• by the end of the course, the student has tangible evidence to take to interviews with prospective employers.
• If a student does not perform adequately in a particular skill area, they may carry forward that core skill to a future placement without failing their placement, providing they make reasonable progress in other respects.
• if a student is in a setting where they do not have the opportunity to demonstrate their skill, they may carry forward that core skill to a future placement without failing their placement, providing they make reasonable progress in other respects.

Potential benefits or liabilities, depending upon the outlook of the student include that

• the student must take extra responsibil ity for ensuring that they have learning opportunities and for ensuring that they collect relevant evidence about their skills.
• there is likely to be some unconscious advantage to those with good presentation skills.
• Students may have to do more work to produce evidence of their skills.

Benefits and liabilities for agency supervisors

• supervisors will be given clear guidance, in the form of the learning contract, which indicates what the student is hoping to learn.
• Performance indicators should clarify expectations of students and make it easier to determine assessment.
• When appointing new entrants to their first position, selection panels will be able to ask applicants for evidence of their skills and experience during training. Graduates will be able to produce 'testimonials' signed by a range of colleagues, who have observed their practice over a period of time and will not be dependent only on the opinions of university staff.

• field supervisors will be asked to produce evidence of particular skills rather than maybe a more general report on the student's progress.
• The field supervisor will have to be clear with the student before the start of the placement about what learning opportunities they can actually offer.

Benefits and liabilities for teaching staff

• portfolio based assessment of practicum offers is a clear alternative pedagogy from which to justify alternative teaching methods.
• the requirement that the student set specific learning objectives and performance indicators, should result in less dissent about assessment and any appeals should be easier to resolve.
• It is possible that the assessment of students may become a more interesting process as students can be more individual and imaginative in the means they use to demonstrate their skills.
• The assessment process should become one of dialogue, in which each stage of assessment forms part of the process of setting the agenda for future learning.

Liabilities for teaching staff include that

• adopting a portfolio assessment system will require a restructuring of practicum preparation units
• The practicum curriculum and syllabus will require revision,
• the patterns of placements and the methods of assigning students to agencies may need changing
• the teaching methods used in practicum preparation sessions may alter.
• There are administrative complications arising from allowing students more flexibility in the length of time they take to complete the practical components of the course, and in ensuring that students have successfully completed all core skills before graduation.
• Teaching staff will also need to specify clearly what range of core skills need to be demonstrated in each practicum. In some professional areas it may be appropriate to identify both core skills, which it is necessary that all students demonstrate and additional skills which may be specific to the particular settings or future career pathways.

Benefits and liabilities for the institution

• research with part time youth worker training courses indicated that there was some evidence (Jackson, M, personal communication) of reduced attrition rates (for people leaving without completing the course), when portfolio assessment was used as compared with previous assessment methods. Whether this will materialise in courses which combine traditional academic courses with portfolio assessed practicums will be interesting to see.
• In a climate of falling student numbers, another possible benefit of a course using portfolio assessment is that students are offered an alternative means of demonstrating their abilities. This may prove attractive to some students who might not otherwise choose that particular institution.
• different method of assessment and a different pedagogic approach, will stimulate debate about appropriate pedagogy for undergraduate teaching.

• concerns that this method of training will be too expensive to implement successfully within current budgets. The same incomplete study by Jackson mentioned above, indicated that within the context of part time youth worker training the portfolio based training was no more expensive than the course based training which it replaced. Whether this would apply to degree courses is unclear.

Implementation: Compromises and adaptations

• We had to work within the constraints of a given budget
• We had to adhere to the institutional guidelines for the structure of degree courses within the faculty.
• We had to devise a scheme that would not place long term extra burdens on academic staff;
• The scheme had to be manageable for the placement coordinator both in terms of being able to place students and in terms of record keeping on student progress.
• The scheme should not disadvantage existing students.

Initial suggestions for practicum arrangements, the proposed pattern of supporting units, the proposed structure of the course, including which units the practicum was attached to, and the way the curriculum was translated into units were modified because of institutional constraints.

Restructuring of the placement

The standard placement pattern for first year students is that students undertake the first five weeks of the course at University and then do a two week block placement. After the two week block placement, they then attend the agency for half a day per week for the next eight weeks concurrently with their academic study. Students are required to work out placement objectives, strategies and performance indicators with respect to both core skills and additional skills before they start their placements. The performance indicators are intended to give students an indication of what evidence they need to collect.

Students are expected to record different kinds of information during their placement to assist their learning and to help make links with theory units. During their first placement, students are asked to keep a reflective diary, an appointments diary and to begin to collect information for a resource file. The appointments diary and the resource file are 'public property' and should be made available to their supervisors on request. The reflective learning diary is their own and not available for inspection by their supervisors. It is intended that they make use of the reflective diary as a source of data for their assignments and as a basis of discussion in their peer groups. However the prime function of the diary is as a learning tool to help the student make sense of their learning experience.

Agency supervisors are asked to:

• Provide confirm the attendance of the student
• To provide comment on their professional conduct,
• To provide suggestions to students on future areas of learning and on their progress whilst with the agency.

Staff prepare students in the classroom setting and routinely visit each student once each semester during their placement. They spend about an hour at the agency, normally half that time is spent with the student and the supervisor, and half with the student alone. If there are difficulties then more frequent visits will be made.

Research questions

Research methodology and sample

Written structured questionnaire have been sent to all agency supervisors and these are currently being processed. Follow up telephone interviews will be undertaken with a purposive subsample from this group. The questionnaire to staff involved in assessing the portfolios has yet to be analysed and the follow up discussion with staff on the findings from placement supervisors and students will take place when the results are available.

Preliminary findings

Students

The follow up survey took place about six weeks after the initial survey and after the students had received feedback on their portfolios. This survey indicated that all six of the students still felt that the placement a useful learning experience, however all except one had some reservations about the preparation process. Two of the students felt that they needed more time in class to discuss the placement and the portfolio use; one felt there were too many objectives (each student should have had six objectives relating to core skills); one student stated that they felt that the objectives were not clearly defined before going on placement; one student felt that they needed more support from the Youth Work staff.

The findings from agency supervisors and from youth work staff have not yet been processed. However, initial staff discussions have raised the following points.

• The process of designing the portfolio assessment resulted in more attention being focused on the practicum curriculum and on our expectations of students at each stage in their learning process.
• That additional record keeping processes have had to be developed to keep track of student progress
• That minor modifications will be made to placement arrangement to reduce the work load of the placement coordinator and to improve the attractiveness of placement arrangements to youth work agencies.
• The portfolios were quite difficult to mark, especially as the commentaries on learning were generally not done very well. Some students appeared to have submitted only a resource file. Most appeared to have put a substantial amount of work into their portfolio.
• There is a need to focus more on encouraging students to produce a commentary on their portfolio.
• There needs to be a change to some of the pre-requisites for the practicum units
• the standard course pattern of units for the mid year intake students has been modified
• There needs to be further development of the practicum curriculum and learning strategies.

Discussions of preliminary findings

It is interesting to note that despite the fact that half the students in the follow up survey were feeling that they were not well enough prepared in terms of identifying their learning objectives the survey found that all the students felt that the objectives were relevant to the placement. This finding is consistent with the experience of staff that, especially for the first placement, some students will never feel well enough prepared before they begin their placement. It is also interesting to note that of the students interviewed more fully, all considered that their portfolio would be either useful or very useful to them in seeking future employment and as a record of their personal learning. None of the students mentioned that they felt that producing a portfolio had been an unduly time consuming exercise,(it replaced an academic assignment). Most felt that the portfolio reflected their learning, (67%) though most also felt that some aspects of their learning were not reflected in the portfolio. (67%).

The findings of the follow up survey and the assessment of the portfolios indicate that it might be useful to spend more time in class in discussion of portfolio building, on the differences between resource files and portfolios and on how to write commentaries on the evidence presented in the portfolios.

Conclusions

Appendix 1: Extract from Student Placement Guide

Youth Work units studied	Practice assessment: Core skills	Practice assessment: Additional skills
Semester 1 1.Intro to YW 2. Ideology & YW Practice 3. Interpersonal skills	Self awareness Applied interpersonal skills Information gathering skills.	Team work: Attendance at a youth camp or residential course as part of the staff team.
Semester 2 1. Helping skills in YW 2. Principles & Practices of YW 3. Intro to Social Analysis	Applied helping skills Applied interpersonal skills Information gathering skills Knowledge of agencies in youth field who provide counselling and individual support to young people.	Team work: Attendance at a youth camp or residential course as part of the staff team.
Semester 3 1. Health & YW 2. Groups in YW 3. Youth Theory	Risk management plans Health & safety Audit Self management Designing appropriate programs Working with groups in a structured setting Working with groups in an unstructured setting Working with groups or individual young people	Group work skills First Aid certificate Intervention skills: preventing crises and responding to crises. Team work: Attendance at a youth camp or residential course as part of the staff team.
Semester 4 1. YW & the Law 2. Youth in Community settings 3. Option: Race & ethnicity or Gender & Sexuality	Professional judgement Handling difficult situations Intervention skills: preventing crises and responding to crises. Group work skills	Working effectively with community members, working with special populations Understanding different communities: profile Project planning and evaluation Team work: Attendance at a youth camp or residential course as part of the staff team.
Semester 5 1. Admin & Management in YW 2. Research in Communities	Skills in administration and management At least one skill area from the optional list	Applied research project Working effectively with community members Working with special populations Understanding different communities: profile Project planning and evaluation
Semester 6 1. Ethics in YW 2. Option: Social change or Policy	Skills in being able to explain and justify the philosophical basis of their work and demonstrate how this can be implemented in practice At least one skill area from the optional list	Strategic planning Policy development Social change project. Applied research project.

Changing levels of responsibility

As you progress through the course you will gradually be expected to take on increasing levels of responsibility.

In your first year placements you will be expected to assist an experienced youth worker in working directly with young people. You should not normally be left by yourself with a group of young people or be expected to take responsibility for locking or unlocking the premises.

In the second year you will be expected to be able to take responsibility for planning and implementing programmes with groups of young people. In this case you may sometimes work by yourself with individuals or groups, but a more experienced youth worker should be available to support you at your request and to help you to develop your skills, by working with you for at least part of the time.

In the final year of the course you will be moving towards being able to take on all aspects of the role of a youth worker. In your third year placements it is quite likely that you will agree tasks with your supervisor and then be expected to organise your own plan of work and to complete this without daily supervision.

Author: Trudi Cooper, Edith Cowan University, t.cooper@cowan.edu.au Supported by the Edith Cowan Innovations in Teaching and Learning Project Please cite as: Cooper, T. (1996). Portfolio assessment in higher education. Proceedings Western Australian Institute for Educational Research Forum 1996. http://www.waier.org.au/forums/1996/cooper.html