|[ Proceedings Contents ] [ Forum 1998 Program ] [ WAIER Home Page ]
Holistic or discrete? A competency based assessment issue in the Certificate of General Education for Adults Reading and Writing StreamJulienne van Loon
School of Communication and Cultural Studies Curtin University of Technology
In 1997, the Admissions Studies section at Hedland College implemented the Certificates of General Education for Adults (C.G.E.A.), in order to replace two locally developed certificate level bridging programs which were not widely recognised. These original bridging programs were based on conventionally taught classes and norm referenced assessment practices. It was generally felt however, that these programs were not meeting the needs of the students, and that the competency based nature of the C.G.E.A. program would be more advantageous for the Aboriginal adults who are the primary clientele for the Admission Studies section. C.G.E.A., because it is competency based, allows a student centred approach to learning which is self paced. It also permits the students to obtain a qualification or clear statement of attainment which is both portable and widely recognised at other TAFE colleges.
In fact, the implementation of the C.G.E.A. has been wonderfully advantageous for the students, but has not been entirely free of problems for the lecturing staff. Many of these implementation problems are to do with assessment. This essay focuses on the Reading and Writing Stream at Level Three, particularly on the assessment of a particular competency to do with the mechanics of writing. According to the curriculum documents, in order to pass any writing assessment task at C.G.E.A. Level Three, a student must be able to "spell, punctuate and use grammar sufficiently for the meaning of the text to be understood" (Adult Community and Further Education Board, Victoria, 1996). This is a simple performance statement, but a difficult one to asess, particularly in light of the complexities of English language usage.
This essay examines this particular 'mechanics of writing' competency, before a backdrop of broader contemporary arguments about the nature of competency based assessment, and assessment generally. It encompasses debates to do with the nature of competency itself, the general education versus vocational training dichotomy, the nature of cultural bias in assessment, and issues of validity and reliability in the case of a student centred approach to general education.
The point at which a lecturer decides whether a student is either able or unable to use the mechanics of writing to a level where the text can be understood, depends not only upon how the phrase 'to be understood' is interpreted, but also upon what the lecturer's definition of competency is. Unfortunately, as Thomson (1992) points out, definitions of competency abound. On the simplest level, Blake and Hanley (1995) posit that "competency can be understood as a capacity or capability in the learner" (p.35). This definition doesn't really enlighten us a great deal, particularly when Blake and Hanley (1995) go on to concede that "the term may defined differently according to context" (p. 35). The Penguin Dictionary of Australian Education (1993) is slightly more forthcoming in its definition of competency based vocational education as "a program emphasising the acquisition and performance of well-defined skills associated with a particular job, trade or profession" (p.79). This definition of competency based programs does highlight two key problems with deciding on a point or level at which competency may be reached in relation to the C.G.E.A.. On the one hand the Penguin Dictionary of Australian Education (1993) definition highlights the fact that a given competency or performance statement needs to be 'well-defined'. The C.G.E.A. competency mentioned earlier is not well defined. A literacy lecturer, for example, may be very good at understanding poorly written texts, so the degree to which a text 'can be understood' is an immediate variable, compelling the question 'understood by whom?'
Secondly, the Penguin Dictionary of Australian Education (1993) definition brings into light the way in which degrees of competency are defined to a great extent by their raison d'etre. That is, competency is generally linked to a particular job, trade or profession. This is really a key issue in understanding why the given C.G.E.A. competency seems to lack detail. The C.G.E.A. is a general education program. So the purpose of a piece of writing for a C.G.E.A. assessment task certainly is not defined by a given profession or vocation. Thus, the degree to which the student's written text needs to be understood cannot be related to a particular, workplace specific situation. The degree of appropriateness of competency based assessment in a field where relevance to a specific workplace is impossible, in fact forms the basis of many arguments against the implementation of competency based assessment in higher education (Henry & Taylor, 1994).
The Western Australian Department of Training (W.A.D.O.T.) is certainly aware of the problematic nature of competency based assessment, and defines assessment in the competency based training context as "the process of collecting evidence and making judgements on whether competency has been achieved" (W.A.D.O.T., 1996). WADOT's discussion paper on the matter goes on to say that
Competency based assessment differs from other forms of assessment in a number of ways. Firstly, it is assessment against a specified standard (usually endorsed competency standards), not comparison with other learners. Secondly, competency based assessment allows the learner to provide a range of evidence to support their assessment process....Thirdly, competency based assessment forms a key part of the training reform agenda and links competency standards, training curriculum, and on-the-job and off-the-job training with the needs of workers and industry (W.A.D.O.T., 1996, p.3.)Again, because the C.G.E.A. is a general education program, its competency standards are not endorsed by a particular industry, thus complicating the first and third points made in this extract. The point regarding evidence then, could be the most important aspect for the C.G.E.A. lecturer to base assessment decisions on. Perhaps then, lecturers simply need to develop a detailed spelling, punctuation and grammar checklist for the C.G.E.A. competency in question, ticking off various pieces of evidence in a text, and moderating the checklist with other lecturers. This perhaps seems adequate, until we take into account the difficulties and complications involved in developing and implementing checklists.
Thomson (1992) reminds us that competency based standards are in fact performance based standards, and that simply describing a performance as competent does not tell us enough. In other words, competent performance must be accompanied by a description of expectations. Thus, a che cklist is formed. One might specify, for example, that in a two hundred to four hundred word piece of writing, there must be no more than six spelling mistakes, no more than six grammar errors, and no more than six punctuation problems. Here is where the complexities of English language usage begin to complicate assessment. A spelling mistake alone, for example, might not have all that much bearing on whether a text can be understood. If competency were to be spelt 'kombietentsie', it would have a great impact on a reader's ability to understand what the writer is trying to express. But if a student was to spell competency as 'compitency', the meaning would still be apparent, and in fact, a Level Three C.G.E.A. student who used this particular word in the correct context, should be applauded for having such broad vocabulary, not failed because it was the sixth difficult word the student had spelt slightly incorrectly. Detailed checklists, by their very nature, can cause an assessor to split an interminable number of hairs, perhaps preventing a pass mark from being issued for a task which just seems, for any number of reasons not specified on the checklist, to be a great success.
Hager (1994) concedes that higher education opponents of competency standards in Australia have virtually all agreed that narrow conceptions of competence, where particular discrete lists are enforced, are undesirable. This is not a particularly new observation. Harvey (1975) cites Ebel (1971), who quite early on in the norm versus criterion referenced debate, saw obvious limitations of criterion referenced measures. He argued that good criteria are extremely difficult to obtain and in general, they require a degree of detail in the specification of objectives or outcomes that are unrealistic and impractical to use. "Consequently," he says, "one must settle for statements of general objectives, unencumbered with the details of what is to be taught, or how to teach it." (p. 36). Ironically, in the general education based C.G.E.A., this lack of detail about what is to be taught, is one of the programs greatest features, allowing the content to be adapted to different groups and different contexts. Yet, in assessment terms, this same flexibility creates a degree of difficulty, particularly in relation to issues of reliability.
Bloch and Thomson (1994), along with Hager (1994), urge that checklists are necessary and can be practical if used in a more integrated way:
All those concerned with the future of competency based education, training and assessment in this country need to work towards finding a balance between over-beuraucratising the assessment process and creating a system that is so 'loose' that it renders absurd the notion of national standards and portable competencies (Bloch and Thomson, 1994, p23).Similarly, Hager (1994) points out if competence consists of a series of observable behaviours, it does not follow that this same series of observable behaviours will be performed by any person competent at a given skill. Obviously, not everyone who is competent in a given field will approach a task in precisely the same way as everyone else. Thus, there is an argument for a holistic approach to competency based assessment, an approach that looks at the process and the product together, one which encompasses not only the list of observable skills, but also some elements around and outside those skills.
Hager (1994) describes the holistic approach to competency based assessment in terms of chemistry, explaining that whilst "atoms are discrete and independent units, they nevertheless combine to form molecules which have quite different properties from those of their constituent atoms" (p.10). Thomson (1992) says this too, when he reminds us that "the whole of a competence is greater than the sum of its parts" (p.38). Discussions about aspects of competency which lay outside the bounds of discrete lists, also put forward the concept of recognising 'non-routine skills' (Thomson, 1992). The identification of non-routine skills make allowances for a student's ability to deal with the unexpected, or demonstrate interpersonal skills. Such skills are often difficult to describe. Consideration of these aspects moves a little closer to a practical level of competency. Actual practice is undoubtedly richer than sequences of isolated tasks which do not allow for any great synthesis. Nowhere is this more so than in the practice of writing.
A further issue which makes competency based assessment in a general education context so difficult, is that the purpose of a course or unit is not itself always so strictly defined as it might be in a vocational context. According to Ashcroft and Foreman-Peck (1994), assessment schemes should closely mirror the aims, objective or intentions of the course. Interestingly, the aims and objectives of the C.G.E.A. program are so broad as to encompass "skill development in reading, writing, numeracy, oral communication and generic skills to meet personal needs and facilitate participation in the community, workplace or further education and training" (Adult Community and Further Education Board, Victoria, 1996). Thus, individual students may be attending a general education course such as the C.G.E.A. for any number of reasons. One elderly women currently enrolled in the program for example, is attending to improve her writing to a level which will allow her to write her life story, a story of both personal significance to her children and political significance to the wider community. Another young student is attending specifically to use the program as a bridge which will enable her to gain entry to a Business Studies Certificate. Yet another student is attending simply to get out of the house and away from the children, and perhaps improve her general skills in the long term, to facilitate employment options in the future. All of these students speak what is commonly referred to as Aboriginal English, and this has a special implications for their writing, especially when completing writing of a personal nature, as is required by the C.G.E.A.'s Writing for Self Expression assessment tasks.
As mentioned earlier, interpreting the statement that a student needs to be able to spell, punctuate and use grammar sufficiently for the meaning to be understood, elicits the question 'understood by whom?' According to Docking (Undated), "what teachers assess, whether formally or informally, is the attainment of 'middle class concepts', using 'middle class language' - on the assumption that all children have been exposed to 'middle class experiences'" (p.199). On the one hand, the C.G.E.A. writing task assessment conditions specify that "subject matter should be everyday", and on the other hand, lecturers at Regional C.G.E.A. moderation argue over the importance of a Level Three assessment task demonstrating a particular standard of English, such as not beginning a sentence with 'and' or 'but', and having a comma placed correctly within the sentence. O'Neill and Roberts (1995), themselves experienced practitioners in the field of Aboriginal education, stress the importance of understanding that learning is not a neutral practice (p.46). The question of what becomes a 'standard' language is often more a matter of politics and relative power than it is of linguistics.
Kriol, Torres Strait Creole and Aboriginal English are neither correct nor incorrect forms of Standard Australian English, and as such are not 'inferior'. Such dialects have arisen out of various patterns of language development, and communicate as effectively as other dialects or other languages. In the genre of Writing for Self Expression, where subject matter encompasses home and identity, Aboriginal students are highly articulate in Aboriginal English, and their writing can be rich as a result. Yet, if a C.G.E.A. lecturer has a middle class background and limited knowledge of Aboriginal English, then perhaps this sort of writing would fail the performance criteria on the basis that Standard English grammar rules are broken, and t herefore the text can not be 'sufficiently understood' by that lecturer.
In contemporary literary circles, Aboriginal English is highly praised. Ruby Langford Giniby's poem The Gubberments, illustrates the effect of this style of writing: "I gib you all rations ta lubra, and kids, ya gotta work for tucker ya git; missionary come he teach us lidgin, ya gotta be good ta go ta heben"(Langford Giniby, 1994). Reflecting on the oral nature of Aboriginal English, another prominent Aboriginal writer, Lionel Fogarty, says "what you should do is read... in an Aboriginal way, take the Aboriginal side of my language, and then reflect back on the English side.... That's the only way (you) can understand all the mosaic, all the patterns of the words" (Fogarty, 1995). It is too easy for assessors, through either conscious or unconscious cultural bias, to underestimate the capabilities of Aboriginal students, through their writing. Good practice in Aboriginal education, as O'Neill and Roberts (1996) argue, respects people's cultural roots and works in ways that make students feel comfortable. Thus if the elderly C.G.E.A. student mentioned previously, who aims to write her family history, wants her own dialect recognised, the lecturer needs to take a broader view of the competency statement which calls on a student to spell, punctuate and use grammar sufficiently for the text to be understood. That is, in this context, it is not appropriate to use Standard English as a yard stick.
Viewing the competency statement this way demonstrates a holistic approach, by allowing for the definition of 'sufficiently understood' to be interpreted according to the purpose of the text, and its intended audience. It also allows for the recognition of an individual's previous learning experiences. Docking (Undated) concedes that the ability to learn something new depends upon what students already know, and that what they already know depends upon their cultural background. Recognition and some understanding, on the part of the lecturer, of those backgrounds, when assessing a competency, is important if Aboriginal students are to progress through the education system.
Language and literacy demands do vary, of course, depending on specific work or community contexts, and some standards need to be set for specific purposes. Yet because of their innate broadness, general education programs, demand something beyond a discrete, 'list of skills' approach to competency based assessment. It is interesting to consider any impact this may have reliability and validity. Whilst Harvey (1975) perhaps rightly insists that criterion must not represent the values and standards of just one teacher, and that meaningful criteria must not be idiosyncratic, he does concede that reliability does not have the same meaning in criterion referenced assessment, as it does in norm-referenced assessment. Somehow, a competency statement needs to allow for some flexibility, that is for some 'non-routine' skills, yet remain particular enough to give a lecturer an idea of a performance standard.
Validity too, is dependent on the quality and appropriateness of the performance statement, as well as how it is interpreted. Determining what constitutes 'sufficient evidence', in order not to over or under assess, is important in a general education context where performance statements can become quite general (Bloch and Thomson, 1994). In this instance, validity may involve assessing only the critical elements of a performance statement. More often, it essentially involves interpreting those critical elements in a broader context. This is a student centred approach, which is precisely what the general nature of many of the C.G.E.A. performance statements really require to be effective. It doesn't necessarily however, result in a sophisticated approach reliability and validity.
Lecturers in competency based education, particularly the expansive field of basic or general education, need to accept that the standards set for competency assessment are arbitrary, subjective decisions, just as they are for all types of assessment (Docking, Undated). An individual lecturer's definition of competency needs to be conceptualised in terms of knowledge, abilities, both routine and non-routine skills and attitudes as they are displayed within the given context. As much as it may be considered dangerous, decisions to do with competency at the ground level, do lie with the individual lecturer. The lecturer should however, be making an informed decision, and be fully aware of the innately difficult nature of 'cut off points' and of 'evidence'. Whilst the particular performance statement examined here is incredibly general, its generality needs to be considered in the context of a general education curriculum rather than a workplace specific one. General education by its very nature, demands this holistic approach to the concept of competency. Word (1987) is certainly correct in stating that switching to criterion-referencing does not make measurement problems any easier. On the contrary, at least for the lecturer, it makes them more difficult. The ability to apply a general performance statement to a variety of general education contexts, yet remain consistent for the purposes of reliability, requires a great deal of professional judgement. Essentially, educators in these non-industry specific fields need to be appropriately trained in the field of competency based training and assessment in order to meet that challenge, and to make those judgements accurately.
Adult Community and Further Education Board. (1996). Certificates in General Education for Adults. (Curriculum document.) Victoria: Adult Education Resource and Information Service.
Blake D. & Hanley, V. (1995). The Dictionary of Educational Terms. London: Arena.
Block, B. & Thomson, P. (1994). Working towards best practice in assessment. Adelaide: National Centre for Vocational Education Research.
Docking, R (Undated). Basic concerns or controversies about assessment. Murdoch University.
Fogarty, L. (1995). Musgrave Park. Scarred for life. RePublica, (3). Sydney: Angus & Robertson.
Hager, P. (1994). Is there an cogent philosophical argument against competency standards? Australian Journal of Education, 38(1), 3-18.
Harvey, T. (1975). Some thought on norm-referenced and criterion-referenced measures. Research in Education, 13, May.
Henry, M. & Taylor, S. (1994). New pathways? Postcompulsory schooling, TAFE and mass higher education. Australian Universities Review, 37(2), 29-33.
Langford Giniby, R. (1994). The Gubburments. All same as family in a big ouse. RePublica, (1). Sydney: Angus & Robertson.
O'Neill, G. & Roberts, J. (1995). Student directed learning within an Aboriginal terms of reference. National Bulletin of Good Practice in Adult and Community Education, No.3, 1995.
Penguin Dictionary of Australian Education (1993). Sydney: Penguin Books.
Thomson, P. (1992). Competency-based training: Some development and assessment issues for policy makers. Australian Journal of TAFE Research and Development, 6(2).
Western Australian Department of Training. (1996). VET assessment framework: Implementation of competency based assessment in the Western Australian vocational education and training sector. Discussion paper. August 1996.
Word, R. (1987). Observations on Criterion Referenced Assessment. Measurement and Assessment in Education and Psychology. Lewes: Falmer Press. 154-167.
|Please cite as: van Loon, J. (1998). Holistic or discrete? A competency based assessment issue in the Certificate of General Education for Adults Reading and Writing Stream. Proceedings Western Australian Institute for Educational Research Forum 1998. http://www.waier .org.au/forums/1998/loon.html|