[ Proceedings Contents ] [ Schedule ] [ Abstracts ] [ WAIER Home ]

Evaluating the effectiveness of online learning using a new web based learning instrument

Vanessa Chang
Curtin University of Technology
In today's educational context, the delivery of education via the Internet or Web based learning is not a new concept. In fact, this medium of education delivery is much sought after by students and many academics now accept and use this environment. The opportunity now exists for educators to look more closely at the effectiveness and appropriateness of a Web based learning environment. Educators need to equip themselves with instruments that allow the effectiveness of Web based learning to be evaluated.

This paper describes the development of a new Web based learning environment evaluation instrument. Apart from demographics and general background information, this instrument is divided into four main aspects. The first three are adapted from Tobin's (1998) clusters of emancipatory activities, co-participatory activities and qualia. The fourth aspect of the instrument focuses on information structure and the design of online material.


Using the Internet as a medium, web based learning can be viewed as an innovative approach for education delivery. The Web learning environment is potentially a powerful teaching and learning arena in which new practices and new relationships can make significant contributions to learning. We now have a shift in learning, unlike traditional face-to-face teaching and learning, attention needs to be paid to online teaching and learning. Designing and delivering instruction on the Web requires thoughtful analysis and investigation of how to use the Web's potential (Khan, 1997). Before we can investigate the effectiveness of web based learning environment, we need to revisit the role of teachers and students in a face-to-face learning environment and compare this to web based learning environment.

Clearly the role of the teacher either in a classroom face-to-face teaching or online setting is to ensure that some type of educational process occurs amongst the learners involved. In the traditional classroom setting, the teacher role generally imparts knowledge to learners (Relan & Gillani, 1997). In the Web environment, the role of the teacher becomes that of an educational facilitator (Sherry & Wilson, 1997). As a facilitator, the teacher provides guidance and allows students to explore the course material as well as related materials without restriction. Students in face-to-face classroom setting see and work with one another and get to know each other well through the learning process. For learning to be successfully delivered online, the learning process on the Web needs to be well facilitated.

Roles of the online teacher

> Collins and Berge (1996) categorise the tasks and roles of the online teacher into four areas: (1) pedagogical, (2) social, (3) managerial, and (4) technical. They described pedagogical as function and task that revolves around educational facilitation. Social function is described as the promotion of friendly social environment which is needed in the process of online learning. The managerial aspects of online learning involves setting agenda, objectives, rules and decision making norms. The technical aspect focuses on the teachers' proficiency with the use of the technology. The teacher in an online course must ensure that they themselves are comfortable with using this technology to transfer knowledge to students.

Roles of the online learner

A successful learner should be active in an online learning environment. The roles of the online learners include (1) knowledge generation, (2) collaboration, and (3) process management (Palloff & Pratt, 1998). In online learning environment, the learner is responsible for actively seeking solutions to problems confined within the knowledge area being studied with guidance from the teacher. They are expected to view problems and questions presented by the teacher and those of other students. Students in the online learning environment are also expected to learn collaboratively and cooperatively (Khan, 1997). Students are expected to work together in order to generate deeper levels of understanding of the course material. Students are also expected to share the resources and other materials that they are finding with other learners. In the role of process management, students are expected to participate with minimal guidelines, interact with one another and speak up when the discussion are moving into an uncomfortable zone. Students must be willing to speak out when they are offended or simply have an opinion on something (Palloff & Pratt, 1998).

Web based learning applications

Finder and Raleigh (1998) explain four ways in which Web applications could be used in a course: (1) Informational, (2) Supplemental, (3) Dependent, and (4) Fully Developed. Informational use is described where course information such as course outline and assignment descriptions are available on the Web. Supplemental use includes students use of the Web to complete part of the course and the learning materials generally includes links to related sources. Dependent use is described as that most learning materials exist on the Web and students use the Web to complete course assignments. Fully Developed use is viewed as using the Web to deliver the entire course. This is deemed as offering the course totally online where students and teacher may never meet face-to-face. The Web based learning environment instrument described in this paper focused on the four Web applications mentioned above. Tobin (1998) in his work on Connecting Communities Learning has described a framework which can be used for evaluation of learning environments in interactive environments. This new web based learning environment instrument, known as WEBLEI, is built on the work of Tobin (1998).

The Web based Learning Environment Instrument (WEBLEI)

This instrument is designed to capture students' perception of web based learning environment. Apart from demographics and background information sections, there are four core aspects in the instrument. The first three aspects are adapted from Tobin's (1998) work on Connecting Communities Learning (CCL) and the final aspect focuses on information structure and the design aspect of the web based material. Each of these aspects is explained in the following section.

WEBLEI Scale I: Emancipatory activities

Tobin (1998) listed three main categories of convenience, efficiency and autonomy for emancipatory activities. The following items are included in evaluating Emancipa tory Activities:
  1. I can access the learning activities at times convenient to me.
  2. The online material is available at locations suitable for me.
  3. I can use time saved in travelling and on campus class attendance for study and other commitments.
  4. I am allowed to work at my own pace to achieve learning objectives.
  5. I decide how much I want to learn in a given period.
  6. I decide when I want to learn.

WEBLEI Scale II: Co-participatory activities

According to Tobin (1998), "co-participation implies the presence of a shared language which can be accessed by all participants to engage the activities of the community, with a goal of facilitating learning". Included under the co-participatory activities are six categories of flexibility, reflection, quality, interaction, feedback and collaboration. The focus of this aspect is on the learning activities that the students will participate in. This includes structuring of activities in which students use their existing knowledge to apply to the present subject and build new understandings from the present subject. This co-participatory aspect is aligned with Laurillard's (1993) analysis of how learners 'come to know' through (1) active learning, (2) feedback, and (3) reflection.

The following items are included in evaluating Co-Participatory Activities:

  1. The flexibility allows me to meet my learning goals.
  2. The flexibility allows me to explore my own areas of interest.
  3. I am encouraged to explore concepts beyond my regular web based lessons.
  4. The asynchronous nature of the interactions enables me to reflect and respond when I had formulated an appropriate response.
  5. This mode of learning enables me to interact with other students and the tutor asynchronously.
  6. I communicate with other students in this subject electronically (via email, fax, bulletin boards, chat line).
  7. I have the autonomy to ask my tutor what I do not understand.
  8. The tutor responds promptly to my queries.
  9. The tutor addresses my queries adequately.
  10. The tutor sends me comprehensive feedback on my assignment.
  11. I have the autonomy to ask other students what I do not understand.
  12. Other students respond promptly to my queries.
  13. In this learning environment, I have to be self-disciplined in order to learn.
  14. I regularly participate in self-evaluations.
  15. I regularly participate in peer-evaluations.
  16. It is easy to organise a group for a project.
  17. It is easy to work collaboratively with other students involved in a group project.

WEBLEI Scale III: Qualia

Tobin (1998) explained qualia by describing knowledge which is considered "as embodied in neural networks as vectors of electric charge that reflect life experiences of individuals". According to Churchlands (1989) and Churchlands (1996) "neural network theory conceptualises knowing in terms of electronic loadings on a matrix of neurons that tightly couples qualia and cognitive ways of knowing".

Tobin (1998) described six categories of qualia. They are enjoyment, confidence, accomplishments, success, frustration and tedium.

The following items are included in evaluating Qualia:
  1. I felt a sense of satisfaction and achievement about this learning environment.
  2. I enjoy learning in this environment.
  3. I could learn more in this environment.
  4. The technology resources enhance learning.
  5. I was supported by positive attitude from my peers.
  6. I was able to access the materials without much difficulty.
  7. I had no difficulty using the technology.
  8. I am confident in using the technology.
  9. I have no problems going through the materials on my own.
  10. I was in control of my progress as I moved through the material.
  11. It was easy to move about in the material.
  12. The web based learning environment held my interest throughout my course of study.
  13. I felt a sense of boredom towards the end of my course of study.
  14. I felt isolated towards the end of my course of study.

WEBLEI Scale IV: Information structure and design activities

Information structure and design deals with how the web based learning materials is structured and organised, and whether the materials presented follows accepted instructional design standards, such a stating its purpose, describing its scope, incorporating interactivity, or providing a variety of formats to meet different learning styles. Included in this section are relevance and scope of content, validity of content, accuracy and balance of content, navigation, and aesthetic and affective aspects.

The following items are included in evaluating information structure and design activities:

  1. The learning objectives are clearly stated in each lesson.
  2. The scope of the lesson is clearly stated.
  3. The organisation of each lesson is easy to follow.
  4. The structure keeps me focused on what is to be learned.
  5. Expectations of assignments are clearly stated in my subject.
  6. Activities are planned carefully.
  7. The subject content is appropriate for delivery on the Web.
  8. There is a logical sequence of presentation of the subject content.
  9. The presentation of the subject content is clear.
  10. The quiz in the web based materials enhances my learning process.
  11. The material shows evidence of originality and creativity in the visual design and layout.
  12. The graphics used in the material are appropriate.
  13. The colours used in the material are appropriate.
  14. The multimedia technology (eg animation, graphics, sound, video) contributes to the affective appeal of the material.
  15. The links provided in the material are clearly visible and logical.
  16. The links provided are relevant and appropriate to the document.
  17. The links provided are reliable ie no inactive links.
  18. The 'Help' system included in the material is context sensitive.
  19. The web based learning approach can substitute traditional classroom approach.
  20. The web based learning approach can be used to supplement traditional classroom approach.


This paper has described a new instrument which assesses student perceptions of four core aspects of the Web based learning environment, namely, emancipatory, co-participatory, qualia, and information structure and design. The availability of this instrument will allow researchers and developers to evaluate the use of the Web as a learning environment.


Churchland, P. M. (1989). A neurocomputational perspective: The nature of mind and the structure of science. Bradford Book. Cambridge, MA: MIT Press.

Churchland, P. S. (1996). Toward a neurobiology of the mind In R. Lilnas and P. S. Churchland (eds.), The mind brain continuum. Bradford Book. Cambridge, MA: MIT Press: 281-303.

Khan, B. H. (1997). Web based instruction. Educational Technology Publications, Englewood Cliffs, New Jersey.

Laurillard, D. (1993). Rethinking university teaching: A framework for t he effective use of educational technology. London: Routledge.

Palloff, R. M. & Pratt, K. (1998). Effective teaching and learning in the virtual classroom. World Computer Congress: Teleteaching 98, Vienna/Austria and Budapest/Hungary, August.

Relan, A. & Gillani, B. B. (1997). Web based instruction and the traditional classroom: similarities and differences In B. H. Khan, Web based instruction, Educational Technology Publication, Inc., Englewood Cliffs, New Jersey, 41-46.

Sherry, L. & Wilson, B. (1997). Transformation Communication as a Stimulus to Web Innovations In B. H. Khan, Web based instruction, Educational Technology Publication, Inc., Englewood Cliffs, New Jersey, 67-73.

Tobin, K. (1998). Qualititative perceptions of learning environments on the world wide web In B. J. Fraser & K. G. Tobin (eds.), International handbook of science education, Kluwer Academic Publishers, United Kingdom: 139-162.

Author: Vanessa Chang, School of Information Systems, Curtin University of Technology, GPO Box U1987, Perth Western Australia 6845. Email: changv@cbs.curtin.edu.au

Please cite as: Chang, V. (1999). Evaluating the effectiveness of online learning using a new web based learning instrument. Proceedings Western Australian Institute for Educational Research Forum 1999. http://www.waier.org.au/forums/1999/chang.html

[ Proceedings Contents ] [ Schedule ] [ Abstracts ] [ WAIER Home ]
Last revised 30 May 2006. This URL: http://www.waier.org.au/forums/1999/chang.html
Previous URL 30 July 2001 to 16 May 2006: http://education.curtin.edu.au/waier/forums/1999/chang.html
Previous URL from 2 Sept 1999 to 30 July 2001: http://cleo.murdoch.edu.au/waier/forums/1999/chang.html
HTML: Roger Atkinson [rjatkinson@bigpond.com] and Clare McBeath [c.mcbeath@curtin.edu.au]