|[ Abstract for this presentation ] [ Proceedings Contents ] [ Schedule ] [ Abstracts ]
Conclusions about the efficiencies and effectiveness of educational research methodologies in investigating how students learnGail Chittleborough
Science and Mathematics Education Centre
Curtin University of Technology
Examining how students learn is not only looking at what they know but how they come to know what they know. In this way this research focused on the process of learning as well as the content of learning. This paper examines the educational research methodologies that were used in investigating both of these aspects of how students learn chemistry. Multiple research methodologies were adopted including surveys, cross-sectional studies and naturalistic inquiries. A variety of both qualitative and quantitative data sources were used. The choice of data sources, the samples and the difficulties and limitations of the research methodology is discussed. The validity and reliability of the interpretive analysis is demonstrated with examples from the research into how students learn chemistry. While the research was focussed on particular research questions, the choice of methodology has some commonality with all research situations. The research process itself is a dynamic and developing process and the parallel between the process and content of learning is made to the process and content of research.
Factors affecting the choice of data sources include the research process, design, situation and subject. I am focusing on data sources because they are the food or fuel of the study. Researchers spend a lot of time preparing data sources, collecting data, analyzing data and drawing conclusions form the data. Consequently the data sources are crucial to the research. Shortsightedly, research can be seen as just the collection and analysing of data sources. This paper aims to demonstrate the important role of data sources, important even crucial, but not independent of the rest of the research process.
My study investigated how students learn chemistry with a particular emphasis on the role of chemical representations and chemical models. Examining how students learn is not only looking at what they know but how they came to know what they know. Because of this, the research focused on the process of learning as well as the content of learning.
My intended methodology of a naturalistic inquiry collecting both qualitative and quantitative data sources was mostly achieved. Aware of the need for triangulation, corroboration and validation, my research design included a variety of data sources. By looking at the detail of the data and focussing on it's analysis it was possible that I ran the risk of loosing sight of the research methodologies. Distinguishing the research methodologies from the data sources is important because the methodologies provide the framework for the research are needed to ground the data sources.
My research questions changed, as I responded to a new opportunity that is not a problem as long as the new questions are consistent with the research problem, methodology and research subject and that the data is needed to address the research question. Research should be fluid and dynamic, but also mindful of the planning that has preceded it. Taking advantage of opportunities provides more and more data, collected easily but the processing and analysing takes time. Table 1 is presented to show the variety of data sources.
|The Number of|
the Data Source
|1||Molecular Chemical Representations (MCR)||V||2|
|1||My Views of Models and Modelling in Science (VOMMS)||V||3|
|1||Group work dialogue||A||4|
|1||Participant researcher's observations||5|
|2||Scientific Models (SM)||V||1|
|2||Students' Understanding of Models in Science (SUMS)||V||2|
|2||My Views of Models and Modelling in Science (VOMMS)||V||3|
|3||Students' Understanding of Models in Science (SUMS)||V||1|
|3||My Views of Models and Modelling in Science (VOMMS)||V||2|
|3||Molecular Chemical Representations (MCR)||V||3|
|3||What do you thi nk this atom looks like?||V||4|
|3||Student Unit Experience (SUE)||A||5|
|3||Participant researcher's observations||7|
|4||Students' Understanding of Models in Science (SUMS)||V||1|
|4||My Views of Models and Modelling in Science (VOMMS)||V||2|
|4||Students' Evaluation of Educational Quality (SEEQ)||A||3|
|* NA Not applicable|
The comparisons among the four studies became complex. While each of the four studies was independent, there was the added opportunity for comparison. The strength of the multiple methodologies, multiple data sources and multiple samples was countered by needless complexity and lack of fair comparison. In hindsight the research became too large and almost unmanageable.
Research methods that allowed the value of the research to be recognised were more fulfilling to me as a researcher. For example action research implementing changes to learning styles, and getting feedback from participants or stakeholders' were more fulfilling than the more descriptive studies that were interesting but had no immediate impact.
The term "understanding" is associated and implied in the term learning. Wandersee and Griffard (2002) define understanding "as a dynamic epistemological status conferred on individuals by a consensually recognised referent group within a community of scholars, based upon criteria of intersubjectivity, parsimony, coherence and conceptual transparency" (p. 29).
As a subject of research, learning appears abstract and illusive however, it is apparent that theoretical frameworks from the literature provide structure and descriptors to shed light on the subject. In my study I had 3 theoretical frameworks - models, 3 levels of chemical representations and learning, connecting the learning tools, the way of thinking about chemicals - the mental models - and the learning. The theoretical frameworks provided the common foundation for the content in all four studies. Even though each study was different the commonality of the theoretical frameworks unified them.
Because the interpretive analysis is made in light of previous research, it is advantageous to write the research question in line with the literature e.g. the research question, "What are students' perceptions of the role and purpose of generic models and scientific models?" is related to the theoretical framework of four types of models - teaching, scientific, mental and expressed models - showing their relationship to learning.
|1.1||What are students' perceptions of the role and purpose of generic models and scientific models?||1, 2, 3, 4|
|1.2||What are the criteria which students identify as being significant when classifying scientific models?||1, 2, 3, 4|
|1.3||How does students' modelling ability affect their use of models and their ability to understand chemical concepts?||1, 3, 4|
|1.4||How and why do models help students learn?||1, 2, 3, 4|
|2.1||What are students' perce ptions of the role and purpose of chemical representations, including chemical models, teaching models, chemical equations, diagrams, and pictures in learning chemistry?||1, 3, (2 & 4 indirectly)*|
|2.2||What are students' understandings of each level of chemical representation in relation to the chemical phenomena they experience?||3, (1 & 4 indirectly)**|
|2.3||How does this understanding enable students to effectively transfer from one representational level to another?||1, 3, 4|
|2.4||How do the variety of representational forms, which students encounter in chemistry, impact on the epistemology, ontology and social factors that have been shown to contribute to conceptual change?||1, 3, 4|
|3.1||What are the factors that influence how and why students learn chemistry?||3, 4|
|3.2||What learning strategies do students use in learning chemistry?||1, 3, 4|
|3.3||How do learning strategies contribute to the development of students' personal mental models of chemical phenomena?||1, 3|
|3.4||How does students' metacognitive awareness influence their learning of chemistry?||4|
|Type of Study||Data Sources*|
and Sample Sizes
|1||High school||Year 11||3-4 weeks||1.1|
|Interpretive research - |
descriptive and explanatory
|Questionnaires - Models n=36|
Class group activity - audiotapes and videotapes, observations
|2||High school||Years 8, 9, 10||1 week||1.1|
|2.1||Quantitative analysis||Questionnaires - SM n=228|
|Interpretive research -|
descriptive and explanatory
|Initial Questionnaire (containing SUMS, VOMMS and MR) n=18|
Worksheet 1 n=9
Worksheet 2 n=8
Worksheet 3 n=9
Worksheet 4 n=10
1st interviews n=7
2nd interview n=12
|Interpretive research -|
descriptive and explanatory
|Introductory Questionnaire (containing SUMS and VOMMS) n=49, SEEQ n=98|
Online Survey n=115
1st interview n=17
2nd interview n=5
|* abbreviations include: MR - Molecular Representations; SM - Scientific Models; SUMS - Students' Understanding of Models in Science; VOMMS - My Views of Models and Modelling in Science; SUE - Student Unit Experience; SEEQ - Students' Evaluation of Educational Quality.|
Investigating the learning and understanding of chemistry used direct sources to gain data about students' understanding of their own learning e.g. What strategies do you use in learning chemistry? And also indirect sources e.g worksheet responses to chemistry questions, to investigate how the student is learning.
Molecular Chemical Representations (MCR)
My Views of Models and Modelling in Science (VOMMS) Scientific Models (SM)
Students' Understanding of Models in Science (SUMS)
Student Unit Experience (SUE)
Students' Evaluation of Educational Quality (SEEQ)
|What do you think this atom looks like?|
1st and 2nd Interview (3)
1st and 2nd Interview (4)
|Indirect||Worksheets 1-4||Group work dialogue Researcher's obs|
1st and 2nd Interview (3)
1st and 2nd Interview (4)
It is worthwhile classifying the data sources to gain an appreciation of the range and type of data sources, providing further validation, and identifying gaps in the research design. Being aware of the number of interpretations within each data source is worthwhile. The qualitative data is more generalised and deals with the sample as a group whereas the qualitative data is more individualistic- and may or may not be representative of the whole group
These data sources are well suited to a statistical analysis: percentages, frequencies, standard deviation, Cronbach alpha reliability, factor analysis, scales, ANOVAs, and correlations. This research highlighted the difficulty in formulating an instrument so that it is unambiguous and statistically valid.
Example 3: Written answersSimilarly an interview question asks students to reflect on their learning. Gabby's response to this interview question illustrates two interpretations- firstly Gabby's interpretation and response and then the researcher's interpretation of her response including the words, intonation and intent.
- What aspects of the pre-laboratory exercises are helpful to your learning of chemistry?
- What aspects of the pre-laboratory exercises are NOT helpful to your learning of chemistry?
Example 4: Interview
Models are frequently used in chemistry teaching and learning, can you recall any chemical model that you have learnt in your chemistry module?
Gabby: I think it is important for someone to understand the basics. If there were no pictures I would not be able understand anything but now I have pictures in my head. I think the pictures that you get are very important because that is what you remember.
Not at all confident
|1.||1.35 grams of Na2CO3 contains (0.05x .25x 6.022 x1023) ie 7.53 x 1021 molecules||1||2||3||4||5||X|
|2.||A standard solution has a known number of particles that is indicated by the molarity of the solution.||1||2||3||4||5||X|
Making interpretations from students' responses required the researcher to assess what and how the student was learning. Wading through the chemistry and identifying the way the student is tackling the learning helps to gain an understanding of how the student is thinking. This is a complex process.
|Interviewer||But when you were doing the lab, you probably didn't have a good understanding of what was happening.|
|Mat||No. I didn't know what was happening at the molecular level. But when I sat down after looking at the procedure that we had carried out in the lab, and looking at the questions, and then what we were asked to find out, I could link the experiment to the question. Try to develop some point of view.|
|Interviewer||All right. So when you did that, how did you go about making that link. Did you actually draw pictures or did you just think about 'well I've got this organic layer and I've got this aqueous layer - Did you do this sort of thing what I've done here?|
|Mat||I did have an idea of, I had an idea of the setups, and what was added to each setup. So, in some way, I didn't have the diagram on paper, but I had an idea in my mind that this was what was happening.|
|Interviewer||And you definitely understood that the equilibrium constant you measured was for which equilibrium system?|
|Mat||The equilibrium that I measured?|
|Mat||Well it was for the iodine moving from the aqueous, the iodine in the aqueous layer reacting with the iodide that was added in the aqueous layer.|
|Interviewer||So it was for the equilibrium just within the aqueous layer.|
|Mat||Just within the aqueous layer. And I think there was some point where we had to subtract the organic layer.|
Mat is a very able student and worked at understanding the changes that were occurring at the sub-microscopic level in the experiment. He demonstrated transferring from the macroscopic level to the sub-microscopic level. This analysis requires two interpretations - the student and the researcher.
Strength of this research is in the variety of data that the four studies provide. The samples of students are diverse, with different motivations, ages and background knowledge.
Over the period of the research, the emphasis on the quantitative data declined and the emphasis on qualitative data increased. Studies 1 and 2 have substantial quantitative data sources, while in Studies 3 and 4 the quantitative data sources are used to support and complement the qualitative data sources. Similarly, the emphasis on the direct data sources declined and the emphasis on indirect data sources increased. This evolution is typical and expected because the qualitative data provided a good overview of the research topic highlighting some misconceptions or weaknesses and then the quantitative data sources provided opportunities to examine these areas in more detail. These aspects add to the rigour and robustness of the research.
Research results can be inconclusive or dubious whether the research is rigorous or not, consequently, research methods must be rigorous, appropriate, reliable and valid so that all results are dependable. Without this certainty, the analysis and conclusions are worthless. I focused on the qualitative and quantitative data sources because of the need for triangulation, corroboration and validation without stepping back to make sure that they were achieving this objective. Infiltrates all levels of research methodology: appropriate methods to answer the research question, the analysis of the data; a mix of quantitative /qualitative and direct/indirect data sources.
Focussing on the structure and process of educational research can improve the efficiency and effectiveness of the educational research. More specifically, narrowing and restricting the research by writing very specific research questions can keep the research manageable and achievable. Identifying the relevant theoretical frameworks and writing the research question consistent with these makes the analysis and interpretation easier. Exploring and selecting the most appropriate research methodology will provide direction and a foundation for the research. Having a range of substantiating data sources and being confident in the validity and reliability of those data sources provides credibility to the research. Linking each data source with particular research question should make the research more efficient and effective.
Atherton, J. S. (2003). Two dimensions of practice. [viewed 6 May 2003, verified 9 Aug 2005] http://www.doceo.co.uk/tools/willwit_1.htm
Ausubel, D. P. (1968). Educational psychology: A cognitive view. New York: Holt, Rinehart and Winston, Inc.
Battino, R. (1992). On the importance of rote learning. Journal of Chemical Education, 69(2), 135-137.
Bereiter, C. & Scardamalia, M. (1989). Intentional learning as a goal of instruction. In L. B. Resnick (Ed.), Knowing, learning and instruction: Essays in honor of Robert Glaser (pp. 361-392). Hillsdale, NJ: Lawrence Erlbaum Association.
Cohen, L. & Manion, L. (1994). Research methods in education. New York: Routledge.
Duit, R. & Treagust, D. F. (1998). Learning in science - from behaviourism towards social constructivism and beyond. In B. J. Fraser & K. G. Tobin (Eds.), International handbook of science education (Vol. 1, pp. 3-25). Dordrecht, The Netherlands: Kluwer Academic Publishers.
Hennessey, M. G. (2003). Metacognitive aspects of students' reflective discourse: Implications for intentional conceptual change teaching and learning. In G. M. Sinatra & P. R. Pintrich (Eds.), Intentional conceptual change. Mahwah, NJ: Lawrence Erlbaum Associates.
Mason, D. S., Shell, D. F. & Crawley, F. E. (1997). Differences in problem-solving by non-science majors in introductory chemistry on paired algorithmic-conceptual problems. Journal of Research in Science Teaching, 34(9), 905-923.
Ramsden, P. (1992). Learning to teach in higher education. London: Routledge.
Schommer, M. (1990). Effects of beliefs about the nature of knowledge on comprehension. Journal of Educational Psychology, 82(3), 498-504.
Skemp, R. R. (1976). Relational understanding and instrumental understanding. Mathematics Teaching, 77, 20-26.
Stephens, S., McRobbie, C. J. & Lucas, K. B. (1999). Model-based reasoning in a year 10 classroom. Research in Science Education, 29, 189-208.
Wandersee, J. H. & Griffard, P. B. (2002). The history of chemistry: Potential and actual contributions to chemical education. In J. K. Gilbert, O. De Jong, R. Justi, D. F. Treagust & J. H. Van Driel (Eds.), Chemical education: Towards research-based practice (Vol. 17, pp. 29-46). Dordrecht, The Netherlands: Kluwer Academic Publishers.
Yager, R. E. (1991). The constructivist learning model: Towards real reform in science education. The Science Teacher, 58(6), 52-57.
|Author: Gail Chittleborough, Science and Mathematics Education Centre, Curtin University of Technology, GPO Box U1987, Perth WA 6845. Email: G.Chittleborough@curtin.edu.au
Please cite as: Chittleborough, G. (2005). Conclusions about the efficiencies and effectiveness of educational research methodologies in investigating how students learn. Proceedings Western Australian Institute for Educational Research Forum 2005. http://www.waier.org.au/forums/2005/chittleborough.html