[ Proceedings Contents 1996 ] [ Abstracts 1996 ] [ WAIER Home ] School effectiveness research in rural schools Deidra J. Young and Darrell L. Fisher Curtin University of Technology

Background

This paper reports some of the findings of the pilot study undertaken in two rural schools in Western Australia, Forest Senior High School and Raner Senior High School. While these schools were from entirely different locations, they provided interesting information about how students, parents and teachers perceived their schools. The names of these schools are aliases, used in order to protect their anonymity for the purposes of this study.

Review of rural educational research

Many educators, researchers, legislators, and the general public believe that students from smaller and rural schools receive an education that is inferior to that of students from larger urban or suburban schools. Until recently, there has been little empirical evidence to challenge that view. Now, however, a growing body of work has begun to examine how well students perform in and after graduation from rural schools. Some of these studies are presented below, and, although the results are far from conclusive, they do suggest that some generally held beliefs about rural student achievement are open to question.

A comparison of the performance on standardised achievement tests of students from small, usually rural, schools with those form larger, often urban, institutions has not produced definitive results. Several studies have not found any significant differences between the two groups. In research completed in the state of New York, Monk and Haller (1986) found that students from smaller (often rural) schools achieved as well as students from larger schools. Kleinfeld and others (1985), in their Alaska study, did not find that high school size determined the quality of a student's education, experience, or achievement on standardised tests. Moreover, in one New Mexico study, which looked at factors affecting performance of selected high school students, those attending schools in rural areas performed as well as those in urban locales (Ward & Murray, 1985).

There is some indication that what is being measured in studies of rural-urban differences is socioeconomic status and/or ethnicity. Easton and Ellerbruch (1985) found that the poorer rural students scored considerably lower on citizenship and social studies tests than did students from upper socioeconomic urban communities. Another study which held socioeconomic level and ethnicity constant revealed no urban-rural achievement gap (Edington & Martellaro, 1984). This has been reaffirmed by an Australian study of students in Years 3, 7 and 10. In this study, the socioeconomic status of the school accounted for most of the variation in student achievement in mathematics, reading and writing (Young, 1994).

Kleinfeld and colleagues (1985) suggest that schools that achieve the best results do exhibit a strong teacher/administration/community partnership and school/community agreement on educational programs. They also have reported that there is a direct relationship between quality education programs and the ability of the staff to work toward an educational partnership with the community. Smaller communities tend to generate more community support for the school, with the school becoming a centre for community activity. This, in turn, theoretically provides the students with a greater feeling of belonging to something in which they can participate, and thus enables them to develop a better self-concept.

Finally, rural education research has been conducted in Louisiana by Stringfield and Teddlie (1991, 1993) for ten years and these researchers have produced some fine and valuable findings. These researchers have found significant variations in what makes a school effective in the rural parts of the USA. We hope to replicate some of this work in Australia, along with a more substantive piece of research in following a number of rural and urban schools for a three year period.

Figure 1: Assessing the classroom environment

That classes and schools differ in terms of their learning environments, which in turn influence student achievement has been demonstrated by Hattie (1987) who showed that 20% of students in desirable climates are better off than students in average classrooms. In the last 25 years there have been instruments developed for a range of classroom contexts, such as individualised classrooms (Fraser, 1990) and constructivist classrooms (Taylor, Dawson & Fraser, 1995). These instruments have been employed in a range of studies, with different instruments and scales used in particular studies. Recently, Fraser, Fisher and McRobbie (1996) began the development of a new learning environment instrument which incorporates scales that have been shown in previous studies to be significant predictors of outcomes (Fraser, 1994) and additional scales to accommodate recent developments and concerns in classroom learning, such as equity issues and the promotion of understanding rather than rote memorisation. The first version of the new instrument contained the following 9 scales, each scale containing 10 items: Student Cohesiveness, Teacher Support, Involvement, Autonomy/Independence, Investigation, Task Orientation, Cooperation, Equity and Understanding. The new instrument employed the same five-point Likert response scale (Almost Never, Seldom, Sometimes, Often, Almost Alw ays) as used in some previous instruments.

For the purposes of this study, we used 5 of these scales in the student questionnaire, that is, Student Cohesiveness, Teacher Support, Involvement, Task Orientation and Cooperation. Subsequent analyses by Fraser, Fisher and McRobbie (1996) have demonstrated that the scales Autonomy/Independence and Equity and Understanding were not reliable.

Student self-concept

That self-concept is related to achievement presupposes that certain classroom environments enhance both aspects (Hattie, 1992, p. 197)

In the literature, the relationships between Self-Concept, Ambition and the Classroom Learning Environment were not well understood. While it was expected that the Classroom may influence student Self-Concept and student Ambition, causality has not been documented. Further, these relationships have not been investigated in rural school settings. This study sought to address these issues using a path analytic technique which did not assume causality and could cope with measurement error.

Included in this study, were two measures of Self-Concept, namely, General Self-Concept and School Self-Concept. The General Self-Concept scale describes the student's feelings about himself/herself. There are both negative and positive statements related to success and failure in life. The School Self-Concept scale measures the student's perceptions about their academic ability and potential to be a success at school. In this paper, this scale will be referred to as academic self-concept or school self-concept.

Student aspirations

The problems rural students face are confounded. Firstly, when these students grow from adolescence to mature adulthood, they also must face the reality that there is little for them in their town/farm/rural area. In order for these students to attain their potential in life choices, they must make a choice. Either they can stay with their families in their rural location and enjoy the rural lifestyle they are accustomed to, or they must move to the city to either look for work or further their education in vocational colleges or university. It is obvious to these students that education will expand and fulfil their lives; often parents send their children to boarding schools in the city in order to prepare them for the new changes which lie ahead. Unfortunately, some of these students, who are accepted into higher education courses, become extremely lonely and disheartened and return to their rural home. Of course many others are keen to leave home and become independent. It appears that this is sometimes related to the social network which rural students develop when they arrive in the city. Hektner attempted to disentangle the rural young person's aspiration for social mobility and preferences for residing in rural locations (Hektner, 1995). In his study of midwestern US schools, Hektner found a substantial amount of conflict experienced by rural students in choosing to leave or stay at home. Rural students were more likely to have conflicting aspirations about wanting to live at or near home and wanting to "move out in order to move up".

Stevens' investigation of influences on vocational choices of senior high school students in a rural community demonstrated that rural students have to make career decisions at an earlier age than urban students (Stevens, 1995). This study also found a significant difference between the rural working class and the rural middle class. That is, parents who are able to send their children to boarding schools in order to complete the final two years of high school did so from a superior financial base. In the rural school which Stevens' studied, there was negligible provision for students to complete their high school education, with the result that the working class families were disadvantaged and unlikely/unable to send their children to boarding schools. Further, Stevens' noted a difference in the students' perceptions of the world and their ability to cope in an urban school environment. Many rural students were supplied with inadequate information and counselling in order to choose their school subjects for their chosen occupations and also experienced conflict regarding the superiority of the urban lifestyle which lay before them.

These findings are similar to McCracken and Barcinas (1991), whose study of rural schools in Ohio revealed that rural students tended to be more homogeneous, come from larger families and have lower socioeconomic status. Rural parents tended to have a lower educational attainment and were less likely to expect their children to attain an education beyond high school. McCracken maintained that these parental and home influences helped to explain why rural students chose lower educational courses. However, rural youth were also more likely to select vocations which they had been able to observe or experience, such as agricultural college or technical colleges. Students in rural areas had lower income expectations, did not observe many high-income workers and those students who were bright and capable tended to be sent away to complete their education. The discrepancy in educational aspirations between rural and nonrural students seems clear, yet the reasons for it are not. Initiatives to raise students' aspirations in rural settings have had limited research foundations. However, it is the hope of a number of researchers that research can be developed which can make a difference both to the research field and to the student (Walberg, 1989; Quaglia, 1989; Cobb, McIntire & Pratt, 1989; Reid, 1989; McCaul, 1989; Pratt & Skaggs, 1989; Breen, 1989; Hansen & McIntire, 1989; Preble, Phillips & McGinley, 1989; Sherwood, 1989).

There appears to be a distinct relationship between socioeconomic status, occupational aspirations and educational aspirations and this theme has been the subject of research by Haller and Virkler (1993). These important relationships framed this research study of the psychological, socioeconomic and classroom influences on occupational aspirations and educati onal aspirations.

Conceptual framework

Research design: Western Australian School Effectiveness Study [WASES]

This research study will be called the Western Australian School Effectiveness Study and abbreviated to WASES throughout the rest of this proposal. This research involves three phases. In the First Phase, reported here, the survey instruments were developed and piloted in two schools (1995).

In the Second Phase, a longitudinal survey is being conducted in 30 West Australian high schools over a three year period. Government, Catholic and Independent secondary schools are being surveyed. The purpose of this survey is to evaluate the school and classroom climate and characteristics of effective schools in differential contexts. Because the growth model is particularly useful for measuring change over time in student outcomes, while controlling for other influencing variables which may also change over time, the same students at the same schools will be surveyed over a period of three years (1996 to 1998). This phase will be called WASES-II in 1996, WASES-III in 1997 and WASES-IV in 1998 and is being funded in part by the Australian Research Council. Finally, in the Third Phase, a case study approach will be used to examine some exceptionally effective schools in the rural and urban locations of Western Australia (1999).

The instruments used in the pilot study included variables measuring those features and characteristics which make a school effective at different levels. These levels emerge from Stringfield and Teddlie's findings, along with Scheerens and Creemer's model and are discussed further in the methodological section.

Sample

From the two schools, a total of 163 Year 10 science students and 8 classes contributed towards trialling the instruments and completing questionnaires. The breakdown by school is presented in Table 1. The pilot study took place in September 1995, with four researchers working within the schools to coordinate the data collection process. This provided us with substantively more useful information regarding student confidentiality and staff problems in completing the questionnaires.

School	Students	Classes
Forest Senior High School*	73	3
Raner Senior High School*	90	5
Total	163	8

* pseudonyms

Methodology

Students were asked to complete the questionnaire and seal it in an envelope, which was given back to the researchers visiting the school. They were constantly reassured about the confidentiality of their responses. The student questionnaire consisted of background and socioeconomic questions, along with questions about the students' rural life. Further, students were asked questions which related to a measure of their Ambition. Students were asked about their career aspirations (expected occupation) and education aspirations (expected education).

Descriptive statistics and item reliabilities are presented in Table 2.

Four variables were used to measure Socioeconomic Status: Father's Occupation, Mother's Occupation, Father's Education and Mother's Education. For both student and parents, occupation coding was done on the basis of the Australian Standard of Classification of Occupations (ASCO, ABS, 1990). The codes are presented in ASCO from 1 to 8, however in this study the 8 was a manager/administrator and the 1 was a labourer/cleaner. Pensioners, unemployed, retired, home duties and missing data were not used in these analyses. For student and parents, education was coded from 1 being primary schooling to 6 being tertiary education (completed university degree).

Two Self-concept scales were used in this study: General Self-concept and School/Academic Self-concept. These measures have been described previously and their items are presented in Table 2. The items had 5 possible responses coded 1 for "the sentence is False", 2 for "the sentence is Mostly False", 3 for "the sentence is neither False nor True", 4 for "the sentence is Mostly True" and 5 for "the sentence is True".

The Classroom Learning Environment scales included items on a five point scale ranging from Almost Never, Seldom, Sometimes, Often to Almost Always (coded 1 to 5). Students were asked to respond to statements describing their classroom. Descriptive statistics for these scales are found in Table 2.

No student achievement data was collected during the pilot study, but rather this researcher sought permission from the International Association for the Evaluation of Educational Achievement (IEA) to use the Third International Mathematics and Science Study (TIMSS) testing instruments for the next phase of this study, that is, Phase II. Data were collected from one principal, some teachers and parents. However, the sample was too small to make valid comparisons and issues to do with confidentiality of the school identity meant that we felt it was unethical to proceed with comparing these two schools. In the large-scale study being collected this year (1996), and over the next two years (1997 and 1998), these issues of school identity are unlikely to arise as there are now 30 schools participating in the longitudinal study.

For the purposes of this study, four latent variables were measured using observed variables. This was important in order to avoid substantial measurement error both by measuring the latent variable and in measuring relationships between the observed variables. The purpose of these analyses was to investigate the relationships between the four latent variables. These variables were:

• LISREL was used in order to determine the measurement error of the latent variables: Ambition, Self-concept, Socioeconomic Status and Classroom Environment. For example, socioeconomic status was initially analysed using six observed variables, that is, mother and father's occ upation and education, number of books in the home and family size. The measurement errors of these latent variables were estimated by the LISREL analysis and observed variables which did not fit were discarded.

• In the next stage, Confirmatory Factor Analysis (CFA) was used to test the following model of latent and observable variables. The model was constructed, the four latent variables plus gender were selected, along with their observed variables, and each relationship between observed and latent variable was estimated with one observed variable fixed at one and the others estimated.

• There were two endogenous latent variables in this model: Ambition and Self-concept. These variables are measured by their observed variables and predicted by the other latent variables. Essentially, the endogenous model is the centrepiece of the structural equation model. There were three exogenous latent variables in this model: Socioeconomic Status, Gender and Classroom Learning Environment. Before combining all of these latent variables into a single structural equation model, the relationships between the exogenous latent variables were estimated.

• Finally, the general structural equation model was estimated. This consisted of a measurement model which specified the relationship between observed and latent variables and a latent variable model which estimated the influence of latent variables on each other variables (Bollen, 1989, p. 319). For both of these models, the covariance matrix was used as the input data. This was important particularly for those variables or parameters which had different scales and resulting in greater measurement error. The major feature of the structural equations with latent variables, as distinct from the standard regression approach, is that these models are more realistic in their allowance for measurement error in the observed (pp. 151-178).

Scales	Number of items	N	Mean	SD	Reliability a
Socioeconomic Status
Father's occupation (1-8)	1	153	5.33	2.48	-
Father's education (1-6)	1	136	2.90	1.39	-
Mother's education (1-6)	1	134	3.14	1.37	-
Gender (Males=1, Females=2)	1	163	1.42	.49	-
Ambition
Occupational aspiration (1-8)	1	153	5.49	2.42	-
Educational aspiration (1-6)	1	144	4.65	1.52	-
Self Concept Scales
Self concept (school)	10	163	3.85	.71	.886
Self concept (general)	10	163	4.17	.60	.876
Classroom Learning Environment Scales
Student cohesiveness	10	159	3.50	.54	.771
Teacher support	10	159	3.55	.67	.857
Student involvement	10	160	3.54	.52	.741
Task orientation	10	154	3.33	.66	.902
Cooperation	10	154	3.29	.50	.748

Results

AMBITION was a general construct not measured, but a measurement model was used with the observed indicators of Expected Occupation (Occupational Aspirations) and Expected Education (Educational Aspirations).

The CLASSROOM LEARNING ENVIRONMENT SCALES, were all reasonably positive. In Table 2, the means are seen to range from 3.29 to 3.55. These measures were of the individual classrooms and comparisons would be more valid at the classroom level of analysis in the large scale study.

SELF CONCEPT was a general construct not measured, but a measurement model was used with the two observed scales school self concept and general self concept, each measured the students' perception of themselves and their ability to succeed. The scales had high reliabilities (.886 and .876, respectively) and were very high in general (Table 2). That is, student self concept statements were mostly true for these students (3.85 and 4.17, respectively).

The Structural Equation Model

In the analyses of this data, gender was found to not only have a weak effect on ambition and self-concept traits, but its only strong, positive correlation was with the observed variables comprising the latent trait Classroom Learning Environment. As boys were coded (1) and girls (2), this means that girls perceived their classroom learning environment differently, and more favourably, than boys. This was true for students residing in the same or different classrooms. However, as gender had negligible effects on the latent traits of interest, it was removed from the model and the other effects were re-estimated. Other effects which were found to be weak an d not significant were Classroom Learning Environment on Ambition and Ambition on Self-Concept. These effects were subsequently removed from the model and re-estimation showed a reasonably tight fit.

Results from the analyses will be presented in Figure 2. This means that all effects are comparable in terms of scale. The final estimated model revealed a strong, significant relationship between the Classroom Learning Environment and Self-Concept, Self-Concept and Ambition, and a weak effect of Socioeconomic Status on Ambition. The chi-square with 61 degrees of freedom was 81.98 (p = 0.038) and goodness of fit index was 0.847.

Results from these analyses suggest that students choose their future occupations and educational career paths more on the basis of their own self-perceptions, than their socioeconomic backgrounds or gender. These self-perceptions or Self-Concept were strongly and positively affected by the Classroom Learning Environment. While these results have limited implications due to the small sample size, the relationships were important for directing further research in this area.

Discussion

There were a number of limitations to this pilot study, the first of which is the small sample size. Missing data further reduced the sample size. Further, only having two schools meant that no multilevel modelling could be undertaken. This is an important technique due to the level of variations which can occur between schools and between classrooms. That is, the statistical tests of variables can be incorrect due to errors in estimating student level variance.

Finally, these analyses were limited to a small number of variables and further analysis which includes more school level and classroom level variables and a larger sample size should improve the consistency and reliability of the results. Other student level variables which should provide further information is student achievement. These problems and limitations will be overcome in the next phase of this study, the Western Australian School Effectiveness Study Phase II.

Notes

An earlier version of this work was presented at the Society for the Provision of Education in Rural Australia Conference, 23 August 1996, Hobart, Tasmania.

I would like to acknowledge the assistance and advice provided by our LISREL guru: Leigh Smith, Head of School, Psychology, Curtin University of Technology.

References

Breen, D.T. (1989). Enhancing student aspirations: A goal for comprehensive developmental guidance programs. Research in Rural Education, 6(2), 35-38.

Byrne, B.M. (1984). The general/academic self-concept nomological network: A review of construct validation research. Review of Educational Research, 54, 427-456.

Cobb, R.A., McIntire, W.G., & Pratt, P.A. (1989). Vocational and educational aspirations of high school students: A problem for rural America. Research in Rural Education, 6(2), 11-16.

Creemers, B.P.M., & Scheerens, J. (1994). Developments in the educational effectiveness research program. International Journal of Educational Research, 21(2), 125-140.

Edington, E.D., & Koehler, L. (1987). Rural student achievement: Elements for consideration. New Mexico State University, Las Cruces, New Mexico. ED 289 658.

Edington, E.D., & Martellaro, H.C. (1984). Variables affecting academic achievement in New Mexico schools. Las Cruces: New Mexico Center for Rural Education. ED 271 267.

Fraser, B.J. (1990). Individualised classroom environment questionnaire. Melbourne, Victoria: Australian Council for Educational Research.

Fraser, B.J. (1994). Research on classroom and school climate. In D. Gabel (Ed.), Handbook of research on science teaching and learning, (pp. 493-541). New York: Macmillan.

Fraser, B.J., Fisher, D.L., & McRobbie, C. (1996, April). Development, validation and use of personal and class forms of a new classroom environment instrument. Paper presented at the Annual Meeting of the American Educational Research Association, New York.

Haller, E.J., & Virkler, S.J. (1993). Another look at rural-nonrural differences in students' educational aspirations. Journal of Research in Rural Education, 9(3), 170-178.

Hansen, T.D., & McIntire, W.G. (1989). Family structure variables as predictors of educational and vocational aspirations of high school seniors. Research in Rural Education, 6(2), 39-49.

Hattie, J. (1992). Self-concept. Hillsdale, NJ: Erlbaum.

Hektner, J.M. (1995). When moving up implies moving out: Rural adolescent conflict in the transition to adulthood. Journal of Research in Rural Education, 11(1), 3-14.

Kleinfeld, J.S., et al. (1985). Alaska's small rural high schools: are they working? ISER Report Series No. 58. Fairbanks: University of Alaska. ED 266 915.

Marsh, H. W. (1992). Self description questionnaire - II manual. Sydney: University of Western Sydney.

Marsh, H.W. (1990). Multidimensional, hierarchical self-concept: Theoretical and empirical justification. Educational Psychology Review, 2, 77-172.

Marsh, H.W. (1993). Academic self-concept: Theory measurement and research. In J. Suls, (Ed.), Psychological perspectives on the self (Vol. 4, pp. 59-98). Hillsdale, NJ: Erlbaum.

Marsh, H.W., & Shavelson, R.J. (1985). Self-concept: Its multifaceted, hierarchical structure. Educational Psychologist, 20, 107-125.

McCaul, E. (1989). Approaches to dropout prevention: The philosopher's stone revisited. Research in Rural Education, 6(2), 25-30.

McCracken, J.D., & Barcinas, J.D. (1991). Differences between rural and urban schools, student characteristics, and student aspirations in Ohio. Journal of Research in Rural Education, 7(2), 29-40.

Monk, D.H., & Haller, E.J. (1986). Organizational alternatives for small rural schools. Cornell: New York State College of Agriculture and Life Sciences at Cornell University, 1986. ED 281 694.

Pratt, P.A., & Skaggs, C.T. (1989). First generation college students: Are they at greater risk for attrition than their peers? Research in Rural Education, 6(2), 31-34.

Preble, B., Phillips, P., & McGinley, H. (1989). Maine's aspirations movement: Reaching out to youth. Research in Rural Education, 6(2), 51-59.

Quaglia, R. (1989). Student aspirations: A critical dimension in effective schools. Research in Rural Education, 6(2), 7-9.

Reid, J.N. (1989). The rural economy and rural youth: Challenges for the future. Research in Rural Education, 6(2), 17-23.

Scheerens, J., & Creemers, B.P.M. (1989). Conceptualising school effectiveness. International Journal of Educational Research, 13(7), 691-706.

Sherwood, R.A. (1989). A conceptual framework for the study of aspirations. Research in Rural Education, 6(2), 61-66.

Stevens, K. (1995). Vocational cho ice for senior high school students in rural Australian communities. Journal of Research in Rural Education, 11(3), 182-186.

Stringfield, S., & Teddlie, C. (1991). School, classroom and student level indicators of rural school effectiveness. Journal of Research in Rural Education, 7(3), 15-28.

Stringfield, S., & Teddlie, C. (1993). Schools make a difference: Lessons learned from a 10-year study of school effects. New York: Teachers College Press.

Taylor, P. C., Dawson, V., & Fraser, B. J. (1995, April). CLES: An instrument for monitoring the development of constructivist learning environments. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Tomlinson, D. (1994). Schooling in rural Western Australia: Report of the ministerial review of schooling in rural Western Australia. Perth: Office of the Minister for Education, Western Australia.

Walberg, H.J. (1989). Student aspirations: National and international perspectives. Research in Rural Education, 6(2), 1-6.

Ward, A.P., & Murray, L.W. (1985, October). Factors affecting performance of New Mexico high school students. Paper presented at the Meeting of the Rocky Mountain Educational Research Association, Las Cruces. ED 271 266.

Young, D.J. (1994). A comparison of student performance in Western Australian schools: Rural and urban differences. The Australian Educational Researcher, 21(2), 87-105.

Young, D.J. (1995). The effect of the science learning environment on science achievement: A comparison of 12 countries from the IEA Second International Science Study. Educational Research and Evaluation, 1(2), 129-158.

Authors: Deidra J. Young, D.Young@smec.curtin.edu.au Darrell L. Fisher, Curtin University of Technology Please cite as: Young, D. J. and Fisher, D. L. (1996). School effectiveness research in rural schools. Proceedings Western Australian Institute for Educational Research Forum 1996. http://www.waier.org.au/forums/1996/young.html