Development, validation and use of the Online Learning Environment Survey

This article reports the development, validation and use of a survey for assessing students' perceptions of their e-learning environments. The Online Learning Environment Survey (OLES) was administered to 325 students, 131 in Australia and 194 in Hong Kong. The data were analysed to examine 1) the reliability and validity of the survey, 2) differences between the perceptions of a) students' actual and preferred environment, b) students and their teacher and c) male and female students and 3) whether associations exist between students' perceptions of their e-learning environment and their enjoyment of e-learning. In addition to quantitative data, unstructured interviews were used to provide a more in depth understanding of the e-learning environments created. These data provide valuable feedback to educators working in e-learning environments to help teachers to evaluate the effectiveness of the environment and to make adjustments and improvements as required.

Objectives

The objectives of the study were:

1. To validate the Online Learning Environment Survey (OLES) as a tool to provide educators with information about students' perceptions of e-learning environments.

2. To use the new questionnaire in investigating differences between the perceptions of:
   1. students' actual and preferred e-learning environment;
   2. male and female students; and
   3. teachers and students.

3. To investigate whether students' perceptions of e-learning environments are associated with their enjoyment of e-learning.

Theoretical framework

Few fields of educational research can boast the existence of such a rich array of validated and robust instruments. Learning environment research has provided a useful focus in evaluations of educational innovations (Fisher, Aldridge, Fraser & Wood, 2001; Fraser & Maor, 2000; Maor & Fraser, 1996; Newby & Fisher, 1997; Teh & Fraser, 1995; Zandvliet, 2003) and more recently web based learning (Jegede, Fraser & Fisher, 1995; Taylor & Maor, 2000; Walker, 2002). Past research has found links between classroom environments and student outcomes (Fraser, 1994, 1998a; Goh, Young & Fraser, 1995). A recent study focused on the effectiveness of outcomes focused and technology rich learning environments in promoting student retention, achievement, attitudes and equity (Aldridge, Fraser, Fisher, Trinidad & Wood, 2003; Trinidad, Macnish, Aldridge, Fraser & Wood, 2001). Such research has shown that students' outcomes are likely to be better when the actual learning environment more closely matches their preferred learning environment (Aldridge, Fraser, Fisher, Trinidad & Wood, 2003; Fraser, 1998b, 1999; Fraser & Fisher, 1983).

There are many factors that can influence the learning experience, including the infrastructure, quality of content and assessment, quality of learner support systems, assumptions made by learners and educators about the learning experience itself and peer support networks for learners and educators (Macnish, Trinidad, Fisher & Aldridge, 2003). It is also suggested that, given the emerging nature of e-learning[1], there is a need for research in this discipline to inform teaching and learning development. Ellis and Phelps (2000) consider that discussion of the application of online technologies has not matured. Reeves (1998) also comments on the need for development and empirical research. As Godfrey (2001) states, "to become confident, critical and creative users of ICT educators must have access to professional development programs that enable them to have multiple skills, both in the use of technology and in task design" (p. 16). Therefore educators need, not only the ICT skills, but also the models of best practice and knowledge to support learning grounded in practical learning theory, especially if they are to develop their own materials. They need to understand the rationale for integrating ICT into learning environments and interpreting curriculum documents to make decisions about designing, delivering, managing and evaluating instruction in e-learning environments.

In the race for educational and training institutions to move into e-learning, there has been a tendency to develop e-learning environments around content. This has resulted in a fairly linear approach to the use of technologies. Learning environments must be built on sound learning principles with communication, activities and problem solving (Albon & Trinidad, 2002). Whilst instructors have become more comfortable with producing e-learning materials, encouraging students to absorb information from them, and then testing students to see if they have, there is now a growing movement towards designing e-learning environment that recognises how the communicative powers of the Internet support an active and constructive role for learners (Albon & Trinidad, 2002; Oliver & Omari, 1999; Salmon, 2000). This article reports the findings from research that uses a tool to help educators assess their e-learning environment.

Research methods

Instruments used

In 1996, Fraser, McRobbie and Fisher (1996) began the development of the What is Happening In this Class? (WIHIC) questionnaire, which incorporates a wide range of dimensions (from a range of questionnaires) that are important to the present situation in classrooms and have also shown to be significant predictors of outcomes (Fraser, 1994, 1998a). A similar approach has been used in the development of the Outcomes Based Learning Environment Questionnaire (Aldridge, Laugksch, Fraser, & Seopa, 2004) and the Technology Rich Outcomes Focused Learning Environment Survey (TROFLEI; Aldridge, Dorman & Fraser, in press) in which scales, relevant to the learning environment to be assessed, in addition to new scales developed for a specific purpose, have been combined to create a new instrument. In each case the new instruments have been shown to be valid and reliable.

The Online Learning Environment Survey (OLES) used this technique when it incorporated scales from four existing instruments, namely, the What Is Happening In this Class? (WIHIC; Fraser, Fisher & McRobbie, 1996) questionnaire, the Constructivist Learning Environment Survey (CLES; Taylor, Fraser & Fisher, 1997), the Distance Education Learning Environments Survey (DELES; Jegede, Fraser & Fisher, 1995; Walker, 2002) and the Technology Rich Outcomes Focused Learning Environment Instrument (TROFLEI; Aldridge, Dorman & Fraser, in press; Aldridge, Fraser, Fisher, Trinidad & Woods, 2003).

The selection of scales from different instruments was made with the unique nature and characteristics of e-learning environments in mind and to ensure that the dimensions are consistent with Moos' (1974) scheme for classifying the dimensions of any human environment[2]. Three scales from the WIHIC were selected, namely, Teacher Support, Student Autonomy and Equity. Four scales from the DELES were selected, namely, Authentic Learning, Student Interaction & Collaboration, and Asynchronicity. Finally, one scale from each of the CLES and TROFLEI were selected, namely, Personal Relevance and Computer Usage, respectively.

The original version of the OLES had 62 items and was developed to include two separate response scales, one allowing students to indicate how often they perceive a classroom practice as actually happening, and the other to indicate how often they would prefer that practice to happen. In each case, students responded on a five point frequency scale of Almost Always, Often, Sometimes, Seldom and Almost Never. The Online Learning Environment Survey (OLES) can be viewed on the web at http://www.monochrome.com.au/oles/oles.htm. Table 1 provides a description of each scale included in the final version of the OLES, a sample item, the name of the questionnaire from which the scale originated and the classification of each scale according to Moos (1974) scheme.

Scale	Description The extent to which	Sample Item	Original question- naire	Moos dimension
Computer usage	... students use their computers as a tool to communicate with others and to access information.	I use the computer to obtain information from the Internet.	TROFLEI	System maintenance and system change
Teacher support	... the teacher helps, befriends, trusts and is interested in students.	The teacher gives me valuable feedback on my assignments.	WIHIC	Relationship
Student interaction and collab- oration	... students have opportunities to interact with one another, exchange information and engage in collaboration.	I share information with other students.	DELES	Relationship
Personal relevance	... there is a connection between students' out of school experiences.	I can relate what I learn to my life outside of this class.	CLES	Personal development
Authentic learning	... students have the opportunity to solve real world problems that are authentic.	I work on assignments that deal with real world information.	DELES	Personal development
Student autonomy	... students have opportunities to initiate ideas and make their own learning decisions, and the locus of control is student oriented.	I make decisions about my learning.	DELES	System maintenance and system change
Equity	... students are treated equally by the teacher.	I am treated the same as other students in this class.	WIHIC	Relationship
Asynch- ronicity	... the asynchronous nature of the discussion forum promotes reflective thinking and the posting of messages at times convenient to the students.	I read the posted messages at times that are convenient to me.	AODLE	System maintenance and system change

A second form of the OLES was developed to assess teachers' perceptions of the e-learning environment. This version contains items parallel to the student version and, like the student version, it allows teachers to express their actual and preferred perceptions.

To assess students' satisfaction with their e-learning environment, an Enjoyment scale was adapted from the Test of Science Related Attitudes (Fraser, 1981).

Quantitative data collection

Qualitative data collection

Qualitative data were gathered through online interviews with students and educational material, including examples of online notes, curriculum, etc, logged from the classes using e-learning. Unstructured interviews were used to help to clarify and expand the survey responses in order to provide a more empathetic understanding of the effectiveness of the learning environment. Online interviews via email were conducted with a total of 21 students, 11 from Hong Kong and 10 from Australia. Interviews were also conducted with 7 teachers.

Findings and results

Validity and reliability of the OLES

Factor Loading
Item No	Computer usage		Teacher support		Student interaction and collab- oration		Personal relevance		Authentic learning		Student auton- omy		Equity		Asynch- ronicity
	Act	Pref	Act	Pref	Act	Pref	Act	Pref	Act	Pref	Act	Pref	Act	Pref	Act	Pref
2	0.79	0.79
3	0.78	0.82
4	0.77	0.78
5	0.83	0.73
6	0.84	0.83
7	0.35	0.48
9			0.78	0.88
10			0.76	0.84
11			0.80	0.95
12			0.74	0.87
13			0.77	0.87
14			0.70	0.81
15			0.50	0.72
16			0.72	0.88
17					0.79	0.77
18					0.77	0.70
19					0.79	0.76
20					0.78	0.77
21					0.77	0.75
22					0.77	0.76
23							0.58	0.77
24							0.61	0.58
25							0.68	0.74
26							0.85	0.81
27							0.41	0.49
28									0.40	0.69
29									0.65	0.83
30									0.83	0.85
31									0.80	0.91
32									0.59	0.85
33											0.38	0.45
34											0.50	0.42
35											0.54	0.38
36											0.77	0.75
37											0.70	0.84
38											0.80	0.75
39											0.71	0.72
40											0.65	0.79
41													0.71	0.77
42													0.79	0.89
43													0.79	0.85
44													0.83	0.88
45													0.82	0.87
46													0.84	0.90
47													0.83	0.91
48													0.82	0.89
62															0.67	0.70
63															0.68	0.72
67															0.79	0.76
68															0.77	0.77
69															0.83	0.77
70															0.46	0.53
% Vari- ance	10.03	7.48	6.77	7.89	3.69	3.37	2.65	2.20	5.64	44.95	4.24	2.75	32.83	4.25	3.56	4.53
Factor loadings smaller than 0.30 have been omitted. The sample consisted of 325 students.

To examine whether the items in a scale assess the same construct, the internal consistency reliability was calculated. For both the actual and preferred forms of the OLES, the internal consistency (Cronbach alpha reliability) estimates ranged from 0.86 to 0.96 for that actual version and from 0.89 to 0.96 for the preferred version (Table 3). These estimates were comparable to those found in studies using the same scales (e.g. Aldridge, Fraser, Fisher, Trinidad & Wood, 2003) and, therefore, were considered satisfactory.

Table 3: Internal consistency reliability (Cronbach alpha coefficient), Discriminant validity (mean correlation with other scales) and Ability to differentiate between classrooms (ANOVA results) for the individual as the unit of analysis

Scale	No of items	Alpha reliability		Mean correlation with other scales		ANOVA eta2
		Actual	Preferred	Actual	Preferred	Actual
Computer usage	6	0.89	0.90	0.21	0.38	0.17**
Teacher support	8	0.93	0.96	0.41	0.44	0.10**
Student interaction and collaboration	6	0.93	0.94	0.43	0.54	0.11**
Personal relevance	5	0.86	0.93	0.45	0.56	0.03
Authentic learning	5	0.89	0.95	0.43	0.59	0.05
Student autonomy	8	0.90	0.95	0.39	0.53	0.05
Equity	8	0.96	0.97	0.36	0.59	0.04
Asynchronicity	6	0.87	0.89	0.38	0.43	0.25**
Enjoyment	8	0.96
** p<0.01 The sample consisted of 325 students in 11 groups. The eta2 statistic (which is the ratio of 'between' to 'total' sums of squares) represents the proportion of variance explained by class membership.

The mean correlation of a scale with another scale was used as a convenient index of discriminant validity (see Table 3). For the actual version, the discriminant validity ranged from 0.21 to 0.59 and for the preferred version the discriminant validity ranged from 0.38 to 0.59. These scores indicate that there is a degree of overlap between the scales, but the factor analysis attests to the independence of factor scores on the eight OLES scales.

An analysis of variance (ANOVA) was used to determine the ability of the actual form of each OLES scale to differentiate between the perceptions of students in different groups. The eta2 statistics were calculated to provide an estimate of strength of the association between class membership and the dependent variable (OLES scale). The ANOVA results (see Table 3) indicate that four of the eight scales are able to differentiate significantly between classrooms (p<0.01), namely, Computer Usage, Teacher Support, Student Interaction and Collaboration and Asynchronicity. These findings suggest that students perceive the learning environments of different classrooms differently for these four scales.

Students' perceptions of actual and preferred e-learning environments

Table 4: Average item mean, Average item standard deviation and Difference (Effect size and MANOVA results) between students' actual and preferred scores on the OLES using the individual as the unit of analysis

OLES scale	Average item meana		Average item standard deviation		Difference
	Actual	Preferred	Actual	Preferred	Effect size	F
Computer usage	2.94	3.54	1.21	1.17	0.50	4132**
Instructor support	3.68	4.19	0.89	1.02	0.53	47.07**
Student interaction and collaboration	3.64	3.93	0.96	1.04	0.29	13.21**
Personal relevance	3.37	4.00	0.85	0.98	0.69	75.26**
Authentic Learning	3.46	3.94	0.89	1.06	0.49	39.42**
Student Autonomy	3.80	4.19	0.78	0.93	0.46	34.31**
Equity	3.90	4.21	0.94	1.01	0.32	16.34**
Asynchronicity	3.12	3.47	1.01	1.14	0.33	17.04**
**p<0.01   N=325 students. a Average item mean = Scale mean divided by the number of items in that scale.

Differences in gender perceptions of e-learning environments

Figure 1: Average item mean for students' actual and preferred scores on the OLES

For each analysis, the multivariate test yielded significant results (p<0.01) in terms of Wilks' lambda criterion, indicating that there were sex differences in the set of criterion variables as a whole. Therefore, the univariate ANOVA was interpreted for each scale. In order to estimate the magnitudes of gender differences, effect sizes were calculated as recommended by Thompson (1998, 2001). Because the number of items in each scale differs, the average item mean, or scale score divided by the number of items in that scale, was chosen to provide a meaningful comparison between scales.

For the actual version of the OLES, the results reported in Table 5 suggest that the perceptions of male and female students are statistically significantly (p<0.01) different for three of the eight scales, namely, Teacher Support, Student Interaction and Collaboration and Equity. For each of these scales, females perceive a more positive classroom environment than do males. The effect size for these three scales for which gender differences are statistically significant are approximately one quarter of a standard deviation (0.24-0.27). These effect sizes suggest a notable difference between male and female perceptions of the actual learning environment.

Table 5: Average item mean, Average item standard deviation and Difference (Effect size and MANOVA result) between male and female students' actual and preferred scores on the OLES using the individual as the unit of analysis

OLES scale	Form	Average item meana		Average item standard dev		Difference
		Male	Female	Male	Female	Effect size	F
Computer usage	Actual	2.96	2.92	1.26	1.16	0.03	0.09
	Preferred	3.56	3.52	1.23	1.12	0.03	0.09
Teacher support	Actual	3.57	3.78	0.98	0.78	-0.24	4.46*
	Preferred	4.06	4.32	1.12	0.88	-0.26	5.42*
Student interaction and collaboration	Actual	3.51	3.77	1.06	0.84	-0.27	5.86*
	Preferred	3.82	4.04	1.12	0.95	-0.21	3.56
Personal relevance	Actual	3.34	3.41	0.91	0.78	-0.08	0.48
	Preferred	3.92	4.08	1.06	0.88	-0.16	2.28
Authentic learning	Actual	3.48	3.43	0.91	0.87	0.06	0.26
	Preferred	3.90	3.98	1.15	0.96	-0.08	0.49
Student autonomy	Actual	3.73	3.87	0.86	0.68	-0.18	2.42
	Preferred	4.09	4.30	1.05	0.78	-0.23	4.08*
Equity	Actual	3.80	4.01	1.01	0.85	-0.23	4.29*
	Preferred	4.04	4.39	1.11	0.87	-0.35	9.88**
Asynchronicity	Actual	3.08	3.16	1.08	0.93	-0.08	0.62
	Preferred	3.48	3.45	1.20	1.07	0.03	0.08
*p<0.05 **p<0.01     N=164 male students and 161 female students a Average item mean = Scale mean divided by the number of items in that scale.

Table 5 and Figure 2 indicate that females would prefer a learning environment that includes significantly more (p<0.05) Teacher Support and Equity than their male counterparts. For the remaining six scales, gender differences are not statistically significant. The effect sizes for the two preferred scales, for which gender differences are statistically significant, are approximately one quarter (0.23) and one third (0.35) standard deviation for Teacher Support and Equity respectively. With the exception of these two scales, it would appear that male and female students prefer similar e-learning environments. Overall, the results suggest that girls perceive a more positive classroom environment than boys, thus replicating earlier studies (Fisher, Henderson, & Fraser, 1997; Henderson, Fisher, & Fraser, 2000; Wong & Fraser, 1996).

Figure 2: Average item mean for male and female students' actual and preferred scores on the OLES

Differences between teachers' and students' perceptions of the e-learning environment

T tests for paired samples, using the class means as the unit of analysis, were used to investigate whether teachers and students had different perceptions of their classroom environment. The results (reported in Table 6) indicated that teachers perceived statistically significantly (p<0.05) more of six of the eight OLES scales than their students (the exceptions being Student Interaction and Collaboration and Student Autonomy). Overall, the results reported in Table 6 and Figure 3 suggest that e-learning environments are perceived more favourably by teachers than by students. The effect sizes (reported in Table 6) for the six scales for which there was a statistically significant difference range between approximately three quarters of a standard deviation (0.73) and two and two thirds standard deviations (2.65). These effect sizes suggest a notable difference between teachers' and students' perceptions.

To try to explain these differences, qualitative information was collected and analysed. For example, in the case of the Teacher Support scale, interviews suggested that some teachers made assumptions about their students' knowledge about how they should use the e-learning environment and how much support they needed to use it effectively. Most of the teachers were confident that their students were able to work independently, expecting them to work on materials after class. However, one teacher stated that "students really do expect to be spoon fed and they are not very self directed in their own learning"[T03-06].

Table 6: Average item mean, Average item standard deviation and Difference (effect size and paired t test result) between teachers' students' scores on the OLES using the class mean as the unit of analysis

OLES scale	Average item meana		Average item standard dev		Difference
	Students	Teachers	Students	Teachers	Effect size	t
Computer usage	3.35	3.98	0.77	0.96	0.73	2.86*
Teacher support	3.84	4.66	0.36	0.26	2.65	5.03**
Student interaction and collaboration	3.81	4.21	0.36	0.38	1.08	2.01
Personal relevance	3.47	3.83	0.22	0.21	1.67	3.63*
Authentic learning	3.55	4.03	0.31	0.53	1.14	2.75*
Student autonomy	3.88	3.91	0.20	0.53	0.08	0.18
Equity	3.97	4.37	0.24	0.38	1.29	3.00*
Asynchronicity	3.44	3.86	0.32	0.24	1.50	6.80**
*p<0.05 **p<0.01     N=325 students. N=7 teachers a Average item mean = scale mean divided by the number of items in that scale.

In contrast, many of the students whom were interviewed felt that they needed more guidance from the teachers. One such student stated that "the teacher does not explain much before I got used to it. Overall speaking, there is not enough support from the teachers during the time when I am using e-learning in the classroom or at home. But I managed to learn by myself" [S03-05]. Two other students explained that "They encourage us to use e-learning, but do not provide enough training to fully utilise its features" [S03-09] and "We used the forum only in class when the teacher asked us to do so. As we were already in the same classroom at that time, the forum did not help much in interaction. It might have been better if we had more time to discuss issues face to face in class. The forum and e-learning work best when students cannot be in the same room" [S03-021].

Such comments illustrate that the expectations of students and teachers when using e-learning can be quite different. Through such interviews, it was possible to discuss ways in which the teachers could improve the use and support during e-learning.

Figure 3: Average item mean for students' and teachers' actual perceptions on the OLES

Associations between student perceptions of the learning environment and their enjoyment of e-learning

The multiple correlation (R) reported in Table 7 for the set of eight OLES scales was statistically significant (p<0.01) and was 0.56. This value of the multiple correlations replicate the findings of the past studies, cited by Fraser (1998a), that indicate associations between student attitudes and dimensions of the classroom environment.

To identify which classroom environment scales contribute most to the variance in student satisfaction, the standardised regression weights (beta) were examined (see Table 7). Five of the OLES scales (Computer Usage, Teacher Support, Authentic Learning, Student Autonomy and Asynchronicity) are significantly (p<0.05), positively and independently related to student enjoyment. These results suggest that more emphasis on these five scales in e-learning environments is linked with learners enjoying their online experiences.

Scale	Enjoyment-environment association (N = 325 students)
	r	b
Computer usage	0.27**	0.13*
Teacher support	0.20**	0.15*
Student interaction and collaboration	0.26**	0.00
Personal relevance	0.32**	0.02
Authentic learning	0.40**	0.14*
Student autonomy	0.39**	0.26**
Equity	0.36**	0.11
Asynchronicity	0.43**	0.26**
Multiple correlation (R)		0.56**
*p<0.05 **p<0.01

The students interviewed were, on the whole, positive about the impact of their e-learning environment, summing up their experiences with statements such as "I like the characteristics of easy accessibility and a boundary free environment [as it provides] rich information sources and 'synchronised' interactions in e-learning" [S03-07]. Another student expressed that there were more opportunities to share work and ideas when he said, "I can see that, with the use of this, we've got a place that we could share the references that we've found, the ideas that we had in mind and the experience that we had"[S03-010].

Discussion

For this study, the OLES was administered to 325 students and seven instructors in 11 e-learning groups at the university and secondary level in Hong Kong and Australia. Qualitative and quantitative data were gathered within the unique e-learning environments that are developing in these countries.

As a first step, the data collected from 325 students were analysed in various ways to support the reliability and validity of actual and preferred versions of the OLES. Principal axis factor analysis with oblique rotation confirmed a refined structure, comprising 52 items in eight scales. The internal consistency (Cronbach alpha reliability) estimates for the actual version of OLES ranged from 0.86 to 0.96 for the actual version and from 0.89 to 0.96 for the preferred version. When the mean correlation of a scale with other scales was used as a convenient index of discriminant validity, the values for the discriminant validity could be regarded as small enough to confirm that each scale generally measures distinct aspects of the e-learning environment. Also, four of the eight scales of the actual form of the OLES were able to differentiate between the perceptions of students in different e-learning groups.

As a second step, data collected using the OLES were used to explore differences between the perceptions of: students' actual and preferred e-learning environment; male and female students; and teachers and students. In some cases, the qualitative data were used to explain the differences found from analyses of questionnaire data.

To examine whether differences exist between students perceptions of the actual and their preferred learning environments, MANOVA for repeated measures was used. The results indicate a statistically significant difference (p<0.01) between students' actual and the preferred scores for each of the eight learning environment scales. The effect sizes range between approximately one third of a standard deviation (0.32) and two thirds of a standard deviation (0.69), thus suggesting that there is an educationally important difference between students' perceptions of the actual and the preferred environment.

The study also involved an investigation of gender differences in students' perceptions of the actual and preferred learning environment. Gender differences were explored using a one way MANOVA with the set of OLES scales as the dependent variables and gender as independent variable. It was found that females perceive statistically significantly more (p<0.05) actual Instructor Support, Student Interaction and Collaboration and Equity and more preferred Instructor Support and Equity than their male counterparts. However, with the exception of the two OLES scales of Instructor Support and Equity, male and females would prefer similar e-learning environments.

A further objective was to examine whether instructors and students perceive the e-learning environment differently. The results indicate that instructors perceive statistically significantly (p<0.05) more favourable e-learning environments than do their students for six of the eight OLES scales (the exceptions being Student Interaction and Collaboration and Student Autonomy). The effect sizes for the six scales for which there was a statistically significant difference range between approximately three quarters of a standard deviation (0.73) and two and two thirds standard deviations (2.65). These findings replicate past research that compares teachers' and students' perceptions of the learning environment (Fraser, 1998a). Qualitative data were collected to help to explain the differences between student and teacher perceptions. It was found that, generally, teachers and students had different expectations, which led to the contradiction in perceptions.

A final objective of the study was to examine whether associations exist between the e-learning environment perceived by the students and their enjoyment of e-learning. Simple and multiple correlation analyses were conducted to investigate associations between student enjoyment and the eight OLES scales. The results of the simple correlation analysis indicated that all eight of the OLES scales were statistically significantly (p<0.01) associated with student enjoyment of their e-learning at the individual level of analysis. The multiple correlation for the set of eight OLES scales was statistically significant (p<0.01). The standardised regression weights suggest that five of the eight OLES scales, namely, Computer Usage, Instructor Support, Authentic Learning, Student Autonomy and Asynchronicity, were significantly (p<0.05), positively and independently related to student enjoyment. These patterns of associations are generally consistent with past research (Fraser, 1998a).

Conclusion

Because the OLES is administered online, it has the ability to provide users with data that depict the actual and preferred learning environments of students and teachers immediately, giving instant and potentially valuable feedback to instructors working in these environments. Such data can then used to support open dialogue between the teacher and students to determine ways in which they might work together to guide educational decision making to improve their e-learning environment.

Also, the present study is significant as it illustrates how learning environment research tools, such as the OLES, can help in evaluating the effectiveness of e-learning environments. Past studies have found links between students' perceptions of the psychosocial characteristics of their learning environments and their learning outcomes (Fraser, 1998a). The present study is important in that it used the OLES to explore ways in which educators can make improvements to their e-learning environments based upon their students' perceptions, thereby enhancing student outcomes.

References

Aldridge, J., Fraser, B., Fisher, D., Trinidad, S. & Wood, D. (2003). Monitoring the success of an outcomes-based, technology-rich learning environment. Paper presented at the annual meeting of the American Educational Research Association, April, Chicago, IL.

Aldridge, J. M., Laugksch, R. C., Fraser, B. J. & Seopa, M. A. (2004). Development of an instrument for monitoring the success of outcomes-based learning environments in classrooms in South Africa. In C. C. Sunal & K. Mutua (Eds), Research on Education in Africa, the Caribbean and the Middle East: An Historical Overview. Greenwich, CA: Information Age Publisher.

Anderson, G. L. & Walberg, H. J. (1968). Classroom climate group learning. International Journal of Educational Sciences, 2, 175-180.

Coakes, S. J. & Steed, L. G. (2001). SPSS: Analysis without anguish. Sydney, Australia: John Wiley.

Ellis, A. & Phelps, R. (2000). Staff development for online delivery: A collaborative team based action learning model. Australian Journal of Educational Technology, 16, 26-44. http://www.ascilite.org.au/ajet/ajet16/ellis.html

Erickson, F. (1998). Qualitative research methods for science education. In B. J. Fraser & K. G. Tobin (Eds), The international handbook of science education (pp. 1155-1173). Dordrecht, The Netherlands: Kluwer Academic Publishers.

Fisher, D., Aldridge, J., Fraser, B. & Wood, D. (2001). Development, validation and use of a questionnaire to assess students' perceptions of outcomes-focused, technology-rich learning environments. Paper presented at the annual conference of the Australian Association for Research in Education, December, Perth, Western Australia. http://www.aare.edu.au/01pap/fis01028.htm

Fisher, D., Henderson, D. & Fraser, B. (1997). Laboratory environments and student outcomes in senior high school biology. American Biology Teacher, 59(2), 14-19.

Fraser, B. J. (1981). Tests of Science-Related Attitudes (TOSRA). Melbourne: Australian Council for Educational Research.

Fraser, B. J. (1994). Research on classroom and school climate. In D. Gabel (Ed), Handbook of research on science teaching and learning (pp. 493-541). New York: Macmillan.

Fraser, B. J. (1998a). Science learning environments: Assessment, effects and determinants. In B. Fraser & K. Tobin (Eds), International handbook of science education (pp. 527-564). Dordrecht, The Netherlands: Kluwer.

Fraser, B. J. (1998b). Classroom environment instruments: Development, validity and applications. Learning Environment Research: An International Journal, 1, 7-33.

Fraser, B. J. (1999). Using learning environment assessments to improve classroom and school climates. In H. J. Freiberg (Ed.), School climate: Measuring, improving and sustaining healthy learning environments (pp. 65-83). London: Falmer Press.

Fraser, B. J. (2002). Learning environment research: Yesterday, today and tomorrow. In S. C. Goh & M. S Khine (Eds), Studies in educational learning environments: An international perspective (pp. 1-26). Singapore: World Scientific.

Fraser, B. J. & Fisher, D. (1983). Student achievement as a function of person-environment fit: A regression surface analysis. British Journal of Educational Psychology, 53, 89-99.

Fraser, B., Fisher, D. & McRobbie, C. (1996). Development, validation and use of personal and class forms of a new classroom environment instrument. Paper presented at the annual meeting of the American Educational Research Association, April, New York.

Fraser, B. J. & Maor, D. (2000). A learning environment instrument for evaluating students' and teachers' perceptions of constructivist multimedia learning environments. Paper presented at the annual meeting of the National Association for Research in Science Teaching, April, New Orleans, LA.

Godfrey, C. (2001). Computer technologies: Scaffolding tools for teaching and learning. Australian Educational Computing, 16(2), 27-29. http://www.acce.edu.au/journal/journals/vol16_2.pdf

Goh, S., Young, D. & Fraser, B. J. (1995). Psychosocial climate and student outcomes in elementary mathematics classrooms: A multilevel analysis. The Journal of Experimental Education, 43, 90-93.

Henderson, D., Fisher, D.L. & Fraser, B.J. (2000). Interpersonal behaviour, laboratory learning environments, and student outcomes in senior biology classes. Journal of Research in Science Teaching, 37, 26-43.

Jegede, O., Fraser, B. & Fisher, D. (1995). The development and validation of a distance and open learning environment scale. Educational Technology Research and Development, 43, 90-93.

Macnish, J., Trinidad, S., Fisher, D., & Aldridge, J. (2003). The online learning environment of a technology-rich secondary college. Paper presented at the annual meeting of the American Educational Research Association, April, Chicago, IL.

Maor, D. & Fraser, B. J. (1996). Use of classroom environment perceptions in evaluating inquiry-based computer assisted learning. International Journal of Science Education, 18, 401-421.

Moos, R. H. (1974). The Social Climate Scales: An overview. Palo Alto, CA: Consulting Psychologists Press.

Newby, M. & Fisher, D. (1997). An instrument for assessing the learning environment of a computer laboratory. Journal of Educational Computing Research, 16, 179-190.

Oliver, R. & Omari, A. (1999). Using online technologies to support problem based learning: Learners responses and perceptions. Australian Journal of Educational Technology, 15, 58-79. http://www.ascilite.org.au/ajet/ajet15/oliver.html

Reeves, T. (1998). The impact of media and technology in schools. A Research Report prepared for the Bertelsmann Foundation.

Salmon, G. (2000). E-moderating: The key to teaching and learning online. London: Kogan Page.

Taylor, P. C., Fraser, B. J. & Fisher, D. L. (1997). Monitoring constructivist classroom learning environments. International Journal of Educational Research, 27, 293-302.

Taylor, P. & Maor, D. (2000). Assessing the efficacy of online teaching with the Constructivist On-Line Learning Environment Survey. In A. Herrmann and M.M. Kulski (Eds.), Flexible futures in tertiary teaching. Proceedings 9th Annual Teaching Learning Forum, 2-4 February 2000. Perth: Curtin University of Technology. http://lsn.curtin.edu.au/tlf/tlf2000/taylor.html

Teh, G. P. L., & Fraser, B. J. (1995). Development and validation of an instrument for assessing the psychosocial environment of computer-assisted learning classrooms. Journal of Educational Computing Research, 12, 177-193.

Thompson, B. (1998). Review of 'what if there were no significance tests?' Educational and Psychological Measurement, 58, 334-346.

Thompson, B. (2001). Significance, effect sizes, stepwise methods and other issues: Strong arguments move the field. Journal of Experimental Education, 7, 80-93.

Tobin, K. & Fraser, B. (1998). Qualitative and quantitative landscapes of classroom learning environments. In B. J. Fraser & K. G. Tobin (Eds.), The international handbook of science education (pp. 623-640). Dordrecht, The Netherlands: Kluwer.

Trinidad, S., Macnish, J., Aldridge, J., Fraser, B. & Wood, D. (2001). Integrating ICT into the learning environment at Sevenoaks Senior College: How teachers and students use educational technology in teaching and learning. Paper presented at the annual conference of the Australian Association for Research in Education, Perth, December. http://www.aare.edu.au/01pap/ald01027.htm

Walberg, H. J. (Ed). (1979). Educational environments and effects: Evaluation, policy and productivity. Berkeley, CA: McCutchan.

Walker, S. (2002). Insight: Distance education learning environments survey. [viewed 10 Jan 2003, verified 15 Jan 2005] http://insight.southcentralrtec.org/ilib/delesa/delesainfo.html

Wong, A. F. L. & Fraser, B. J. (1996). Environment-attitude associations in the chemistry laboratory classroom. Research in Science and Technological Education, 64, 29-40.

Zandvliet, D. (2003). Learning environments in Malaysian "Smart School" classrooms. Paper presented at the annual meeting of the American Educational Research Association, April, Chicago.

Endnote

1. Electronic learning or e-learning, as defined by Jackson (2002) Defining eLearning - Different Shades of "Online", can be technology enhanced learning and/or technology delivered learning. Both dimensions describe e-learning for the purpose of this paper.

2. Moos identified three basic dimensions including: the Relationship Dimension, which measures the nature and intensity of personal relationships; the Personal Development Dimension, which measures the directions in which personal growth and self enhancement occur; and the System Maintenance and System Change Dimension, which measures the extent to which the environment maintains clear objectives and control and responds to change.

Authors: Suzanne Trinidad Faculty of Education, Languages and Social Work Curtin University of Technology GPO Box U1987, Perth, 6845, Western Australia Email: S.Trinidad@curtin.edu.au Tel: +61 8 9266 2552 Fax: +61 8 9266 2547 Jill M. Aldridge Science and Mathematics Education Centre Curtin University of Technology GPO Box U1987, Perth, 6845, Western Australia Email: J.Aldridge@curtin.edu.au Tel: +61 8 9266 3211 Fax: +61 8 9266 2503 Barry J. Fraser Science and Mathematics Education Centre Curtin University of Technology GPO Box U1987, Perth, 6845, Western Australia Email: B.Fraser@curtin.edu.au Tel: +61 8 9266 7896 Fax: +61 8 9266 2503 Please cite as: Trinidad, S., Aldridge, J. and Fraser, B. (2005). Development, validation and use of the Online Learning Environment Survey. Australasian Journal of Educational Technology, 21(1), 60-81. http://www.ascilite.org.au/ajet/ajet21/trinidad.html