An evaluation of computer assisted learning in geography in Singapore

Barry J. Fraser
Curtin University of Technology, Perth

Six computer assisted learning (CAL) modules in geography were developed by the investigators and evaluated in the unique milieu of Singapore schools in terms of impact on achievement, attitudes and classroom psychosocial environment. A sample of 671 students from the second year of high school were assigned randomly to either a CAL or control group. In contrast to past research, effect sizes were relatively large and ranged from 1.0 to 3.5 standard deviations. Compared with control students, CAL students had higher achievement and attitude scores and perceived their classes as having greater gender equity, investigation, innovation and resource adequacy.

This evaluation of computer assisted learning (CAL) courseware is distinctive in two important ways. First, the research was conducted in the unique milieu of the Singapore school system. Second, in addition to investigation of the impact of CAL on traditional achievement and attitude outcomes, the present evaluation broke new ground in that it also involved the development and use of an instrument which assessed the psychosocial environment in CAL classrooms.

The Singapore education system is different from that of many other countries in that it is highly centralised (with almost every school using the same prescribed textbooks), the mode of instruction is essentially expository and achievement-oriented, and schools are graded for excellence based solely on the students' academic performance. Educational applications of computers were introduced fairly extensively into Singaporean schools only in 1980, with the focus initially on computer science and low-level computer literacy courses at the secondary school level. Computer assisted learning (CAL) was introduced to the schools in 1986, mainly for remediation and enrichment, but no Singaporean school develops its own CAL courseware.

In Singapore, there is a dearth of research on the effects of CAL. Moreover, the only three research studies that have been conducted (Ong & Lee-Leck, 1986; Low, 1988; Woo-Tan, 1989) involved small and non-random samples, a short duration of the experimental treatment, and a narrow scope of the content selected. Because of the lack of dependable research information, and in view of the potential that research conducted in the unique educational context of Singapore has for our understanding of computer assisted learning, the present study was conceptualised and conducted.

The main purposes of the present study were to develop six CAL modules on the topic of decision-making in geography, and to evaluate this innovation in terms of its impact upon student achievement, student attitudes and the classroom environment.

The CAL and traditional teaching approaches

The CAL approach consisted of a sequence of instructions consisting of learning activities for the concept of decision-making, followed by a prescribed set of exercises. During the CAL lessons, students were presented with the necessary information. Students also were guided with instructions on the computer screens, were provided with practice exercises, and had their learning assessed. Feedback was provided contingent upon correct and incorrect responding.

In order to enhance the validity of conclusions from the evaluative study, a control group, which studied the same topic via the direct expository teaching methods common in Singapore, also was included in the study. The traditional expository method used by the control group was similar to the CAL approach in that it consisted of six modules (lasting six weeks) of two lessons per module. In terms of curriculum time, each module covered two 35-minute lessons. Thus, the CAL and control groups learned the same content within the same time span. The same teacher taught the lessons for both groups. Furthermore, similar sets of exercises and worksheets were given to both groups in order to enhance internal validity. The exercises for the CAL group were stored in the computer, whereas the exercises for the control group were given out as worksheets.

The traditional method consisted of a sequence of instructions with the teacher using the direct expository teaching method. The teacher initiated, directed and controlled the learning. Teaching resources such as slides, charts, diagrams and transparencies were used. Students were provided with worked examples and solutions displayed on the overhead projector. Students then were required to complete exercises in the worksheets. Any support for the students in their understanding of what they were doing came from the teacher, who had provided the initial explanation and the worksheets which served to reinforce the topics taught. Interactions within the class were generally between the teacher and the class as a whole, with limited one-to-one interaction between teacher and student, or between student and student.

Method

Sample

Geography Achievement test (GAT) and Semantic Differential Inventory (SDI)

Geography Classroom Environment Inventory (GCEI)

The final version of the new instrument, the Geography Classroom Environment Inventory (GCEI), has four Likert-type scales. The response alternatives for each item are 'almost never', 'seldom', 'sometimes', 'often' and 'very often'. The initial GCEI instrument contained the eight scales of teacher concern, participation, gender equity, investigation, innovation, differentiation, organisation and resource adequacy; but four scales (namely, teacher concern, participation, differentiation, and organisation) were omitted after item and factor analyses. Each scale was selected because of its relevance to the unique environment of CAL classes (Teh & Fraser, 1993). For example, gender equity was included because a decade of research in the CAL environment has shown that the use of computers maintained and exaggerated inequities (Sutton, 1991), that equity issues are complex (Schubert, 1986, Sutton, 1991), and that there are gender differences in achievement and attitudes towards computer usage (Hattie & Fitzgerald, 1987; Sutton, 1991). Although a strong tradition in CAL research at all school levels has been to investigate the effectiveness of the usage of CAL in a specific cognate area (Bangert-Drowns, Kulik & Kulik, 1985; Roblyer, Castine & King, 1988; Kulik & Kulik, 1991), past CAL research seldom has examined the potential of computer usage in facilitating and promoting student investigation in the classroom. Investigation therefore was another dimension that was incorporated into the GCEI instrument.

Eight items are contained in the final version of each scale. Typical items contained in the GCEI are "The teacher pays more attention to boys' questions than to girls' questions" (gender equity), "Students carry out investigations to answer questions coming from class discussions" (investigation), "New and different ways of teaching are used in this class" (innovation), and "There are enough computer programs available for our lessons" (resource adequacy). The scoring direction is reversed for almost half of the 32 items in the GCEI.

In developing the GCEI, cognisance was taken of Moos's three general dimensions as they apply to all human environments (Moos, 1974). These three general dimensions are relationship dimensions (the nature and intensity of personal relationships within the environment), personal development dimensions (the basic directions along which personal growth and self-enhancement tend to occur) and system maintenance and system change dimensions (the extent to which the environment is orderly, clear in expectations, maintains control and is responsive to change). Table 1 shows that the four scales in the final version of the GCEI provide reasonable coverage of the three different basic types of dimensions proposed by Moos.

Table 1. Descriptive information for GCEI scales
Scale name	Description	Moos's classification
Gender equity	Extent to which boys and girls are treated equally by the teacher	Relationship
Investigation	Extent to which the skills and processes of inquiry are used in problem-solving and investigation	Personal development
Innovation	Extent to which the teacher plans new and varying activities and techniques, and encourages students to think creatively	System maintenance
Resource adequacy	Extent to which the computer hardware and software are adequate	System maintenance

The development of the GCEI also followed the following steps. First, a comprehensive review of the literature on computer assisted learning was undertaken to identify scales considered important in this unique environment (Plomp & Pelgrum, 1991). Second, extensive interviewing of teachers and students ensured that the GCEI's initial scales and individual items were considered salient by teachers and students. Third, several computer education experts vetted scales and items and agreed that the constructs were relevant. Fourth, following extensive field testing, item and factor analyses were used to refine the original 80-item version with 8 scales to form the final 32-item version with 4 scales (Teh & Fraser, 1993).

The Appendix contains a copy of the final version of the GCEI. The first, second, third and fourth item in each block in the Appendix assesses, respectively, gender equity, investigation, innovation and resource adequacy. The number circled represents the score allocated except for items with R in the for teacher's use column; these items are scored in the reverse manner. Omitted or invalidly answered items are scored 3.

Validation statistics for the GCEI

Table 2. Internal consistency reliability (alpha coefficient), discriminant validity (scale intercorrelations), and ANOVA results (F and [eta squared]) for class membership differences for each scale in the modified version of GCEI for the experimental group.
Scale	Number of items	Alpha reliability	Scale intercorrelations				ANOVA results
			GE	IV	IN	RA	F
Gender equity	8	0.67		0.01	0.22	0.34	17.46**	0.38
Investigation	8	0.65			0.45	0.10	49.30**	0.64
Innovation	8	0.52				0.20	42.12**	0.60
Resource adequacy	8	0.68					31.27**	0.53
** p<0.01

Table 2 also provides information about each scale's ability to differentiate between the perceptions of students in different classrooms. These results were obtained by performing for each scale a one-way ANOVA, with class membership as the main effect and using the individual as the unit of analysis. Results of these analyses reported in Table 2 indicate that each of the four scales differentiated significantly (p<0.01) between the perceptions of students in different classrooms. The statistic, which represents the proportion of variance in environment scores accounted for by class membership, ranged from 0.38 to 0.64 for the various scales. The results for the control group were fairly similar.

Impact of CAL on achievement, attitudes and classroom environment

To investigate differences in students' scores on the achievement, attitude and classroom environment measures between the experimental (computer) and control (non-computer) group, ANCOVA procedures were computed separately for each of the six outcome measures. Three different measures (namely, students' national primary school leaving examination scores, their semestral assessment scores and their pretest scores) were used as covariates. A summary of the results for the significance of differences between the computer and non-computer groups are reported in Table 3. The effect size (ie., the number of standard deviations of difference between the experimental CAL group and the control group) for each outcome also is reported in Table 3.

Table 3. A comparison of experimental and control groups on achievement, attitude and classroom environment outcomes
Scale		Number of items	F	Effect size [1]
Achievement		30	5824.59**	3.5
Attitude		20	630.13**	1.4
Classroom Environment
	- Gender equity - Investigation - Innovation - Resource adequacy	8 8 8 8	219.66** 809.14** 703.63** 504.28**	1.0 1.9 1.7 1.5
** p<0.01 [1] The effect size is the difference between the means of the experimental and control groups divided by the standard deviations for the control group.

Table 3 shows that the present evaluation of the use of computer assisted learning revealed statistically significant differences (p<0.01) for all six outcome measures. A massive effect size of 3.5 standard deviations (of difference between the experimental CAL group and the control group) emerged for the achievement outcome and an effect size of 1.4 standard deviations was found for the attitude measure. These differences favoured the CAL group.

Also significant differences emerged in the students' perceptions between the computer and non-computer group for all of the four GCEI scales, namely, gender equity, investigation, innovation and resource adequacy. Table 3 reveals that the use of computer assisted learning was associated with an effect size of 1.0 standard deviations for gender equity, 1.9 standard deviations for investigation, 1.7 standard deviations for innovation and 1.5 standard deviations for resource adequacy. These differences favoured the CAL group in every case. Relative to control classes, the CAL classes were perceived to have greater gender equity, investigation, innovation and resource adequacy. The present study's findings in the context of computer assisted learning are consistent with past studies on non-CAL classroom environments (Walberg, 1975; Fraser, 1979, 1981, 1986) in which learning environment measures have proved useful in curriculum evaluation.

Discussion

This study suggests that appropriately-designed CAL courseware can be an effective instructional method in the classroom milieu, and that the educational application of appropriately-designed CAL courseware in social science classrooms has the potential to be efficacious. Hopefully, this study will serve as a catalyst for further research into the use of CAL courseware in the social science classroom.

No systematic attempts previously have been made to examine the effects of CAL on learning environment characteristics. Little has been studied or is known about the impact of microcomputers on learning climates in education (Ellett, 1986). This study responds to the plea by Ellett (1986) and Lancy (1987) to fill the lacunae represented by the study of computers and their impact on students. This study is significant because, in contrast to previous research, it uses courseware developed by the researchers in investigating computer learning environments in schools.

One of the study's major contributions is that a new classroom environment instrument has been developed and validated specifically for the unique setting of computer assisted learning. The scales in this instrument display adequate factorial validity, internal consistency reliability, discriminant validity and predictive validity (in terms of being significantly related to student outcome scores). Also each scale differentiates significantly between the perceptions of students in different classrooms. It is likely that other researchers will find this new instrument useful in future studies of CAL classroom environments.

The major finding was that, in contrast to past research, the use of CAI in this study led to a large impact in terms of achievement (effect size of 3.5 standard deviations), attitudes (1.4 standard deviations) and classroom environment (ranging from 1.0 to 1.9 standard deviations). The large effect sizes arising from this study could be attributed to the peculiar situation inherent in the Singapore education system. That is, the highly meritocratic, technologically-biased, centrally-controlled and achievement-oriented system might produce students who perform well. The large effect sizes seem to suggest that appropriate computer-based teaching can be effective with slow learners (ie., the Normal students), a finding which is consistent with the meta-analysis of CAL effectiveness reviewed by Kulik and Kulik (1991). However, there is a need for replication because of the uniqueness of the Singapore milieu and because results for slow learners might not be generalisable to other students.

The specific content area for this study, namely, the topic of decision making in geography, was selected because practitioners (the teachers and principals) felt that this topic was the hardest to teach to slow learners. Furthermore, in accordance with the scheme of work prepared by schools, this topic would have to be taught at the time when the study was implemented. Because of this purposive and specific content area selected and its limited scope, the results of this study should be generalised with caution to other geography topics at the Secondary Two level. The findings of this study reflect the effectiveness of the present topic on decision making in geography; other CAL programs implemented under different conditions or using different geographical topics might not lead to the same results.

There is considerable scope to make use of the new instrument for computer assisted learning environments in replicating the present evaluation of innovations in CAL, as well as in investigations of the effects of CAL classroom environments on student outcomes. In addition, it is hoped that researchers will make use of the new questionnaire specifically in CAL settings in pursuing some of the other lines of classroom environment research reviewed by Fraser (1994). These include the use of classroom environment research in school psychology (Burden & Fraser, in press), person-environment fit investigations of whether students achieve better in their preferred classroom environment (Fraser & Fisher, 1983), practical attempts to improve classroom setting (Fraser & Fisher, 1986), studies of links between classroom-level and school-level climate (Fisher & Fraser, 1993) and research which combines qualitative and quantitative methods in the study of learning environments (Fraser & Tobin, 1991).

References

Burden, R. L. & Fraser B. J. (in press). Use of classroom environment assessments in school psychology: A British perspective. Psychology in the Schools.

Ellett, C. D. (1986). Conceptualizing the study of learning environments. In B. J. Fraser (Ed.), The Study of Learning Environments, (pp.34-40). Salem, OR: Assessment Research.

Fisher, D. L. & Fraser, B. J. (1993). Interpersonal teacher behavior style and school environment. In Th. Wubbels & J. Levy (Eds.), Do You Know What You Look Like: Interpersonal Relationships in Education, (pp. 103-112). London: Falmer Press.

Fraser, B. J. (1979). Evaluation of a science-based curriculum. In H. J. Walberg (Ed.), Educational Environments and Effects: Evaluation, Policy, and Productivity, (pp. 218-234). Berkeley, CA: McCutchan.

Fraser, B. J. (1981). Learning Environment in Curriculum Evaluation: A Review. Evaluation in education: An international review series. Oxford, England: Pergamon Press.

Fraser, B. J. (1986). Classroom Environment. London: Croom Helm.

Fraser, B. J. (1994). Research on classroom and school climate. In D. Gabel (Ed.), Handbook of Research on Science Teaching and Learning, (pp. 493-541). New York: Macmillan.

Fraser, B. J. & Fisher, D. L. (1983). Student achievement as a function of person-environment fit: A regression surface analysis. British Journal of Educational Psychology, 53, 89-99.

Fraser, B. J. & Fisher, D. L. (1986). Using short forms of classroom climate instruments to assess and improve classroom psychosocial environment. Journal of Research in Science Teaching, 23, 387-413.

Fraser, B. J. & Tobin, K. (1991). Combining qualitative and quantitative methods in classroom environment research. In B. J. Fraser and H. J. Walberg (Eds.), Educational Environments: Evaluation, Antecedents and Consequences, (pp. 271-292). Oxford, England: Pergamon Press.

Fraser, B. J. & Walberg, H. J. (Eds.) (1991). Educational Environments: Evaluation, Antecedents and Consequences. Oxford, England: Pergamon Press.

Fraser, B. J., Walberg, H. J., Welch, W. W. & Hattie, J. A. (1987). Syntheses of educational productivity research. International Journal of Educational Research, 11, 145-252. (whole issue)

Hattie, J. & Fitzgerald, D. (1987). Sex differences in attitudes, achievement and use of computers. Australian Journal of Education, 31(1), 3-26.

Kulik, C. C. & Kulik, J. A. (1991). Effectiveness of computer-based instruction: An updated analysis. Computers in Human Behavior, 7(1-2), 75-94.

Lancy, D. F. (1987). The message is the medium: Studies of computer learning environments in schools. In B. J. Fraser (Ed.), The Study of Learning Environments, (Vol. 2, pp. 64-71). Perth: Curtin University.

Low, K. G. (1988). Investigation of various modes of computer-based learning for secondary schools. Singapore Journal of Education, 9(2), 27-32.

Moos, R. H. (1974). The Social Climate Scales: An Overview. Palo Alto, CA: Consulting Psychologists Press.

Moos, R. H. & Trickett, E. J. (1987). Classroom Environment Scale Manual, (2nd ed.). Palo Alto, CA: Consulting Psychologists Press.

Ong, S. T. & Lee-Leck, M. K. (1986). The effects of computer assisted instruction on attitudes and achievement in mathematics on preservice primary school teachers. Selected Proceedings of Educomp '86: Impact of Microcomputers in Education - Issues and Techniques, (pp. 196-201). Singapore: Ngee Ann Polytechnic.

Plomp, T. & Pelgrum, W. J. (1991). Introduction of computers in education: State of the art in eight countries. Computers and Education, 17(3), 249-258.

Roblyer, M. D., Castine, W. H. & King, F. J. (1988). Assessing the impact of computer-based instruction: A review of recent research. Computers in the Schools, 5(3/4), 1-149.

Schubert, J. G. (1986). Gender equity in computer learning. Theory into Practice, 25, 267-275.

Sutton, R. E. (1991). Equity and computers in the schools: A decade of research. Review of Educational Research, 61(4), 475-503.

Teh, G. P. L. & Fraser, B. J. (1993). A study of computer assisted learning environments in Singapore. Paper presented at annual meeting of American Educational Research Association, Atlanta, GA.

Walberg, H. J. (1975). Educational process evaluation. In M. W. Apple, M. J. Subkoviak & H. S. Lufler (Eds.), Educational Evaluation: Analysis and Responsibility, (pp. 237-268). Berkeley, CA: McCutchan.

Welch, W. W. & Walberg, H. J. (1972). A national experiment in curriculum evaluation. American Educational Research Journal, 9, 373-383.

Woo-Tan, L. B. (1989). Effects of Computer assisted instruction on the learning of transformation geometry. Unpublished Master's dissertation, National University of Singapore, Singapore.

DIRECTIONS

There are no 'right' or 'wrong' answers. Your opinion is what is wanted.

Please do not write on this booklet. All answers should be given on the separate Answer Sheet.

Think about how well each statement describes what actual classroom is like. Draw a circle around:

1	If the practice actually takes place	ALMOST NEVER
2	If the practice actually takes place	SELDOM
3	If the practice actually takes place	SOMETIMES
4	If the practice actually takes place	OFTEN
5	If the practice actually takes place	VERY OFTEN

Be sure to give an answer for all questions. If you change your mind about an answer, just cross it out and circle another.

Authors: Dr George Teh is Senior Lecturer at Singapore's Nanyang Technological University's National Institute of Education. The work described here was part of his doctoral research at Curtin University of Technology, Perth. Dr Barry Fraser is Professor of Education and Director of the national Key Centre for School Science and Mathematics at Curtin University of Technology, Perth. Please cite as: Teh, G. P. L. and Fraser, B. J. (1994). An evaluation of computer assisted learning in geography in Singapore. Australian Journal of Educational Technology, 10(1), 55-68. http://www.ascilite.org.au/ajet/ajet10/teh.html