Internet based teaching in geography at Macquarie University: An analysis of student use

The design of a first year course with enrolments of about 400 students which is delivered using information technology via the Internet or Intranet is reviewed. The course is composed of a variety of information technology based applications including course materials, bulletin boards, email, quizzes, access to the World Wide Web and multimedia packages linked together using a single Internet browser interface. Access statistics through the entire course were collected hourly over the sixteen weeks of semester and separated on the basis of gender and final grade. Statistical analysis is presented which shows that different groups of students access the course in different ways. Performance in the course is statistically related to the number of times the student accesses the package. We find no evidence that males are advantaged or disadvantaged compared to females through the use of information technology although females appear to use the communications part of the course more than males. Based on the access statistics, we make recommendations on the components which seem to be well received by students and we note some components which will require more careful integration into a teaching program. Overall, the delivery of a first year course via information technology works well provided the components are carefully integrated into the course framework.

Introduction

Curricula can be developed incorporating a vast array of different learning resources and experiences designed to match specific educational contexts. Students can access databases, research and commercial software programs, computer based instructional programs, models, simulations, games, quizzes, virtual laboratory sessions and field trips, or communicate individually and collaboratively with staff and fellow students. This can all be achieved from both on campus or off campus, thereby providing students with viable choices in how, when and where they wish to study.

Owing to the newness of the technology and the infancy of research informing the design and development of IT based teaching and learning resources, the full creative potential of IT and its capacity to enhance student learning has yet to be fully explored. For academic staff, IT based approaches provide the means to develop flexible study programs for both on and off campus students without necessarily increasing workloads. IT can enable the development of a different range of teaching resources to meet the needs of an increasingly diverse student body. IT can provide alternative, efficient and cost effective access to resources that previously were only available through on campus computer laboratories or through the library (eg. online newspapers, data bases, libraries, museums and journals). Finally, IT can provide efficient and effective alternative forms of communication with students (eg. bulletin boards and email).

For students, the potential advantages are comparable to those for staff. Firstly, there is the potential for enhanced flexibility in both the timing and location of access to educational programs. Secondly, the diversity in curriculum resources has the potential to accommodate individual learning styles. Lastly, online communication via email means that students are not restricted by on campus consultation hours or face to face contact with staff.

While these potential advantages exist for both staff and students, attempts to introduce change must be accompanied by monitoring and evaluation throughout all stages of the design, development and implementation of new programs. At Macquarie University we have been exploring the use of IT in first year geography teaching and we have been monitoring how students interact with IT in order to attempt to realise the potential that we believe IT offers. This paper provides a quantitative assessment of how students in one large first year course interact with a variety of IT components. The course includes a number of different IT based components which have been developed and systematically evaluated over a number of years. These components are delivered electronically via either an Intranet (on campus) or the Internet (off campus) from a common WWW interface. The impetus for such development has come from a variety of strategic, educational and economic concerns (Rich et al, 1997) including the need: to provide a flexible learning environment for students; to maintain or enhance existing educational standards in the face of declining teaching resources and; to diversify the information and learning resources available to students to accommodate different learning styles.

The focus of this paper is to review this integrated approach and provide insights into how a large group of students use the technology in relation to four main themes which we believe have important strategic and educational implications for the University and individuals. The themes relate to: flexible learning; possible gender biases in the use of IT; relationships between the use of IT and student performance; and resource design, in that flexible learning programs are underpinned by a resource based learning approach to curriculum design (NBEET, 1997).

In the body of this paper we will review the structure of the course and describe the different IT elements integrated into it. This will be followed by a brief assessment of some of the access statistics and a discussion of what we believe we can learn from these statistics in terms of the design and implementation of these types of teaching and learning packages.

The course and teaching package

Information and content provision

Internet collection

Multiple choice quiz

Multimedia packages

These two packages provide a framework to explain, revise and test knowledge and understanding of key issues. Text based explanations are supported by pictures, illustrations and short animations which have the potential to support learning (Burton et al, 1995; Clark and Pavio, 1991; Winn, 1994) specifically when the text and/or graphics and animations are explicative, learners lack prior knowledge of the subject, and the focus of instruction is understanding rather than repetition of factual information (Mayer, 1997). In addition, students can explore in more detail and at their own pace aspects of the course content not fully dealt with elsewhere.

Email

Bulletin board

Summary

Evaluation of the GEOS114 teaching package

In undergraduate teaching at Macquarie University, we employ the grading scheme of A (obtained by 33 students in GEOS 114 in 1996, or 9.5%), B (obtained by 49 students, or 14.1%), C (obtained by 205 students, or 58.9%), CQ (obtained by 22, or 6.3%) and F (obtained by 39, or 11.2%). The broad distribution of results is similar year to year. Of the 348 students included in this analysis, 152 (43.7%) were female and 196 (56.3%) were male.

Access statistics over the semester

Figure 1(a)

Figure 1(b)

Figure 1(a) and (b): Access statistics (individual accesses on Internet based pages) over the sixteen weeks of semester. The Y axis is in units of normalised percent of total accesses. This is the number of accesses for all students obtaining a particular grade or belonging to a particular gender divided by the total accesses for all students multiplied by 100. This is then normalised by the number of students obtaining a particular grade or belonging to a particular gender and smoothed with a seven day moving average), (a) for student grade, (b) for male and female students.

Table 1: t score statistics calculated on the basis of grade anomalies (accesses by all students in grade x divided by the number of students in grade x minus accesses by all students divided by the total number of all students). Those in bold are significant at the 5% level.

From Figure 1(a) it appears that A grade students accessed the material more than any other students during the first seven weeks and the remaining groups of students cannot be separated. A statistical analysis can be performed on just this seven week period, the results of which can be found in Table 2.

This shows that A students do access the Internet more than any other group of student and that the differences in the access statistics are significant at the 5% level. However, the only other statistically significant difference in this seven week period is between B grade and F grade students. We cannot say, for instance, that B students access the course materials more than CQ students. The F grade students accessed the materials more than CQ students in the first seven weeks but both F and CQ grade students noticeably failed to access the material as much through the period of highest access. Whether this related to interest, a lack of patience in waiting for access to a computer during periods of peak load or other factors is unknown. Two further points are noteworthy: first the relationship between grade and access statistics can also be seen at the very beginning of the course where A and B grade students look over the computer based resources immediately, while it takes the students who eventually obtain a lower grade a week or two to get going. Second, during the mid semester break, the A grade students were accessing the material almost as much as the CQ students had in weeks 1 through 5.

The final period of interest is the last four weeks of the semester where Figure 1(a) indicates that B students' access is anomalous. This might be interpreted as a group of students successfully "cramming" in the last couple of weeks before the examinations. Table 3 shows the results of the statistical analysis.

During this period, B students access the Internet significantly more than A students and C students. However, the peak in the access statistics from CQ and F students in week 14 means that there is no statistically significant difference between the B students and the CQ and F students in this four week period. A grade students access the course materials significantly less than B grade students (at the 5% level) and numerically less than CQ and F grade students. They had presumably managed to "get ahead" and were able to revise effectively without accessing the computer resources. B grade students invested a great deal of time in the two weeks just prior to the examinations.

Separating the access statistics by gender (Figure 1(b)) shows that in week 1, females access the package twice as much as males. Other than this difference, there are no statistically significant differences (at the 5% level) between the access statistics separated by gender hence we find no statistically significant gender based differences in the use of IT in this course.

Access statistics aggregated by day of week

Figure 2: Access statistics by day of week. (a) for A students, (b) for B students, (c) for C students, (d) for CQ students, (e) for F students, (f) for male and female students (male right hand side bar, note different scale on Y axis). The scale on the Y axis is as for Figure 1 summed over each day of the week over the sixteen weeks of semester.

Finally, Figure 2(f) shows aggregated daily access for male and female students. Excluding Monday, where male students access the package rather more than female students, there are no obvious differences between the access statistics. It should be noted that the pattern of access through the week is largely determined by the pattern of lectures (Tuesday and Wednesday).

Access statistics aggregated by time of day

Figure 3: As for Figure 2 but for time of day.

Key component utilisation

Figure 4(a)

Figure 4(b)

Figure 4: (a) the number of messages posted to the bulletin board (left bar) and accessed on the bulletin board (right bar) as a function of grade and gender; (b) the number of email messages sent as a function of grade and gender. Both figures are normalised by the number of students obtaining a particular grade or by the number of students in a gender class.

The reading of posted items on the bulletin board is encouraging, although we need to implement measures to encourage more students to submit items (note the large difference seen in Figure 4). This is a new mode of communication for the majority of students and there may be some reluctance to communicate through an electronic means rather than verbally. Harasim et al. (1995) notes that this is a common experience for first time users.

In terms of email use, 420 email messages were read or sent over the semester. Overall, 49% were directed to the course convenor. Module 1 saw the greatest use (24.6%) with a drop off to Module 2 (8.7%), Module 3 (14.1%) and Module 4 (3.82%). While we note that we would probably not have received 420 knocks on the doors of staff, the ability of students to send us questions, and the ability of staff to mail all students with clarifying information before too many asked the same question in person, significantly eased the convening of this course.

There were relatively few differences in the use of the email by students obtaining different grades. Figure 4 (b) shows that while there was more utilisation of email by A grade students, B , C , CQ and F grade students all accessed email with a fairly similar frequency. Females sent, on average, 1.45 emails and males sent 1.05 which given the size of the course is a significant difference.

Discussion

Flexibility

Performance

Gender

A number of studies conducted at school (rather than higher education) level have found that girls tended to use computers differently, focussing more on such activities such as word processing and collaborative work while boys liked game playing and competitive work (Kerr, 1996). Our results, indicate that when access is clearly defined and structured then there is no significant difference between the way males and females use the technology, except in the use of some of the communications where females appear to use the technology more than males. This may be due to a maturity factor with usage changing as students move from the school to the higher education sector or it may be, as Whitely (1997) surmised, that female university students are a self selecting group that are more comfortable with using computers.

Use of resources

Communications package

Students benefit from articulating their knowledge by giving explanations and responses to questions (Katz and Lesgold, 1993; Chi et al. 1989; Chi et al. 1994). Although the Bulletin Board provided an electronic mechanism to do this through its capacity to support students' discussing issues, the expected benefits remain to be proven to be an effective instructional tool in our courses. We find that they are read but not contributed to very often by students (although non-contributors may still be benefiting from passive learning). The relatively low contribution rate may be because we included the bulletin board as a supplementary resource but did not integrate it well into the overall teaching and learning package as effectively as we might. To evaluate the relative merits of bulletin boards fully, they must be integrated into the rest of the course such that they become an integral component which the students see as advantageous to their personal performance.

Conclusions

We have shown how the monitoring of access patterns can provide insight into student use. These data can also be used to help to identify problem students, to identify successful work patterns which can then be turned into strategies to assist all students in managing their own learning; to identify the resources that are not being used as intended; and to identify access patterns which can then be used to help plan for the provision of flexible learning infrastructure such as computer laboratories and off campus access.

Overall, the kind of integrated approach we have presented here offers advantages over aspects of more traditional teaching. We do not argue that IT can replace all traditional methods in all circumstances, but it is possible to provide an IT based approach without compromising standards. There are also cost efficiencies in staff time resulting from the use of email and the ability to contact all students as soon as one student identified a problem. While there are cost efficiencies, there are obviously also costs in relation to developmental and technical support. Developmental costs are falling dramatically as new software comes onto the market, but hardware and human resource costs remain high.

In essence, IT can give students the flexibility they want. IT gives staff better communication lines to the student body and access to all students equally and not just to those with the confidence to come and ask questions. When integrated into more traditional forms of teaching, IT can improve the educational experience and we now see the technology to be a valuable tool in enhancing the quality of teaching and learning. At the same time we recognise that, like a lot of early IT based teaching initiatives, the resources described in this paper were developed on the basis of intuitive beliefs about teaching and the capacities of the latest technologies, rather than being grounded in research on cognition and student learning. The results reported in this paper are encouraging, however, they are focussed on the evaluation of student access and usage. The question of whether and how the resources developed assist students in their learning is still largely unanswered. To answer this, we are now in the process of undergoing a major review and evaluation of the course to determine the effectiveness of the resources in assisting students to achieve the desired learning outcomes and to further investigate some of the issues that have emerged from our evaluations to date such as relationships between usage, performance and individual differences including motivation, attitudes towards IT, age, and gender.

Acknowledgments

References

Burton, J. K., Moore, D. M. and Holmes, G. A. (1995). Hypermedia concepts and research: An overview. Computers in Human Behaviour, 11(3-4), 345-369.

Busch, T. (1995). Gender differences in self-efficacy and attitudes towards computers. Journal of Educational Computing Research, 12(2), 147-158.

Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P. and Glaser, R. (1989). Self explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.

Chi, M. T., De Leeuw, N., Chiu, M-H. and LaVancher, C. (1994). Eliciting self explanations improves learning. Cognitive Science, 18, 439-477.

Clark, J. M. and Pavio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3(3), 149-210.

Fletcher-Flinn, C. M. and Gravatt, B. (1995). The efficacy of computer assisted instruction (CAI): A meta-analysis. Computing Research, 12, 219-242.

Harasim, L., Hiltz, S. R., Teles, L. and Turof, M. (1995). Learning networks: A field guide to teaching and learning online. Cambridge, Mass: MIT Press.

Harrison, S. R. and Tamaschke, H. R. (1984). Applied Statistical Analysis. Prentice Hall, Sydney, Australia.

Jacobson, M. J. and Spiro, R. J. (1995). Hypertext learning environments, cognitive flexibility and the transfer of complex knowledge: An empirical investigation. Journal of Educational Computing Research, 12, 301-333.

Katz, S. and Lesgold, A. (1993). The role of the tutor in computer-based collaborative learning situations. In Lajoie, S. P. and Derry, S. J. (eds), Computers as cognitive tools. Hillsdale, New Jersey: Lawrence Erlbaum.

Kerr, S. T. (1996). Towards a sociology of educational technology. In Jonassen, D. H. (Ed.), A handbook of research for educational communications and technology. New York: Simon and Schuster Macmillan.

Khine, M. S. (1996). The interaction of cognitive styles with varying levels feedback in multimedia presentation. International Journal of Instructional Media, 23(3), 229-237.

Mayer, R .E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32(1), 1-19.

McInnis, C., James, R. and McNaught, C. (1995). First Year on Campus: Diversity in the initial experiences of Australian undergraduates. Canberra: AGPS.

NBEET, Higher Education Council (1997). Quality in resource based learning. Canberra: AGPS.

NCSA (1997a). http://hoohoo.ncsa.uiuc.edu/docs/tutorials/user.html

NCSA (1997b). http://hoohoo.ncsa.uiuc.edu/cgi/

Rich, D. C., Pitman, A. J., Gosper, M. and Jacobson, C. (1997). Restructuring of Australian higher education: Information technology in geography teaching and learning. Australian Geographer, 28, 135-157.

Winn, W. (1994). Contributions of perceptual and cognitive processes in the comprehension of graphics. In W. Schnotz and R. W. Kulhavey (Eds.), Comprehension of Graphics. Amsterdam: North Holland.

Whitley, B. E. Jr. (1997). Gender differences in computer related attitudes and behaviour: A meta-analysis. Computers in Human Behaviour, 13(1), 1-22.

Appendix: Technical provision of IT components

The software we use to allow email to be sent between student and staff relies on NCSA user authentication protocols. We can identify individual students without the need for individual user accounts. All messages are stored in a single location, but each message has an extra tag linking it to a specific student. We also use NCSA user authentication protocols to run the bulletin board. This permits us to identify individual students without the need to set up individual user accounts. All messages are initially stored in a single file and a moderator views and decides whether to accept, edit or delete. If the message is accepted, it becomes accessible to the students.

The quiz was written in C and generates html which displays each question onto the screen, receives and processes the student's response. The program logs each quiz attempt by each student. The list of questions, possible answers and responses to each answer are stored as a text file, so that writing questions for the quiz does not require any programming knowledge. Since the students use the quiz via Netscape, it is fully accessible across the Internet.