Learning human biology: Student views on the usefulness of IT materials in an integrated curriculum

Given the significant investment by tertiary institutions in the design and creation of computer based teaching and learning resources, it is important to continually evaluate outcomes from their implementation. Within this context, this study reports on the use and perceived usefulness of educational multimedia resources and communications technologies within a single course in a first year biology program. Using an action research model as the basis for the evaluation, data of expected and actual use and usefulness of the resources were collected from students using surveys and focus groups. While the majority of students indicated the multimedia resources were useful for learning activities through providing off campus access to supplementary and relevant materials, others did not find the resources useful, and some did not use them at all. In addition, the use of communications technologies was greatest for social interactions rather than course specific activities. Use was not a function of students' access to computers or the Internet. These findings highlight that online resources will not necessarily generate value added learning for all learners, and that programs will need to offer a variety of learning resources that target different learning styles and enable a mix of off campus and on campus opportunities. Other important factors also emerged from the study, in particular the barriers that learners encounter when working with computers and the integration of these resources into the teaching curriculum. In addition, the study highlighted the value of action research as a means of conducting evaluations of computer based learning resources.

Introduction

The current study examined the first year course, human biology, which integrates a range of computer based learning modules, online materials and communications strategies with more traditional learning resources, such as lectures and practical sessions. The aims of the course include helping the students develop a familiarity with foundation issues in human biology, with the ability to relate learning to real life and to enhance the development of those life skills required of a science graduate. The teaching methods use lectures, practical classes in which the students are encouraged to work in small peer groups to foster collaborative learning strategies and good inter-group communication, and independent study opportunities which can be undertaken at a time to suit the student and do not require attendance on campus. The environment therefore provides a learning paradigm where students are provided with a range of resources to cover the curriculum of the course and this range has been designed to cater for a variety of learning styles. We also acknowledged that students have different learning styles and therefore, as teachers, we need to use a variety of techniques and resources, embedded in the teaching and learning environment, to assist their learning. The recent work of De Vita (2001) and Heffler (2001) endorses this perception.

The purpose of this study was to provide both a reflective and analytical assessment of a broad range of learning resources integrated through web based technology. Three specific factors led to the adoption of this approach. First, the students enrolling in the course constitute an heterogenous group with widely varying academic backgrounds and interests in biology, a situation that now appears to be the norm for large first year science classes. Second, the computer based modules used within this course have already been demonstrated as effective (Peat, 1999; 2000b; Peat, Franklin & Mackay-Wood, 1997), having been developed over a number of years with ongoing formative evaluation enabling each resource to be enhanced as it was being integrated into the curriculum. Students (and staff) were asked questions such as "Was it easy to use?", "...accessible?", "...enjoyable" and "Were there any bugs?". Individual resources were modified according to student and staff feedback. Many of these products were developed initially for use on the University Intranet but have now been launched on the Internet. Use of these resources within the learning process has been investigated by asking students for what purpose (tutorial, remedial learning, self assessment) they are using the materials, and how the materials help them in their learning. Part of this current project is to determine the accessibility of these Internet resources to the students and when and how they are being used. Third, the research agenda within instructional technology has gone beyond that of comparing resources to one of making them work better (Reeves, 1999). Over the past 25 years a vast quantity of research has been conducted to assess the comparative benefits of computer based learning, with ambivalent results (cf Reeves, 1993; Russell, 1999). For this project the focus was not on the individual resources as effective learning tools, but on the perceptions of both students and teaching staff as to their importance in the overall teaching and learning process; this is particularly significant with the increased emphasis by tertiary institutions upon online learning. In relation to this factor, the project emphasised the student perceptions of their learning experiences as advocated by Prosser (2000), with the focus on the ways in which resources, using information and communications technologies (email, discussion groups, computer based learning) had been effectively integrated into the curriculum.

Given the amount of time and money expended on the development of all the teaching and learning resources and within the context of the three factors identified (increased heterogeneity of users, accepted mature materials, and making the resources work better), the aim of this project was to determine how the computer based modules and information and communications technologies influence the learning process. The resources in question are being sustained within the curriculum and have been used by more than 800 students each year for several years. While this has been accepted as cost effective, the effectiveness of the materials has not been sufficiently demonstrated with respect to student learning.

Available online resources

In 1997, communications technologies in the form of email and chat were introduced as a resource for learning, enabling asynchronous electronic student/staff and student/student contact. The use of communications technologies as a resource was enhanced in 1999 when The University of Sydney provided all students with a free email account. The current communications links available via CyberTutor, CyberAdmin and CyberTechSupport through the VLE allow students to contact staff about academic, administrative and technical matters.

Evaluation plan

In addition, as the study was integral to a larger Federal Government research grant (Phillips, Bain, McNaught, Rice & Tripp, 2000) it was important to ensure the evaluation study was consistent with the Learner Centred Evaluation model (Alexander & Hedberg, 1994) being assessed by the grant recipients. Given that the resources being evaluated had undergone pedagogical testing (Peat et al., 1997), it was clear that the evaluation was associated with implementation factors and that the process for investigation best suited to the study was action research, as it would enable the investigation to "drill down" into relevant outcomes based on data collected during the study. In particular we wanted to review how the resources were used at different stages of the course and therefore the evaluation questions were a function of the prior stage of the investigation rather than predetermined. This was considered particularly important as research is continually highlighting the complexity of factors which enable the success of computer based learning (for example Hedberg & Sims, 2001; Sims, 2001) and as such we were wishing to learn more about the learner-computer dynamic.

Participants

The target population of human biology students (n=800) is typically recent high school leavers enrolled in science based degree programs (83%). They are predominantly female (65.5%), full time enrolled (98.5%), within the school leaver age range (91.5%), having attended school in Australia and coming from predominantly university oriented backgrounds. Most (92%) had completed the previous semester's biology course and this previous experience of the learning environment within first year biology would have been expected to give them an informed perspective on the way in which the course would be presented, in particular the use of computer based resources.

Methodology

The overall study looked at how successful the learning resources were that have been implemented in a large first year human biology course. The perceptions of the major stakeholders (students, academic staff and courseware developers) involved in the course were sought, in order to enable alignment of responses and derivation of common themes in terms of the understanding, potential and use of learning resources within the program. This paper, however, focuses on the students' perceptions of their use of computer based learning resources and communications technologies within an integrated curriculum. Data were collected at four different times (Figure 1) during the semester length course, with the information obtained from each data collection informing a subsequent collection (by the use of an action inquiry method).

Figure 1: Student data gathering chart at DC1 - DC4

A combination of qualitative and quantitative survey questions as well as focus group discussions were used at the data collection points (DC1-DC4), to target student perceptions of the usefulness of the materials to their learning. Open ended questions were thematically analysed and categorised (Denzin and Lincoln, 1994).

The first data collection point (DC1) was at the commencement of the course and consisted of a survey (S1) to establish a benchmark of understanding and perceptions prior to any teaching and learning influences, as well as students' expectations of learning resource usage. At S1 all students (n=800) were surveyed, during the fourteen laboratory sessions that accommodate this large group of students each week and the data collected are from this entire stakeholder group. Students are assigned at random by the university timetabling computer into one of these fourteen sessions.

Subsequent data collection by survey was of a subset of this stakeholder group with S2, S3 and S4 each surveying half the total number of students. These students were in seven randomly selected laboratory sessions. The information from S1 helped inform both S2 and the focus group. In particular students were asked further questions in S2 about their use of email and the Internet in their learning, and ease of access to first year online modules. In addition S2 targeted their perceptions of the use of virtual versus real dissections in enhancing learning. The focus group questions concentrated on the use of online tutorial material. On the basis of the focus group material the third survey (S3) revisited some of the material from the second survey as it was felt that some of the S2 questions had been ambiguous and the data collected were not easy to interpret. It also focused specifically on students' perceptions of the usefulness of the resources to their learning and understanding. At this stage a focus group was asked to comment specifically on the use of the self assessment modules. Whilst survey S4 repeated much of the ground covered in the initial survey (S1), it did so by asking students what resources they had actually used and how useful they had found these resources.

Due to the holistic and inclusive nature of the investigation, the quantity of data generated was huge and time became an issue, given that preceding data had to be analysed quickly in order to inform each subsequent survey instrument. Similarly there was too little time to reflect on the data before moving on to the next phase. This led to some data being hard to interpret. Although the students were involved in the project and were provided with information as it came out of the data, there was a perception that students were being over surveyed in human biology and university courses generally. The use of an action inquiry method exacerbated this problem, which was reflected in a difficulty in obtaining volunteers to participate in focus group discussions.

Results and discussion

This section reports on the major factors emerging from this research process and examines student:

• access to computers and the Internet;
• use and perceptions of the Internet;
• access to online materials;
• views of communications technologies;
• perceptions about using online tutorials in general; and
• profile of use of electronic resources.

Student access to computers and the Internet

Student use and perceptions of the Internet

		Expectation (S1)	Actual (S4)
Access computer	never/rarely weekly daily	13% 54% 33%	16% 71% 13%
Access Internet	never/rarely weekly daily	11% 55% 34%	20% 64% 16%

The majority of students (81%) had a preconception that they would use the Internet to support their learning during the course but only 76% actually used the Internet to support their learning (Table 2). With 99.5% of students indicating access to a computer but in fact 20% not accessing the Internet at all during this course, the implication for universities considering putting more materials online is one of caution. Similar data for lack of uptake of web based teaching has been reported by Oliver & Omari (2001) who found that 20% of students were not comfortable with using the Web as their learning environment and suggested that this number of students (50 in their cohort) is too big to be ignored when making decisions on delivery of materials. We need to be aware that students will continue to require a diverse range of learning resources both online and offline. Given these data it might be useful to investigate further the reasons for non usage and whether this relates to access issues and competitive access at home, whether there are issues with learning styles and resource preferences, or maybe an interrelationship between how people wish to learn and how the material is presented.

Students found the Internet as a resource to support learning was as they expected, but 24% of them were not using it as a resource (Table 2).

		Expectation (S1)	Actual (S4)
			Use	Do not use
Use of Internet to support learning		81%	76%	24%
Usefulness of Internet in supporting learning	not useful useful extremely useful	9% 58% 33%	11% 55% 34%

Again these data indicate that a considerable number of the students (24%) are not using the Internet as a resource and those that do use it indicate it is useful (55%) but not extremely useful. This reinforces the idea that within the student cohort there are a variety of learning styles which require the provision of a variety of resources, and not necessarily computer based. Half the students who used the Internet perceived that it was useful in supporting their learning, and this encourages us to continue to provide web based resources.

Other studies have reported that learning styles may not be the only reason why students do not use such resources to their full extent. Sims (2001) argued that limitations in the communication design of the application could affect the maintenance of engagement and Hedberg & Sims (2001) proposed that the familiarity of the environment would impact on the success of a student's encounter with the particular learning resource or environment. While the resources we employed have clear pedagogical integrity, more needs to be understood of the learner-computer interface and, as suggested by our results, the potential effect on learning with computers.

Student access to biology online materials

		Expectation (S1)	Actual (S4)
			Use	Do not use
Use of VLE to support learning		85%	85%	15%
Usefulness of VLE in supporting learning	not useful useful extremely useful	6% 47% 47%	6% 44% 50%

Whilst general access to computers and the Internet is good, there is some concern within the student body about access to the special materials made available via the VLE on the Internet for this course. A number of students (16%) indicated they had difficulties in accessing these materials and open ended question methodology was used to find out why. In response to the question "Do you have any difficulties accessing First Year Biology Internet resources? If, yes, please indicate why?" there were 50 responses and these were categorised within themes. The major difficulties identified were difficulties with software (28% of all responses), insufficient RAM (22%) and download times (18%). The most frequently stated comment was about the difficulties with the "plugins" needed to view the educational multimedia modules online. The remaining 32% of responses covered competition for access, navigation and hardware problems.

We can provide the technology but we need to be careful that we match this with student abilities and experiences. Some universities provide all students with electronic toolkits to help them in using the Internet but this is only of use if the students are able to access the toolkit materials and use them. We are considering providing the students with our tutorial and self assessment modules on a CD.

Student use and perceptions of communication technologies

On the other hand, email, as a form of asynchronous communication, was considered to be a more useful resource to support learning in human biology. The student expectation of their probable use of email and its potential usefulness was however greater than the reality. At the commencement of the course 59% of students expected to have access to email at least weekly in order to participate in and successfully complete the course, whereas only 29% actually used email weekly. Interestingly, 22% initially expected to be in daily email contact but only 5% actually used email daily, with the majority of students only using email weekly. Similarly 41% of students expected email to support their learning in human biology but in reality only 22% used it for this purpose. This is similar to the data in the McInnis study (2000) which found that only 20% of students had used electronic access to tutoring support. Of those students who did use email, 57% found it useful/extremely useful in supporting their learning, which means that approximately 10% of the entire cohort found email useful in supporting student learning.

		Expectation (S1)	Actual (S4)
Access to email	never/rarely weekly daily	41% 37% 22%	71% 24% 5%
			Use	Do not use
Use of email to support learning		41%	22%	78%
Usefulness of email in supporting learning	not useful useful extremely useful	41% 48% 11%	43% 47% 10%
Use of chat groups to support learning		16%	5%	95%

The student expectation for using email was much higher than the reality of using it and this needs to be viewed in the light of the other stakeholders' perceptions (which will be reported in a future paper). It may be that as teachers we have unrealistic expectations and that there is a mismatch between what we think as providers and how the students perceive the provisions. It may be that the students, whilst expecting to use the technology, find they do not like using it, do not know how to use it or that they do not see the purpose in using it for course related matters. Students need a purpose for using a resource and this needs to be made clearer to them.

Whilst Table 4 reports on expectations of use (from S1) and actual use (from S4) of email to support learning in the human biology course, a mid course survey (S2) indicated that the overall student use of email was high, with 97% of all students surveyed indicating some use. Most of this use (75%) was for other than course related activities. This was further investigated within focus group discussions where students indicated that they appreciated and expected information to be sent to them via email but that they would rather talk face to face with staff as this gives immediate feedback and allows for follow up questions. Students found email responses to be "not fast enough", expecting immediate responses to their questions.

Student use and perceptions of computer based online tutorials

The data indicate that, from a cohort of 800, there are potentially 200 students not using the educational multimedia materials. Given that 91% of those students that did use the materials found them useful, it is important to explore why the group of non-users do not avail themselves of these learning resources. Core resource material should be emphasised both to the students and the laboratory teaching staff; additional/remedial/revision materials should also be suitably identified so that students (and staff) are aware of the purpose of the materials.

		Expectation (S1)	Actual (S4)
			Use	Do not use
Use of CBT to support learning		73%	75%	25%
Usefulness of CBT in supporting learning	not useful useful extremely useful	9% 60% 31%	9% 53% 38%

The focus group at DC3, which investigated student perceptions of the online self assessment modules, revealed that the students were using them as a learning tool as well as for the original purpose.

.. get to know what you understand not just rote learn
...probably use more as a learning tool but also for revision at the end.

Type of understanding/learning	Usefulness (%)
	Of no use	Of some use	Useful	Very useful	Essential
Developing an independent approach to learning	7	14	33	29	17
Developing discipline based research skills	11	19	32	27	11
Understanding the structure of body systems	5	17	33	31	14
Understanding the functions of body systems	5	18	28	33	16

The results in Table 6 have a bell shaped distribution with some students perceiving the materials as of no use while others found them essential, indicative of the use of different learning preferences by students and the need for a variety of learning resources. This is the same theme as before where some students will benefit from this type of material whilst others will not, and this is probably true for all the different learning resources. This will be the subject of further investigations.

As part of Survey 4, students were given the opportunity to comment on the computer based learning resources via a number of open ended questions. Some responses to the question: "Explain how you think the CBLs help in your learning, understanding and revising" are seen below indicating students appreciate the interactive nature of the modules and the ability for self assessing.

CBL allows interactive study from home, which certainly assists in learning, understanding and revising.

They clearly explain and demonstrate principles otherwise confusing and vaguely explained (if at all) in the textbook.

Allow visual reinforcement. Visual aids/animation helps clarify and understanding. Self tests at end helpful.

Allow problem based learning and to learn interactively.

I don't believe they do. The information is much more successfully provided elsewhere, as it is more organised and direct. These are only useful once everything else has been memorised to ensure everything has been covered.

It is a useful tool for reinforcement of lecture material.

It is a huge help with personal study that can be conducted at home.

Yes, it allows me to further understand the topic, but sometimes, it involves you spending a lot of time for it.

It is a very effective method of independent study and is very helpful in being another complement to lectures and the text.

Excellent idea - step by step demonstration, explains material carefully and thoroughly - great way to learn.

I'm not good with computers, don't find it useful.

Don't use computers.

I don't agree with putting all important information on the web, it was difficult for me to access the internet, hence I miss out on a lot of learning material.

Student profile of use of resources

Educational implications

The evaluation of computer based learning resources is becoming even more critical now that institutions are placing significant investments in online teaching and learning, and the need for competent teachers and learners to use the environments effectively (Sims, Dobbs & Hand, in press). In this study, the action research approach elicited a range of factors which informed the investigators about the integration of these resources into the educational environment. While their educational value can be demonstrated from an instructional design perspective, their use is the critical element; from the results obtained it is clear that students are willing to use the resources, but only if they are effectively integrated into the curriculum and relevant to the assessment strategies. By using an evaluation strategy relevant to the environment we have demonstrated that the onus is on the teacher to ensure such resources are integrated and that the students are aware of their purpose and function within the course.

In terms of the evaluation project itself, it is useful to reflect on both the impact and unanticipated consequences of the action research undertaken. First, the process of using the outputs of each data collection point to inform subsequent data collection and analysis strategies provided a continuity for the evaluation as well as enabling the analysis to focus on the extent to which the resources made a difference (impact). For this study, the major outcome was that a difference was noted, but not for all participants, highlighting the importance of aligning teaching and learning environments with the student cohort and their expectations of that environment and its integrated resources. Second, the data most unexpected was the extent to which participants did not use the available resources, regardless of their explicit value to the overall program, and the extraneous factors that may have also had an impact on their access. For higher education departments, it is clear that while online resources can be demonstrated to elicit favourable learning outcomes, there can be no guarantee that they will be used and therefore investment in their development must be carefully monitored.

The message coming out of this study is that the majority of students embrace educational multimedia and information and communications technologies and find these learning experiences valuable. However, there is a proportion of the cohort (15-20%) that, for whatever reason, are not taking up the challenge of these new(er) technologies. In addition there are students, who use the materials but do not find them particularly useful. We should take care when replacing viable (traditional) resources with computer based ones that the replacement will be acceptable. This reinforces the requirement for academic departments to continually review teaching resources to ensure they meet current student needs and learning styles. The outcomes of this current study will inform other curriculum developments within first year biology courses.

Acknowledgments

References

De Vita, G. (2001). Learning styles, culture and inclusive instruction in the multicultural classroom: A business and management perspective. Innovation in Education and Training International, 38(2), 165-174.

Franklin, S. & Peat, M. (2001). Managing change: The use of mixed delivery modes to increase learning opportunities. Australian Journal of Educational Technology, 17(1), 37-49. http://www.ascilite.org.au/ajet/ajet17/franklin.html

Hedberg, J. & Sims, R. (2001). Speculations on design team interactions. Journal of Interactive Learning Research, 12(2/3), 189-204.

Heffler, B. (2001). Individual learning style and the learning style inventory. Educational Studies, 27 (3) 307-316.

Lewis,R. (1999). The role of technology in learning: Managing to achieve a vision. British Journal of Educational Technology, 30(2), 141-150.

Lim, K. F. and Lee, J. (2000). IT skills of university undergraduate students enrolled in a first year unit. Australian Journal of Educational Technology, 16(3), 215-238. http://www.ascilite.org.au/ajet/ajet16/lim.html

McInnis, C., James, R. & McNaught, C. (1995). First Year on Campus: Diversity in the initial experiences of Australian undergraduates. Canberra: AGPS.

McInnis, C., James, R. & Hartley, R. (2000). Trends in the First Year Experience in Australian Universities. Canberra: AGPS.

Morrison, G. R., Ross, S. M. & Kemp, J. E. (2001). Designing Effective Instruction. New York: John Wiley & Sons.

Oliver, R. & Omari, A. (2001). Student responses to collaborating and learning in a web-based environment. Journal of Computer Assisted Learning, 17(1), 34-47.

Peat, M. (1999). Virtual communication for lab-based science teaching: A case study. In Proceedings of the Computer Based Learning in Science International Conference, ISBN 80-7042-144-4.

Peat, M. (2000a). Towards First Year Biology online: A virtual learning environment. Educational Technology & Society, 3(3), 203-207. [verified 24 Jul 2002] http://ifets.massey.ac.nz/periodical/vol_3_2000/b06.html

Peat, M (2000b). Online assessment: The use of Web-based self-assessment. In A. Hermann and M.M. Kulski (Eds), Flexible Futures in Tertiary Teaching. Curtin University of Technology, Perth, WA (119-127). [verified 24 Jul 2002] http://lsn.curtin.edu.au/tlf/tlf2000/peat.html

Peat, M. & Franklin, S. (2000). Student Part-time Work Survey, 2000. http://fybio.bio.usyd.edu.au/SOBSFYB/fyb/FYBhome.html [verified 25 Jul 2002 at http://fybio.bio.usyd.edu.au/FYB/tdg/AnnReports/wk_sur00.htm]

Peat, M., Franklin, S. & Mackay-Wood, R (1997). The development of self-assessment modules: Use of tailor-made templates. In Virtual Conference on Computers in University Biology Education 1997 (CUBE97). http://www.liv.ac.uk/ctibiol/vCUBE97/html/rob_mackay-wood.html [verified 25 Jul 2002 at http://science.uniserve.edu.au/mirror/vCUBE97/html/rob_mackay-wood.html]

Phillips, R., Bain, J., McNaught, C., Rice, M. & Tripp, D. (2000). Handbook for Learning-Centred Evaluation of Computer-Facilitated Learning Projects in Higher Education. CUTSD http://cleo.murdoch.edu.au/projects/cutsd99/handbook/handbook.htm

Prosser, M. (2000). Evaluating the new technologies: A student learning focused perspective. Paper presented at the UniServe Science National Workshop, University of Sydney, 28 April 2000. [verified 25 Jul 2002] http://science.uniserve.edu.au/pubs/procs/wshop5/prosser.html

Reeves, T.C. (1993). Pseudoscience in computer-based instruction: The case for learner control research. Journal of Computer-Based Instruction, 20(2), 39-46.

Reeves, T.C. (1999). A research agenda for interactive learning in the new millennium. In B. Collis & R. Oliver (Eds), Proceedings of the 1999 EdMedia Conference. Association for the Advancement of Computing in Education.

Russell, T.L. (1999). The No Significant Difference Phenomenon. Office of Instructional Telecommunications, North Carolina State University.

Simonson, M., Smaldino, S., Abright, M. & Zvacek, S. (2000). Teaching and Learning at a Distance: Foundations of Distance Education. Upper Saddle River, NJ: Prentice Hall.

Sims, R. (2001). The online learning alchemist: Preventing gold turning into lead. In C. Montgomerie & J. Viteli (Eds), Proceedings of Ed-Media 2001. World Conference on Educational Multimedia, Hypermedia and Telecommunications. Tampere, Finland, June 25-30, 2001. Association for the Advancement of Computing in Education (AACE).

Sims, R., Dobbs, G. & Hand, T. (in press). Enhancing quality in online learning: Scaffolding design and planning through proactive evaluation. Distance Education.

UniServe Science (2000). Survey of computing experience and skill development needs of incoming first year students at The University of Sydney in 2000. Prepared for the Information Technology Committee by UniServe Science. http://science.uniserve.edu.au/pubs/survey00.html

Wilson, A. (1996). Multimedia Maclabs. Education in Chemistry, 33(5), 134-136.

Wilson, A. & Cavallari, B. (1995). Ozchem: An Australian chemistry laboratory simulation. Active Learning, 3, 45-50.

Authors: Mary Peat, Sue Franklin, Alison Lewis, The University of Sydney Rod Sims, Deakin University Please cite as: Peat, M., Franklin, S., Lewis, A. and Sims, R. (2002). Learning human biology: Student views on the usefulness of IT materials in an integrated curriculum. Australian Journal of Educational Technology, 18(2), 255-274. http://www.ascilite.org.au/ajet/ajet18/peat.html