| Australian Journal of Educational Technology 2002, 18(2), 169-186. |
AJET 18 |
A marine science virtual environment was developed as a CD-ROM to simulate marine sampling procedures for abundant and rare species. Students participated as scientists in the process by conducting fish counts for species of differing abundances and sampling unit sizes, calculating the most cost effective sample unit size for common and rare species, and then presenting their findings and interpretations in scientific report format.This paper reports an evaluation of the CD-ROM as it was incorporated into an assessable activity in a third year undergraduate and postgraduate Marine Biology subject. The evaluation evidence indicated that: (a) most students were able to navigate and use the CD-ROM effectively; (b) most students experienced an improvement in their understanding of the sampling process, particularly with rare species; (c) staff confirmed that there was an improvement in students' understanding of the sampling process, including how to choose and undertake the most cost effective sampling method; but (d) this understanding did not always improve students' ability to interpret their results critically and to relate them to what they had observed in the field. These findings are discussed in terms of what is required to provide students with an authentic learning experience as marine scientists, and how the CD-ROM could be incorporated into such a learning experience in the future.
The assessment task attempted to integrate content with practical processes (fieldwork and laboratory) but failed to do so due to financial and legal considerations which prohibited taking more than 100 students on a marine science research trip. The teaching staff and the educational designer (the writer of this paper) felt that a simulation of the practical aspects of the process would address some of the learning problems.
A CD-ROM resource, An Ocean in the Classroom, was developed to give students an insight into designing and implementing an appropriate sampling regime for species of different abundance. It was felt that this would address the discipline specific skill gaps identified during curriculum analysis. It would also provide the context for students to use as a practical benchmark of 'own experience', through which they could interpret quantitative findings in terms of what they actually saw in the field. This in turn would ensure the students were better placed to interpret their observations and data in the context of the literature. The visual transect technique chosen for the simulation is the main method of sampling used by marine ecologists across a broad range of ecological investigations. Knowing how this technique is conducted, and its limitations, is foundational knowledge that cannot be gained by reading literature alone.
The CD-ROM was presented to the students during a tutorial situation where they received some brief instruction on its use together with assignment guidelines. Students then conducted a trial run during the tutorial class, where tutors were available to provide assistance if required. The CD-ROM was made available to students to take from the class and use for collection of data for their written assignment.
The evaluation of the improved CD-ROM and assessment task was guided by the Alexander & Hedberg (1994) and Bain (1999) framework. This was used as a learner centred tool for planning the product development and for guiding the structure of the evaluation and the implementation of the CD-ROM within the teaching situation. The evaluation was conducted as a means of ascertaining the usability of the CD-ROM as a tool for practice. This was considered an important part of the evaluation as we wanted to ensure that no technical barriers blocked a student's ability to achieve the learning outcomes. Given the amount of work that had been undertaken to develop the resource we also felt it important to find out whether the resource provided the solution to the learning problems identified and assisted the students in their achievement of the learning outcomes for the assessment task.
The navigation and interface (the path through the material)
What were the direct learning gains?
Some students who considered themselves to be familiar with interactive software expressed frustration at having to enter sampling details on screen. However, this was intentional because the simulation activity was designed to replicate how a marine scientist would conduct the process in the field. We felt that this component could be automated, but that students might not then appreciate the need to process this detail in the same way they would in the field.
A number of students indicated that the video was too small for definitive identification of one of the target species. This was due to technical constraints in laboratory situations that forced the use of smaller scale video than originally intended. An upgrade of the computers along with advances in video software and playback ability mean that we can rework the CD-ROM to allow full screen video.
"Am I counting the right species and should I see this many fish in one transect?" andGuidance was originally omitted in order to emulate the realities of data collection as closely as possible. However, it was apparent that this instructional consideration was restricting the students' ability to conduct the exercise. Therefore, we decided to guide the student through the activity by providing a trial transect with a comparison table for results before requiring the students to begin their data collection. This is the basic premise of much of constructivist thinking, the goal of which is to guide students so that they begin to think and act like the practitioners of the discipline (Brown et al 1989).
"I am not sure if I am counting the fish right - how would I know?"
The visual aspect I suppose. To actually SEE the distributions of the fish in the flesh and not get a piece of paper composed by someone else dictating where THEY saw individuals along the transects.With this group of students, it seemed that the novelty value of using the resource prevented them from progressing beyond this visual representation of a marine sampling process. Their descriptions were clearly oriented around their observer status and not their practitioner role.
'Actual' sampling. The majority of students commented on the importance they felt in actually being able to do the sampling themselves rather than receiving a set of data points. Their engagement in the 'actual' sampling process placed them in the role of scientist where they were better able to appreciate why the technique was chosen and the practicalities of performing the technique. Some students indicated that it went further than this and provided a means of reinforcing concepts and theories presented in the literature.
Because you had to "swim" them and count, it was a lot more real then just being given the data. You could see how rare species abundance varied; none or few on some transects, more on others etc... Thus you knew as you read the papers that rare species would have more variable counts, need bigger transects to sample more of them, and also more replicates. It reinforced the information from the papers we read.Others stated that information from quantitative data is often more clearly understood when you are involved in the process that leads to the development of the data set, such as determining the clumping of species.
The simulation helped with my understanding of clumping of fish species. This type of understanding is difficult to obtain by just looking at data.During interviews a number of students alluded to the concrete experience, the importance of being able to carry out the process, instead of being presented with facts.
It's the hands on stuff where you learn through doing it yourself. You grasp stuff more that way or I at least do. Like if you dissect something you understand the structure more than if you just read it in a book cause your actually doing more hands on.Another student commented on the linking of the process to the content of the discipline:
The fact that you actually got to experience the sampling, and it provided a very strong visual link with the learned information.It was clear that, particularly for the students who had not collected marine data before, the experience of being placed in the role of practitioner was a useful one. It enabled them to trial a process they would not normally have engaged in and it meant they were better able to understand the reasons for choosing that particular approach to sampling. Many of the students who were subsequently interviewed also indicated an increased feeling of ownership of the data because they had collected it themselves. This seemed to provide additional motivation for completion of the related assessment task because they were more interested in viewing the results of their first experience of collecting 'real' data. The student responses reinforced their need to be engaged in the activities of the discipline as a means of authenticating the experience for better understanding.
Difficulties. A number of students referred to the collection of that amount of data and the time involved, but with the simulation, they were able to conduct the sampling themselves in a one hour computer tutorial.
We found out how hard it was to sample species out in the natural environment.........Going through that and understanding the frustration that goes behind field study in marine biology and the extreme concentration in trying to figure it out and how long it really takes.The assignment was more beneficial. A few students indicated that the analysis conducted as part of the assignment was more beneficial to their understanding than their use of the CD-ROM.
Just the fact that we had to crunch numbers, generate the graphs, and then loosely interpret them gave a better knowledge of how data sets turned into conclusions. It was not the CD-ROM, we could have done almost (but perhaps not quite) as well just by working on sets of raw data, sorry guys. Counting the fish on the computer is cool, but the program is just not that enlightening as to the science aspect.However, some students commented on how the resource increased their understanding of data collection techniques, and provided the basis for them to describe why they had reached that particular understanding in the subsequent assignment.
... one of the species was more dispersed in the environment and that's why you have to use longer transects ...... when you have just numbers on a page you don't quite have that same appreciation whereas two of the species were similar numbers but one was a little bit lower numbers, so it was the clumping. So I guess that was the biggest thing and that was the climax question on the assignment, and someone can tell you that but until you actually see it you don't understand.Students who had been involved in marine data collection in the past seemed to gain least from the resource.
I did a conservation expedition and I did it for 3 months counting fish so I'm a bit of an exception, there aren't that many people who are going to have done the research I have done. I guess it gave an idea to people who didn't have any idea how to do fish transects what a fish transect was.
I thought students with some field work experience gained less than students with no fieldwork experience. They thought that the CD-ROM was unnecessary for them, so they missed its' relevance and did not critically analyse the data in the assignment well, they got the crude message but were unable to tease out the detailed interpretation of their results. Part of this problem, though, was caused by the way in which we conducted the tutorial, it was rushed. When we have improved the CD we will give them to students so they can do the exercise in a more relaxed atmosphere and have time to make qualitative observations rather than just collect the raw data.On completion of the assignment, the lecturers had the following comments to make about what had improved relative to previous years and what had not improved:
The CD helped students explain their results, with reference to the literature and in terms of what they observed when collecting the data. The literature isn't always right or applicable, so it is vital to compare what you might expect to what you actually see. But I was disappointed that more students did not ask 'what do I think is happening?'. I had hoped that simulating the data collection would help them absorb the visual distribution information while counting and then in the assignment they would be able to discuss this, but I am not sure that we actually achieved this. They still missed the significance of relating their statistical results to what they actually saw when collecting the data.The lecturers were clearly disappointed that the students were not better equipped to interpret their results in light of what they had observed while using the CD-ROM. They referred to the students' inability to explain their results, particularly where patchy distribution and clumping influenced the results.
Students were told that they were going to use different transect sizes for fish of differing abundances but they seemed reluctant to inquire beyond this.
However, it seems clear from the student responses, that the patchy distribution and clumping of species is a message that they understood as a result of using the CD-ROM. The students said:
However, there were some barriers to progress and a small percentage of mature age students experienced problems with the navigation structure. Changes that will be made to the CD-ROM will ensure that less instruction is required at the outset of using the resource. This along with some simplification of the navigation will assist students whose IT skill levels may obstruct their initial use of this resource. Even so, it is clear that if we are to adopt and use this type of technology, then we should also ensure that the students who will be using it have the IT skills required to do so. General moves to provide for the teaching of generic skills, of which use of ICTs is one, will hopefully reduce the level of this problem amongst mature age students.
The evaluation indicated that the CD-ROM did indeed assist the students to gain an insight into the design and conduct of a marine sampling regime. Furthermore, they were better able to choose the best (most cost effective) method for sampling each species. In addition, they gained an experience of a particular sampling technique which they could refer to knowledgably in the subsequent assignment. On marking the assignment, the teaching staff indicated that the CD-ROM did not assist most students in their ability to interpret and explain their quantitative results in terms of their qualitative observations when they were asked to do so in the assignment. However, when questioned during survey and interview, the students were able to point to the patchiness of species distribution and the difficulties in recognising it in their quantitative data. This was the primary factor that also influenced the accuracy and precision of density estimates, and was usually omitted from their written report.
In order to understand this problem, we should consider the core principles of scientific inquiry. These are the ability to formulate hypotheses, collect and evaluate literature and place a value judgment upon it, to gather evidence through data collection, interpret that evidence, and defend conclusions based on the evidence collected and what the literature describes. A gap area is often identified in the gathering of evidence and this CD-ROM simulation assisted in bridging that gap.
Simulations as a tool for learning can authenticate experience for the aspects of discipline content where the practical experience is difficult (Corderoy et al.,1993). Simulations are commonly used to integrate process with content, by engaging the learner in the meaningful practices of the discipline (Brown, et al., 1989; Corderoy et al., 1993; Laurillard, 1993; Alexander & Hedberg, 1994). The Ocean in the Classroom CD-ROM simulates the process of data collection in the marine environment. It 'leads the student to water'. The question is whether a simulation alone can provide the situation that enables the student to 'think'.
If we accept the elements of tertiary teaching and learning as presented by Laurillard's conversational framework model (1993), then we can identify the need for students to re-describe and re-define their knowledge within a 'goal-action-feedback' cycle. It is this component that a 'simulation' based resource cannot by its nature influence. This is within the realm of the teaching and learning domain as identified by Laurillard (1993) when she refers to simulations:
None of them succeed in supporting all the activities in the complete learning process, but they are the only media so far to offer interaction at the level of action in the real world, albeit a simulated one. The link between this and the student's redescription...... makes them pedagogically very valuable, as long as the remaining learning activities can be covered by other means.In this study, the simulation clearly provided a valuable means of engaging students in the real practices of the discipline; it also assisted the students to describe the sampling processes used, and to transfer their knowledge beyond the current situation to other research activities. Nevertheless, it is clear that when we also expect the student to interpret and describe in scientific report format, with reference to the literature, they experience problems. This situation could be improved by the students having the opportunity to present their preliminary findings for consideration and discussion by their peers prior to completion of the written component. Indeed, sceptical questioning and imaginative speculation to reflect on experience have been shown to deepen knowledge (Brookfield, 1987). This area could be accommodated in the subject by introducing the use of a reflective journal, or what Edelson (2000) describes as a progress portfolio, used by the learner to facilitate reflective inquiry. However, this would remain within the realm of the individual and may need some intervention within the world of the expert, to be a more powerful learning tool. A short presentation and discussion of findings could reinforce reflective inquiry on how the knowledge has come to be known, that is somewhat less confrontational for the students than presenting their written work for the scrutiny of experts before they have tested its validity.
Many technology based teaching innovations have been criticised for lack of rigour in the evaluation processes (Alexander and McKenzie, 1998). If we are to improve the learning outcomes and processes we aim to achieve, then a robust investigation resulting in improvements in the teaching situation as well as the innovation must occur. Changes to the CD-ROM, as well as changes to the teaching situation, will be made in an attempt to address some of the areas of concern raised in the evaluation study. However, if we are to truly address the concerns raised by the evaluation there is a need to extend our thinking beyond the resource and the teaching situation towards a change in the way in which we teach scientific inquiry. This is clearly an institutional matter, because the teaching of science is situated within the institution and bound by institutional practices that constrain the way the curriculum is entered into by students.
Our methods of teaching place the content as the focus of curriculum through lectures in which 'the facts' of the discipline are presented to students. A tutorial type activity usually follows, where students are involved in debate and defence of particular assertions or viewpoints presented in the literature. The laboratory then introduces students to established techniques, where precision and accuracy are paramount and a testament to the existing knowledge. Students are rarely, if ever, encouraged to analyse the inadequacies of the foundations of that knowledge and so do not discover its limitations, and cannot adequately address solutions or even explore other possibilities as alternatives to that knowledge (Schwab, 1962). Assessment activities complete the cycle, engaging the student in 'measuring' their results through quantitative processes, thus removing them from the realities of what is actually happening. The basic abilities of a scientist to question and critically analyse are at the core of scientific inquiry and yet they are the skills that our current methods of teaching place least value on.
The conceptual principles of inquiry render scientific knowledge unstable and subject to change because research is not guided by the first principle that formed the inquiry, but is further extended by new conceptual principles (Schwab, 1962). Indeed, institutions are frequently under pressure to present new knowledge as it becomes available. In biological science at James Cook University, the results of recent research quickly find their way into the curriculum content as the lecturers teaching the discipline are also heavily engaged in research. This means that curriculum is very current and relevant and it also places the scientist in a highly esteemed and quite powerful position. This is particularly true where the 'facts' that are being presented to the students in lectures are those of the lecturers, acquired through the hard toils of their research. In this evaluation project the teaching staff indicated that students were quite prepared to rebut these facts in a tutorial situation. However, when students were required to question the literature evidence in a written assignment, they seemed unable or unwilling to do so. There seems to be a tendency by the student to place a higher value on the evidence in the literature than their own evidence, even when their quantitative findings, and what they are seeing, conflicts with what they read.
Providing the expectation that students will fail as they engage in learning, and that this is a normal and expected part of the learning process, could provide the basis for students to contest the evidence they are presented with. It will absorb them in knowledge construction and scientific inquiry that is less confrontational. It will ensure that they do not continue down the well trodden path of hypothesis to theory to fact, but see it as a process of investigation and discovery, learning how to hypothesise and critically analyse. This concern with what is known rather than how it has come to be known is outlined by Laurillard in her analyses of teaching and learning in a tertiary context (Laurillard, 1993). What is needed is a change in the role of teacher to that of facilitator and a shift from a content focused curriculum, where the content is not presented as a rhetoric of conclusions but where students view the 'facts' of science as a search for meaning, concept seeking and concept forming as the main objective (Schwab, 1962).
Hedberg, J.G., Harper, B. and Brown, C. (1993). Reducing cognitive load in multimedia navigation. Australian Journal of Educational Technology, 9(2), 157-181. [verified 21 Jul 2002] http://www.ascilite.org.au/ajet/ajet9/hedberg.html
Apple, M. W. (1979). Ideology and Curriculum. London: Routledge.
Bain, J. D. (1999). Introduction: Learning centred evaluation of innovations in higher education. Higher Education Research and Development, 18(2), 165-172.
Brookfield, S. (1987). Developing Critical Thinkers: Challenging adults to explore alternative ways of thinking and acting. San Francisco: Jossey-Bass.
Brown, J. S., Collins, A. and Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-42.
Bruner, J. S. (1966). Toward a Theory of Instruction. New York: WW. Norton & Co.
Connelly, M. F. (1977). Scientific Enquiry and the Teaching of Science. Canada: Ontario Institute for Studies in Education.
Edelson, D. C. (2000). Learning-for-use: A framework for the design of technology-supported inquiry activities. Journal of Research in Science Teaching, 38(3), 355-385.
Jonassen, D. (1998). Designing constructivist learning environments. In C. M. Reigeluth (Ed), Instructional Theories and Models, 2nd ed. Mahwah, NJ: Lawrence, Erlbaum.
Jonassen, D. H., Mayes, T. and McAleese, R. (1993). A manifesto for a constructivist approach to technology in higher education. In T. Duffy, D. Jonassen and J. Lowyck (Eds), Designing Constructivist Learning Environments. Heidelberg, FRG: Springer-Verlag.
Jonassen, D. et al (1995). Constructivism and computer-mediated communication in distance education. The American Journal of Distance Education, 9(2).
Kolb, D. A. (1984). Experiential Learning: Experience as the Source of Learning and Development. Englewood Cliffs, New Jersey: Prentice-Hall.
Laurillard, D. (1993). Rethinking University Teaching: A Framework for the Effective Use of Educational Technology. London: Routledge.
Lave, J. and Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation. Cambridge, MA: Cambridge University Press.
Kemmis, S. and McTaggart, R. (Eds) (1988). The Action Research Planner, third edition. Victoria: Deakin University.
Piaget, J. (1970). Science of Education and the Psychology of the Child. London: Longmans.
Reeves, T. C. (1991). Ten commandments for the evaluation of interactive multimedia in higher education. Computing in Higher Education, 2(2), 84-113.
Reeves, T. C. (1992). Effective dimensions of interactive learning systems. Proceedings Information Technology for Training and Education Conference (ITTE '92) (pp. 99-113). Brisbane, Australia: University of Queensland.
Reeves, T. C. Multimedia development tools. [verified 21 Jul 2002] http://mime1.marc.gatech.edu/MM_Tools/
Reeves, T. C. (1999). A research agenda for interactive learning in the new millennium. EdMedia World Conference on Educational Multimedia, Hypermedia and Telecommunications (EdMedia'99). Seattle, Washington USA.
Schwab, J.J. (1962). The Teaching of Science. Harvard University Press, Cambridge, Massachusetts USA.
Slack, S. and Stewart, J. (1989). Improving student problem solving in genetics. Journal of Biological Education, 23(4), 308-312.
| Author: Roisin O'Reilly, Head, Teaching Development, James Cook University, Townsville Qld 4811. Email: Roisin.Oreilly@jcu.edu.au
Please cite as: O'Reilly, R. (2002). You can lead a student to water, but can you make them think? An evaluation of a situated learning environment: An Ocean in the Classroom. Australian Journal of Educational Technology, 18(2), 169-186. http://www.ascilite.org.au/ajet/ajet18/oreillyr.html |