| Australian Journal of Educational Technology 1999, 15(3), 242-256. |
AJET 15 |
Denise Kirkpatrick
Charles Sturt University
Simulations can be used to familiarise participants with the complexities of decision making and negotiation. The infusion of computer mediated communication strategies with simulation offers the potential for creating educationally rewarding learning experiences in a cost effective, flexible and realistic manner. A simulation using electronic dialogue about decision making processes at a hypothetical contaminated site has been developed and evaluated. The simulation was found to be an effective tool for supporting learning about the social, political, economic and scientific dimensions involved in managing contaminated sites. The participants particularly valued the opportunity for negotiation and communication skill development within the simulation. The use of a facilitated group reflection process improved learning outcomes. This was evidenced by higher order learning assessed from a SOLO analysis of reflective essays. While electronic dialogue using group discussion software was an extremely useful communication strategy, face to face meetings were required at critical points in the simulation. These were in the introductory part of the simulation and the debriefing phase.
In simulations, participants adopt a functional role or persona within a simulated environment or scenario and it is interaction between these two that leads to learning about the subject or problem (Errington, 1997). Documented examples of simulations to address the issues surrounding decision making about contaminated sites are scarce. A simulated town hall meeting was used to address issues surrounding nitrate contamination of groundwater as an educational experience (Pan, 1996). This involved public authorities (eg. mayor, Soil Conservation Service), industry representatives and staff and students from a university. Simulations can also be used proactively to encourage cooperation and consensus building among stakeholders. A simulation utilising a board game and involving the public was used during the development of remediation plans at a contaminated site (Applegate & Sarno, 1997).
Technology is increasingly being used to supplement traditional face to face communication in educational and business settings. In particular, computer mediated communication (CMC), enabling either asynchronous (eg. email) or synchronous (eg. real time chat sessions) electronic dialogue, is being more widely employed. These communication techniques can allow flexibility in learning activities by facilitating learner participation that would otherwise be problematic because of constraints associated with distance, time or the financial cost of attending a face to face meeting. The inclusion of these communication techniques in teaching can allow extended simulation time frames, which more closely parallel professional practice, where such decisions are usually made reflectively over a period of weeks or months. The United Nations simulation is one example of a traditional simulation using a face to face meeting for participant interaction. An adaptation of this simulation using only electronic dialogue is also available (Electronic United Nations, 1998). Electronic dialogues (Project ICONS, 1998) and teleconferencing (Vincent & Shepherd, 1998) have also been used to involve distant participants in political science simulations. While electronic dialogue can support interactions such as information exchange, opinion and suggestions which are integral to such simulations, it is less suited for communicating agreement and disagreement and for social-emotional tasks involving conflict and negotiation (Hiltz & Wellman, 1997). The theory of media richness (Trevino et al, 1990) suggests that face to face is the richest medium because it allows rapid mutual feedback, natural language, multiple cues to convey meaning, and can convey emotions. Electronic messaging systems such as email are considered a leaner medium. The design of decision making simulations which use CMC needs to take into account the limitations of the communication media.
The focus of this paper is on the development and evaluation of an electronically supported simulation of the decision making process at a hypothetical contaminated site. The project was evaluated continuously over two years using participant feedback, evidence of participant learning, observation of the operations of the simulation, and technical considerations. Given the few reported examples of CMC enhanced simulations, the information within this paper may be useful to those considering using this approach to learning.
Content was structured into 3 modules addressing the concepts: contaminant fate; contaminated site investigation; and management. The modules were designed to move from micro-level knowledge (which was context independent) through the application of processes in real life-work context, to a holistic view which took a broad view of factors that impact on subject knowledge and application. All modules were developed using principles of problem oriented learning (Felletti, 1993; Ross, 1984; Wilkerson & Hundert, 1991) and experiential learning (Reason & Heron, 1986), which required students to solve problems and tackle real life professional questions and tasks, developing related skills and knowledge in context. McLoughlin and Oliver (1995) caution against the use of computer mediated communication technology leading to didactic teaching and learning approaches. To overcome this possibility, an element that was consciously experientially based, and learner directed was included in the subject design.
The contaminant fate and investigation modules were presented completely in a web based mode, using individual and group based site investigations to develop understanding of the key scientific concepts and processes involved in the management of contaminated sites, the "what" of the subject. The application of these processes and knowledge was the "how" of the subject. The Contaminated Site Management module comprised 40% of the assessment for the subject and used a simulation to develop students' understanding of the decision making processes involved in managing a complex environmental problem. The simulation was intended to provide a context where the identification and understanding of issues related to the management of a contaminated site could be developed by engaging in the processes of exploring, inquiring and constructing personal representations. Simulations are problem based learning experiences that are set in motion by a particular task, issue, policy, crisis or problem and are effective in familiarising participants with the complexities of decision making and negotiation (Gredler, 1992). Learning about the role, problem and the situation specific to the subject area occurs through the interaction between participants and the scenario (Errington, 1997). All modules were web based and all necessary resources were available on the subject website with links to additional related sites.
The subject was intended to develop necessary scientific disciplinary knowledge to provide a base, from which students could engage in professional decision making. There was a significant diversity in the background knowledge of the participants, in terms of their understanding of the scientific and social dimensions of the conflict inherent in the simulation. Students were given the option of completing the management module through the simulation or through independent study. For personal reasons one student elected to study independently in 1997 and in 1998 all students enrolled in the subject participated in the simulation. The simulation was run in 1997 with 15 participants and a revised version was run with 22 participants in 1998. Participants were from various courses, including Coastal Resource Management, Civil and Environmental Engineering and Groundwater Management.
The scenario was designed to allow conflict among the participants on the following issues:
The TopClass web based learning software (TopClass, 1998) was used to provide communication support for interactions necessary to resolve the simulation problem. This involved the creation of a group discussion area for the posting of emails, private email and a real time chat facility.
The simulation involved the following phases:
A survey of student computer ownership and access indicated that almost 90% of students had regular computer access and were familiar with basic computer use. A two hour face to face tutorial was designed to familiarise participants with navigating TopClass and using both the group discussion area and private email. Groups of 3 participants played a training game involving a 'shipwreck survival scenario' over 2 weeks using CMC. This encouraged familiarity with using CMC to state a point of view and work as a group to reach consensus as they agreed on the equipment they would need to survive after a shipwreck.
Briefing phase
In this phase participants familiarised themselves with the scenario, identified their persona preference and researched their selected persona. Allowing participants to self select roles encouraged participant "ownership" of the persona. Information on the simulation objectives, environmental issues, the scenario, basic persona information and simulation tips were provided as web pages. Relevant background reports on contamination assessments, company financial statements, and contamination remedial options and costs were provided to particular personae.
The personae available in the simulation represented an Environmental Protection Agency, Water Resources and National Parks agencies, Mayor, Mayoral Candidate, Industrial Site manager, factory workers, Union delegates, ecotourism company, business lobby groups, news reporters, conservative and extreme environmental groups, and a range of residential communities. The Environmental Protection Agency and the Industrial Site Manager roles involved several participants collaborating since they were key roles and were expected to be demanding in terms of the nature and quantity of responses required. Single participants took all other personae. The simulation resources included a one paragraph description of each persona and the web addresses of organisations that would be aligned with the ideology of the persona. Participants were required to fully develop their persona profile and publish it on the web. This profile included statements about the objectives, information needs and constraints in achieving the needs of their persona. Basing the persona and the site to real organisations and a genuine location was intended to create an authentic social and cultural perspective for the participants as well as minimising the amount of background material that had to be created for the simulation.
Interaction phase
The interactive phase began with a face to face meeting at which information was released that created the potential for conflict between the participants. Participants then introduced their persona and began interacting to achieve their persona needs. A second face to face meeting was held 6 weeks later to help progress to the later stages of the decision making process where options to clean up the site were canvassed. Apart from these scheduled meetings all interaction took place in a distributed fashion using CMC (in 1998 there were 99 messages exchanged between 20 personae over 66 days). Where several people shared a persona, private telephone discussions were often used to communicate. Real time chat was provided but not utilised by participants.
Participants were not prevented from meeting in a face to face setting during their other subjects. However, as the students came from 3 different courses and many worked part time it was unlikely that this would be convenient. The facilitator engaged in minimal interaction during the simulation and acted mainly as an adviser when characters sought resources or ideas on how to handle a situation. Posted messages were not moderated and the face to face meetings were run by the relevant personae. An anonymous login was provided for participants to express points of view with which their persona may not have wanted to be associated.
Debriefing/reflection phase
Debriefing is an essential part of the experiential learning process. It is a structured post hoc method involving guided recall, reflection and analysis of the experience involving three phases (Gredler, 1992; Lederman, 1992). This involves a systematic reflection and analysis of the experience followed by an intensification and personalisation of the experiences by the participants. Finally participants generalise their experiences and discuss the application and implications of the experience. The debriefing for the simulation included a face to face meeting and consisted of both large group and small group discussions. The essays and concept maps developed as assessment tasks were used to support the debriefing process and direct attention to the content and process of the simulation. There was opportunity for private reflection over the following two weeks before the assessment tasks for the simulation were due.
The simulation was also intended to develop understanding of the responsibilities and appropriate responses for characters in the simulation, and to develop communication, negotiation and decision making skills. Achievement of these objectives was evaluated using peer assessment of: persona participation; goals and strategies; use of appropriate language; preparation; prompt responses; and the creation of opportunities to further the resolution process.
An analysis of the content from the electronic dialogue exchanged by participants showed that in both years most interactions related to the negotiation of positions and making decisions based on rhetoric rather than scientific data. There was little evidence of extra curricular study into the scientific basis for making decisions. It appears that the simulation was successful at allowing participants to integrate and communicate their existing knowledge rather than collecting in depth knowledge specific to the scenario. Only those participants with no training in groundwater contamination felt that they had learnt some of the scientific dimensions associated with this management problem.
A second learning objective was that participants identify the responsibilities of various characters and exhibit appropriate responses for their persona in that context. Examination of the electronic messages during the simulation showed that all participants stayed within appropriate boundaries for their character without facilitator intervention. The role profiles provided by the participants indicated general awareness of professional responsibilities associated with their roles and the use of external web sites in 1998 to supplement persona development assisted participants to collect the information they required to develop this understanding.
The final learning objective of the simulation was to develop communication, negotiation and decision making skills in the participants. A key indicator of the level of these skills was the quality of interactions within the simulation, judged by participants' use of appropriate information and strategies to achieve their aims and the effectiveness of communication in achieving desired outcomes within the scenario. Participant feedback indicated that all students believed the simulation gave them the opportunity to develop these skills. Most participants demonstrated an awareness of the limitations and benefits of both face to face and electronic dialogue in achieving different communication and decision making tasks.
Most participants were aware of the impact of the available communication medium for achieving their communication needs. In the first simulation run there were no face to face meetings scheduled for the interactive phase. However, in response to a perceived need the participants scheduled their own meeting. They reported that although electronic dialogue provided the flexibility to communicate when it was convenient to the participants, it sometimes led to a stilted dynamic compared with face to face communication. Participants believed that face to face meetings allowed them to draw out fundamental problems through body language and the expression of passion. This reflects limitations in the use of electronic media to facilitate consensual understanding (Hiltz & Wellman, 1997; Trevino et al, 1990). Participants felt that strategic face to face communication sessions could create greater participant engagement during the intervening electronic dialogue. Participants also found that they could convey a vast amount of information at face to face meetings. In the second run of the simulation a face to face meeting was scheduled both at the beginning and middle of the interactive phase. The participants had identified that it was at the beginning of the simulation when they were identifying their objectives and positions and during the debriefing phase that face to face communication was important. Although real time chat was available within the CMC environment it was not utilised. It is possible that better training with this media may substitute for some of the face to face communication by allowing instant feedback.
Participants noted that the use of assigned login names based on their persona was beneficial. Harasim et al. (1995) suggest that an online environment is particularly conducive for simulations and role plays because networking allows for anonymity and the use of pseudonyms which can support the adoption of roles and personae.
Measuring participant engagement and contribution through the level of interaction can be problematic. The quality and quantity of the messages posted by participants has been used to assess their contribution and the degree of engagement of participants in simulations (Vincent & Shepherd, 1998). This may not be appropriate for this type of simulation where the different personae had significantly different levels of interaction expected from them. Some personae had key roles and could be expected to comment on most issues (eg. Environmental Protection Authority) while other personae had a limited sphere of jurisdiction. In 1998 there were 89 messages posted over the 2 month duration of the interactive phase. The number of messages per character varied from 1 to 16 with an average of 6 while 4 characters contributed over half the exchanges. More investigation is required to establish the extent to which these differences were a consequence of variation in the characters, levels of computer access and participant engagement. One potential measure of participant interaction may be to record the amount of time that each participant is logged into the discussion area. This is possible using a feature in the TopClass software.
Creating conditions for adequate learner interaction and experience while maintaining a realistically simulated environment presents a significant design challenge. The participants stressed the need for changes that improved character dynamics. No multimedia resources were provided on the physical setting of the simulation, since this was considered to be resource intensive and not likely to significantly improve participant engagement. This approach was consistent with Hays and Singer (1988), who argue for the provision of the minimum of realism necessary to simulate function.
The SOLO taxonomy was a useful tool in analysing aspects of student learning. This was used to identify that student learning outcomes were improved through the use of a facilitated group reflection process during the debriefing phase of the simulation.
The use of electronic dialogue through group discussion software was found to be an extremely useful communication strategy. It provided a flexible and reflective communication medium. It was found that electronic dialogue via email was not always able to satisfy the communication needs of the participants. Face to face meetings were identified as desirable at the start of the interaction phase and during the debriefing. It is important that the design of a simulation allows access to communication facilities suited for the various types of interactions required by participants.
The participants rated the enthusiasm of other participants as the most important feature of the simulation. Improvements to character dynamics were identified as more important than multimedia enhancements. Measuring participant engagement through measuring message quantity and quality, although used in other simulations, was not appropriate in this simulation, due to differences in the responsibilities of the various characters.
Applegate J.S. and Sarno, D.J (1997). FUTURESITE: An environmental remediation game-simulation. Simulation and Gaming, 28(1), 13-27.
Biggs, J.B. (1992). A qualitative approach to grading students. HERDSA News, 14(3), 3-6.
Biggs, J.B. and Collis, K.F. (1982). Evaluating the Quality of Learning: The SOLO Taxonomy. New York, Academic Press.
Electronic United Nations (1998). http://www.simulations.com/eun/glos.htm (accessed 23 Nov 99).
Ehrman, S. (1991). Gauging the educational value of a college's investment in technology. Educom Review, XXVI, 3(4), 24-28
Errington, E. (1997). Role Play. HERDSA Green Guide No. 21, Canberra, Higher Education Research and Development Society of Australasia
Felletti, G. (1993). Inquiry based and problem based learning: How similar are these approaches to nursing and medical education? Higher Education Research and Development, 12(2), 143-156.
Gredler, M. (1992). Designing and evaluating games and simulations: A process approach. Houston, Gulf Publishing.
Harasim, R., Hiltz, S. R., Teles, L., and Turoff, M. (1995). Learning networks: A field guide to teaching and learning. Cambridge, MA: MIT Press.
Hays, R.T. and Singer, M.J. (1988). Simulation fidelity in training system design: Bridging the gap between reality and training. New York, Springer-Verlag.
Hiltz, S.R and Wellman, B. (1997). Asynchronous learning networks as a virtual classroom. Communications of the ACM, 40(2), 44-49.
Kirkpatrick, D. L. (1994). Evaluating training programs: The four levels. San Francisco, Berrett-Koehler.
Kirts, C.A, Tumeo, M.A. and Sinz, J.M. (1991). The COMMONS GAME: Its instructional value when used in a natural resources management context. Simulation & Gaming, 22(1), 5-18.
Lederman, L.C. (1992). Debriefing: Towards a systematic assessment of theory and practice. Simulation & Gaming, 23(2), 145-160.
Pan, W.L. (1996). Role playing and mind mapping issues on nitrate contamination. J. Nat. Resour. Life Sci. Educ., 25(1), 37-42.
Project ICONS (1999). http://www.icons.umd.edu/about/index.html (accessed 23 Nov 99).
Reason, P. and Heron, J. (1986). Research with people: The paradigm of cooperative experiential enquiry. Person Centred Review, 1, 457-476.
Ross, B. (1991). Towards a framework for problem based curricula. In D. Boud and G. Felletti (eds), The challenge of problem based learning. London, Kogan Page.
Swinerton, L.N. (1979). The Dead River; A simulation game of river restoration. In Exline, C.H. and Larkin, R.P, Simulation Review, Simulation & Games, 10(1), 87-90.
TopClass (1998). http://www.wbtsystems.com/ (accessed 23 Nov 99).
Trevino, L. K., Lengel, R. H., Bodensteiner, W., Gerloff, E.A., Muir, N. K. (1990). The richness imperative and cognitive style: The role of individual differences in media choice behavior. Management Communication Quarterly, 4(2), 176-197.
Vincent, A. and Shepherd, J. (1998). Experiences in teaching Middle East politics via Internet-based role-play simulations. Journal of Interactive Media in Education, 98(11). http://www-jime.open.ac.uk/98/11/vincent-98-11-paper.html (accessed 23 Nov 99).
Webster, J. and Hackley, P. (1997). Teaching effectiveness in technology mediated distance learning. Academy of Management Journal, 40(6), 1282-1309.
Wilkerson, L. and Hundert, E. M. (1991). Becoming a problem based tutor: Increasing faculty awareness through faculty development. In D. Boud and G. Felletti (eds), The challenge of problem based learning. London, Kogan Page.
| Authors: Robert McLaughlan, National Centre for Groundwater Management, University of Technology, Sydney. PO Box 123, Broadway NSW 2007, AUSTRALIA Email: Robert.McLaughlan@uts.edu.au Phone +61 2 95142614 Fax +61 2 95141985
Denise Kirkpatrick, Centre for Enhancing Learning and Teaching, Charles Sturt University, Panorama Ave, Bathurst NSW 2795, AUSTRALIA Please cite as: McLaughlan, R. and Kirkpatrick, D. (1999). A decision making simulation using computer mediated communication. Australian Journal of Educational Technology, 15(3), 242-256. http://www.ascilite.org.au/ajet/ajet15/mclaughlan.html |