Australian Journal of Educational Technology
2002, 18(2), 127-146.
AJET 18

An evaluation of tertiary language learning through student constructed multimedia: The Interactive Stories approach [1]

Michelle de Souza and Michael Fardon
The University of Western Australia
Rob Phillips
Murdoch University
The Interactive Stories [IS] approach offers language students an alternative way to learn language skills, through authoring and producing their own 'choose your own adventure' stories using software for interactive multimedia. This paper describes an evaluation study of the Interactive Stories approach. The main focus of the study was to investigate the learning processes used by students, and how well they met the desired learning outcomes. The study also investigated how well students coped with using new technologies and ways to improve the approach.

The study established that students attained skills in reading, writing, speaking and listening, even though they did not realise this at the time. Students were able to creatively use multimedia to place their language in an authentic context. Because students worked in groups, a sense of community was achieved within the class. The areas of improvement identified are mainly administrative, and do not point to any fundamental weakness in the approach. However, the IS approach seems to be most applicable to intermediate and advanced students.


Introduction

Background

It's nice to do a project that you can show off to people, something that you have creative control over... It's interesting and you learn new skills. (Interview with advanced level French student participating in project)
For several years, the Multimedia Centre (MMC) in the Faculty of Arts at The University of Western Australia has been exploring ways of promoting creative and innovative use of multimedia, especially by students. One major focus has been in integrating multimedia work into mainstream courses of study in the humanities and social sciences. One aspect of this is the Interactive Stories [IS] model, in which students create their own language learning projects. The model utilises the StoryTime authoring tool (Fardon & Kinder, 1997) developed by the MMC, which allows the author to configure relationships between text, visual and audio media on screen using a variety of interfaces. The generic skills associated with student multimedia projects are becoming increasingly important in a society where "changing work practices have increased demands for information technology [IT] skills within non-IT professions" (Lawson & de Matos 2000, p.87).

Students who participate in the IS program work in groups to create an Interactive Story for their peers to use. One side of the 'conversation' is provided by digital video segments played on the computer. After each segment, the user is presented with a set of branching points, usually presented as text. As a choice is made, a subsequent video segment is played, resulting in a complex interweaving of narratives and responses. The end result is a multimedia version of a 'choose your own adventure' story.

Figure 1 shows an example of an Interactive Story created by students using StoryTime. The user chooses from the text options on the left hand side of the screen and a response to the chosen statement appears on the movie screen, which, in turn, links to more text options and so on.

Figure 1

Figure 1: A student produced Interactive Story

Subsequent to the perceived success of the initial trial of Interactive Stories used in a French language class (Jaccomard, 2000, 2001), the authors presented the student projects in a seminar with staff members from the other language departments in the Faculty, four of whom decided to integrate the approach into their courses in 2000. It became evident that a growing number of language teaching staff were interested in using the IS approach. At the same time, the MMC decided to evaluate the use of IS and the student learning resulting from the approach.

The Interactive Stories approach actively involves students in scripting a narrative or dialogue, filming appropriate segments, editing them and then constructing a multimedia version of their script. Students further their technological aptitude through exposure to the StoryTime authoring software, iMovie and QuickTime software, digital video, and the digitisation and manipulation of audiovisual resources. By creating their own material and bringing it to life, observing their creation in action, it was felt that students would gain "a greater appreciation of the subtlety and nuance of linguistic interaction, and learn some of the skills necessary to shape and motivate their own language learning progress." (Riggenbach, 1988, p.118).

The script produced by students is refashioned in a number of editing phases, one of which is a result of feedback provided by the class teacher in the first round of marking. It is important to note that unit coordinators have chosen to ask intermediate and advanced students to create situations that hold a particular social, historical or cultural relevance to the target language. In order to meet this requirement students may choose to consult a number of resources: other students, teachers, native speakers, texts or other audiovisual aids. Students are asked to pay attention to the registers of language and the delivery of the spoken language so as to recreate an ambience which is true to the contextual atmosphere of their chosen topic.

The Interactive Stories approach

The overarching essence of the IS approach is defined by constructionism and social constructivist theories of learning inasmuch as it is learner centred, project based and collaborative. Papert (1993) describes constructionism as giving students things to do so they can learn it as pertaining to something real. This makes the learning experience more meaningful and memorable. Jonassen (1996) points out that what constructivism and constructionism have in common is that they focus upon the learner as an active constructor rather than a passive receiver. Current thinking supports student centred approaches in the teaching and learning process, where the student is an active participant and where there is a greater emphasis than before on learner interaction (Meskill, 1999; Jager et al. 1998).

The first element to define the IS approach is that it is learner constructed. By this it is meant that students use computer based composition applications to create their projects. Jonassen explains that certain computer applications "require students to think in meaningful ways in order to use the application to represent what they know" (Jonassen, 1996, p.3). He defines these applications as 'Mindtools'. StoryTime and iMovie are two such Mindtools that students utilise as part of the IS approach. StoryTime is a hypertext based composition tool, while iMovie is video based:

With a little experience, individuals can become their own artists, publishers or video producers. (Jonassen, 1996, p.3)
The principal aim of any strategy implemented in a language teaching program is to advance the linguistic aptitude of the learner. In IS, students are not only applying their language skills but they also have the added benefit of being able to construct stories that are non-linear, thus giving them the experience of engaging in language and logic in a real world that is not linear but multidimensional. This is an important aspect of teaching language interaction, as Gambhir (1995) points out.

The second element which defines the IS approach is that it is project based. The primary outcome is a physical product - a multimedia program that students produce and present to their class. Research supports project based learning (Debski, 1997; Barson, 1999; Meskill, 1999):

In a profound way, physical products make learning public. Having a physical product during as well as after a learning activity keeps learners thinking, communicating and speculating about what might come next. (Fisher et al, 1996, p.122).
As students will be working in groups, the third element which defines the IS approach is collaboration which assumes that learning is constructive and social. In addition to this, "in collaborative learning situations, students generally experience a shift in their intellectual development as they learn to articulate their own point of view and listen to the view of others" (Goodsell et al, 1992, p.11).

Evaluation design

In designing an evaluation of Interactive Stories, the focus was on investigating the pedagogical status and implications of the approach, as it imposes a new structure upon language teaching.

A series of evaluation questions was investigated in the context of an evaluation framework appropriate for the implementation of technology innovations in higher education. The evaluation questions required that the framework cater for both formative monitoring and summative evaluation processes. It was also important that the evaluation process be reflective given the interdependence of the questions identified, ie. the learning that is taking place will affect the assessment procedures that will be adopted.

Evaluation framework

The development of the evaluation plan of this study was informed by the Learning Centred Evaluation [LCE] Framework described in Phillips et al, (2000) and Bain (1999). This framework, derived from earlier work by Alexander & Hedberg (1994), has three main characteristics: The LCE framework acts as a scaffold for the development of specific questions, by breaking down the lifecycle of an educational innovation into phases, and explicitly distinguishing the roles of the learning environment, the learning process and the learning outcome.

In this work, the LCE framework has been expressed as a matrix (see Table 1). The left hand side of Table 1 contains the phases and foci of the LCE framework. The third column of Table 1 displays the specific evaluation questions that we developed by working through the framework. Use of the LCE framework aided us in focussing on the types of questions to ask.

This research is grounded in an interpretive paradigm, which was felt to be most appropriate for eliciting information about student learning in complex situations. The intention was to understand the learning taking place at different levels in a natural and non-manipulated environment. An eclectic approach has been taken to choosing the methods used to obtain information to answer the questions posed in Table 1. However, qualitative sources of data were used in most cases, because the richness of such data can yield insights about the actual learning processes used by students. This is discussed further in the following section.

Data sources

For each question in Table 1, we considered the most appropriate source(s) of data to provide evidence to answer the question. This analysis resulted in six generic data production methods: examination of final assessment results, documentation of events, staff group interviews, student interviews, journals and observation of staff and students. The questions to which the data production methods are appropriate are shown in the right-most six columns of Table 1. The range of data production methods enables us to check the internal consistency of the data, for purposes of triangulation. Patton affirms that triangulation is an "important way to strengthen a study design" (1990, p.187). In this study various data sources have been used to answer the evaluation questions. For instance, in order to gather information on how easy it was for the students to use the technology, the following data production methods were used:

Key to abbreviations of data collection methods in Table 1

   
A
F1
F2
I1
I2
MO   
assessment
first round focus group
second round of focus group
interview 1
interview 2
class observation by MMC staff member  
SJ      
SP
TJ
TO
VF
student journal
student projects/final product
teacher journal
teacher's observation of events
video footage collected by students
See also Table 2 for explanation of terms.

Table 1: Evaluation phases of Interactive Stories model as represented in the Learning-Centred Evaluation framework (Alexander & Hedberg, 1994; Bain 1999; Phillips et al., 2000). See text for explanation of terms. Data production methods
(see key in text)
PhaseFocusQuestionsAssess-
ment
Docum-
entation
Staff focus gpStudent
interview
Analysis
and design
Curriculum analysis1. What learning outcomes are appropriate? What do students need to be learning?

F1
Teaching for learning
analysis, and
Specification of
innovation
2. How will the learning outcomes be achieved through the Interactive Stories approach?

F1
3. What assessment processes and marking criteria?A
F1&F2
4. What is a good script and why?

F1&F2
5. What are the pros and cons of the IS approach?

F1
6. What evidence from the 1999 trial can be used to improve the Interactive Stories approach?
SPF2
DevelopmentFormative monitoring of learning environment
and
Formative monitoring of learning process
7. How easy is it for the students to use the technology?


I2
8. How does lecturer input throughout the process influence the students' work?

F2I2
ImplementationSummative evaluation of learning process9. How are dialogues constructed in the IS environment?
SP
I1&I2
10. How much (and what sort of) team work and reflection occurs?
VFF2I2
Summative evaluation of learning outcome11. How well do students meet the learning objectives?ASP;VFF2
12. What are educational benefits of the IS approach?AVF;SPF2I2
Summative evaluation of innovation appropriateness13. Do the educational benefits match the workload of the students?

F2I2
14. Could the IS approach be improved and how?

F2I2
Institutional-
isation
Maintenance evaluation15. Can this model be used equally well across a variety of language units?

F2
16. What factors determine the sustainability of the "interactive conversations" model?

F2

Project participants

The project group consisted of two project evaluators, an evaluation mentor, the teaching staff, and a selected sample of students.

The teaching staff consisted of four language teachers, all of whom were the unit coordinators of the language units in which the Interactive Stories project was being conducted. The teaching staff participated in the focus groups, kept journals of their observations as well as helping with some of the administrative tasks such as handling the ethics approval forms and following up the students for interviews. They also contributed to the planning of future implementations of Interactive Stories.

A group of twelve students was chosen to form a study sample: three students were selected from each of the four language units. The selection of students ensured that an adequate range of gender, age, background, and experience was sampled. Each of these students participated in the interviews and kept journals throughout the project duration. The language units which participated in the IS program and their levels of study are shown below in Table 2.

Table 2. Characteristics of the areas in which Interactive Stories was being taught. There are two levels of study, depending on whether students have had prior experience in the language prior to enrolling.
SubjectYear of StudyLevel of Study
Chinese1st yearBeginners
French3rd yearAdvanced
German1st year
2nd year
Advanced
Intermediate
Italian2nd year
3rd year
Advanced
Intermediate

As well as contact with the class teachers, the students in the study sample had significant contact with the evaluators who led the interview sessions, and acted as a first call for support. The primary evaluator had most contact with the students and observed and made notes about their experiences and behaviour.

All students gave signed, written consent for their material and information to be used for evaluation and publication purposes.

Data analysis

Focus in analysing qualitative evaluation data comes from the questions generated at the very beginning of the evaluation process (Patton, 1987, p.145)
An inductive cross-case analysis was applied to this study (Patton, 1990). The data collected were classified into predetermined categories generated by the evaluators. In this case, the categories were dictated by the questions listed in the LCE framework in Table 1. The first stage of sorting the data into patterns according to the outlined themes was undertaken by the primary evaluator, while the second stage of reviewing the resulting classification scheme was undertaken by the secondary evaluator.

Results

Analysis and design

The Interactive Stories approach addresses all four macro language learning areas in the one project: reading, writing, speaking and listening. Students would be actively involved in writing dialogues, memorising and performing scripts, as well as directing and editing their multimedia projects. The language teachers stressed that the learning outcomes should be achievable for students at different language levels. The following four learning outcomes were identified: The staff focus group participants agreed that the desired learning outcomes were likely to be achieved through the IS approach, because: The use of a variety of media in the IS approach allowed students to create a virtual environment that, in turn, generated the linguistic and cultural features of the language's environment in ways not otherwise possible. For example, one group of students worked on a project about the Italian painter Caravaggio. In one scene, the students created a film of a prostitute who frequented Caravaggio's circle of acquaintances. The actress was dressed in period costume and was filmed outdoors before a pond in the oldest part of the UWA campus. She was in a seductive pose and she used terms in her speech which were reflective of the time period she belonged to. For example, she spoke of "scudi" instead of "lira" to refer to the Italian currency of the time. In another project, the students created a simulation of a ghost, by using a special 'fade in and fade out' effect in iMovie.

An assessment checklist was provided to students with explanations and examples, so that they knew what was expected of them. Most teaching staff used the same style of assessment checklist, but there were some variations between classes. For example, in order to assess the correct use of language in context, one unit coordinator allocated a mark for the filmed setting. Another unit taught novice, first year students. These students could not be expected to portray language in context, because of the limits of their ability in the target language, nor were they able to produce a lengthy script. A segment of a script is shown in Figure 2.

Figure 2

Figure 2: An example script

Each number on the tree at the left refers to a sentence or utterance on the right. Structured in this way, it is easy to view the overall flow of the script, and its scope.

The following characteristics of an 'ideal' script were identified by the staff focus group:

Development

The development evaluation focussed on formative evaluation of both the learning environment (the IS model) and the learning process (what were students doing?), to answer the specific questions identified in Table 1.

The majority of students stated that they had experienced problems in using the technology and they all found that this was unnecessarily time consuming:

We lost a couple of hours trying to get things to work and we found that we would lose half of the morning before we got to the point of asking [for help]
However, not all of the students expressed frustration at the difficulties encountered and, even though they may have been faced with challenges, they didn't feel it was an obstacle. A great deal of technical assistance was required by the students, especially during the editing stages of the project.

Observation of class teachers indicated that they tended to use a show and tell technique in their classes. Teaching staff commented that they regularly attempted to draw the attention of the students to important aspects of the project. However, the class observer often noted that some students were either not devoting full attention to the class teacher (hence they could have missed out on some important bits of information) or they were just confused. Teaching staff sometimes did not fully understand or explain all technical procedures, especially important ones, asking students to contact the MMC staff for support.

MMC Observer: Teacher X can't get camera to download into iMovie. Then X says, "When you do it, it will work for you." He doesn't ask me for help and I won't interfere. It is the end of the lesson so X tells everyone to read their notes.
There was a need for teaching staff to be technically proficient in the use of Storytime, iMovie and the Macintosh operating system software before using the IS approach in their course.

Students asked to be provided with ideas and more instructions to aid their creativity:

I still would have liked to have known what a computer can do because I don't know anything about that. There should be a board with all sheets on things you can do - sound effects and things like that.
There was a further need for reminders to students throughout the semester about deadlines, etc.

The shortcomings identified so far are concerned with administrative aspects of the Interactive Stories approach, and can be largely rectified through documentation and training.

One language issue was identified by students and staff. That was a lack of opportunity to speak in the target language. Teaching staff agreed to incorporate a greater amount of speech work in their classes, in two ways: (a) to use the language more frequently in class and incorporate it into class instructions (especially for advanced students); and (b) by asking students to present critical reflective comments in the target language, either in front of their classmates, or in a journal throughout the course of the semester. As one teacher commented:

Students are getting carried away with the technology. Not much emphasis on German dialogue. Need to work on putting more emphasis on use of language in class.
Both of the unit coordinators who have undertaken this project in 2001 have been using more of the target speech in class.

Implementation

The implementation evaluation focussed on summative evaluation of learning - both the learning process (what were students doing?) and the learning outcomes (what did the students learn?). In addition, the appropriateness of the IS approach was investigated.

There was a difference in the way that students at different levels constructed their scripts. All three beginners to the language who were interviewed said that they had consulted textbooks for help with grammar when constructing dialogues. They also designed and wrote their scripts in English instead of the target language.

Beginner: We were limited to what we know but it was time consuming to go through the dictionary and look up words that we were not familiar with. We thought it would be too simple to copy a dialogue out of a text. We'd work out what we wanted to say and then translate it.
The more advanced students, on the other hand, thought and wrote in the target language:
Advanced Student A: Thought and written in Italian!
Advanced Student B: Also, composed in Italian and French.
In both cases, students were not simply copying dialogues from textbooks because written dialogues in regular textbooks are not linear and could not be directly copied into their project.

The implicit structure of the IS approach suggests a dialogue construct with a conversational question-answer style, rather than a narrative style. However, it is possible to use IS in more sophisticated ways. In reality, not everyone converses in a question-answer type manner. The students who felt compelled to use this style of interaction in their scripts found it difficult to write dialogues that seemed natural:

Student: It was hard to come up with options for linking information. You had to come up with a question that you knew would work well.
One alternative is to not to give an example in the training. The second alternative is to provide different examples of Interactive Stories, using both narrative and dialogue techniques.

All students but two reported that they had successfully worked in unison with their partners to produce the script for their dialogue:

Student: It does help [working with a partner] but... and... there are always things that one person doesn't know that the other does.
Students grappled with the issue of expressing constructive criticism:
Student: Often, with the pronunciation you don't know the person that well enough. You don't feel comfortable saying to that person, "Stop you're wrong."
Both students and staff reported that the project circumstances brought all the participants closer together and overall, staff identified 'group work' as a benefit of the program:
Some students hate group work - particularly high achievers. Particularly one student wouldn't allow the other students to do any work because they wouldn't get a high distinction. Yeah... a real problem but... and again, if I had been aware of that earlier, I would have been able to step in and work with that group. Yet they are so aware that one of the things that will always come up when they apply for a job is being able to work with other people in a team.
Table 3: Student and staff perceptions of language skills learnt using Interactive Stories

SkillsStudent feedbackStaff observations
Speech: pronunciation, presentation, delivery and intonation.The only scene we had a problem with was a long speech, the last word 'raccappriccianti'. The more X tried, the more they tripped up. We left it and came back to it last - got it on take six.Not in all groups but some groups took care to make sure that they were working on their pronunciation.
WritingI think it did a lot for our language learning because we wrote the script and the person spoke on the camera...
Registers of language and forms of addressWhich skills are the focus depends on your choice of topic. For example, the Italian was more familiar language use and the French one was a more formal interview.They paid a lot of attention to the script they handed out to me. Writing skills, finding the right register of language and expanding on semantic skills.
Semantic and idiomatic expressionYou're finding useful information and there's a whole range of vocab associated with it and a whole range of expressions, grammar and registers.
Conversational structuresI think for Chinese, it was good to be able to speak freely beyond the class. Even if it was quite short it did help. It's good to speak and listen - especially to listen. You have to read as well.Some of them said that because it was an interactive conversation, they had to adopt a conversational tone and they learnt more informal structures and language than they would normally pay attention to in class.
Reading and listening to languageStudent X [when asked how language had improved] By listening. By reading. Listening skills.My students said that researching the culture you had to look up sources that were in the language.
Asking questionsWe felt compelled to put questions in there as well.I was surprised to see how many students didn't know how to ask questions in French.

When they were interviewed, half of the students did not perceive that any language learning or development had taken place. For example:

I think whatever level students were before the project they would be just the same at the end of the project.
All staff agreed with the view that the technical problems experienced by the students were so overwhelming that they were not focussed on the language learning that was taking place. However, other data sources indicate that the students were indeed applying their knowledge of language structures and learning from each other. Most students contradicted themselves either in their descriptions of interactions with their partners, or as they captured each other on video while filming, often in heavy debate over language structures:
From out-takes of video footage:
Student being filmed: Shouldn't we be using 'Sie' though?
Student behind the camera: Nah, you'd be using 'Du'.
The evidence suggests that students were learning language skills, but the students did not notice this, because they were engaging with the complexities of the project work. The language learning was an implicit outcome of the authentic task. The Interactive Stories model clearly enables specific linguistic skills of individual students to be enhanced as demonstrated in Table 3.

The students' language skills were also assessed in terms of their performance in their projects, according to the assessment criteria specified for the project. From the results, it is evident that the teachers found that the students performed very well. In the Italian class, scores ranged between 67 and 89 percent, with an average of 80. Overall, the scores of the group of advanced French students ranged from 79 to 87 percent. Overall, the scores of the group of intermediate German students ranged from 67 to 84 percent. Scores were not made available for the Chinese class.

Aspects of creativity in language skills were expressed in terms of researching, applying logic to the script and creative presentation of the subject matter. Teachers of the advanced classes agreed that students went to particular effort to pay attention to the correct use of language, including language in context. Creative thought was applied in a range of ways to the dialogues and scripts in order to embellish the atmosphere of the stories:

We tried to speak a strong Austrian-German accent like the guys on Cool Runnings in our Jamaican pavilion, and we also tried to liven things up like when talking to Hitler.
Overall, the projects produced by the students demonstrated that they made use of technical features creatively, in order to enhance the overall presentation of their ideas. In one German project, for example, the students introduced their topic by filming a scene from outer space on their television screen. They superimposed scrolling credits in German onto this footage using iMovie, as a means of introducing the concept of their story. The authors of this project commented that they were trying to get the same effect as the first Star Wars movie. It is clear that students participating in Interactive Stories projects were able to place their application of language in an authentic context, and were able to creatively use both language and multimedia.

Students also gained skills in group work and collaboration. Interestingly, the students found that the difficulties involved in creating their projects drew the class together into a learning community. Most students also enjoyed the project and said they would recommend the project to future students regardless of any of the difficulties they encountered along the way.

I'd recommend it. I think it was fun as much as it was annoying.
All teaching staff felt that the project had been successful educationally, and indicated that they would repeat the project in future.

An initial concern of the project team was that the IS approach would impose too much work on the students for the educational benefits obtained. However, most students interviewed and observed did not have significant concerns about the workload.

Conclusion

Improvements

The areas of improvement in IS identified by the stakeholders are mainly administrative, relatively simple to achieve, and do not point to any fundamental weakness in the approach.

The technical proficiency of all participants in the project was an issue, particularly the technical proficiency of the teaching staff. Teaching staff need to become more proficient in the technology to reduce the support load on technical staff. Improved documentation will assist in this aim.

Students also wanted clearer information about what was required of them by their teachers, and how they were to be assessed. In addition, they wanted to be provided with ideas and more instructions to aid their creativity. Students were observed watching the work of other groups to see how they were doing things and asking how they could do something like that as well.

The other improvement which was identified was to have more opportunity for spoken language in the classroom.

Applicability

The IS approach may not be suitable for novice language learners. Novices were faced with two very new areas to deal with at the same time - grammar and technology - and this was overwhelming for them. The IS approach seemed to lend itself better to the application of language skills and the refining of linguistic competence, than to learning new language structures.

The IS approach seems to be very applicable to intermediate and advanced students. It gives students opportunities to creatively apply language in an authentic context. In addition, students gain generic skills in negotiation and teamwork, with the added benefit that they 'publish' something, which has a motivational effect (Riggenbach, 1988, p.118). Furthermore, IS should continue to be offered in language courses, but only as long as the students are still acquiring 'new' and useful skills and not ones with which the majority are familiar.

Sustainability

The Interactive Stories model may not be sustainable in the longer term with the current, or a larger, number of language units. There was an obvious need to negotiate resource and support issues between teaching and support staff prior to adoption of Interactive Stories in a given semester. For example, there is only a limited number of available video cameras. If facilities and resources could not be improved then the projects would have to be offered to fewer groups of students each semester.

Major findings

The major findings of this study have been summarised in Table 4 which provides answers to the specific evaluation questions relevant to the Development, Implementation and Institutional evaluation phases of the evaluation framework summarised in Table 1.

In conclusion, the evaluators have identified three areas for further research. Firstly, there is a need to investigate assessment procedures and to look at the best ways of communicating these to the students especially when they are faced with new activities. Secondly, the MMC would like to extend the application of the IS approach beyond its present use in language teaching for example, in creative writing, philosophy or law. Thirdly, the MMC would like to research other applications, or MindTools (Jonassen, 1996) which allow students to represent their ideas and concepts in a meaningful way.

Table 4: Summary of major findings

Evaluation questionsFindings
How easy is it for the students to use the technology?Students found it difficult to use the technology, but rose to the challenge.
How does lecturer input throughout the process influence the students' work?Better documentation and training needs to be provided.
Lack of technical skill was an issue.
How are dialogues constructed?Students had to think non-linearly and creatively to design their scripts.
Advanced students worked in the target language.
How much (and what sort of) team work and reflection occurs?The team aspect was largely seen as positive.
A sense of community was fostered in the class.
Little evidence of reflection was found.
How well do students meet the learning objectives?Skills were attained in the four macro language learning areas (reading, writing, speaking and listening).
The language learning was an implicit outcome of the authentic task.
What are the educational benefits of the Interactive Stories approach?It is clear that students participating in Interactive Stories projects were able to place their application of language in an authentic context, and were able to creatively use both language and multimedia.
Do the educational benefits match the workload of the students?Largely, yes.
Could the Interactive Stories approach be improved and how?The areas of improvement identified by the stakeholders are mainly administrative, relatively simple to achieve, and do not point to any fundamental weakness in the approach.
Can this model be used equally well across a variety of language units?The IS approach seems to be most applicable to intermediate and advanced students.
What factors determine the sustainability of the "interactive conversations" model?The Interactive Stories model may not be sustainable in the longer term without extra resources.

Endnote

  1. The evaluation study of IS described here is guided by the principles and methods outlined in the Handbook for Learning Centred Evaluation of Computer Facilitated Learning Projects in Higher Education (Phillips et al, 2000), which arose from a staff development project funded by CUTSD (Committee for University Teaching and Staff Development) and facilitated by ASCILITE (Australasian Society for Computers in Learning in Tertiary Education). A longer version of this paper appears in the project report at http://cleo.murdoch.edu.au/projects/cutsd99/

References

Alexander, S. and Hedberg, J. G. (1994). Evaluating technology-based learning: Which model? In K. Beatie et al (Eds), Interactive Multimedia in University Education: Designing for Change in Teaching and Learning. Melbourne, IFIP.

Bain, J. D. (1999). Introduction: Learning centred evaluation of innovation in higher Education. Higher Education Research and Development, 18 (2), 165-172.

Barson, J. (1997). Space, time and form in the project-based foreign language classroom. In R. Debski et al (Eds), Language Learning Through Social Computing. Melbourne, ALAA and The Horwood Language Centre, pp 1-37.

Debski, R. (1997). Support of creativity and collaboration in the language classroom: A new role for technology. In R. Debski et al (Eds), Language Learning Through Social Computing. Melbourne, ALAA and The Horwood Language Centre, pp 39-65.

Fardon, M. and Kinder, J. (1997). Partnership in multimedia production: A model that works. In R. Kevill et al (Eds), What works and why: Proceedings of ASCILITE 97. Perth, Curtin University, pp.175-180. [verifed 21 Jul 2002] http://www.ascilite.org.au/conferences/perth97/papers/Fardon/Fardon.html

Fisher, C. et al (Eds) (1996). Education and technology: Reflections on Computing in Classrooms. San Francisco, Jossey-Bass.

Gambhir, S. K. (1995). Interactiveness in spoken language. In V. Gambhir (Ed), The Teaching and Acquisition of South Asian Languages. Philadelphia, University of Pennsylvania, pp.57-75.

Goodsell, A. S. et al. (1992). Collaborative Learning: A Sourcebook for Higher Education. Pennsylvania, National Center on Postsecondary Teaching, Learning, and Assessment.

Jaccomard, H. (2000). "Tu parles..." : Conversations interactives en cours de français avancé. Carnet austral, 13, pp.10-16.

Jaccomard, H. (2001). Interactive conversations for advanced learners of French. In A. Herrmann and M. M. Kulski (Eds), Expanding Horizons in Teaching and Learning. Proceedings of the 10th Annual Teaching Learning Forum, 7-9 February 2001. Perth: Curtin University of Technology. [verified 21 Jul 2002] http://lsn.curtin.edu.au/tlf/tlf2001/jaccomard.html

Jonassen, D. H. (1996). Computers in the Classroom: Mindtools for Critical Thinking. New Jersey, Merrill.

Laurillard, D. (1994). The role of formative evaluation in the progress of multimedia. In Beatie, K. et al. (eds), Interactive Multimedia in University Education: Designing for Change in Teaching and Learning. Melbourne, IFIP.

Laurillard, D. (1993). Rethinking University Teaching: A framework for the effective use of educational technology. London, Routledge.

Lawson, R. and de Matos, C. (2000). Information technology skills in the workplace: Implications for Bachelor of Arts degrees. Australian Journal of Educational Technology, 16(2), 87-103. http://www.ascilite.org.au/ajet/ajet16/lawson.html

Levy, M. (1997). Project-based learning for language teachers: Reflecting on the process. In R. Debski et al (Eds), Language Learning Through Social Computing. Melbourne, ALAA and The Horwood Language Centre, pp 179-199.

Levy, M. (1999). Design Processes in CALL: Integrating Theory, Research and Evaluation. In K. Cameron (Ed), Computer Assisted Language Learning (CALL) Media Design and Applications. Lisse, Swets & Zeitlinger, pp 83-107.

Meskill, C. (1999). Computers as tools for socio-collaborative language learning. In K. Cameron (Ed), Computer Assisted Language Learning (CALL) Media Design and Applications. Lisse, Swets & Zeitlinger, pp 141-162.

Morrison, G. R. et al (1999). Integrating Computer Technology Into the Classroom. New Jersey, Prentice Hall.

Patton, M. Q. (1987). How to Use Qualitative Methods in Evaluation. California, SAGE.

Patton, M. Q. (1990). Qualitative Evaluation and Research Methods, second edition. California, Sage.

Patrikis, P.C. (1997). The evolution of computer technology in foreign language teaching and learning. In R. Debski et al (Eds), Language Learning Through Social Computing. Melbourne, ALAA and The Horwood Language Centre, pp 159-178.

Phillips, R. et al (2000). Handbook for Learning Centred Evaluation of Computer Facilitated Learning Projects in Higher Education. Perth, Murdoch University and ASCILITE. [verifed 21 Jul 2002] http://cleo.murdoch.edu.au/projects/cutsd99/

Riggenbach, Heidi (1988). Tapping a vital resource: Student-generated materials. In K. Bikram (Ed), Materials for Language Learning and Teaching, pp.117-128.

Stewart, D. C. (1998). Collaborative learning and composition: Boon or bane? Rhetoric Review, 7(1), 58-83.

Authors: Michelle de Souza and Michael Fardon, Arts Multimedia Centre, University of Western Australia, Nedlands WA 6009
Rob Phillips, r.phillips@murdoch.edu.au
Teaching and Learning Centre, Murdoch University, Murdoch WA 6153

Please cite as: de Souza, M., Fardon, M. and Phillips, R. (2002). An evaluation of tertiary language learning through student constructed multimedia: The Interactive Stories approach. Australian Journal of Educational Technology, 18(2), 127-146. http://www.ascilite.org.au/ajet/ajet18/desouza.html

Print version errata 8 Aug 2002: References to Table 5 on pages 143 and 144 should be to Table 4.

[ AJET 18 ] [ AJET home ]
HTML Editor: Roger Atkinson [rjatkinson@bigpond.com]
This URL: http://www.ascilite.org.au/ajet/ajet18/desouza.html
Created 16 Aug 2002. Last revised 27 Aug 2002.

Page access count