| Australian Journal of Educational Technology 2002, 18(3), 293-307. |
AJET 18 |
This paper discusses how teachers of English as a Second Language (TESOL) and educational multimedia experts collaborated in order to design a CD-ROM aimed at assisting international students to prepare for the International English Language Testing Service (IELTS) English Language Speaking test. The research focuses on the synthesis of content and multimedia knowledge and expertise. It shows how this synthesis occurred, and makes suggestions for how the collaboration necessary for such design work can be facilitated.
First we provide an overview of instructional design research literature relating to interactions with subject matter experts. We outline the research methods used in this paper to collect data bearing on interactions of this sort. A discussion of the data follows, focusing on aspects of content, multimedia design and their synthesis. Particular attention is paid to how the boundaries separating content and multimedia knowledge and expertise gradually weakened during the design process, removing barriers to design and paving the way for design to proceed. We conclude by suggesting how content and multimedia experts can facilitate the process of design.
Tessmer (1998) draws a distinction between the design requirements for multimedia products that are designed to develop specific skills, and those that have learning goals more akin to the exploration of knowledge. This paper reports on research into the design of an ESL multimedia product that is highly skill specific and not exploratory in nature.
According to Block (1981), the relationship between CEs and instructional designers must be collaborative and must effectively share responsibility for the successful completion of the project. Mutual respect is also important, as these working relationships are often quite torrid and can test even the strongest relationships (Ingram et al 1994). In particular, while the design model adopted for a specific project is the result of an instructional designer's accumulated knowledge and is analogous to a script (Keppell, 2001), the ability of the instructional designer to convey this script is one of the key characteristics of a successful multimedia project.
Ausubel's construct of "intellectual scaffolding" (Ausubel, 1960, 1963, 1968) represents another important feature of interactions between CEs and instructional designers. Intellectual scaffolding provides a form of graphic organiser that allows the instructional designer to develop the content and explore the relationships between concepts, in a way that can be understood by both parties (Wedman 1987). Holley and Dansereau (1984) further suggest that re-formatting the content into a knowledge map causes CEs and instructional designers to process the information both verbally and spatially, as well as adding "greater semantic depth" to the content. This idea of verbal and spatial processing is particularly important to the project this paper reports, as it was the very verbal nature of language knowledge, and the predominantly visual nature of screen based learning, that made the boundaries of knowledge so strong and hindered progress.
Data were gathered in two ways. The first involved audiotaping the weekly meetings between the CEs and the EMD. All tape recordings were transcribed in full. The second involved the participants in the design process keeping a diary reporting their own observations of points of blockage and facilitation, as they went about designing the CD-ROM. The meetings data were cross-referenced with data collected via diaries. All data were then categorised according to how academic and technical design was conceptualised and progressed in relation to the major problems or blockages to the design process that arose and were solved.
According to Bernstein (1990; 1996), 'boundaries' around knowledge are typical in the case of subject disciplines, mathematics for example. The knowledge base of mathematics, its internal conceptual structure and the meaning of the particular concepts internal to it are bounded such that mathematics cannot be mistaken for other disciplines such as history. The same applies to English language and educational multimedia design knowledge. Crucially, discipline specific knowledge cannot be understood in any immediate way by one who has no formal knowledge of it. Rather, it has to be made comprehensible to the uninitiated and this was the circumstance confronting the CEs and the EMD involved in this project. CEs were without formal knowledge of educational multimedia design and vice versa.
EMD: ...so you are basically using the movement, the audio and visual stimulus to bring that altogether. Our job is to bring that together in a meaningful way that enhances the learning...In the first sentence of the extract above two multimedia concepts, 'audio' and 'visual', are used without elaboration on their special multimedia meanings. The EMD then goes on to suggest that the concepts be merged in order to enhance learning. In theoretical terms, then, the taken for granted way in which the EMD uses these concepts in effect illustrates the boundary around them. How so? The meaning of the concepts is tacit rather than explicit and as such their multimedia specific meaning is obscured unless one has the same knowledge of them as an EMD. In this example the boundary around multimedia knowledge could be called 'strong', because the meaning of the concepts is both formal and tacit.
In addition, again following Bernstein (1990, 1996), when two distinct areas of knowledge such as ESL and multimedia design are juxtaposed there is a degree of insulation between these areas of knowledge. That is, ESL and multimedia design knowledge are separated by the boundaries surrounding each and in effect these boundaries insulate the two areas from each other.
Thus, when interpreting data, evidence of insulation between content and multimedia design knowledge was generated by how the content and multimedia experts interacted with their counterparts' knowledge and expertise. The following extract of data (selected from a transcript of interaction that occurred later in the project and discussed in more detail later in this paper), is an example of how interaction occurred:
EMD: Ok so what does lexical resource mean in relation to the test?In this extract of data, the EMD questions the CE about the meaning of particular English language concepts. Because this extract is selected from a transcript representing a point more advanced than the first example, but still in the first half of the project's life, like the previous example it focuses on particular concepts. However, it illustrates this focus in relation to both content and multimedia knowledge and the EMD's questioning of the CE makes the theoretical insulation between the two knowledge areas explicit. That is, the need to question indicates that there is insulation between the two areas of knowledge.CE: It means the breadth of language that the student - the candidate is able to draw on in order to say whatever it is that they have to say.
EMD: Ok so when you say breadth of language do you mean breadth of vocab?
CE: Not only vocab well I guess, in a broad sense but not just single words, but appropriate use of phrases and register...
EMD: ...Ok so that's, what about grammatical range and accuracy? In terms of the test what does that mean?
CE: Ok so what we'd be looking at is appropriate use of tenses, correct formation of word forms, (PAUSE) word order, correct word order...
There is a difference of relevance to later discussion, between the two examples we have given insofar as elaboration is requested and given in the second example. In theoretical terms, comparison of this second example with the first shows how the explicit provision of information that would otherwise be tacit and contained within the boundary of one area of knowledge can in effect begin to weaken the insulation between two distinct areas of knowledge.
The details of the analysis are now discussed. The discussion first emphasises the boundary around content knowledge and expertise.
In theoretical terms, then, the boundary relations around the scoring coda employed for the IELTS test can be called 'strong'. The following data demonstrate this boundary strength in relation to a CE's explanation of the 4.5 score required for entry into an IELTS preparation program.
R [1] So for EMD's benefit - what's 4.5 sound like?This explanation could not have been elicited from a person without intimate knowledge of, and experience with, ESL learners and IELTS test candidates in particular. That is, the differences between 4, 4.5 and 5 scores are highly specialised and are not explicit. Thus, this specialist content knowledge had to be communicated explicitly to the EMD and incorporated into design, or else the end product would have been of little use.CE: Just looking - pretty basic. A student who could speak in sentences but not string a lot of sentences together very well and making a number of errors...They'd have some difficulty understanding questions that were more than just basic questions...They would be tested according to the structure of the test which is asking them to speak about things that they are familiar with and then later to talk about the future or speculating with something like opinions. That's where they fall down at that level. So if I could just tell EMD from here, if it's a 4 (that's classed as a limited user), which says basic competencies limited to familiar situations...has frequent problems in understanding expressions...is not able to use complex language. And a 5 (so we're looking at between these two), is a modest user who has partial command of the language coping with overall use in most situations, though is likely to make many mistakes...should be able to handle basic communication. So that's what we are looking at, somewhere between those ranges, 4.5.
A design point these data thus imply is that when designing flexible learning materials, CEs need to communicate to EMDs a profile of the students to whom the flexible learning materials are targeted. Moreover, in terms of the theoretical framework used for analysis, the extract of data illustrates how the boundary relations surrounding basic content knowledge had to be weakened by the CE in order for the EMD to better understand the task at hand.
Indeed there was a dual need for CEs to explain and demonstrate concepts and skills sets to the EMD. These two needs were highlighted by the fact that the IELTS test targets students' understanding and performance in things such as lexical resource and grammatical range and accuracy. These specialist language concepts are not self explanatory. In our theoretical terms they are strongly bounded and insulated from each other. Therefore, in order for the CD-ROM design to progress these concepts had to be understood by the EMD. The extract below is an example of how this process of developing understanding took place.
EMD: Ok so what does lexical resource mean in relation to the test?The interchange between the EMD and the CE above involves a progressive broadening of the conceptual schemes on which English language concepts are built. Theoretically, the extract illustrates how the progressive weakening of boundaries around content knowledge added understanding to the EMD's limited content knowledge.CE: It means the breadth of language that the student - the candidate is able to draw on in order to say whatever it is that they have to say.
EMD: Ok so when you say breadth of language do you mean breadth of vocab?
CE: Not only vocab well I guess, in a broad sense but not just single words, but appropriate use of phrases and register...
EMD: ...Ok so that's, what about grammatical range and accuracy? In terms of the test what does that mean?
CE: Ok so what we'd be looking at is appropriate use of tenses, correct formation of word forms, (PAUSE) word order, correct word order...
The extract below in turn extends this understanding of weakening boundaries around knowledge to the development of pedagogy. It shows how discussing ways of teaching students to reflect on their own levels of conceptual proficiency led to insight on the part of the CE into the pedagogic differences between CD-ROM and face to face modes of delivery.
CE: So sometimes what we might do in the classroom is put four or five sentences on the board and ask them to say which ones are right and which ones are wrong.The CE's first two turns above illustrate her generic thinking about pedagogy in a face to face context. However, her last three sentences immediately above highlight a crucial pedagogic difference between face to face and flexible modes of delivery. Flexible delivery conventionally targets unknown students with whom the designer may never interact in a face to face way (Land, 1997). Therefore, clarity and meaning cannot simply be added piecemeal as they usually are in face to face teaching. Instead, the structure, order and sequence of main and sub-concepts and skills must be consciously designed so as to assist and direct the learner explicitly, in the absence of a teacher. Put another way, in the absence of a teacher it is necessary to build explicit directions and instructions (pedagogy) into the design so that the learner can navigate the technical and conceptual dimensions of the learning materials.R: So they can see why?
CE: Even set up pair work which can be recorded where you have a question and answer session and then you go back, listen to it, and think about ways you can improve on it...making the learner aware enough that they can actually pick their own mistakes. Sometimes you get to the point where you say this is a mistake but they have to be able to start recognising things like that in their language themselves.
R: Can you recognise...can you see a person who is about to make the transition?
CE: Yes, I think often you can, if you know the students well enough. But it would be more difficult in this sort of situation, wouldn't it? Because you don't know them.
The data below in turn illustrates how the building-in of pedagogy extends to the incorporation of feedback for the learner.
CE: ...ok, so if we work all the way through this and then we say 'now you'll do the same thing for hobbies and interest', but they haven't actually mastered these things. So, is there a way that when they've done the activity if it turns out that the evaluation is not very positive they can go back and relook at this and try the activities again? So there is, you know, there is a procedure where we say we suggest 'you have a look at this again' and 'go back here' and 'go back and redo this'. Is that possible?From a design perspective, in both this and the previous extract the CE initially thinks in terms of face to face delivery, then in terms of flexible delivery. These data about thinking in face to face terms and then in multimedia terms suggest that the collaborative design of flexible learning materials necessitates that CEs think their face to face strategies out aloud, so that these strategies can be understood by EMDs and then reconfigured into multimedia modes of delivery.EMD: Yeah, I'm just thinking how to do it. You could have them proceed all the way through to the end and have some feedback for them in terms of...acknowledging their mistakes.
In theoretical terms, strategies such as thinking aloud weaken boundaries around content and thereby assist pedagogy to be made explicit in the process of design and ultimately to the learner. More generally, the extracts presented thus far also point toward boundaries around multimedia knowledge and expertise that are themselves just as integral to the process of building instruction, feedback and conceptual clarity into flexible learning materials design.
First extractIn this extract multimedia terminology, 'audio' and 'visual', is implicit in meaning. However, in lay usage these terms can mean almost anything. In other words, to the uninitiated such as CEs the terms are too vague to be applied in any unmediated way to pedagogic strategies aimed at explaining content.
EMD: ...so you are basically using the movement, the audio and visual stimulus to bring that altogether. Our job is to bring that together in a meaningful way that enhances the learning...
This is less the case with the data extract below, which involves the EMD explaining what is meant by audio and visual stimulus and how they can be brought together. The extract is taken from data collected later in the project, when the EMD was explaining how a video segment could be incorporated into design.
Second extractIn contrast with the first extract, this second extract unpacks how visual and auditory multimedia might be used step by step and in detail in a pedagogic sequence. Comparison of the second extract with the first in effect demonstrates that for design purposes the EMD's verbal descriptions and concrete examples of multimedia capabilities needed to be expressed explicitly, in lay and educational terms. Notably, the first extract is taken from data collected early in the study while the second is selected from data collected later in the study. This time differential implies the potential value of using explicit explanation early, rather than later on in design collaborations.
EMD: ...I was thinking of an instance where you could set up a situation where a student is taking the test, you could break that up into a number of portions. You could use a video to show the student coming in, how they would be treated in the test, sitting down and having the rules explained to them and then we might say 'ok - well I'd like you to talk about a particular interest' and then stop it at those particular points and say 'would you choose this, this, or this?' You might take it a little bit further and say 'what would your choice be?' So they can then choose their paths. They might choose a particular path that continues on and they can observe the reactions of the interviewer and the interviewee, in terms of that outcome. So we can do a multi path option.
To extend on this point about the value of explicit communication, the data extract below illustrates how actually viewing samples of multimedia while simultaneously having technical terminology and capabilities explained by EMDs enhances understanding for CEs.
CE: That was really evident in all of the little videos or whatever you want to call them, that you showed us, because not only the introduction gave us a good idea of the sorts of things that we might include and how we might go about it, but seeing those types of animation and having you tell us which are the more complex ones helped us to have a clearer understanding of what sorts of things we might do...In the extract of data above the key passage is the CE's comment "seeing those types of animation and having you tell us which are the more complex ones helped us to have a clearer understanding of what sorts of things we might do". This passage demonstrates how a basic multimedia concept such as 'animation' is not self explanatory. In theoretical terms, this concept's meaning is strongly insulated from non-multimedia knowledge. In showing the CE's improved understanding the extract highlights the theoretical value of weakening the boundaries around concepts that insulate them from understanding and hinder collaboration.
In the first section of this paper dealing with the content dimension of design, the pedagogic utility of multimedia was highlighted. Picking up on this point, methods of multimedia design can be understood as types of pedagogy that in turn can enhance academic design; multimedia methods can be thought of as vehicles for delivering content and skills. The following extract of data makes this point more explicit.
CE: you see this is the difficulty that we had when we started getting involved in this; we realised how big a job it is just to do Part One. And we didn't really want to do, well we were a bit unsure. Did we really want to do exactly the same sort of activities for another one? In our thinking about that bit, well, ok, you can have the same sort of activity that the candidate is familiar with. But if you keep changing the activities, then they not only have to get used to new language, they also have to get used to a new type of activity.In this extract of data the CE's initial quandary about how to convey content in an interesting way (pedagogy) illustrates that there are two aspects of any flexible learning package that the learner must understand, namely, the technical operation of the package and its academic content. At this stage the CEs are beginning to understand the script (Keppell, 2001) that the EMD is proposing for the way in which the product will operate. The CE's quandary in effect illustrates that, if flexible delivery is to assist learners then its technical functions must be obvious to the learner. Put another way, technical operations need to be designed in such a way that they are clearly separable in the mind of the learner from the academic content and skills contained in the package and thus become functional rather than counter-productive in a pedagogic sense. Moreover, the CE's response in the extract above to the EMD's comment about methodology in turn illustrates the need for CEs themselves to understand methods of technical operation. The extract thus illustrates two design points in theoretical terms.EMD: so you're talking about the methodology of this basically...you're talking about Drag and Drop, Tick the Box.
CE: it was that methodology that you were pointing out that needed to be said.
First, once the meaning behind the names of multimedia techniques or methods such as Drop and Drag is operationalised by CEs, they can then visualise ways of conveying their subject matter via multimedia (pedagogy). That is, weakening the boundary around multimedia knowledge enhances pedagogic possibilities for delivering content. Multimedia methods and terminology then become analogous to pedagogic tools that CEs can use to carry the conceptual structure of academic content.
Second, developing an understanding of multimedia as pedagogy allows designers to concentrate on making the technical operation of a package explicit to learners. In this way multimedia methods can also be used to build assessment and learner self evaluation into flexible learning materials.
The following data extract illustrates how these sorts of goals can be achieved. In the extract the CE can visualise or verbalise two possibilities for feedback. In light of these possibilities the EMD then offers technical solutions to the problem of how to activate feedback via multimedia. These solutions lay out ways by which the learner could use multimedia to achieve the outcome that the CE visualises, in effect showing how multimedia can serve as assessment tools in an educational design sense.
CE: so at some stage the user is going to have to put this information to work, where they come up with their own responses. Fluency Practice One was formulating those responses, but the only problem I saw was well how...how do you check that? How do you get feedback on that sort of thing?....and then I thought that possibly, I don't really know but you could have an activity where you evaluate it, the responses from the video, and you evaluate them in terms of either just a simple yes/no...The other option would be to write it down. But remember that even if they've tape recorded they're still going to have to go back and to some extent analyse what was said. Sometimes the best way to do that is to write it down.In theoretical terms, these data illustrate how weakening of boundaries around multimedia knowledge progresses understanding of multimedia as a pedagogic enabler. The verbal and spatial processing of information adds "greater semantic depth" (Holley and Dansereau, 1984) and weakens the knowledge boundaries. The data also highlight the pedagogic import of technical design knowledge. They show how multimedia can complement academic knowledge and expertise.EMD: what you could actually do is a check-list that has those things. You could do it terms of both text and voice. 'Did you have this; tick yes or no'. At the end of that you could make a decision. It might say 'looks like you could probably do a little bit more practice'.
Of course, the analysis presented here has the benefit of hindsight. The project itself did not progress smoothly to this realisation. In this regard the empirical task of making progress in design by synthesising content and multimedia knowledge and expertise was theoretically dependent on weakening boundaries between content and multimedia knowledge and expertise, but we did not realise this fully at the time.
If this is the case, empirically, the analysis of data indicates that when considering collaborative flexible learning materials design involving CEs and EMDs, it is imperative that:
In particular, the data presented here suggest that the theoretical key to developing design collaborations is the boundary between content and multimedia knowledge and expertise. Specifically, design collaborations between content and multimedia experts are likely to be facilitated by encouraging the demystification of disciplinary knowledge and expertise and an engagement with one's counterpart's knowledge and expertise. Our data indicate that this engagement can have the effect of weakening theoretical boundaries between content and multimedia knowledge and thereby enhance the practical development of explicit pedagogies that in turn are likely to benefit learners. Moreover, in a retrospective sense our data suggest that the earlier in the course of the project these things happen the better. Indeed, some preparation on the part of CEs and educational EMDs aimed at making their own knowledge and expertise explicit and understandable to others before attempting design might get collaborations off to a good start.
Ausubel, D. P. (1963). The psychology of meaningful verbal learning. New York: Grune & Stratton.
Ausubel, D. P. (1968). Educational psychology: A cognitive view. New York: Holt, Rinehart & Winston.
Bernstein, B. (1990). Class, Codes and Control (Volume 3): Towards a theory of educational transmissions. London: Routledge & Kegan Paul.
Bernstein, B. (1996). Class, Codes and Control (Volume 4): The structuring of pedagogic discourse. London: Routledge & Kegan Paul.
Block, P. (1981). Flawless Consulting. University Associates. San Diego. CA.
Holley, C. D. & Dansereau, D. F. (1984). Special learning strategies: Techniques, applications and related issues. Orlando, FL: Academic Press.
Holt, D. M. & Thompson, D. J. (1998). Managing information technology in open and distance higher education. Distance Education, 19(2), 197-216.
Ingram, A., Heitz, K., Reid, C., Walsh, M., Wells, C. (1994). Working with subject matter experts. P&I, 33(8), 17-22.
Keppell, M. (2001). Optimizing instructional designer - subject matter expert communication in the design and development of multimedia projects. Journal of Interactive Research, 12(2-3), 209-227.
Land, R. (1997). Virtually there: Flexible learning in the new millennium, In C. Bell, M. Bowden & A. Trott (Eds), Implementing Flexible Learning, 15-42. London: Kogan Page.
Sinclair, M. & Smith, R. (2001). Developing Flexible Learning Materials Design Protocols, Using an ESL Example. Unpublished Report on a Central Queensland University Focus Grant project, Central Queensland University, Rockhampton.
Singh, P. & Sinclair, M. (2001). Diversity, disadvantage and differential outcomes: An analysis of Samoan students' narratives of schooling. Asia Pacific Journal of Teacher Education, 29(1), 73-92.
Tessmer, M. (1998). Meeting with the SME to design multimedia exploration systems. Educational Technology, Research and Development, 46(2), 79-92.
Tessmer, M., Jonassen, D. (1993). Evaluating computer-based training for repurposing to multimedia: A case study. Paper presented at the annual meeting of the Association for Educational Communications and Technology, New Orleans, LA.
Torrisi-Steele, G. & Davis, G. (2000). A website for my subject: The experiences of some academics' engagement with educational designers in a team based approach to developing online learning materials. Australian Journal of Educational Technology, 16(3), 283-301. http://www.ascilite.org.au/ajet/ajet16/torrisi-steele.html
Wedman, J. F. (1987). Conceptualizing unfamiliar content. Journal of Instructional Development, 10(3), 16-21.
| Authors: Dr Mark Sinclair, Lecturer (Research), Faculty of Education and Creative Arts, Central Queensland University. Email: m.sinclair@cqu.edu.au
Scot Aldred and Richard Smith, Central Queensland University Please cite as: Sinclair, M., Aldred, S.and Smith, R. (2002). Facilitating the collaborative design of flexible learning materials: Investigating the synthesis of content and multimedia knowledge and expertise. Australian Journal of Educational Technology, 18(3), 293-307. http://www.ascilite.org.au/ajet/ajet18/sinclair.html |