Using interactive multimedia to improve operator training at Queensland Alumina Limited

Peter Reilly
Queensland Alumina Limited

Queensland Alumina Limited (QAL) and the Interactive Multimedia Unit (IMU) at Central Queensland University (CQU) have recently completed a joint project involving the design, development and implementation of a multimedia training package. The multimedia tutorial forms an integral part of the training curriculum for alumina production workers at QAL's Fluid Bed Calciner facility. This paper describes the progress of the project to date and the current and future strategies for evaluation of the training program. An initial evaluation of learner attitudes shows a high degree of acceptance and enthusiasm for the new package. The quality of the multimedia training materials was recently recognised at 1997 Queensland IT&T awards for excellence.

Multimedia training

Training in many industries has traditionally involved two predominant teaching strategies - on-the-job training by peers and face-to-face instruction (Plant and Shannon, 1993). Recently many companies have become interested in the potential of new technologies to offer more effective and efficient training for their staff. The use of technology is a means by which training can be made more flexible and supportive of the principles of adult learning (Hosie, 1993).

Interactive multimedia can offer a range of benefits over traditional training approaches by providing improved flexibility, cost- and time-effectiveness, consistency and availability (Fankhauser and Lopaczuk, 1996; Murphy, 1993). Forman (1995) identifies benefits and values in four major areas - organisational benefits, instructional benefits, learning effectiveness and business efficiency. Multimedia can also provide improved and more consistent testing and administration (Sims, 1991). Keppell and Richards (1996) also suggest that self-paced multimedia materials offer a private environment which enables trainees to review the material as many times as they wish.

Hosie (1993) identifies the need for courses which are professionally designed and evaluated, adopt effective learning strategies and encourage a self-directed approach. This strategy is also supported by Rossett and Barnett (1996) who maintain that basic instructional design principles must guide the development of computer-based training and recommend an approach based upon analysis, scenarios, discovery and expert advice.

Industry context

The Calcination section is supported by 74 employees who comprise four operating teams, two maintenance teams and one management team. The section houses the Fluid Bed Calciner - a complex apparatus used to wash, filter and calcine precipitated hydrate to form alumina powder. The calcining process occurs in a complex arrangement of pipes and vessels through which alumina hydrate, air, gas and fine alumina dust circulate.

The operation of the Fluid Bed Calciner is both monitored and controlled by computer. Control room operators analyse trends in the computer readouts and make adjustments to ensure efficient and safe operation. Fluid Bed Calciner operators, known as alumina producers, maintain the operation of the Calciner by responding to requests from the control room operators, carrying out maintenance tasks and undertaking housekeeping duties.

Needs assessment

Queensland Alumina believed that interactive multimedia could provide an additional tool to enhance the operating skills and knowledge of its Fluid Bed Calciner operators and thereby reduce the loss of production caused by operator error.

The specific training issues which could be addressed through the use of multimedia in this case were:

• Effectiveness. By using the multimedia tutorial trainees could familiarise themselves with the basic concepts and procedures and make better use of practical training with their mentors.

• Instructional needs. The multimedia package could be designed to address the section's specific learning objectives through the use of animation and self-testing.

• Flexibility. The multimedia package could be made available on demand and used by an individual at their own pace.

• Consistency. The use of a multimedia package could assist in dispelling inaccuracies by providing an authoritative source of information.

The project began with an analysis of the training environment and the needs of the trainees. Fluid Bed Calciner operators must have an accurate understanding of the internal workings of the system to correctly diagnose faults and implement troubleshooting techniques. They must react quickly, correctly and safely when responding to requests from the control room operators.

An examination of the existing training resources - one print manual, two Powerpoint presentations and a Quality Assurance procedure manual - indicated a need for updating and expanding to improve relevance to the practical setting and provide better support for visualisation of the calcination process.

Figure 1: The Fluid Bed Calciner training program.

A new training structure was designed which incorporated a multimedia tutorial which could be used before, during and after practical training sessions with an experienced operator. While maintaining the importance of procedural knowledge and technical skills the new training program aimed to develop a better understanding of the underlying operating principles of the Fluid Bed Calciner to achieve a better balance between physical and conceptual skills.

To be assessed as competent in the Fluid Bed Calciner program trainees must complete all lessons in the multimedia tutorial and achieve a score of 100% for the assessment tasks. Trainees must also undertake a verbal assessment for the practical component and a written assessment at the end of the tutorial during which they are required to apply their knowledge to realistic situations and identify troubleshooting strategies.

Development approach

The project followed a development cycle which involved needs assessment, collection of content, design of planning grids (storyboards), editorial review, client review, media design and collection, authoring, beta testing, implementation and evaluation. Rather than being a strictly linear process, the development cycle acts a guide for project managers and stages may overlap or be reiterated as necessary.

Each of the lessons were subjected to 'formative experimentation' (described by Reeves, 1992a) through expert review and beta testing. This process confirmed the accuracy of the content and the functionality of the interactions. Feedback was collected from the testing group through observation and informal discussion.

Figure 3: The IMU development cycle

Design and structure

• the use of simple direct language
• limiting screen information to a single concept
• logical explanations for procedures
• use of graphics to support and explain text
• use of large buttons
• avoidance of icons
• limited navigation options (next, back, help, quit, menu and settings).

Further development of this model has seen the inclusion of optional, full narration which matches the on-screen text and text/audio help which provides an explanation of the features on each screen. Kenworthy (1993) recommends that information be both visualised and verbalised for poor readers and that supporting audio match on-screen text exactly to allow the identification of unfamiliar words.

To further develop the model to suit the needs of the QAL trainees a number of new features were introduced specifically for this project.

Clark (1989, p. 120) suggests that 'processes can be displayed on computer effectively through the use of animation and colour to illustrate the dynamic nature of the process'. The first lesson of the Fluid Bed Calciner training package features a large overview animation which clearly shows all parts of the Calciner apparatus and the flow of hydrate, air etc. throughout. Learners can display or hide the flows, and can select a particular section to view a close-up with a further textual/audio explanation. This allows learners to develop an understanding of the process as a whole and the relationship of each stage to the others. It was envisaged that trainees would return to and consult this overview many times as they progress through the more detailed information in subsequent lessons.

Each lesson also features safety and environment information to encourage trainees to consider these as integral to their duties. Activities included throughout each lesson are designed to allow learners to assess their own understanding of the material as it is studied by providing immediate feedback. At the end of each lesson a series of assessment questions highlights the most important topics covered and these must be answered correctly to complete the lesson. These tasks concentrate on factual and procedural information that a trainee needs at call.

One of the aims of this project was to equip trainees not only with an understanding of how the Fluid Bed Calciner functions but to enable them to apply their knowledge to solve common operational problems. Troubleshooting and problem-solving are heavily emphasised throughout the multimedia tutorial with explanations of common malfunctions, and methods for locating the cause and identifying possible solutions.

Data collection

The course material and media were then collated in a planning matrix which was specifically designed for multimedia development at the Interactive Multimedia Unit (Keppell & Buschgens, 1995). This matrix documents the structure, content and interactions for each lesson and can be reviewed by the client and the subject matter experts. After the iterative process of review was complete the matrix was then used by technical staff to assemble the lessons. This method of storyboarding has proved very successful in increasing efficiency, reducing team member frustration, reducing quantitative error and improving the quality of the final product (Keppell & Buschgens, 1995).

Implementation and initial evaluation

The initial evaluation of the package has focussed upon the attitudes of the learners who have used the multimedia tutorial. This information will be complemented by further investigations.

To date 26 alumina producers and a number of interested section staff members have completed an anonymous questionnaire after using the tutorial. A range of scaled, yes/no and open-ended questions focus on the degree of acceptance by the trainers and trainees and the perceived suitability of the training materials.

Results from the initial survey have been positive with learners indicating that they found the package easy to use, the instructions clear, the tasks and activities valuable for learning, the graphics and images effective and the feedback appropriate.

The multimedia tutorial was also compared favourably with other instructional materials on the same subject and all users agreed that they would use a multimedia tutorial in a different area and would recommend multimedia training to others. Only one respondent did not believe that the tutorial improved his understanding of the material.

A series of open-ended questions provided an opportunity for further feedback. The following sample comments provide both encouragement and suggestions for improvements:

'The software is an excellent training/learning tool, but there is no replacement for hands on experience. A picture says a thousand words, this is a step in the right direction for QAL.'

'The animations are very helpful.'

'I thought that it was well worth the money. I learned a few things I didn't really know.'

'Quality of photos is rather poor. As offered this package is quite good. Further development is needed. I would like to see another package up and running, but with more detail.'

Ongoing evaluation

Interviews will be conducted with trainees selected from each production team to generate an open-ended discussion, particularly about issues raised in the survey process. An interview guide will be used, although some flexibility will be allowed for interviewees to pursue topics about which they feel strongly. It is envisaged that the interviews will encourage learners to draw their own conclusions about the learning experience.

Quantitative information will also be collected by the management system which tracks student progress through the package. A log file records information such as the number of times the tutorial is used, the length of time a trainee spends on a lesson and the trainee's success rate on the activities and assessment.

Information will also be collected about the Fluid Bed Calciner operation to ascertain whether the number of unscheduled outages attributable to operator error have decreased since the new training package was introduced. This may provide an indication as to the effectiveness of the materials in increasing operator skill and knowledge. Statistics on damage to the inner lining of the Calciner and consumption rates of materials will also be collected and examined as another possible indicator of training effectiveness.

Future directions

The project managers and coordinator have also received a small grant from Central Queensland University to carry out a detailed case study focussing on project management and communication, content collection, design standards, the development process and evaluation methodologies. This sort of intensive study is recommended by Reeves (1992b) and has been used by Burgess (1995) to describe a similar collaboration between industry and an educational institution.

Conclusion

An initial evaluation of the software shows a high degree of acceptance by the target users and has identified some areas for improvement. The continued evaluation of this project over the next year should provide further information about the appropriateness and effectiveness of multimedia training in this environment which will be used to improve the current training package and contribute to on-going research into effective training design.

References

Beattie, K. (1994). How to avoid inadequate evaluation of software for learning. In K. Beattie, C. McNaught and S. Wills (eds), Interactive multimedia in university education: Designing for change in teaching and learning. Elsevier Science BV, Holland.

Burgess, J. (1995). An evaluation of an industry training project based on the design, development and use of self-paced multimedia courseware: A case study. MEd thesis, University of Southern Queensland.

Clark, R. C. (1989). Developing technical training: A structured approach for the development of classroom and computer-based instructional materials. Addison-Wesley, Reading, Mass., USA.

Forman, D. (1995). The use of multimedia technology of training in business and industry. Multimedia Monitor, July, pp22-27.

Fankhauser, R. and Lopaczuk, H. (1996). Exploring the multimedia landscape from a training and professional development perspective. In J. G. Hedberg, J. Steele and S. McNamara (eds), Learning Technologies: Prospects and Pathways, 48-54. Selected papers from EdTech '96. Canberra: AJET Publications. http://www.aset.org.au/confs/edtech96/fankhauser.html

Hosie, P. (1993). Technologically mediated learning: The future of training in Australia. Australian Journal of Educational Technology, 9(1), 69-86. http://www.ascilite.org.au/ajet/ajet9/hosie.html

Keppell, M. J. & Richards, J. (1996). Choosing multimedia as a training option: Client Considerations. Paper presented at the Fifth Annual Multimedia in Education & Industry Conference, Charleston, South Carolina, July 22-28.

Keppell, M. J. & Buschgens, R. A. (1995). Optimising communication between instructional designers, graphic artists and computer programmers in the development of multimedia materials. In A. C. Lynn Zelmer (ed), Higher Education: Blending tradition and technology, Proceedings of the 1995 Annual Conferences of HERDSA. Central Queensland University, Rockhampton, Australia, pp. 439-451.

Kenworthy, N. (1993). When Johnny can't read: Multimedia design strategies to accommodate poor readers. Journal of Instruction Delivery Systems, Winter, 27-30.

Milheim, W. D. and Lavix, C. (1992). Screen design for computer-based training and interactive video: Practical suggestions and overall guidelines. Performance and Instruction, 31(5), 13-21.

Murphy, L. (1993). DVI in Organizational Information Retrieval and Training. In D. M. Gayeski (ed), Multimedia for Learning - Development, Application, Evaluation. Educational Technology Publications, New Jersey, USA.

Plant, H. & Dekkers, J. (1994). The use of multimedia to train mine operators at Curragh Queensland Mine, Blackwater Central Queensland. Central Queensland Journal of Regional Development, 3(2), 53-61.

Plant, H. & Shannon, B. (1993). The use of multimedia to train mine operators at Curragh Queensland Mine, Blackwater Central Queensland. Internal report to the Division of Distance and Continuing Education, Central Queensland University.

Reeves, T. C. (1991). Ten commandments for the evaluation of interactive multimedia in higher education. Journal of Computing in Higher Education, 2(2), 84-113.

Reeves, T. C. (1992a). Evaluating interactive multimedia. Educational Technology, 52(5), 47-53.

Reeves, T. C. (1992b). Research foundations for interactive multimedia. In Proceedings of the International Interactive Multimedia Symposium, Perth, Western Australia, January 27-31, 1992, Promaco Conventions Pty Ltd, Applecross, Western Australia, pp. 177-190. http://www.aset.org.au/confs/iims/1992/reeves.html

Rossett, A. & Barnett, J. (1996). Designing under the influence. Training, 33(12), 34-42.

Sims, R. (1991). Computer based training: A handbook for professionals. Knowledgecraft Pty Ltd, West Chatswood, Australia.

Wills, S. and McNaught, C. (1996). Evaluation of computer-based learning in higher education. Journal of Computing in Higher Education, 7(2), 106-128.