| Australian Journal of Educational Technology 2001, 17(1), 96-114. |
AJET 17 |
| Rob Phillips Murdoch University |
| Romana Pospisil Edith Cowan University |
| Jennifer L Richardson Murdoch University |
Some of the priorities of this project were to maintain the detail and the depth of the hard copy radiographs in the QTVR images, to simulate the problem solving process used in reading radiographs, and to improve the learning outcomes by highlighting and annotating important areas of images. It was hoped that this solution would provide a more cost effective and convenient method of delivery of large numbers of images to external students. In the subsequent cost benefit analysis, it was found that the innovation described here offers many economic advantages to the School of Veterinary Clinical Science.
The Masters degree enrolment is small (49 students in both 1999 and 2000) and it is offered in traditional print based mode, with some online and technology mediated components. Individual units include the use of email, some online tutorials and discussion forums, in conjunction with print based materials, video, radiographs, ultrasound images and microscope slides. In some units, a custom built computer program is used for case based problem solving exercises. Currently, enrolment numbers are restricted by the cost of producing sets of unit materials such as radiographs and pathology slides. The Masters unit V620, Veterinary Diagnostic Imaging, is typical of a unit where the number of sets of radiographs limits the enrolment numbers, even though there is a growing demand for places. In 1998, the number of students enrolled was 25, but in 2000 it was limited to 22, because damaged and lost materials had limited the amount of available resource materials. It was thought that Masters degree enrolment numbers could increase if appropriate use was made of online and Interactive Multimedia (IMM) technologies.
By using the radiographs and ultrasound images, students learn the principles of recognising normal and abnormal structures and learn to define the patterns of change present in disease processes, and make diagnoses from these. If students experience difficulties in recognising radiographic or ultrasound changes or diagnosing an abnormality, they can discuss the case with their lecturer via telephone. Once the non-assessable worked case studies component is complete, students undertake case evaluations using radiographs or ultrasound images and complete case related assignment questions.
| Fixed costs Cost of new radiograph sets (200 hard copies) |
$2,748 | |
| Cost per new hard copy image | $14 | |
| Marginal costs | 1998 Unit | 2000 Unit |
| Number of students | 25 | 22 |
| Cost of maintaining radiograph sets | $1,030 ($41) | $1,000* ($45) |
| Postage | $3,510 ($140) | $5,340 ($243) |
| Administrative handling | $1,000 ($40) | $1,000* ($45) |
| Total marginal cost | $5,540 ($222) | $7,340 ($334) |
* estimate | ||
Since 1998, the average cost of providing radiographs to students has increased from $222 per student to $334 per student, mainly due to postage costs for overseas enrolments. It is clear that these high costs limit the potential of the unit to increase enrolments.
Studying with the hard copy radiographs can also be inconvenient. Students enrolled in the Masters degree are mostly practising veterinarians seeking further to develop their skills. Using hard copies of images means that students require access to a viewbox, usually located at their workplace, while most of their coursework can be done from home. Students therefore tend to be transporting radiographs in bulky files from one location to another, which can be a considerable inconvenience.
The external mode presents difficulties in achieving a satisfactory level of interaction between teacher and student. There is an unavoidable delay in students receiving feedback on assignments sent by post. In previous years, attempts have been made to improve contact between teachers and students by using email and individual telephone discussions. Teleconferencing has been used for group based discussion of case study radiographs. However, this proved to be costly and often unsatisfactory, as it was difficult to discuss images without being able to pinpoint or highlight areas of interest.
Online learning environments, such as WebCT (WebCT Educational Technologies, 1999), can facilitate communication between the student and tutors, as well as communication with other students (Collis, 1996; Harasim et al., 1995, summarised in Phillips & Luca, 2000). At this stage, we have included a WebCT discussion forum to improve teacher-student and peer-peer communication. However, it was recognised that other approaches needed to be used to provide for:
The rest of this paper is about the implementation of a radiographic database and CD-ROM and describes the technical design and formative evaluation of the two trial CD-ROMs produced for the unit in first semester 2000.
The brief for the project was to make the radiographs and ultrasound images available to off campus external students through the creation of an image database delivered on CD-ROM. We sought to decrease the mailing and reproduction costs, as well as reduce the inconvenience of handling large files of radiographs. There was also the potential to add value to the educational experience by providing greater flexibility in selecting cases to be included in the unit, and adding tools for highlighting or annotating images.
In our explorations of alternatives to the use of hard copies of radiographs and ultrasound images, we started with a detailed analysis of how radiographs are actually used by student veterinarians in case evaluations. In practice, the radiographs are examined with a general light source (viewbox), initially from a distance, subsequently at close range, and finally in fine detail using a focal bright light source. Images and anatomical regions can be displayed side by side for comparison. Students learn to identify an abnormal region, which then requires closer examination, and then locate appropriate items for comparison.
From this analysis, we were able to identify several key requirements that had to be met when presenting images in the new format:
The rest of this section describes technical details of the implementation, starting with the scanning and storage of the individual radiographs.
The interpretation of X-ray images requires fine discrimination between diffuse areas of grey. To replicate this on the computer requires high resolution scanning at high colour depth, and careful attention to brightness and contrast settings to ensure relevant areas are visible. The combination of large physical size and high resolution implied that scanned images would take up large amounts of storage space on the computer. A 17 in x 14 in radiograph scanned at 300 dpi (dots per inch) in millions of colours would occupy 63 megabytes of disk storage, uncompressed. Only a limited number of such images could be stored on a CD-ROM, and loading a single image would place significant load on even the most up to date computer. In V620, students are provided with approximately 200 images and are required to view several side by side for comparison.
At first sight, it appeared that it would be impossible to achieve the requirements (large size, high image quality, moving within the image, zooming in and out) with file sizes which were usable on standard personal computers. However, we then considered Apple Computer Inc.'s QuickTime Virtual Reality (QTVR) technology (Apple Computer Inc., 2000).
QTVR is a variant of the QuickTime digital video format, allowing the user to experience a form of virtual reality on their computer screen. There are two types of QTVR movies, panoramas and objects. A QTVR panorama is where the camera is looking outward, and 'panning' through a 360 degree arc. Objects are where the camera is looking inward, and give the impression that the object being viewed is turning around, as if it is held in your hand. A QTVR object is often created by standing the object on a turntable with a fixed camera location. One frame of the object is taken at increments, as the turntable is turned.
In both panoramas and objects, the QTVR system software enables the user to move and zoom within the image. QTVR also allows important areas of images to be highlighted by hotspots, which enable annotation of images or the inclusion of hyperlinks to other images. This was seen as particularly helpful for external students.
QuickTime VR offered the image manipulation features needed for the radiographs and provided inbuilt compression algorithms to reduce image file sizes. However, the question remained whether QTVR could work with a single radiograph or ultrasound image. After some research, we determined we could achieve this by creating a virtual QTVR object, with only one frame (setting the horizontal sweep angle to 0 degrees). We were then able to perform a feasibility study to determine whether the image quality was adequate and how great the image compression would be.
An obsolete XRS RSU1 X-ray Scanner (with a resolution of 146 dpi) and the Ray Vision software was used for scanning most large images. The resolution was marginal for some images, particularly for bone detail and ultrasound images. Consequently, some were scanned with a smaller UMAX A4 flatbed scanner at 300 dpi resolution. This gave a higher resolution, but with the limitation that insufficient light was able to pass through the image to provide the depth of grey scale across the image, to make the features visible. The radiologist needed to perform a quality check on each scanned image and, frequently, several different scans with different contrasts had to be made, to ensure that clinical features were visible. Eventually, the radiologist was satisfied that the scanning and QTVR technique was suitable for all but the abdominal and some thoracic images. However, a higher resolution X-ray scanner would have increased quality and reduced the amount of image preparation time.
Scanned images were 'cleaned up' in Adobe Photoshop, then imported into the VR Worx application to create the QTVR files.
Once it had been established that the approach described here was feasible, the remainder of the images was scanned. Although there were originally 200 hard copies, not all of these were scanned, but some new images were added. In total, 250 images, of varying type, quality and size, were scanned. In some cases, multiple images were displayed on the one X-ray film, but these were produced as separate QTVR files. In total, 319 individual QTVR movies were produced.
The compression offered by QuickTime also resulted in considerable reductions in file sizes. Given that there was a large range of image sizes, the average size of the scanned images was approximately 2MB, while the average size of the QTVR images was 300kB. This size makes it feasible to deliver images over the web, as well as on CD-ROM.
The authors are unaware of any previous use of QTVR to display X-rays for learning purposes. The Anatomedia project (Kennedy, Eizenberg & Kennedy, 2000) contains radiographic images, but these are of low resolution, intended for the learning of anatomy. The particular advantage of the approach described here is that the QTVR radiographic images are of sufficient detail to enable diagnosis.
Kraft et al. (1998) reported on a multimedia program for teaching veterinary orthopaedic radiology. Relatively low resolution images were displayed, and specific areas of interest were displayed as magnified images when clicked on. This program took a didactic approach, leading students linearly through explanations and annotations of the radiographs. Instead of treating the computer as a teacher, in our case we have seen it as a tool to assist advanced students to work through the process of determining the area of importance themselves, using a problem solving approach.
However, the timeline was very tight. The project started in November, in the context of a looming deadline. The unit commenced in mid-January, and students had to have access to their materials by early February. The tight timeline restricted the amount of prototyping and bug testing that was possible. It was anticipated that any unexpected delays would impact severely on the ability to meet the deadline.
An initial version, containing approximately one quarter of the images, was distributed to students by the deadline. As more information became available, a second version was developed, building on student experiences with the first.
Version 1
Once it became clear that the QTVR images were suitable for teaching purposes, effort turned to finding the most effective way to deliver the material to students. The typical way by which students engage with the printed materials is through case studies. An image or images are presented, together with history and clinical examination information and students are guided through the radiographic features, leading to a radiographic diagnosis of the condition.
Because text information is associated with each image and case, a database solution was indicated. The database would enable related information to be kept together, simplifying maintenance, and enabling further cases to be added or deleted in subsequent years. Because of the file sizes and internet bandwidth issues, it was decided to provide the images on CD-ROM. The requirement was therefore for a database which could be distributed on CD-ROM.
FileMaker Pro was chosen as the most appropriate database for this purpose, due to its potential for use on the web, and prototyping of the CD-ROM commenced.
The database structure was built, but the process of entering data into the database was delayed for non-technical reasons. The administrative staff member responsible for managing the case data was on leave, and, because of version control problems, it was difficult to identify the most accurate case data for inclusion in the database. Eventually, data for the first set of case studies was finalised and written to CD-ROM. The intention was that the rest of the images could be sent out subsequently in version 2.
When testing the first prototype of the CD-ROM, it was discovered that the database would not run directly from a CD-ROM. The short term solution was to install the database from the CD-ROM onto the user's computer, and run it from there.
The impact of the delay in entering database content was that the CD-ROM was not able to be tested on a wide range of computers and hardware configurations before being sent out to students. Subsequently, a number of students reported difficulties in being able to use the CD-ROM on their existing hardware. The CD-ROM provided to students was relatively resource intensive, compared to IT requirements associated with previous units in the Masters degree, and those disseminated to students at the start of the project. In particular, it was very time consuming on some computers to install the images onto the hard disk. In addition, some students could not get the QTVR images to display, either because there was insufficient RAM available for the QuickTime software, or because it was not installed. The initial problems clearly affected the views of some students towards the unit.
However, students having computers with the specifications defined for the unit reported few difficulties with the first CD-ROM. Nevertheless, there were clear perceptions from a number of students that the technical difficulties impeded their learning. The provision of technical support by the development team may have ameliorated these feelings.
The development team anticipated problems in implementing a new piece of software at short notice, and provided students with complete hard copy materials for the first section of the unit. In addition, a total of 69 cases using 152 images was used for the second section and provided in QTVR format on the CD-ROM. Unfortunately, due to financial constraints, only 34 hard copy radiographs were provided for this second section.
Version 2
The development of the second version was also pursued in the context of tight timelines. The slow installation process and inability to work from the CD-ROM resulted in the FileMaker Pro database being dropped for the time being. Instead, all images were stored on the CD-ROM, and a simple navigational shell was written in Macromedia Director to enable students to locate and open QTVR images. This solution had the advantage of speed and ease of use, but at the expense of the contextual material stored previously in the database. In this version, students referred to paper based information about the cases, and most of the images were also supplied as hard copy.
We always envisaged that version 2 would be a short term solution to an immediate problem, and that a more robust solution would be developed based on student feedback.
| CD-ROMs | Radiographs |
| Useful for studying and self tests | Better for marked assignments |
| Better image quality | |
| Good for gross detail | Good for fine detail |
| Could be viewed at home or work - preference for home | Had to be viewed at work |
| Students unfamiliar with this approach | Students familiar with this approach |
| Convenient in terms of time and place | Cumbersome and time consuming |
| Initial technical problems | |
| Good for ultrasound images | |
| Need relatively powerful computer | |
| Relatively cheap to distribute | Very expensive to distribute |
| Hotspots very helpful | Hard to identify abnormalities from written descriptions |
| Zooming capability helpful, but not as good as viewboxes |
I hate the CD-ROM's. Maybe my views are tainted by the problems I had originally with viewing the CD-ROM's and the fact that as a result I am behind and struggling to get up to date again.However, other students overcame their early poor perceptions:
Initially when I was having computer problems I was very frustrated with the CD-ROM, however once the problem was rectified I found it very useful and an efficient use of time. I think it would be good for future students to utilise.Students tended to use the CD-ROMs for studying and self tests. However, all respondents reported that they used the hard copy radiographs for the marked assignments. This was because the students felt that the radiographs had a better image quality.
I found the hard copy radiographs easier and better to look at as these are more realistic and easier to image - particularly found the orthopaedics difficult to really look at due to the small size.However, the CD-ROMs were seen as more convenient in terms of time and place, and some students enjoyed the flexibility of having both methods available.
I felt that both methods were important and used both at different times and in different places. The radiographs had to be viewed at work and these I tried to do at quiet times in between consults and in breaks so that I didn't have to go in to work too often on days off.Two students reflected that their preference for the radiographs may have been due to their familiarity with this technique.
I probably feel comfier with the radiographs as that is what we are all used to the most.In addition, the radiographs had the drawbacks of being cumbersome and time consuming. Students had to transport books and notes to a vet treatment room with a viewer, and then try to write their reports from there, when the room was not suitable for that purpose.
The feedback from students indicates that they would like to receive images in both modes. The CD-ROM images were found to be suitable for study purposes, because of the flexibility and hotspots. However, students wanted hardcopies for the assignments, because of a perceived higher image quality. A mixed approach will significantly reduce unit costs the next time it is offered.
I have enjoyed the unit and found it informative and have learnt a lot from both hard copy and CD-ROM X-rays - a well structured and well written unit.Students reported some difficulties in studying externally, but recognised that there were few alternatives to them if they wanted to continue their veterinary practice. They had found that the most restrictive part of studying other external units in the Masters degree was the lack of practical work. They found that the multitude of radiographic and ultrasound images provided in this unit met some of their practical needs:I have found the course very informative and have enjoyed the teaching material that you have sent... It has been a great opportunity for people like me who have been out in practice for over 20 years to catch up with all aspects of their subject and to enjoy the benefits of new teaching methods.
I think the practical component of this unit is great.Several students found that the workload in this unit was greater than in previous units in the Masters degree, and one student also found it more difficult:
This course would be wonderful if we had the luxury of twice the amount of time to do it in!
This is my second year on this course and therefore my second year at using a computer so I am getting over the shock of it all now and coping much better. I feel quite happy about it now...Students saw important advantages in working in a technology supported environment. The use of email and the web for communicating with lecturers was seen as very positive and better than telephone contact. The asynchronous nature of email was identified as important given the widespread, and busy, student cohort.
I think the major benefit of the technology is being able to work independently in a variety of situations - it allows for greater flexibility.Despite the range of problems experienced by the students, they felt that future students would benefit from the use of the CD-ROM based approach.I think it is essential in allowing for continuous feedback and access to tutors.
The technology is definitely broadening the horizons for the future.I think the growth of your ideas with the computer programs will make the teaching even better in the future. It will probably enable more information to be sent to us.
It can be seen that the majority of the effort in the project went into the scanning of the images, followed by the development of the version 1 database. The project management component is consistent with other relatively sophisticated multimedia development projects (Canale & Wills, 1995; Phillips, 1997, p57)
| Phase | Hours | Percentage of effort |
| Project Management | 75.5 | 16% |
| Prototyping | 36 | 8% |
| Scanning | 213.5 | 45% |
| Version 1 | 101 | 21% |
| Version 2 | 22.25 | 5% |
| Website | 24.75 | 5% |
| Total | 473 | 100% |
Table 4 summarises the cost of production of the 319 QTVR images. It can be seen that the cost metric for the production of a new QTVR image is $27. This contrasts with a cost of $14 for each hard copy image. However, from Table 1, each subsequent hard copy also costs $14, while the marginal cost of production of new CD-ROMs is quite low. This argument is carried further in Table 5, which compares the fixed and marginal costs of producing the teaching resources in both hard copy and CD-ROM formats. The figures in the hard copy column are derived on a per student basis from the rightmost column of Table 1. The up front electronic cost is derived from the 473 hours worked by the development team at $40/hour. The other figures are best guess estimates. Using these figures, the CD-ROM method becomes more cost effective at enrolments of 7 students, and each additional student returns a benefit of approximately $4000.
| Total scanning expenditure | $8,540 |
| Number of original hard copy images | 250 |
| Cost per hard copy image | $34 |
| Number of QTVR images | 319 |
| Cost per QTVR image | $27 |
| Hard copy | Electronic | |
| Up front costs (1 set) | $2,748 | $18,920 |
| Reproduction costs for extra copies | $2,748 | $10 |
| Postage | $243 | $8 |
| Administrative handling | $45 | $1 |
| Maintenance costs | $45 | ? |
From Table 1, the average cost of providing hard copy radiographs to students in 2000 (postage and handling) was $334 per student ($7340 in total). However, in 1998, three sets of materials were damaged or lost. These should have been replaced, at $2748 per set.
The real cost of maintaining full sets of materials for 25 students is $7340 + 3 x $2748 (=$15,583). This equates to $623 per student. On the other hand, the cost of burning and distributing CD-ROMs is estimated as $19 per student, plus an undetermined maintenance cost.
There is a benefit to the school of $12100 ($604 per student) each year in using the electronic images. This could be spent each year in maintaining the electronic images without costing the School any more than it does at present. At the same time, extra student income can be earned at a marginal cost of $19 per student (assuming that extra students do not cause extra teaching load).
The CD-ROM solution offers the school more flexibility and opportunities for income generation.
There are acknowledged problems with both versions of the CD-ROM, and work is proceeding on development of a third solution. In this version, HTML pages will be generated automatically from a database, and stored in a static form, which allows the images to be annotated with contextual information. These pages may be delivered entirely on CD-ROM, or using the WebCT CD-ROM tool.
In the section entitled Teaching Problems we identified problems experienced by students working alone with hard copies. Work has begun to circumvent some of these problems, by including interactive tutorials and quizzes. It is intended that interactive activities will replace print based worked case studies and teleconferences, thereby:
We are confident that the approach described here will lead to increased enrolments in the Masters degree, as enrolment numbers will no longer be limited to the sets of physical resources available.
The project team members from the Educational Design Group of the Teaching and Learning Centre at Murdoch University were:
Nick Castle - Multimedia Programmer
Eleanor Chaos - Scanning and QTVR
Christine Bailey - Project Coordinator
Lisa Masiello - Graphic Designer
Terri Sheehan - HTML Author
Carol Adair - HTML Author
Bain, J. D. (1999). Introduction. Higher Education Research and Development, 18(2), 165-172.
Canale, R., & Wills, S. (1995). Producing professional interactive multimedia: Project management issues. British Journal of Educational Technology, 26(2), 84-93.
Collis, B. (1996). Tele-learning in a Digital World: The Future of Distance Learning. International Thomson Computer Press.
Harasim, L., Hiltz, S. R., Teles, L., & Turoff, M. (1995). Learning Networks: A Field Guide to Teaching and Learning Online. Cambridge Massachusetts: The MIT Press.
Apple Computer Inc. (2000). QTVR Home Page. Apple Computer Inc. [viewed 6 Jun 2000, verified 3 Mar 2001] http://www.apple.com/quicktime/qtvr/
Kennedy, D., Eizenberg, N., Kennedy, G. (2000). An evaluation of the use of multiple perspectives in the design of computer facilitated learning. Australian Journal of Educational Technology, 16(1), 13-25. http://www.ascilite.org.au/ajet/ajet16/kennedy.html
Kraft, S., Hoskinson, J. J., Mussman, J. M., Michaels, W. E., McLaughlin, R., Gaughan, E. M., & Roush, J. K. (1998). Development of interactive patient-based multimedia computer programs in veterinary orthopaedic radiology. Veterinary Radiology and Ultrasound, 39(2), 98-104.
Phillips, R. (1997). The Developer's Handbook to Interactive Multimedia - A Practical Guide for Educational Applications. London: Kogan Page.
Phillips, R., Bain, J., McNaught, C., Rice, M., & Tripp, D. (2000). Handbook for Learning-centred Evaluation of Computer-facilitated Learning Projects in Higher Education (1.0). Australasian Society for Computers in Learning in Tertiary Education. [viewed 17 Apr 2000, verified 18 Apr 2004] http://www.tlc.murdoch.edu.au/archive/cutsd99/handbook/handbook.html
Phillips, R. A. (2001). A Case Study of the Development and Project Management of a Web/CD Hybrid Application. Journal of Interactive Learning Research, in press.
Phillips, R. A. & Luca, J. (2000). Issues involved in developing a project based online unit which enhances teamwork and collaboration. Australian Journal of Educational Technology, 16(2), 147-160. http://www.ascilite.org.au/ajet/ajet16/phillips.html
Phillips, R. A., Pospisil, R., Bell, J., & Patterson, A. (1999). Meeting Needs: A staff development resource for redesigning sociology courses according to an outcomes-based model. Proceedings Australasian Society for Computers in Learning in Tertiary Education Conference, Brisbane, Australia. http://www.ascilite.org.au/conferences/brisbane99/papers/phillipspospisil.pdf
WebCT Educational Technologies (1999). Web CT: World Wide Web Course Tools. http://www.webct.com/
| Authors: Rob Phillips Teaching and Learning Centre Murdoch University, Murdoch WA 6150 r.phillips@murdoch.edu.au
Romana Pospisil
Jennifer L Richardson Please cite as: Phillips, R., Pospisil, R. and Richardson, J. L. (2001). The use of a QTVR image database for teaching veterinary radiology and diagnostic ultrasound to distance education students. Australian Journal of Educational Technology, 17(1), 96-114. http://www.ascilite.org.au/ajet/ajet17/phillips.html |