|Australian Journal of Educational Technology
2003, 19(1), 72-86.
A national study in Australia in the late 1990s explored barriers to the adoption and reuse of computer facilitated learning (CFL) in Australian universities. These barriers will be summarised. One of these barriers is that it is hard to find information on courseware that is educationally sound; usually such courseware is expensive to produce and so reuse is especially desirable. However, even when information and access to electronic courseware exists, reuse may still not occur. Two cases will be described to illustrate the complexity of reuse. These cases are: 1) a collection of 169 plastic surgery websites; and 2) an international consortium of veterinary microbiology resources based on a well-evaluated case study design. Some strategies for improving reuse are suggested.
Pearson Education's reach extends across the globe through its seventy regional Web sites and twenty-five publishing centers, developing educational products for children, schools, universities, adults and corporations in thirteen languages. Long renowned as the world's market leader in English Language Training, Pearson Education maintains publishing operations in seven regions: the U.S., Europe, Middle East and Africa, Canada, Latin America, North Asia, South Asia, and Australia & New Zealand. Global imprints include Longman, Scott Foresman, Prentice Hall, Addison-Wesley, Prentice Hall-Financial Times, Markt & Technik, CampusPress, Direct English and Éditions du Renouveau Pédagogique. (http://www.pearsoned.com/globalPub.htm)Many of the Pearson texts have electronic companions as CDs or websites; both open sites, Blackboard and WebCT are used. But the text remains the primary market commodity and the electronic materials are just enhancements.
In order to get similar models operating with purely electronic courseware, what is needed? Is it just a question of organising electronic resources better? Or are there deeper issues about the use of electronic media in education? We are now in the position that there has been a great deal of development of electronic educational resources in universities world wide. This has occurred together with substantial investment of IT systems and infrastructure. However, there is little evidence of dissemination of these electronic resources and practices. Greater collaboration and sharing of resources is becoming an increasingly urgent issue. There are several existing databases of computer facilitated learning (CFL) materials, but these databases do not appear to have significantly increased the take up of CFL materials and strategies. In order to make the most of the valuable resources which exist, a range of educational, technological and management factors needs to be addressed. Let's look briefly at one study which examined adoption of CFL across Australian universities.
This study found that the issues surrounding the adoption of CFL at university are complex, and no single factor will result in adoption. Instead, there is a range of factors, all of which must be addressed. Several universal factors in relation to widespread use of CFL were identified:
Commercial search engines were used to identify plastic surgery websites in July 2000. One year later all sites were revisited. Those no longer available were excluded and the remaining sites were scored according to content, complexity, technical difficulty and readability using the Flesch-Kincaid Grade Level Score (e.g. Johnson, 1998). This readability index computes readability based on the average number of syllables per word and the average number of words per sentence. The score in this case indicates a grade school level. For example, a score of 8 means that an eighth grader would understand the document. Standard writing approximately equates to the seventh to eighth grade level. The maximum score is 10. Two hundred and five sites were identified in July 2000 and one year later 169 were still online. Less than 10% of sites scored 8 and above, 60% scored 4 and below. There is clearly a wide range of material available on the Internet but the question remains how to identify the sites with the greatest educational value, especially for students for whom English is a second language? Is reading ease a key factor in a visually rich environment? Are the objectively high scoring sites the best?
In order to begin to answer these questions, we asked 150 students to evaluate these 169 Internet sites according to their subjective perception of educational value, including language and content. Students related to the sites according to their own personal judgments of their educational interests and needs. They were not given specific criteria but ranked them on subjective impressions of their value to themselves at the time of ranking. The rankings from the objective (reading ease) and the subjective (personal interests and needs) scoring did not correlate. Clearly the reading ease of the text was not a key decisive factor in deciding students' preferences. While this is not a rigorous study, it has highlighted the wide variety of student preference about the style of web material they prefer.
We are left with a wealth of material but no clarity at this point about how to organise and frame this material so as to support individual student learning styles (Coates & Rowsell, 1998). How does one design the basic shell which points to these sites? How do we order and groups these sites? Certainly a list of 169 URLs is not likely be an educationally engaging site! Should detailed descriptions be added, pointing out a range of ways in which the material on each site might be used? Should we add questions which can be answered by searching a number of sites? Should we use these sites as a basis for case study questions? Some or all of the above? All we know at this stage is that an enormous amount of work will be needed before this potential wealth of the Internet can be realised for the benefit of medical students studying plastic surgery. And we are unclear about how to find the time and technical resources to do this work.
This case shows clearly that good educational design is one important element in integrating existing electronic resources into a coherent learning resource for students. Knowing about resources and having access to them is important, but knowing what to do with them in order to facilitate learning provides another set of complex challenges.
In January 1996 an international consortium of over 30 veterinary bacteriology and mycology teachers from Australia, Canada and the USA was formed to provide new online materials for veterinary courses. One of the main focuses in the formation of this 'consortium' was the emphasis on learning microbiology as a tool for preparing professional veterinarians, rather than learning microbiology for its own sake. PBL was to be a foundation principle for all projects. As a pilot, a range of materials was produced to allow members of the international consortium to comment on issues such as content and style. In order to gauge student response to these initial VetSource materials, five students were interviewed in September 1997. Student response to the initial VetSource materials was very encouraging (McNaught, Whithear, Browning, Hart & Prescott, 1998).
All this is very positive, but two years later the consortium had petered out. Only three universities actually provided materials and this is not enough to build the foundation for a true multi-institutional consortium. The funding that the consortium hoped to obtain through the US Department of Agriculture did not eventuate. The University of Melbourne was building a web based platform that would be highly suitable for the interactive nature of the VetSource modules (Goschnick, 1998). Despite a public launch of Creator (as the platform was called), funding for completion of the project was withdrawn. The lack of buy-in in terms of producing materials, the lack of a technical platform and the limited funding took its toll. The few VetSource modules were incorporated in the VetBac project at the University of Melbourne, which still continues, but these materials were not used by any of the other consortium members. The model of VetBac has influenced the development of other computer facilitated PBL veterinary work at the University of Melbourne (McNaught, Whithear & Browning, 1999) but the scale of operation has a much more limited scope than we hoped in the mid-1990s.
In this story there were three barriers-funding, a technical platform and insufficient production of the modules. A conclusion we might draw about why there was insufficient production of the modules is that the work environments in the universities in the consortium did not support the time and energy these academics needed to devote to making the consortium a sustainable entity. The models of how to do PBL exist; the willingness to engage can be kindled; but the work environment of higher education needs to be more supportive towards the time needed for innovation and collaboration.
|Student learning need|
(after Open University, 1998)
|Examples of appropriate use of online strategies|
|Building and maintaining motivation|
|Information handling skills|
|Independent learning skills|
|Linking theory to practice|
|Practising discussion, argument, articulation of ideas|
|Rehearsing skills and procedures|
|* Clear and current information includes items such as:
Improving the understanding of how electronic resources might work with other design elements of the student learning experience
Several people are working in this area. One model is given by McNaught, Kennedy & Majoor (2002). They describe a model of how different student learning needs can be supported by the various functions of and strategies for online learning. This model will hopefully further the understanding of how the various components of an online learning site, such as information areas, interactive tutorials, quizzes, and access to threaded discussions and chat can support learning for more students most effectively (Table 1). This may make it easier for teachers to see how a variety of electronic materials they have collected (such as the plastic surgery resources described earlier) might be used in designing a coherent student learning environment.
The development of generic designs
There is certainly a major interest in whether it is possible to develop generic designs. Generic designs can be described as a combination of clearly articulated learning designs combined with templates into which media elements can be inserted. The Australian University Teaching Committee (AUTC) has funded a project titled: 'Information and Communication Technologies (ICTs) and their role in flexible learning' The project is a two year endeavour and began in November 2000. The project's aims as stated at the end of 2001 were:
Access to information about CFL resources
While we have said this is not the determining factor in the use and reuse of CFL, it is important, and the current work being done on metadata and interoperability standards needs to be followed so that Internet architecture is developed with learning in mind. A good site to keep on eye on is the Australian IMS project site, hosted by EdNA Online at [http://ims.edna.edu.au/] It provides information on metadata standards and projects in Australia and elsewhere. Though not recently updated (most of the pages are August 2000), it does provide contact points for further searching.
AustLit, [http://www.austlit.edu.au/] (Australian Literature Gateway) and UniServe Science [http://science.uniserve.edu.au/] are examples of well-known information points and serve as continuing models of provision of access to relevant information and resources. In addition, UniServe Science organises events such as national symposiums in order to enact a role of fostering "a sense of community among tertiary teachers of science" (Johnston & Peat, 2002).
A recent Australian survey by the Department of Education, Science and Training (DEST) (Bell, Bush, Nicholson, O'Brien & Tran, 2002) investigated the extent of the use of online technologies by Australian universities. Fifty-four percent of university units now include an online component. The report describes the nature of online courses, what disciplines they are in, and how they are supported by online services. This is useful information and may hopefully be the beginning of more detailed sharing of information about online resources being used in Australian universities.
Policies and processes to enable collaborative work, both within and across institutions
Just how much activity is there in this area? Just how high up on the priority list for our governments is this type of policy item? If it is seen as a having a high priority at government level, it is much more likely to get attention at institutional level. Here we will focus on the Australian government. In order to get a rough idea of the number of studies and reports in the area of collaborative policy in higher education commissioned by the Australian government in recent years the Australian Department of Education and Science (DEST) website was searched using InfoCat. The InfoCat ('Information Catalogue' http://infocat.dest.gov.au/IE/) covers most research and statistical information published by DEST since 1995, with some information for work done prior to 1995. EdNA Online (http://www.edna.edu.au/) was also searched looking for Australian sites. The search words and results are shown in Table 2.
While this is hardly an exhaustive or rigorous search, it demonstrates two important points.
|'All words' search on 23 July 2002||Site and no. of results||Comment|
|Collaboration policy||DEST 5||Four of these are research related, rather than related to teaching and learning policy; the fifth is McNaught et al. (2000), which is largely an analysis of existing issues.|
|Collaboration process||DEST 6||One is research related, one is a 1992 curriculum document, one is about academic writing support; one is an equity report, and the fifth is on youth issues. The final is Taylor & Richardson (2001), which is a proposal for peer review of ICT resources. If implemented (and there is no certainty about this), this proposal could support reuse.|
|Collaboration policy||EdNA Australian sites. 20 out of 382,331 in the collection||Eight of these are focused more on research, industry or the general community. The other 12 do provide examples of organisations/ networks which illustrate collaborative policy.|
|Collaboration process||EdNA Australian sites. 19 out of 382,331 in the collection||Eleven of these relate to teaching and learning in some way (the others relate more to research or industry); a couple of examples of actual collaborative processes exist here.|
One government initiative in Hong Kong is worth sharing because of its explicit recognition of the importance of inter-institution collaboration. In Hong Kong there are eight higher education institutions (see http://www.ugc.edu.hk/english/fund_inst.html [verified 5 Feb 2003]), each with a distinctive character. To date, there have been three rounds of Teaching Development Grants which are awarded on a competitive basis. These are substantial grants in the range $HK1-10M. The criteria on which grant proposals are judged are impact (60%), outcomes (20%), collaboration (10%) and alignment with institutional goals (10%). Within the collaboration category, the following scale is used,
|5.||The project involves two or more institutions, all of which have contributed to the project design and will have a significant and well-defined role in the project's implementation and resulting benefit.|
|4.||The project involves two or more institutions, all of which have contributed to the project design, have a significant and well-defined role in its implementation, although the benefit for involved institutions is not equivalent.|
|3.||The project was designed by the submitting institution, but more than one institution will have a significant and well-defined role in its implementation, but benefit for all participants will not necessarily be equivalent.|
|2.||Individual staff members from outside the submitting institution appear to have contributed to the project design, but only one institution has a significant role in its implementation and will in all likelihood be the prime beneficiary of input upon completion.|
|1.||The project involves individual staff members from outside the submitting institution, but they appear to have only a pro forma involvement and thus benefit to their institution is questionable.|
|0.||The project does not involve more than one institution and benefit is restricted to the submitting institution.|
While this might mean that some 'surface' collaborations are formed at grant writing time, this policy has resulted in several real collaborations where resources are produced and shared more widely.
Australian IMS project. http://ims.edna.edu.au/ [23 July 2002].
Australian Literature Gateway. http://www.austlit.edu.au/ [23 July 2002]
Bell, M., Bush, D., Nicholson, P., O'Brien D. & Tran, T. (2002). Universities Online: A survey of online education and services in Australia. Department of Education, Science and Training Higher Education Group, Occasional Paper Series 02-A. http://www.dest.gov.au/highered/occpaper/02a/02_a.pdf [23 July 2002, verified 5 Feb 2003]
Coates J., & Rowsell, A. (1998). Plastic surgical training and the Internet: How plastic surgeons learn. British Journal of Plastic Surgery, 51, 74-77.
EdNA Online (Education Network Australia). http://www.edna.edu.au/ [23 July 2002]
Goschnick, S. G. (1998). Design and development of Melbourne IT Creator TM - A System for authoring and management of online education. http://www.solidsoftware.com.au/Information/Paper/Tools98/Creator.html [23 July 2002, verified 5 Feb 2003]
Harper, B., Oliver, R. & Agostinho, S. (2001). Developing generic tools for use in flexible learning: A preliminary progress report. In G. Kennedy, M. Keppell, C. McNaught & T. Petrovic (Eds), Meeting at the Crossroads. Proceedings of the 18th Annual Conference of the Australian Society for Computers in Learning in Tertiary Education. (pp. 253-262). Melbourne: Biomedical Multimedia Unit, The University of Melbourne. [23 July 2002, verified 5 Feb 2003] http://www.ascilite.org.au/conferences/melbourne01/pdf/papers/harperb.pdf
InfoCat (DEST 'Information Catalogue'). http://infocat.dest.gov.au/IE/
Johnson, K. (1998). Readability. [23 July 2002, verifed 5 Feb 2003] http://www.timetabler.com/readable.pdf
Johnston, I. & Peat, M. (2002). Scholarly inquiry and flexibility. In Scholarly inquiry in flexible science teaching and learning, (pp. 1-2). Proceedings of Symposium, 5 April. UniServe Science: The University of Sydney. [verified 4 Feb 2003] http://science.uniserve.edu.au/pubs/procs/wshop7/schws001.pdf
McNaught, C., Kennedy, D. & Majoor, J. (2002). Designing online learning sites to cater for learners' needs. In Proceedings of the International Conference on Computers in Education (ICCE 2002). Auckland, New Zealand, 3-6 December.
McNaught, C., Phillips, P., Rossiter, D. & Winn, J. (2000). Developing a framework for a usable and useful inventory of computer-facilitated learning and support materials in Australian universities. Evaluations and Investigations Program report 99/11. Canberra: Higher Education Division Department of Employment, Education, Training and Youth Affairs. [verified 5 Feb 2003] http://www.detya.gov.au/highered/eippubs1999.htm#99_11
McNaught, C., Whithear, K. & Browning, G. (1994). The role of evaluation in curriculum design and innovation: A case study of a computer-based approach to teaching veterinary systematic bacteriology and mycology. In K. Beattie, C. McNaught & S. Wills (Eds), Interactive multimedia in university education: Designing for change in teaching and learning, (pp. 295-308). Amsterdam: Elsevier.
McNaught, C., Whithear, K. & Browning, G. (1999). Systems not projects: Focusing on evaluating the overall student experience, rather than isolated innovations. Higher Education Research & Development, 18(2), 247-259.
McNaught, C., Whithear, K., Browning, G., Hart, G. & Prescott, J. (1998). The best of both worlds: Redeveloping a multimedia project for the web. In T. Ottmann & I. Tomek (Eds), Proceedings of Ed-Media & Ed-Telecom 98. 10th World Conference on Educational Multimedia and Hypermedia and World Conference on Educational Telecommunications, volume 1, (pp. 946-951), Freiburg, Germany, 20-25 June.
Open University (1998). Technology Strategy for Academic Advantage. [23 July 2002, verified 5 Feb 2003] http://www2.open.ac.uk/ltto/tsaa/index.htm
Pearson Education Global Publishing. http://www.pearsoned.com/globalPub.htm [23 July 2002]
Prater, M. A. & Smith, D. J. (1989). Determining undergraduate curriculum content in plastic surgery. Plastic Reconstructive Surgery, 84, 529-533.
Shazly, E. L., Mohamed, M., & Maiwald, G. (2000). The Internet: A new friend to plastic surgeons. Plastic Reconstructive Surgery, 106, 235.
Taylor, P. G. & Richardson, A. S. (2001). Validating scholarship in university teaching. Constructing a national scheme for external peer review of ICT-based teaching and learning resources. Evaluations and Investigations Program report 01/3. Canberra: Higher Education Division Department of Employment, Education, Training and Youth Affairs. [23 July 2002, verified 5 Feb 2003] http://www.dest.gov.au/highered/eippubs/eip01_3/01_3.pdf
UniServe Science. http://science.uniserve.edu.au/ [23 July 2002]
Van Heijningen, R. I., Mannaerts, G. H. H., Blondeel, P. H. N. & Spauwen, P. H. M. (1998). PLink, Plastic surgery and the Internet. British Journal of Plastic Surgery, 51, 86-89.
Whithear, K. G., Browning, G. F., Brightling, P. & McNaught, C. (1994). Veterinary education in the era of information technology. The Australian Veterinary Journal, 71, 1-3.
|This article was nominated for an Outstanding Paper Award at ASCILITE 2002, gaining the additional recognition of publication in AJET (with minor corrections). The reference for the Conference version is:
McNaught, C., Burd, A., Whithear, K., Prescott, J. and Browning, G. (2002). It takes more than metadata and stories of success: Understanding barriers to reuse of computer facilitated learning resources. In A. Williamson, C. Gunn, A. Young and T. Clear (Eds), Winds of Change in the Sea of Learning: Proceedings of the 19th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education, pp451-460. Auckland, New Zealand: UNITEC Institute of Technology.
Authors: Carmel McNaught, Centre for Learning Enhancement and Research, Chinese University of Hong Kong, Hong Kong. firstname.lastname@example.org
Andrew Burd, Department of Surgery, Chinese University of Hong Kong, Hong Kong. email@example.com
Kevin Whithear, Faculty of Veterinary Science, The University of Melbourne, Australia. firstname.lastname@example.org
John Prescott, Ontario Veterinary College, University of Guelph, Canada. email@example.com
Glenn Browning, Faculty of Veterinary Science, The University of Melbourne, Australia. firstname.lastname@example.org
Please cite as: McNaught, C., Burd, A., Whithear, K., Prescott, J. and Browning, G. (2003). It takes more than metadata and stories of success: Understanding barriers to reuse of computer facilitated learning resources. Australian Journal of Educational Technology, 19(1), 72-86. http://www.ascilite.org.au/ajet/ajet19/mcnaught.html