Australian Journal of Educational Technology
2000, 16(3), 201-214. 		AJET 16

Online development using WebCT: A faculty managed process for quality

Sally Burford and Lesley Cooper
Faculty of Social Sciences, Flinders University
In a new initiative, and in response to technological changes, Flinders University is delivering educational programs to distance education students using WebCT as its preferred course tool. With concern for maintaining high quality educational programs, the Faculty of Social Sciences has approached the implementation of this new form of delivery with a management plan that involves and integrates many facets of the Faculty's existing structures with some additional resources. The Faculty promotes a development process that emphasises the use of WebCT as a prototyping tool and a team approach, which values the varied skills of all contributors.

Introduction

Australian universities have undergone profound changes over the past 15 years. This includes expansion of student numbers taking in more diverse students, the implementation of improved management practices and direct competition between universities for students and research funding. Of critical importance are changes in the teaching and learning environment, which are being rapidly reshaped by the impact of new technology and rapid communications. This paper begins by reviewing the conceptual and practical understanding of quality in teaching and learning and then outlines how these frameworks and principles have been implemented within a faculty structure and management plan.

Quality, a conceptual view

The term quality is notoriously difficult to define. Green (1994:15) defines quality as fitness for purpose by which she means that quality is judged in terms of the extent to which a product or service meets its stated purpose. There are particular problems associated with this definition especially in understanding what purpose means and who should define it - government, consumers, academics or employers? Nevertheless when Green defines quality is this way she is deliberately pragmatic recognising that universities having multiple services, products and priorities. She argues that the best universities can do when defining quality is to identify as clearly as possible the best standards and criteria used by various stakeholders when judging quality.

Another view of quality and one, which fits easily under the fitness for purpose approach, is that of defining quality according to conformance to particular standards. Standards can be determined by the inputs to the service, the process by which the product or service is made and the outcomes achieved. The evaluation of quality is determined by the extent to which the product or service reaches the desired standards. It is this view of quality, which underpins the development of work within the Faculty of Social Sciences. In our approach to quality management using WebCT, standards are embedded in the process for development.

Quality, the government's agenda

In the late 1980 and early 1990s two forces combined. There was an increase in the numbers of students attending higher education and a government committed to improving the performance, management and quality of the public sector. Institutions were to achieve greater efficiencies but also a greater responsiveness to customers and taxpayers. These demands for increased accountability and better performance have put issues of quality high on government's policy agenda. Whilst the framework is still evolving there are a number of key policy initiatives.

In the early 1990s the Commonwealth Government announced a set of measures to improve the quality of higher education especially that of teaching and research. As a result of this initiative a non-statutory Ministerial Advisory Body was established to assist in the implementation of the Quality Assurance Program from 1993 to 1995. This body was called the Committee for Quality Assurance in Higher Education (CQAHE.) Between 1992 and 1995, CQAHE conducted three rounds of quality reviews which included a review of teaching and learning in two of the three reviews. While these review processes were criticised, the end result was that universities more clearly focused on the need for continuous improvement and the importance of stakeholders and customers.

After the terms of reference of CQAHE expired, the 1996 Commonwealth Budget proposed a new approach to quality improvement in higher education. With the aim of improving quality institutions, the Higher Education Council was asked to report on structures required for reporting progress and achievements (through the educational profiles process) and reviews on particular aspects of higher education amongst other things. These recommendations were consolidated in 1998 with universities required to include quality assurance and improvement plans in documentation provided as part of the educational profiles process. At a minimum, these plans required that universities get feedback from employers about the quality of graduates, data on the employment of recent graduates and through the Course Experience Questionnaire, the graduates' perceptions of teaching. All university profiles for 1999 - 2001 are publicly available including that of Flinders University. In an overview of the report, Quality of the Higher Education System, DETYA (1999) notes a number of areas where quality can be improved. In the teaching area significant ones include:

In 1996 on average 77% of students from Australian Universities were satisfied with the quality of their teaching (DETYA 1999). At Flinders during this period, the overall results on the good teaching scale in 1996 was 80% increasing to 82% in 1997 (Flinders University 1999).

In reviewing these government approaches, there is a top down pressure to improve educational quality. Nevertheless, it has also sponsored bottom up initiates to improve the quality of teaching and learning. These teaching initiatives were funded through the Ministerial Committees of CAUT and CUTSD. Many of these initiatives were in the area of web delivery. The outcomes of these projects will be reviewed in the subsequent section.

Table 1: Milestones in quality in Australian higher education

DateEvent Purpose/Effect
1988Dawkins White Paper (DEET 1988)Announcing policy changes to the funding, structure and management of higher education
1993-95Committee for Quality Assurance in Higher Education (CQAHE, 1995)Review the quality assurance practices and outcomes in publicly funded universities
1993-96Committee for Advancement of University Teaching (Moses et al 1995)Project funding to improve the quality of teaching and learning
1996Budget Statement (Vanstone, 1996)Foreshadowed the development of quality improvement plans and application of performance indicators
1997-99Committee for Teaching and Staff Development (CUTSD 1999)Project funding for teaching and organisational grants
1997HEC: Quality in Resource Based Learning (HEC, 1997)This sets out a quality assurance framework for resource based learning including that of web based material
1997The West Report, Higher Education Review (Higher Education Financing and Policy Review Committee, 1998)Vision for higher education that includes student centred funding, research priorities, financing and regulatory framework and foreshadows the digital revolution
1998HEC: Quality Implementation and Reporting in Australian Higher Education 1998 (HEC, 1998)This report provides advice to government on the approach to quality improvement in higher education; and reporting on quality assurance outcomes
1999The Quality of Higher Education: An Overview (DETYA 1999)The publication of 1999-2001 quality assurance and improvement plans of 36 of Australia's 38 publicly funded universities
2000Benchmarking: A Manual for Australian Universities (McKinnon et al 2000)Identifies the most important aspects of contemporary university life in changing times and ways of benchmarking them

Quality in teaching and learning

Quality in resource based education has been specifically addressed by the Higher Education Council (1997). They established a set of principles on which quality improvements in resource based learning programs could be developed. Amongst other things these principles include managing and maintaining the technical infrastructure, evaluating for continuous improvement, the effective design, development and implementation of programs for active learning and the provision of effective and efficient administrative systems (HEC 1997:2).

In a comparative analysis of the principles of good technological teaching practice from the Western Cooperative for Educational Telecommunications in the USA and the Open University, Garson (no date) notes three key themes. These are:

He notes that these standards are limited as they fail to take into account In his paper, Garson is particularly critical of the latter point arguing that the cost of web based technology may thwart empowerment of staff and quality student to faculty interaction.

Most recently in Australia, Shirley Alexander and her colleagues at UTS evaluated Information Technology Projects funded by CAUT in 1994 and 1995 (Alexander, McKenzie and Geissinger 1998). Whilst the aim of this evaluation was to determine how the use of technologies produces student learning, these results especially their profiles of successful and unsuccessful projects provide useful insights into quality. They cite the importance of good educational design, a team approach that includes a skilled project manager, a process of development taking account of useability, student need, continuous evaluation throughout the development process and a supportive Head of Faculty.

Thus far we have examined quality conceptually and reviewed the government's agenda. We have also examined quality of teaching and learning generally and reviewed the evaluations of teaching initiatives using technology. There are several recurring themes. Firstly, governments will increase their demands for high quality teaching and learning and set in place mechanisms to assure the community that these standards have been achieved. Benchmarking practices across universities will become part of contemporary university life. Secondly, any quality improvements will be achieved in a climate of constrained funding. Thirdly, any new teaching and learning initiative will require that we attend to quality at the outset. In online delivery this means that any faculty development requires a clearly stated policy, standards and guidelines, a specified development process and multi skilled team approach that encourages efficient use of resources.

Flinders University in context

Flinders University is traditionally a strong research institution with small, localised offerings in distance education. It has recognised the need to compete for its existing student population and is a member of the International Network of Universities - a global network, which offers online delivery of learning material to students of member institutions.

The University comprises four faculties and promotes a decentralised organisational structure, whereby each faculty holds a large degree of autonomy. With some centralised resources in place, the management and development of policy and procedures for online delivery of courses is a faculty responsibility and all development occurs within the Faculty.

The Faculty is a latecomer to online delivery but there are opportunities in this. It enters the arena, at a fairly sophisticated level, when other tertiary institutions have laboured long and hard to develop their own technologies and directions. There exists a large body of knowledge in the use of educational technologies and directions and expertise gathered at other institutions can be drawn upon.

Strong commitment from senior faculty staff to provide managed resources and processes in online course development is held in tension with the ongoing autonomous traditions of universities.

WebCT - A double edged sword

WebCT is a commercially available integrated set of web based course tools. WebCT claims that the product offers instructors the ability to easily access and create content and interactive web based learning experiences (WebCT site, March 2000) and Goldberg and Salari (1997) tell us that WebCT is a tool that facilitates the creation of sophisticated web based educational environments. This is certainly the case but requires some qualification. The quality and sophistication of the final product is dependent on the skills and the experience of the 'instructor' in all aspects of web site design and educational design. A comparison of WebCT and other similar products is not a point of discussion here - what is important is how such a course development tool is used.

Taylor (1995) rightly points out that the instructional media does not automatically enhance the quality of teaching and learning. In fact, skilled and innovative educators in the face to face mode, when confronted with the online conversion and a range of technologies may not be able to 'map' their strategies to online delivery and may resort to linear, didactic teaching styles (Brahler 1999).

WebCT allows academics in the Faculty to retain ownership and control of their online teaching environments - this in itself is supportive of the ethos of the Faculty. It must however, be balanced with concern for quality and the recognition that most academics are not skilled in interface design, multimedia, selecting appropriate technologies for online teaching nor project management.

A team approach

It is well documented (Bates 1995, Brahler 1999, Sims 1997) that a team approach to online course development is a reliable model. Nouwens and Robinson recommend 'a flexible but systematic, participative and teambased approach'. McKey (1999) encourages us to think of online delivery from a student's perspective and to consider a framework - 'The Total Student Experience' - that includes presentation, function, education and administration. If all these facets of online delivery combine to produce a quality learning environment, it is clear that an individualistic approach to development is not a possibility. A team of people is required to put together a 'Total Student Experience'.

Bates (1995:17) encourages a break from traditional university teaching and promotes a change in teaching methods using technology to improve the quality of teaching and learning. Controversially, he describes the traditional lecture as an anachronism - in Medieval times, a single copy of a beautifully, hand written book was read to students because there was just one copy. 'Lecture' is derived from the Latin verb 'to read'. With concern for quality, we cannot take existing face to face resources and 'put them on the web'. For many academics, the required changes - from 'traditional' university teaching to the development of online materials are dramatic and challenging. New approaches to teaching and learning, the strengths and limitations of different technologies, catering to differing styles of learning and designing effective learning environments require a new set of skills.

Alexander, McKenzie and Geissinger (1998) reported that the use of a particular information technology did not, in itself, result in improved quality of learning nor productivity of learning and that the most critical factor for successful outcomes was the design of the students learning experience. Mindful of this, the Faculty has employed an educational designer with experience in information and communication technology, online delivery, educational practice and the tertiary arena. The team also comprises a Computer Systems Officer providing technological assistance in multimedia and a project officer with a high level of technological literacy and skills. There is no physical, centralised unit within the Faculty - flexible delivery staff are located in the same offices as academics and much communication takes place in the corridors and on an impromptu basis.

Faculty flexible delivery staff are involved in online development of learning materials. The university, centrally, administers the WebCT server and conducts staff development workshops in WebCT.

The development process

Educational Systems Development has understandably evolved and borrowed from the traditional information Systems Development Life Cycle and has in place the phases of Analysis, Design, Production, Delivery and Evaluation. Sims (1997) has expounded upon the Interactive Instructional Influence Development model proposed by Aubrey in 1992. This model proposes four interconnected deliverables: Proposal, Prototype, Production and Package.

Whatever the development model, it must be achievable within the confines of established practice and available resources. The Faculty has customised and refined established development methodologies to meet local conditions and culture. It has removed development cycle terminology and has documented a very structured and colloquial planning requirement for academics to follow.

Acknowledging that any development is not a linear process and initial planning is merely a starting point for subsequent iterations of the process, the following working plan is required of any topic convenor attempting online delivery to remote students. The working plan should contain the following information:

  1. names of members of the development team
  2. topic aims
  3. learning outcomes for students
  4. the proposed timing of delivery (as an intensive or concurrent)
  5. the proposed date for on line delivery
  6. a statement about prospective students (demographic information, special needs, access to computers and the Internet)
  7. availability of academic staff for student consultation (when and how)
  8. arrangements with subject librarians about the delivery of reading material and reference material in electronic form if possible
  9. any requests for and any written statements of permission, for use of copyright material
  10. a statement of educational strategies which outlines how the aims and learning outcomes will be achieved and assessed using web based technology
  11. a design outline for web based delivery
  12. the extent to which web based material will be tested for its interface and useability
  13. address a method of academic peer review and production review prior to delivery to students. xiv. any academic administrative arrangements made with the Operations Manager, Academic and Student Services
The working plan incorporates the traditional analysis, design and evaluation requirements of educational systems development. Whilst remaining the responsibility of the academic, the working plan is developed in conjunction with the Faculty Educational Designer, especially in the areas that require educational and web site design. Every course of study is acknowledged as a unique project and there is no enforcement of structure for course delivery, although a suggested starting point is made available.

Rapid prototyping has been an invaluable and successful development strategy. Despite the completion of the analysis and initial design phases, many novices in online learning development are not able to conceptualise the end product. To overcome this obstacle, evolutionary prototyping is employed and a concrete working model is quickly developed. A small working portion of the total learning environment built with the educator's specific content enables communication between the team members. It is at this point that a preliminary review is requested from experienced online educators outside the development team. WebCT is to be acknowledged as an enabling tool for prototyping. With expertise in the WebCT development environment, it has proven to be remarkably easy and quick to develop an overall structure and detailed sections for communication purposes.

Usability testing during development is recommended by Anand and Zaimi (2000) and is a crucial aspect of developing a quality learning environment that benefits the learner. 'Students' are asked to participate in usability testing of WebCT environments developed for distance education. Scenarios/requirements are developed and introduced to the participants - they are observed and interviewed as they attempt to meet these requirements.

Whatever the model used, however well defined the process, the development process itself is a diffuse and difficult one. From its first conception to a final product, a development process is fraught with undefinable influences and unpredicted contributing factors. Participating in such a development is a skill developed by experience and willingness and involves the integration of many different skills and perceptions. Heuristic knowledge and experience of such a development process is invaluable in developing online learning environments.

It was fortunate that development of our first online learning project allowed time and attention to detail in terms of experiencing and reflecting on the process itself and in revision of educational and interface design. The experience and the product are now serving as a springboard for current projects. To date, the team approach and the process itself have proven to be successful.

Pragmatics - the development cost

Economic constraint is now a well accepted component of the tertiary culture and quality online learning materials are expensive to produce. Due attention must therefore be paid to efficiencies and care of limited resources. A considerable investment of time and finance needs to be protected by a management process that requires educational design, documented planning of all aspects of the project and evaluation of the materials at critical junctures in the development process.

A team approach offers a collection of skills with depth and variation outweighing by far that of most individuals. Specialised hardware and software are accessible for all to use and libraries of reusable, customisable software can be developed.

Another consideration is the experience of the development process itself. Brahler (1999) reports that development time and cost decreases with increasing developer experience. A team approach brings that experience to every project and captures the expertise as a Faculty Resource. Brahler warns us that an inexperienced individual, at best, produces an inferior product at an escalated price, or more frequently than not, no product at all. To protect its considerable investment, the Faculty of Social Sciences acknowledges a level of centralisation of resources, skills and processes whilst respecting the traditional ownership, decision making and autonomy of the educator and subject matter expert.

Other strategies for quality

The Faculty's concern has always been that online development is educationally driven, and technology itself is subservient. A decision was made not to rigidly template an environment but to allow each topic to be developed in terms of curriculum design, specification of learning outcomes, strategies to achieve those outcomes and then the choice of appropriate technologies. Within this freedom however arose the need to standardise some aspects of interface design. Faculty standards specify colour of text and background, graphics and positioning of some icons. It has assumed responsibility for student support documentation, provided guidelines for consideration of students with disabilities and developed a set of criteria against which evaluation of an online development is made. Currently in draft form, are a set of quality statements that define an optimal presentation of online learning.

End of semester evaluation seminars and ongoing Faculty reflection in the form of strategic options meetings in flexible delivery also address a concern for quality. In providing a platform of direction, policy, guidelines and standards for development, the Faculty is striving for quality outcomes in a particularly individualistic culture.

Conclusion

The use of information and communication technologies in teaching offers the opportunities and challenges to alter the established work practices of many and to revisit quality outcomes in tertiary teaching and learning. The Faculty of Social Sciences has attempted to build a set of development procedures that foster excellence and innovation. It acknowledges that quality is inherent in the development process. An increase in competition, productivity and accountability in tertiary teaching forces our concerns. WebCT has proved to be a tool that allows the flexibility to readily build learning environments that meet the challenges of quality in online teaching and learning.

References

Alexander, S., McKenzie, S. and Geissinger, H. (1998). An Evaluation of Information Technology Projects for University Learning. AGPS, Canberra. [see executive summary verified 11 Nov 2000 at http://www.detya.gov.au/cutsd/publications/exsummary.html and on 21 Mar 2001 at http://www.autc.gov.au/in/in_pu_cu_ex.htm]

Anand, P. G. and Zaimi, A. (2000). Computer-managed instruction: Evaluation of alternate methods of technology integration in higher education. Educational Technology and Society, 3(1). http://ifets.ieee.org/periodical/vol_1_2000/anand.html [verified 11 Nov 2000]

Bates, A. W. (1995). Technology, Open Learning and Distance Education. Routledge, London.

Brahler, C. J., Peterson, N. S. and Johnson, E.C. (1999). Developing online learning materials for higher education: An overview of current issues. Educational Technology and Society, 2(2). http://ifets.gmd.de/periodical/vol_2_99/jayne_brahler.html [verified 11 Nov 2000]

Committee for Quality Assurance in Higher Education (1995). Report on 1994 Quality Review, Vol 1 and 2. AGPS, Canberra.

Committee for University Teaching and Staff Development (1999). Committee for University Teaching and Staff Development 1998 Annual Report. Department of Education, Training and Youth Affairs, Canberra. http://www.autc.gov.au/in/documents/arfull.pdf [verified 21 Mar 2001]

Department of Education Training and Youth Affairs (1999). The Quality of Australian Higher Education: An overview. http://www.detya.gov.au/archive/highered/pubs/quality/overview.htm [verified 21 Mar 2001]

Department of Employment, Education and Training (1988). Higher Education: A Policy Statement. AGPS, Canberra. (This is known as the Dawkins White Paper.)

Flinders University of South Australia (1999). The Quality Process. http://www.detya.gov.au/archive/highered/pubs/quality/flinders.htm [verified 21 Mar 2001]

Garson, G. D. (no date). The Role of Technology in Quality Education. http://hcl.chass.ncsu.edu/sscore/garson2.htm [verified 11 Nov 2000]

Goldberg, M. W. and Salari, S. (1997). An Update on WebCT (World-Wide-Web Course Tools) - a Tool for the Creation of Sophisticated Web-Based Learning Environments. Proceedings of NAUWeb'97, June 12-15, 1997 Flagstaff, Arizona. http://star.ucc.nau.edu/~nauweb97/papers/goldberg/goldberg.html

Green, D. (ed) (1994). What is Quality in Higher Education? Society for Research into Higher Education and Open University Press, Buckingham.

Higher Education Council (1997). Quality in Resource Based Learning. AGPS, Canberra. See also http://www.detya.gov.au/nbeet/publications/hec/rnl/rbl.htm

Higher Education Council (1998). Quality Implementation and Reporting in Higher Education. AGPS Canberra.

Higher Education Financing and Policy Review Committee (1998). Learning for Life. AGPS, Canberra. (This is known as the West Report.) http://www.detya.gov.au/archive/highered/hereview/default.htm [verified 11 Nov 2000]

Nouwens, F. and Robinson, P. (1991). Evaluation and the development of quality learning materials. Australian Journal of Educational Technology, 7(2), 93-116. http://www.ascilite.org.au/ajet/ajet7/nouwens.html

McKey, Paul (1999). The total student experience. Key note address at ASCILITE 99. http://www.nexted.com/news/papers/tse_McKey.html [verified 21 Mar 2001 at http://www.ascilite.org.au/conferences/brisbane99/papers/mckey.pdf]

McKinnon, R. H., Walker, S. H. and Davis, D. (2000). Benchmarking: A Manual for Australian Universities. http://www.detya.gov.au/archive/highered/otherpub/bench.pdf [verified 21 Mar 2001]

Moses, I. and Johnson, R. (1995). Review of the Committee for the Advancement of Teaching and Staff Development. http://www.detya.gov.au/cutsd/caut/reports/finrep.pdf [verified 11 Nov 2000]

Sims, R. C .H. (1997). Interactive learning as an "emerging" technology: A reassessment of interactive and instructional design strategies. Australian Journal of Educational Technology, 13(1), 68-84. http://www.ascilite.org.au/ajet/ajet13/sims.html

Taylor, J. C. (1995). Distance education technologies: The fourth generation. Australian Journal of Educational Technology, 11(2), 1-7. http://www.ascilite.org.au/ajet/ajet11/taylor.html

Vanstone, A. (1996). Higher Education Statement 9 August 1996. [verified 11 Nov 2000] http://www.detya.gov.au/archive/publications/budget/budget96/budget_statement/hedst.htm

Authors: Sally Burford
Faculty of Social Sciences, Flinders University
GPO Box 2100, Adelaide 5000
Email: sally.burford@flinders.edu.au

Lesley Cooper
Faculty of Social Sciences, Flinders University
GPO Box 2100, Adelaide 5000
Email: lesley.cooper@flinders.edu.au

Please cite as: Burford, S. and Cooper, L. (2000). Online development using WebCT: A faculty managed process for quality. Australian Journal of Educational Technology, 16(3), 201-214. http://www.ascilite.org.au/ajet/ajet16/burford.html

[ AJET 16 ] [ AJET home ]
HTML Editor: Roger Atkinson [rjatkinson@bigpond.com]
This URL: http://www.ascilite.org.au/ajet/ajet16/burford.html Last revision: 29 Aug 2002.
Previous URL to 28 Apr 2002 http://cleo.murdoch.edu.au/ajet/ajet16/burford.html
Page access count