Home

Newsletter - July 2003

Welcome to the mid-year edition of the ascilite newsletter. The theme for this edition is online assessment and included throughout are many links to previous papers and published works on this theme by ascilite members and other researchers.

The ascilite newsletter is produced three times a year and we welcome suggestions for themes and lead articles from our readers. Please send these to any of the editors.

Editor for this edition: Meg O'Reilly
Editorial team: Cathy Gunn, Gerry Lefoe, Meg O'Reilly, Russ Pennell
Web Editor: NetSpot

Online Assessment
What's interesting on the ascilite website?
What's interesting elsewhere?
Regular Links



Online Assessment

Meg O'Reilly

As networked computer technologies become more integrated into teaching and learning in higher education, questions of supporting online assessment are becoming of greater interest to both students and staff. Are the principles of assessment affected when such activities take place online? What are the advantages and challenges of assessment online? How can the technology help or hinder the underlying pedagogy? What training or professional development do students and staff need to be able to successfully engage with assessment online?

Principles of good assessment

Firstly, it is clear from the literature that the following principles are fundamental to good assessment regardless of the medium for delivery:

  • validity (does it measure what was intended?),
  • reliability (can it repeatedly provide the same outcomes?),
  • flexibility (are the diversity of student needs being taken into account?)
  • fairness of assessment (is the assessment applicable to all enrolled students?)
    (Juchnowski && Atkins, 1999; Kerka, Wonacott, Grossman, && Wagner, 2000)

Furthermore, alignment of assessment tasks with the learning objectives and the learning activities is a long-standing sign of good assessment practice (Biggs, 1999) that most certainly applies in the online context. Learners will always benefit from clear guidelines on what is being required and explicit criteria on how their performance will be assessed.

Advantages and challenges of online assessment

Some of the advantages of utilising the online environment in the assessment process include the capacity to support greater student choice in terms of time and place of assessment, enhanced responsiveness of feedback especially in auto-graded tasks, and efficiency in managing and distributing the results. Both students and staff also value the immediacy of online announcements of the availability of a test and the release of test results. The benefits of such an automated management system are particularly appreciated by teachers of large classes who find they can readily establish tutorial groups online for discussions, group work and presentations, and manage both graded and ungraded activities with the use of live chat and all the discussion archives.

If considering using automated quizzes and computer based exams, there are many Web resources available to assist in the design and development of such online assessments and tests (University of Leicester, 2002). If choosing commercially available tools and tests, take time to consider the quality of the questions, the structure and nature of the feedback and the ownership of copyright and intellectual property especially when tailoring a test based on questions from publishers or commercial providers ((Freewood, 2001). The technical requirements of implementing online exams and tests remain a challenge to both the institutional programming staff and those who oversee the exam process to ensure rigour and security (James, McInnes, && Devlin, 2002). Be sure that the technical environment in which you plan to run your exams is robust and reliable.

In addition to the auto-graded options already mentioned where the course management system provides pre-prepared answers as the student proceeds through their quiz or online exam, if the academic wishes to transform the task of marking to the online context, their practices will need to change to screen based annotations and commentary. In auto-graded or semi-automated systems, feedback can also be given to students in terms of model answers with which they compare their own responses, and hints or pointers to the relevant sections of content which the studnet would need to revise in order to perform better on the exam or quiz in future. Once the quiz has been developed, these automated methods can significantly reduce staff workload in terms of marking and tailoring feedback to each students.

In a broader sense, supporting the processes of assessment in the online environment can take account of the particular affordances of the medium. Formative assessment is readily facilitated online, where timely (indeed instant) feedback allows for students to both practice their responses and amend or improve their performance before receiving a grade or mark. Continuous forms of assessment as found in problem-based approaches are supported by the immediacy of online communications. Mutual support and an atmosphere of collaborative learning can be encouraged among students in preparation for vocational or professional practice, and these assessment strategies often need to incorporate the involvement of industry-based specialists in the online interaction between student and assessor.

Technology - helping or hindering pedagogy?

Academic staff who embrace the practices of online assessment are clearly able to have technology serve their needs. If the technology were to interfere with your pedagogical approaches, both your students and markers would be quick to identify the need for improvement. Clearly using technology to assist in online assessment needs to be seen as relevant to the process itself.

One of the most commonly adopted enhancements that technology can provide is where assessment enables distributed classes of learners to engage in text-based discussion. We know from recent research that both part-time and full-time students in Australian higher education and most likely to also be working (Long && Hayden, 2001). As learners with busy lives, they might participate in online activities as time permits within the prescribed period. The flexibility this offers can meet the requirements of both independent and collaborative learning tasks, and an assessment of the quality and nature of participation ensures that a breadth of learning objectives can be achieved.

While more comparable to classroom activities, assessment online that is based on student interaction can now be designed for inclusion in off-campus courses. The online environment is therefore often claimed as able to support a constructivist approach to teaching, learning and assessment. Some guidelines that can be given to online facilitators of discussion might include:

Marks are given for responses containing one or more of the following:

  1. offering up ideas or resources and inviting critique of them
  2. asking challenging questions
  3. articulating, explaining and supporting positions on ideas
  4. exploring and supporting issues by adding explanations and examples
  5. reflecting on and re-evaluating personal opinions
  6. offering a critique, challenging, discussing and expanding ideas of others
  7. negotiating interpretations, definitions and meanings
  8. summarising previous contributions and asking the next question
  9. proposing actions based on ideas that have been developed." (Salmon, G, 2000, p.143)

In addition to collaboration, communication and leadership, the specific skills for working in the online environment now also include Web design, and publishing - often best assessed through evidence of practice. Online technology is ideal for enabling these activities and enriches the authentic experience of the student.

Up-skilling staff and students for online assessment

The role of the academic staff in adopting these new approaches to assessment online includes undertaking professional development as well as preparing students for working productively online. Such preparation for assessment online needs to address the following areas:

  • Plagiarism - what is it, how to avoid it and what are the consequences when it has been detected
  • Authentication - how to design engaging assessment tasks that defy cheating by remote users
  • Security - finding strategies for keeping online tests and results safe and secure, this includes archiving discussions and all work submitted electronically, and reminding students to keep copies of their own work
  • Equity - designing assessment tasks that are accessible and manageable by all students, or providing options
  • Network reliability and robustness - designing within limits of current infrastructure, encouraging students to pay for private ISP accounts
  • Format of tests - ensuring the online format of assessment matches the intended learning outcomes in both level of demand and hierarchy of expected achievement
  • Students' technology skills - providing time for familiarisation with the online environment and necessary competencies prior to assessments.

Case studies and more...

Collaborative group work:
http://www.medfac.unimelb.edu.au/Ascilite2001/pdf/papers/baskinc.pdf

Discussion:
http://www.atimod.com/presentations/
http://www.emoderators.com/moderators.shtml

Online exams:
http://www.scu.edu.au/schools/sawd/moconf/mocpapers/moc33.pdf

Self and peer assessment:
http://www.aset.org.au/confs/aset-herdsa2000/procs/freeman.html
http://www.medfac.unimelb.edu.au/Ascilite2001/pdf/papers/peatm1.pdf
http://cea.curtin.edu.au/tlf/tlf2002/crockett.html

General articles on online assessment practices: http://www.ascilite.org.au/conferences/coffs00/papers/amanda_kendle.pdf
http://www.aset.org.au/confs/aset-herdsa2000/procs/mcloughlin1.html
http://cea.curtin.edu.au/tlf/tlf2001/radloff.html
http://cea.curtin.edu.au/tlf/tlf2000/taylor.html

References

Biggs, J. (1999). What the Student Does: teaching for enhanced learning. Higher Education Research && Development, 18(1), 57-75.
Freewood, M. (2001). Computer Assisted Assessment: Practical Guidance. Retrieved 23 June 2003, from http://www.shu.ac.uk/services/lti/resources/caapracticalguidance/qbcommerciallyavailable.htm
James, R., McInnes, C., && Devlin, M. (2002). Assessing Learning in Australian Universities. Retrieved 29 November 2002, from http://www.cshe.unimelb.edu.au/assessinglearning/04/index.html
Juchnowski, M., && Atkins, P. (1999). Online assessment: Let's do it. Retrieved 23 June 2003, from http://learnwebct.vetonline.vic.edu.au/letsdoit/2002/
Kerka, S., Wonacott, M., Grossman, && Wagner, J. (2000). Assessing Learners Online. Retrieved 5 June 2003, from http://ericacve.org/docs/pfile03.htm#principles
Long, M., && Hayden, M. (2001). Paying their way: A Survey of Australian Undergraduate Student Finances, 2000: Australian Vice-Chancellor's Committee.
Salmon, G (2000). E-moderating: the key to teaching and learning online London: Kogan Page University of Leicester. (2002). The CASTLE Toolkit. Retrieved 23 June 2003, from http://www.le.ac.uk/castle/index.html

What's interesting on the ascilite website?

ascilite campus reps have been wondering what they are meant to be doing... please have a look through the new information on the ascilite website and find the latest list of campus reps and some suggestions about the role.

Campus Reps.

What's interesting elsewhere?

Perhaps you are not too late to take off for the 7th International Computer Assisted Assessment (CAA) Conference, to be held at Burleigh Court International Conference Centre, Loughborough University, UK 8-9 July. http://caa2003.lboro.ac.uk/caaconference/f1.asp

For an Australian conference on assessment (with more lead time) have a look at the Evaluations and Assessment conference to be held on 24-25th November 2003 at the Hilton Hotel, Adelaide. http://www.unisa.edu.au/evaluations/


Regular Links

EdNA

EdNA What's New
A comprehensive Australian online educational resource network. What's New service provides material which has recently been added into EdNA Online. Australia

To subscribe: http://www.edna.edu.au/edna/page1469.html
IRN Internet Resources Newsletter A-Z collection of the latest Internet Resources for academics, students, engineers, scientists and social scientists. Heriot-Watt University, UK


To subscribe: http://www.hw.ac.uk/libWWW/irn/irn.html
INFOBITS Each month the Information Resources Consultant selects from a number of information technology and instructional technology sources and provides brief notes on key topics of interest. University of North Carolina USA

To subscribe:
send email to listserv@unc.edu with the following message:
SUBSCRIBE INFOBITS firstname lastname
SCOUT Report The Internet Scout Project provides information about the Internet to the U.S. research and education community. University of Wisconsin-Madison, USA

To subscribe:
send email to listserv@cs.wisc.edu In the body of the message type:
subscribe SCOUT-REPORT
ALT: Association of Learning Technology ALT is an educational organisation which seeks to bring together all those with an interest in the use of learning technology in higher and further education. UK

ALT is affiliated with ascilite and we have a reciprocal arrangement for conference attendance at the membership rates.
DEST's Higher Education Division The Department of Education, Science and Training provides a free electronic newsletter on issues and developments in Australia's higher education sector. Australia

To subscribe: http://www.dest.gov.au/highered/enews/subscribeenews.htm