| Australasian Journal of Educational Technology 2008, 24(5), 487-504. |
AJET 24 |
Technological support for assignment assessment: A New Zealand higher education survey
John Milne, Eva Heinrich and David Morrison
Massey University
This article presents selected aspects of a large study on the use of e-learning tools in support of assignment assessment, which was comprised of a substantial literature review, an investigation into electronic tools, and interviews with 90 academics at New Zealand tertiary institutions. The article makes two main contributions. Based on the educational literature it provides an outline of good practice in assignment management and marking, demonstrating what needs to be done to best support student learning and showing how e-learning tools can assist. The second main contribution stems from the interviews that have provided insight into current practices of using e-learning tools for assignment assessment. The analysis of these interviews shows how such tools are used and what experiences the academics have had. Summarising the overall study it can be said that there are strong advantages for student learning and staff workloads in using e-learning tools in support of assignment assessment. These advantages are so far only exploited by a minority of academics and a huge potential exists for further application of e-learning tools and approaches.
To answer these questions a comprehensive literature review and interviews with 90 academics at five New Zealand tertiary institutions were undertaken. General and specialised etools suitable for supporting assignment processes were investigated. This article focuses on the assignment process, which includes assignment management and marking, showing educationally sound practice and the support of such practice through etools. The data collection method is explained first, followed by the description of good practice in assignment marking as given in the educational literature. The analysis of interview data that was guided by the steps of good practice is presented. The full study details and findings are available at the project website, http://etools.massey.ac.nz/
The participants were selected by the type of assignments they set, which are referred to as 'essay type' assignments. This term refers to course work where learners have freedom in how they construct their response and that require human attention and judgement in assessment for grades and for providing feedback. Some examples of essay type assessments are essays, design projects, spreadsheets, programming projects, or reports with free text, diagrams, or calculations.
The participants were selected because they use etools for assignments. These etools are software that help staff manage assignment submission and return, administration, assisting with the provision of individual feedback and the communication with students. Some participants used software that provides a basic level of support for these tasks. For example learning management software to manage the assignment submission and return process, with word processors to help provide feedback. Others used specialist software to provide more comprehensive support for the marking and management of assignments. These specialist tools automate some of the administrative tasks of the marking process and provide features that assist staff to give students well targeted feedback. These features include the integration of scoring rubrics with feedback and the ability to easily compare feedback across a course or across a marking team. Examples of these etools are available at the project website, http://etools.massey.ac.nz/.
Automatic marking systems were not part of this study. Software to mark restricted response questions such as multiple choice questions is not relevant to the type of essay type assignments. Software that automatically marks free form text is not part of the study as it requires a large number of essays to setup and calibrate, the quality of the feedback is variable, and they are restricted to text based student work (Dikli, 2006).
The interviews were guided by questions about the context of the assessment, the use of etools, views on advantages and disadvantages of etool use, and factors determining their use (see Figure 1 for the interview questions). Information was collected on demographics such as years of teaching, class size, level of degree, subject (based on New Zealand standard classification of education, Ministry of Education, 2007), course type (campus or distance), and institution. The interviews occurred from November 2006 to February 2007.
The interviews were recorded and transcribed. Of the 90 participants interviewed, 88 transcripts were available due to two interviewees not consenting to be recorded or not releasing the transcripts. The participants were predominately experienced lecturers. About half of the sample had taught in tertiary education for more than ten years, 40% had taught between 3 and 10 years, while less than 10% had taught less than three years. The courses discussed by the participants were an even mix of campus based and distance courses: 39% of courses were campus based, 39% distance and the remaining 22% were either both campus based and distance or the participant discussed both modes using different course examples.
| ||||||||||||||||||||||
Assignment marking needs to be seen in the wider context of an assignment process. This process includes steps like the design of the assignment tasks, submission and return of assignments, marking, providing feedback and general administration. The educational literature provides guidance for each of these steps, as for example formulated in the key principles of good assessment as defined by Nicol and Macfarlane-Dick (2006). These principles include clearly clarifying goals, delivering guidance and high quality feedback and setting up of communication channels between teacher and student. The issues analysis focused on the assignment process and the practices and etools employed by the interviewees. The prevalence of the various e-learning technologies was investigated as were the factors influencing the lecturers' choices.
To conduct the issues analysis a checklist was developed. While working through the interview transcripts the checklist was used to record the key steps in each interviewee's assignment process and their methods of dealing with specific issues. The checklist acted as a basis from which to compare actual practice to the ideal. For example, Nicol and Macfarlane-Dick recommend that lecturers allow students to discuss the assignment goals and criteria. The analysis by issues checklist quantifies the number of interviewees who actually allow students to do this. During the interview process most of the issues outlined on the checklist were either prompted for or came up in conversation. Issues were recorded as either 'yes', 'no' or 'unassigned'. The unassigned value was only given when the interviewee did not speak on or failed to clearly clarify an issue. Key quotes and insightful comments from the interviewees were also noted and kept track of.
The next section of this article introduces the steps of good practice in assignment marking that has formed the basis for the analysis by issues. This section is followed by an overview of the issues analysis results and a more in depth treatment of selected aspects.
Assignment processes and marking occurs in a wide variety of forms. This section assumes the following:
| Supporting students with assignments
Submission of assignments Preparation of marking Marking Keeping records Release results and feedback to students Using the assignment experience for future teaching |
Nicol and Macfarlane-Dick (2006) emphasised that good feedback practice involves clarifying what good performance is in terms of goals, criteria and standards. Effective assessment should help students to understand what is required of them when submitting assignments and appreciate what high quality work looks like. Furthermore, it has been suggested that assignment criteria need to be clearly linked to learning outcomes, and analytic or holistic scoring rubrics should be developed and made available to students before an assignment is due to give them a clear idea of what is expected of their assignment (Gronlund, 2006; Hanna & Dettmer, 2004; Linn & Miller 2005; Nitko, 2004).
Issue 1.2: Allow students to discuss the goals so they understand them
Students need to understand the goals and criteria for achieving them. As well as good explanations by the teacher, students need to explore and discuss the goals and criteria (Gronlund, 2006; Hanna & Dettmer, 2004; Linn & Miller 2005; and Nitko, 2004).
Issue 1.3: Make electronic communication part of the assessment
Communication tools such as wikis, blogs, photoblogs and online discussion can be assessed. Assessment will direct student activity with greater time and effort if the electronic communication is assessed (Goodfellow & Lea, 2005).
Issue 1.4: Use etools to communicate with students about the assignment
Online discussion forums allow students and tutors to exchange information and viewpoints. This can be particularly important on distance courses where face to face discussions may not be possible.
Stephens, Sargent and Brew (2001) suggest features tutors would need in an ideal computer marking and management tool. Among the items suggested is the support of all types of submission including students' electronic submission of various file types. In general assignments can be submitted by learning management systems, specialist assessment systems or email.
Issue 3.2: Use etools to organise markers
Electronic systems that assist teams mark and manage assignments are available (Baillie-de Byl, 2004; Campbell, 2005; Denton, 2003; Edwards, Fernandez, Milionis & Williamson, 2002; Moreale, Whitelock, Raw & Watt, 2002; Plimmer & Mason, 2006; Wells, 2006; Zhang & Heinrich, 2005a, 2005b). These systems vary in their approach but many allow lecturers to do the following: sharing the marking criteria, allocating assignments, allocating notes, double marking to help consistency, and preparation of markers.
Issue 3.3: Do lecturers print assignments?
It is essential that suitable etools are available that are efficient and easy to use. A complete approach must be offered that not only covers assignment submission and management but as well the actual marking process (Jones, Cranston, Behrens & Jamieson, 2005). If this is not the case staff are faced with the decision of either to print student work and mark on paper or to use unsuitable tools for the marking of the electronic assignment copies. Printing out assignments will be a time consuming activity and will reduce the benefits of using technology.
Issue 3.4: Use etools to check for plagiarism
Academic dishonesty and plagiarism occur frequently in tertiary education (de Lambert, Ellen, & Taylor, 2006; Parameswaran & Devi, 2006) and are more common in essays and programming assignments (Alam, 2004). Detection systems, along with telling students what academic dishonesty is and promoting values that institutions want, are recommended ways to manage this issue (de Lambert, Ellen, & Taylor, 2006; Macdonald & Carroll, 2006). Submitting work electronically will allow the use of detection systems. Some systems allow students to submit work to the detection system for checking so they can act on the problem areas the detection system indicates before submitting the work for marking and human feedback.
Issues 4.2-4.4:
Some software allows comments to go directly into PDF documents. This is good as the comments are placed in separate textboxes that are visible right beside their reference points in the essay, without changing the essay layout. It is not possible to inadvertently modify the student's work. Additionally, the comments can be hidden from the essay view and comment summary pages are available. The appropriateness of an etool depends on the specific marking situation and personal preferences.
Issue 4.5: Do lecturers/markers mark on paper?
To enable the benefits of technology markers should mark electronically. A very practical issue relates to the readability of handwritten comments. Students have difficulties in deciphering the handwritten comments put on their work (Blayney & Freeman, 2004; Bridge & Appleyard, 2005; Higgins, Hartley & Skelton, 2002). Typed comments are easier to read and, if looked at on screen, have the additional advantages of various display sizes and of searching and sorting.
Issue 4.6: Use etools to assist with the quality control of one's own marking
The advantages of such etools in general to assist tutors include: improved legibility (Bridge & Appleyard, 2005); staff can edit the feedback as they work through the assignments (Bridge & Appleyard, 2005); markers individually marking an assignment can check feedback and identify if it has changed as the marking proceeded (Barrett & Luca, 2002; Campbell, 2005); the etools can save tutors time in the marking and management of assignments (Baillie-de Byl, 2004; Denton, 2003).
Issue 4.7: Use etools to track the marking status of assignments
Etools such as learning management systems can handle bookkeeping and storage. Systems are available that can handle assignments with multiple files and efficient return to students.
Issue 4.8: Use etools to manage the work of markers
The instructor and their marking team need to work together closely as a team to achieve marking that is reliable and high quality. Even a simple electronic environment facilitates this teamwork and brings major advantages compared to a paper approach. Etools allow the lecturer to allocate assignments to markers, compare markers' feedback and marks, to help ensure consistency and identify markers progress. Some etools allow markers to add comments to each assignment that will not form part of the feedback given to the student. This will assist the markers in organising the marking process. A marker can write a note, indicating the status of the marking. This can be used to manage the sequence of the marking, e.g. each essay question at a time, or to convey information like 'review again' or 'check with co-marker'. If holistic marking is used, the common sorting of essays into 'piles' can be simulated via the comments. This presents a convenient way of modifying the marking sequence for multiple passes through the assignments.
Issue 4.9: Use etools to assist with moderation
Etools can help streamline moderation between markers and also help individual markers maintain consistency. For instance for double marking of assignments it is simple to electronically copy an assignment and make it available to different markers and then compare comments. Etools help marking teams to be consistent as the markers can see each others' comments.
Issue 5.2: Use etools to record or analyse marks
Etools are available to record and track assignment details. These can help lecturers with class lists, the marks and other information about the student assignments. It is often possible to link to other systems and automate some administration tasks.
Issue 5.3: Transfer marks into other systems
Easy linking to university central systems is an important feature in an ideal computer marking and management tool (Stephens, Sargent & Brew, 2001). Marking systems system will not work in isolation and should be compatible and interoperable with existing systems such as student records (Jafari, McGee & Carmean, 2006).
Issue 6.2: Return feedback and then marks
There is preliminary evidence that the mere presence of summative information on student work (such as a grade or a mark) diverts student attention away from the more detailed comments provided. There are numerous reports, from teachers and students, that students often pay little attention to specific feedback if a mark or grade is also provided. These reports are accompanied by a few tantalising pieces of research. Black and Wiliam (1998) cite the research of Butler (1988), who found little learning benefit from feedback that consisted of marks alone or marks together with written comments, but substantial learning benefit where the feedback consisted solely of written comments.
It can be suggested that feedback should be sent first and then marks later. The reverse may reduce student motivation for looking at the feedback.
Issue 6.3: Lecturers discuss the marks and feedback with students or provide opportunities for the students to meet with lecturers
Nicol and Macfarlane-Dick (2006) emphasise the importance of learner's self regulation. One of the seven principles of good feedback practice is encouraging teacher and peer dialogue around learning.
Issue 6.4: Method of returning marks and feedback
The benefits of electronic return of assignments were listed earlier and include the efficient return of marked work, legibility of comments and the ability to integrate with the package of benefits that using an electronic system offers (Plimmer & Mason, 2006; Edwards et al., 2002). Students also prefer electronic feedback (Denton, 2008).
Issue 6.5: Lecturer provides feedback to the class about performance of the class in general
Students learn from individualised feedback that outlines their strengths and weaknesses and guides them towards further learning (Linn & Miller, 2005; Nitko, 2004; Torrance & Pryor, 1998; Tynjala, Mason, & Lonka, 2001). However, general feedback can be useful. Handley and Cox (2007) identified that students preferred generalised feedback identifying what to avoid, how to think through the problem and key issues that provide a framework for thinking. Handley and Cox (2007) suggest that there is a danger that model answers could give the impression that there is one right answer which is not always the case. Students may also rote learn the model answers rather than use the feedback in more educationally effective ways. However, model answers can be useful if the student has to do something with them, such as use the model answer to mark their own or others' assignments.
Issue 6.6: Provide opportunities to close the gap identified in the feedback
This could be resubmission, or building in time to reflect on feedback, or exercises on what they will do for the next assignment. Nicol and Macfarlane-Dick (2006) emphasise the importance of learner's self regulation. Among their seven principles of good feedback practice is providing the opportunity to close the gap between current and desired performance.
Sadler (1989) argued that self assessment is a vital component in learning. He said that if students were to be able to improve, they must have the capacity to monitor the quality of their own work during its production. Feedback on assessment cannot be effective unless students accept that their work can be improved and identify important aspects of their work that they wish to improve. If students are asked and encouraged to critically examine and comment on their own work, assessment can become more dialogue than monologue, and can contribute powerfully to the educational development of students.
Orsmond et al. (2005) report that a number of students read and reread comments. The students keep their marked assignments so that they are able to refer back to the feedback provided to them by their teachers. Electronic documents can be conveniently stored and electronic copies of marked assignments can facilitate students in referring back to previous work. While this is possible in principle with standard file storage systems, a lot more could be done in assisting students to create annotated repositories of marked work.
Issue 6.7: Use etools to assist the appeals process
Providing evidence is an important part of the appeals process. Etools that store marks, feedback and the student work will help the lecturer explain to students why they got their marks and what is required to get better results.
Issue 7.2: Use the assignment experience for future teaching
The storage of feedback will help lecturers review comments. They can look for common misconceptions that can help refine the assignment for future students. McKenzie (2004) proposes to use etools to support markers by allowing the marking team access to each other's marking comments. This allows newer staff to learn from more experienced colleagues and has the additional benefit of this learning occurring in a discipline specific context.
Issue 7.3: Analyse for strengths and weaknesses across all assignments
Collecting all feedback the marker can identify strengths and weaknesses of answers across the whole class. This information can be used as a guide for further teaching (Nitko, 2004).
| Issue | Yes | No | Unass- igned | |
| 1 Supporting students with assignments | 1.1 Clarify what is good performance | 51 | 3 | 34 |
| 1.2 Allow students to discuss the goals | 56 | 2 | 30 | |
| 1.3 Make electronic communication part of assignment | 27 | 45 | 16 | |
| 1.4 Use etools to communicate about assignment | 62 | 10 | 16 | |
| 2 Submission of assignments | 2.1 see Table 1a | |||
| 3 Preparation of marking | 3.1 Prepare electronic submissions for marking | 68 | 15 | 5 |
| 3.2 Use etools to organise markers (Note 1) | 23 | 24 | 41 | |
| 3.3 Do lecturers print assignments | 35 | 53 | 0 | |
| 3.4 Use etools to check for plagiarism | 31 | 38 | 19 | |
| 4 Marking and providing feedback | 4.1 Use an electronic repository of frequently used comments | 32 | 33 | 23 |
| 4.2 Provide electronic feedback in body of the assignment | 49 | 37 | 2 | |
| 4.3 Provide feedback with general comments at end of assignment | 79 | 8 | 1 | |
| 4.4 Provide electronic feedback separate to assignment document | 36 | 42 | 10 | |
| 4.5 Do lecturers/markers mark on paper | 51 | 37 | 0 | |
| 4.6 Use etools to assist with the quality control of one's own marking | 26 | 42 | 20 | |
| 4.7 Use etools to track the marking status of assignments | 29 | 40 | 19 | |
| 4.8 Use etools to manage the work of markers (Note 1) | 21 | 26 | 41 | |
| 4.9 Use etools to assist with moderation | 13 | 29 | 46 | |
| 5 Keeping records | 5.1 Keep historic records of marked assignments | 17 | 8 | 63 |
| 5.2 Use etools to record or analyse marks | 85 | 3 | 0 | |
| 5.3 Are marks transferred into other systems | 66 | 4 | 18 | |
| 6 Releasing marks and feedback to students | 6.1 Return electronic assignment annotated with feedback | 48 | 37 | 3 |
| 6.2 Return feedback then marks | 5 | 69 | 14 | |
| 6.3 Lecturers discuss marks and feedback with students | 68 | 1 | 19 | |
| 6.4 see Table 1a | ||||
| 6.5 Lecturers provide general feedback to the class | 50 | 1 | 37 | |
| 6.6 Provide opportunity to close the gap identified in the feedback | 48 | 7 | 33 | |
| 6.7 Use etools to assist the appeals process | 6 | 10 | 72 | |
| 7 Using the assignment experience for future teaching | 7.1 Extract examples of student work for future teaching | 5 | 2 | 81 |
| 7.2 Use the assignment experience for future teaching | 12 | 2 | 74 | |
| 7.3 Analyse for strengths and weaknesses across all assignments | 13 | 0 | 75 | |
| Note 1: For issues 3.2 and 4.8. There were 47 interviews that discussed assignments that were marked by more than one person and 41 interviews where the lecturer marked alone. | ||||
| LMS | Specialised systems |
Paper | Paper and electronic | |||
| 2.1 | Method for assignment submission | 29 | 9 | 15 | 4 | 31 |
| 6.4 | Method for returning marks and feedback | 20 | 8 | 23 | 10 | 27 |
Popular ways of providing electronic feedback in the body of assignments are Microsoft Word's track changes feature and Adobe Acrobat Professional's PDF commenting etool. The use of such etools seems very much based on the individual. There was no correlation between faculty, level or class size and the use of these etools. In fact, the percentage of interviewees using these etools (about 50%) remains the same across all faculties except the arts and social sciences in the sample population. This indicates that tracking tools do have widespread application possibilities.
An interesting dichotomy exists between those using an electronic repository for assignment feedback and those who do not (Table 1, issue 4.1). A few interviewees voiced their concern about such feedback saying that it leads to homogenised comments and fails to provide 'personal' guidance. Conversely, some see efficiency, consistency and educational benefits in using electronic repositories of feedback comments:
| R71: | I think it might just ease moderating a bit, to try and be consistent and also it's quick and dirty if you will, to cut and past comments if you've got, you know, folks who are making the same error. |
| R31: | My observation is, that were you annotating and adding comments electronically, that there is probably a tendency to provide more feedback to students rather than less. |
The effectiveness of electronic repositories is linked to class size. With increasing class size the time saving benefits multiply. The use of repositories of feedback comments reflects this (Table 2).
When asked 'what do you find most challenging in providing feedback to students on assignments?' the resounding response was 'time'. 'More time' was the key motivating factor in lecturers making the switch from paper to electronic marking. Lecturers widely acknowledged the importance of feedback and accept that it takes time to do well. They did not necessarily want to reduce the total time spent on marking assignments, just the time spent on administration tasks, thus leaving more time for providing the personal 'feedback that counts'. Legibility of comments, reduction of the paper trail and departmental pressure to keep up with colleagues were secondary factors.
| Uses an electronic repository | Class size | |||
| 0 to 10 | 11 to 40 | 41 to 100 | >100 | |
| No | 3 | 18 | 5 | 4 |
| Yes | 4 | 5 | 10 | 10 |
| Unassigned | 1 | 12 | 4 | 5 |
| Total | 8 | 35 | 19 | 19 |
| Note: For this table N=81 as there were 7 interviews for which class size could not be assigned. | ||||
The primary motivation for marking on paper was unanimously the 'feel' and 'look' of paper and the ease in providing 'inline', context based comments. Of those marking on paper a significant number transferred the feedback into an electronic form by either typing in or scanning to PDF. The advantages seen by interviewees who marked electronically included improved consistency, better legibility of comments, efficiency gains and reduction of the 'paper trail'.
| R2: | If electronically submitted then we can moderate each other's work. |
Forty-seven interviewees use markers, of these twenty-one use etools to manage them, such as shared drives or open access databases (Table 3). Some incorporate flags into the etool to organise the allocation of work to markers and indicate the marking status of assignments. Once markers had returned their results the LMS, spreadsheets or databases can be used for moderation:
| R18: | I also double check any assignments that have extremely high and extremely low marks, particularly fails, so they all get checked just to make sure that that's on track and if any marker goes, particularly up or down between their first and second or third assignments then I check those as well. |
| Issue | Yes | No | Unass- igned | No markers |
| Use etools to assist with the quality control of one's own marking | 26 | 42 | 20 | |
| Use etools to track the marking status of assignments | 29 | 40 | 19 | |
| Use etools to manage the work of markers | 21 | 26 | 41 | |
| Use etools to assist with moderation | 13 | 29 | 46 |
A few interviewees mentioned that they erase, or 'clean up', all assignments at the end of each semester. Seventeen though are purposely keeping historic records of marked assignments (Table 1, issue 5.1) and saw benefits for marking and moderation from doing so:
| R72: | It's also quite handy, just occasionally I like to refer to the previous year or the one before that while marking. It enables me to get some sort of check especially when you've only got a small group, to see how they compare with stuff that was done last year. So it's very handy to have the whole thing and with my comments. |
The release of assignment results and feedback followed a pattern similar to that of submission (Table 1, issue 6.4). The electronic methods, consisting of email, LMS and specialist systems, outweigh the returns by paper. A combination of paper and electronic means was the single most favoured method for returning assignments and marks. The common procedure was to publish the marks on the LMS and make the assignment documents, annotated with detailed feedback, available either online or by physically picking up a paper copy.
If a lecturer receives an assignment through an etool this does not necessarily mean return by the same method. Twenty-nine interviewees used an LMS for assignment submission, but only twenty for returning of assignments (Table 1a). The popularity of email increased from fifteen interviewees for submission to twenty-three for return. Assignment return by paper increased compared to paper submission, as a result of lecturers printing electronically submitted assignments for marking. Return method had no relation to class size but was related to course locality and faculty (see Tables 4 and 5). More lecturers on distance courses return assignments electronically (Table 4). There were faculty differences. Assignment return via LMS was the favoured method by computer science, email return for social sciences and a combination of paper and electronic for physical sciences, art and business/management.
| Return method | Course locality | ||
| Campus | Campus and distance | Distance | |
| Electronic | 17 | 10 | 24 |
| Paper and electronic | 11 | 6 | 10 |
| Paper | 6 | 3 | 1 |
| Total | 34 | 19 | 35 |
| Return method | Faculty | |||||||
| AR | CS | CA | ED | HS | MB | SC | SS | |
| 0 | 7 | 0 | 3 | 2 | 5 | 0 | 4 | |
| LMS | 0 | 9 | 0 | 1 | 0 | 8 | 0 | 0 |
| Paper | 0 | 2 | 1 | 2 | 3 | 2 | 0 | 0 |
| Paper and electronic | 2 | 2 | 0 | 3 | 3 | 9 | 4 | 3 |
| Specialised systems | 0 | 1 | 2 | 1 | 0 | 1 | 3 | 0 |
| Total | 2 | 21 | 3 | 10 | 8 | 25 | 7 | 7 |
| Notes: AR - Arts; CS - Computer Science/IT; CA - Creative Arts; ED - Education; HS - Health Sciences; MB - Management/Business; SC - Sciences; SS - Social Sciences. For this table N = 83 as there were 5 interviews for which 'Faculty' could not be assigned. | ||||||||
| R11: | When they see their marks online, they don't see the feedback ... normally they get the grade and, in my experience, many of those students don't often read all the feedback you write. They look more at the grade. |
The issues analysis indicated that lecturers are concerned about good learning practice, quality and efficiency. Comparing survey participants' practice with Nicol and Macfarlane-Dick's 'seven principles of good feedback' (2006) showed that the majority of lecturers are fulfilling these. Interviewees are clearly clarifying assignment goals and criteria, delivering quality feedback, allowing, and often, encouraging student/teacher dialogue and providing students with the chance to close the gap.
The survey identified a range of approaches to assessment. There is a core set of etools such as email, word processors, spreadsheets and learning management systems that participants are using and receiving both efficiency and quality benefits from. In the sample the use of 'first level' technology, LMS, discussion forums, email and 'track changes' is high with over 90% utilising these etools. The extent to which 'secondary level' or more specialised etools for plagiarism, quality control, moderation, etc., are applied is limited (about 10%), even amongst this enriched sample. Despite the general prevalence of LMS submission and return of assignments, this method was not the most popular. The favoured approach was to use a combination of paper and electronic means. This reflects the fact that 'paper' assignments are still widespread. Fifty-one of the interviewees in the sample (Table 1, issue 4.5) still mark on paper to some degree. In many of these cases paper marking is a minor component, used only as a backup or in unusual situations.
The choice to use etools has more to do with the individual than with faculty, class size, course level or institution. Clearly, not everybody is comfortable reading or marking on a computer screen and many enjoy the benefits that paper provides (familiarity, mobility, ease of providing 'inline' comments). From this data set there is little evidence of departmental or institutional encouragement to switch to electronic technology. Motivation comes from the individual. There are clusters within some faculties where a 'culture of technology' exists. Often an 'early-adopting' or 'technology minded' person leads the uptake of new technology, sometimes even designing their own system. This eventually trickles down to the other staff members.
The survey identified several areas in which improvements could be made. Participants using technology often feel that there is a lack of support from their institution. This highlights the need for a support framework.
The existing technologies and practices are not perfect. Several interviewees voiced frustrations about using electronic submission and many take extra steps to prepare electronically submitted assignments for marking. In the sample there are opportunities for improvements. For example those staff who use email for submission or return spoke of difficulties, many of which could be overcome by using a LMS or a specialised marking etool.
Lecturers wishing to integrate etools into the assignment process will be faced with learning some new technologies, acquiring some new skills, and slightly altering their current practices. However, the learning curve is not too steep and in the sample set, once the etools had been adopted, lecturers definitely did not want to revert back to a traditional paper based approach.
Baillie-de Byl, P. (2004). An online assistant for remote, distributed critiquing of electronically submitted assessment. Educational Technology & Society, 7(1), 29-41. http://www.ifets.info/others/download_pdf.php?j_id=3&a_id=27
Barrett, C. & Luca, J. (2002). Open online assessment: Keeping the tutors honest! In Winds of change in the sea of learning. Proceedings ASCILITE 2002, 783-786. http://www.ascilite.org.au/conferences/auckland02/proceedings/papers/104.pdf
Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5, 7-74.
Blayney, P. & Freeman, M. (2004). Automated formative feedback and summative assessment using individualised spreadsheet assignments. Australasian Journal of Educational Technology, 20(2), 209-231. http://www.ascilite.org.au/ajet/ajet20/blayney.html
Bridge, P. & Appleyard, R. (2005). System failure: A comparison of electronic and paper-based assignment submission, marking, and feedback. British Journal of Educational Technology, 36(4), 669-671.
Bridge, P. & Appleyard, R. (2008) A comparison of electronic and paper-based assignment submission and feedback. British Journal of Educational Technology, 39(4), 644-650.
Campbell, A. (2005). Application of ICT and rubrics to the assessment process where professional judgement is involved: The features of an e-marking tool. Assessment & Evaluation in Higher Education, 30(5), 529-537.
de Lambert, K., Ellen, N. & Taylor, L. (2006). Chalkface challenges: A study of academic dishonesty amongst students in New Zealand tertiary institutions. Assessment & Evaluation in Higher Education, 31(5), 485-503.
Denton, P. (2003). Evaluation of the 'Electronic Feedback' marking assistant and analysis of a novel collusion detection facility. Proceedings 7th International Computer Assisted Assessment Conference, Loughborough, U.K. http://www.caaconference.com/pastconferences/2003/procedings/denton.pdf
Denton, P., Madden, J., Roberts, M. & Rowe, P. (2008). Students' response to traditional and computer-assisted formative feedback: A comparative case study. British Journal of Educational Technology, 39(3), 486-500.
Dikli, S. (2006). An overview of automated scoring of essays. Journal of Technology, Learning, and Assessment, 5(1). [verified 13 Sep 2008] http://escholarship.bc.edu/jtla/vol5/1/
Edwards, K. I., Fernandez, E., Milionis, T. M. & Williamson, D. M. (2002). EAST: Developing an electronic assessment and storage tool. Assessment & Evaluation in Higher Education, 27(1), 95-104.
Goodfellow, R. & Lea, M. (2005). Supporting writing for assessment in online learning. Assessment & Evaluation in Higher Education, 30(3), 261-271.
Gronlund, N. E. (2006). Assessment of student achievement. Boston: Pearson.
Handley, K. & Cox, B. (2007). Beyond model answers: Learners' perceptions of self-assessment materials in e-learning applications. ALT-J: Research in Learning Technology, 15(1), 21-36.
Hanna, G. S. & Dettmer, P. A. (2004). Assessment for effective teaching using context-adaptive planning. New York: Pearson.
Hattie, J. (1999). Influences on student learning, Inaugural professorial lecture. University of Auckland, New Zealand. [viewed 9 Sep 2008]
http://www.education.auckland.ac.nz/uoa/fms/default/education/staff/Prof.%20
John%20Hattie/Documents/Presentations/influences/Influences_on_student_learning.pdf
Higgins, R., Hartley, P. & Skelton, A. (2002). The conscientious consumer: Reconsidering the role of assessment feedback in student learning. Studies in Higher Education, 27(1), 53-64.
Jafari, A., McGee, P. & Carmean, C. (2006). Managing courses, defining learning: What faculty, students, and administrators want. Educause Review, July/August, 50-70. http://net.educause.edu/ir/library/pdf/ERM0643.pdf
Jones, D., Cranston, M., Behrens, S. & Jamieson, K. (2005). What makes ICT implementation successful: A case study of online assignment submission. In Breaking Down Boundaries. Proceedings ODLAA 2005. University of South Australia, 9-11 November. [verified 14 Sep 2008] http://www.odlaa.org/events/2005conf/ref/ODLAA2005Jones-Cranston-Behrens-Jamieson.pdf
Linn, R. L. & Miller, M. D. (2005). Measurement and assessment in teaching. Columbus: Pearson Merrill Prentice Hall.
Macdonald, R. & Carroll, J. (2006). Plagiarism - a complex issue requiring a holistic institutional approach. Assessment & Evaluation in Higher Education, 31(2), 233-245.
Ministry of Education (2007). New Zealand Standard Classification of Education (NZSCED). [viewed 13 Feb 2007, verified 14 Sep 2008]
http://www.educationcounts.govt.nz/technical_info/code_sets/
new_zealand_standard_classification_of_education_nzsced
McKenzie, S. (2004). Assessing quality of feedback in online marking databases: An opportunity for academic professional development or just Big Brother? In Beyond the comfort zone. Proceedings ASCILITE 2004, Perth. http://www.ascilite.org.au/conferences/perth04/procs/mckenzie.html
Moreale, E., Whitelock, D., Raw, Y. & Watt, S. (2002). What measures do we need to build an electronic monitoring tool for postgraduate tutor marked assignments? Proceedings 6th International Computer Assisted Assessment Conference, Loughborough, U.K. [viewed 9 Sep 2008] http://hdl.handle.net/2134/1885
Nicol, D. J. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218.
Nitko, A. J. (2004). Educational assessment outcomes. Columbus: Pearson Merrill Prentice Hall.
Orsmond, P., Merry, S. & Reiling, K. (2005). Biology student's utilization of tutor's formative feedback: A qualitative interview study. Assessment & Evaluation in Higher Education, 30(4), 369-386.
Parameswaran, A. & Devi, P. (2006). Student plagiarism and faculty responsibility in undergraduate engineering labs. Higher Education Research and Development, 25(3), 263-276.
Plimmer, B. & Mason, P. (2006). A pen-based paperless environment for annotating and marking student assignments. Proceedings Seventh Australasian User Interface Conference (AUIC2006), Hobart, Australia. [viewed 9 Sep 2008] http://portal.acm.org/citation.cfm?id=1151762
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144.
Stephens, D., Sargent, G. & Brew, I. (2001). Comparison of assessed work marking software: Implications for the ideal Integrated Marking Tool (IMT). Proceedings 5th International Computer Assisted Assessment Conference, Loughborough, U.K. [viewed 9 Sep 2008] http://hdl.handle.net/2134/1835
Torrance, H. & Pryor, J. (1998). Investigating formative assessment: Teaching, learning and assessment in the classroom. Buckingham: Open University Press.
Tynjala, P., Mason, L. & Lonka, K. (2001). Writing as a learning tool: An introduction. In P. Tynjala, L. Mason & K. Lonka (Eds), Writing as a learning tool: Integrating theory and practice (Vol. 7, pp. 7-22). Dordrecht, Boston, London: Kluwer Academic Publishers.
Wells, J. (2006). Markers Assistant - a software solution for the management of the assessment process. International Journal on E-Learning, 5(3), 439(420).
Yorke, M. (2001). Formative assessment and its relevance to retention. Higher Education Research and Development, 20(2), 115-126.
Zhang, J. & Heinrich, E. (2005a). A system designed to support formative assessment of open-ended written assignments. 5th International Conference on Advanced Learning Technologies (ICALT), 88-92.
Zhang, J. & Heinrich, E. (2005b). Using computers to support formative assessment of assignments. ED-Media 2005 World Conference on Educational Multimedia, Hypermedia & Telecommunications, 4510-4515.