|[ ASCILITE ]
[ 2004 Proceedings Contents ] |
Metacognition is widely considered integral to effective learning. However environments that support metacognition can be difficult to develop. This paper proposes an approach to assessment through student contracts that are designed to both address issues of fairness of assessment as well as promote the planning, monitoring and evaluation integral to enhancing metacognition. By negotiating their assessment in ways that involve feedback that is internal, parallel and external to students, they have the opportunity to develop clearer understandings of themselves as learners and their own learning processes. The model is supported with online technology to help create an easy and confidential manner in which peer feedback can be collated. Initial findings suggest that students perceive themselves to be metacognitive, however, they may not be fully engaged in the processes that underpin this psychological state. It is proposed that online environments that support negotiated assessment expose these processes in ways that may enhance metacognitive outcomes, and lead to further research that identifies the nature of such processes and their value in the assessment approach.
Weinstein & Mayer (1986) see all metacognitive activities as partly the monitoring of comprehension, and it would appear that this ability to monitor oneself is what distinguishes metacognitive activity from domain specific cognition. Wilson (1999) argues that the term metacognition can be used in 'vague, confusing, and often contradictory' ways and can be used to describe a range of disparate higher level cognitive skills. She goes on, however, to define metacognition as "awareness individuals have of their thinking and their evaluation and regulation of their thinking". In this definition it is both a state and a process, with 3 functions:
This apparently contradictory position of being a both a state and a process can be reconciled by acknowledging the dependence of metaknowledge upon domain dependent cognitive processes. In their research on metamemory, Nelson and Narens (1994) identify the relationship between the meta-level and the object level of cognition through a reciprocal flow of control and monitoring (Figure 1). While the model itself is perhaps a little simplistic, it does give some hope to those floundering in the problem of how to encourage metacognition in students. In Nelson and Naren's concept, one can view metacognition as a pet puppy - in order to grow and become strong it must be fed and exercised. The process of monitoring nurtures metacognition, likewise the activation of control processes exercises it.
Figure 1: A model of metacognition
It can be argued, then, that metacognition is not something that can be tackled as a discrete entity - just as well, since it is quite inaccessible as such - but can be enhanced through engaging at the subordinate levels of monitoring and control of thinking processes.
Blakey and Spence (1990) cite Dirkes' synthesis of much of the literature on metacognition into the following features:
One of the ways of promoting metacognition is through assessment. Haefner (2004) describes an approach to assessment that engages planning monitoring and evaluation, through three different mechanisms of assessment feedback. These engage students in setting goals, evaluating their performance and monitoring their understandings through techniques that are internal, such as self assessment, parallel such as through peer collaboration, and external, such as tutor feedback.
This study builds on this approach by engaging these forms of feedback in a formative way, where the criteria for students judging the value of their work is negotiated over a semester. The study is based around a final year undergraduate unit in Project Management Methodology for Interactive Multimedia development. As with most final year courses (both graduate and undergraduate), teamwork is often needed to complete developmental projects that illustrate the students' technical/content skills learnt throughout the course of their study, as well as professional skills needed by industry practitioners. However, students often complain about team based assessment, as they perceive that if they work harder than others, the whole team is still given the same mark. A learning environment was developed to help promote fair and equitable teamwork, while at the same time integrating the planning, monitoring, and evaluation inherent in metacognitive development through internal, external and parallel feedback.
Students are encouraged to select their teams and allocate their own tasks based on their skill strengths. Teams are required to create a project proposal, design specification, metrics, evaluation report, post mortem and a web site. Decisions need to made on:
Figure 2: Learning environment design
Within this learning environment, students use the "Team Contract" (Table 1) to negotiate their assessment items. Each row in the Team Contract represents a key assessment point that students can consider how heavily they want to contribute to, based on their aims for future employment and current skill sets.
If there are four students in a team, then each student should contribute 25% of the overall marks. However, this is not mandatory, and students can specify how much of the "assessment pie" they want! This team based negotiated assessment process is completed in two separate stages. Students are to consider:
Beyond the issues of fairness and equitability of marks distribution, however, this negotiation also involves students in planning their learning, by setting goals and estimating their performance both in terms of outcome (mark) and process (quality of work), they are then required to evaluate these goals against actual achievement. Through a cyclical process and through the internal, parallel and external feedback mechanisms of peer, tutor, and self assessment, students are engaged in a continuous process of self monitoring.
|%||Name 1||Name 2||Name 3, etc|
|EM EQ AM AQ||EM EQ AM AQ||EM EQ AM AQ|
|PM Doc 1||2|
|Presentation & Online CV||2|
|PM Doc 2||2|
|We agree that the assigned assessment values & quality are acceptable (Tutor ____________________)|
|EM = Estimated Mark, EQ = Estimated Quality, AM = Actual Mark, AQ = Actual Quality|
The implementation of the team contract is based on planning, monitoring and evaluation (Dirkes, 1985). By week 3 students must plan and negotiate with their team members which assessment items they are responsible for ("EM" in Table 1), as well as predicting the quality of these ("EQ" in Table 1). As the semester progresses, students are required to continually monitor their own performance in terms of their stated plans as well as their team members' commitments (as agreed to in the contract). If they feel the team isn't progressing as agreed, they can inform the tutor through the peer assessment tool. As well as ongoing modification of their initial plans, students formally evaluate their performance and that of their peers when the assessment item is submitted. They do this by entering the actual mark and quality of their contributions into the team contract ("AM" and "AQ" in Table 1).
|1.||I know how I learn best||0||0||3||10||3||General|
|2.||I know what I am good at as well as the things I have difficulty with||0||0||1||10||5||General|
|3.||It is useful for me to think about my studies before I go to class||0||0||2||8||6||Planning||3|
|4.||I find it useful to set myself goals for learning||0||0||3||11||2||Planning||3|
|5.||I understand concepts better when I imagine them in practice||0||0||1||5||10||Monitoring||4|
|6.||It is important for me to find ways of applying what I am |
studying to real settings
|7.||Making notes helps me understand what I am studying||0||1||4||9||2||Monitoring||4|
|8.||I find it helpful to compare my ideas with other students to |
make sure I'm on the right track
|9.||I compare what I've achieved in learning to what I planned to achieve||0||2||4||8||2||Evaluation||5|
|10.||Using the student contract will help me plan my learning||0||0||6||10||0||Value of student|
|11.||Using the student contract will help me monitor my performance||0||0||3||12||1||Value of student|
|12.||Using the student contract will help me evaluate my performance||0||3||12||1||Value of student|
|13.||Using the student contract will help me understand myself |
as a learner better
|0||2||4||10||0||Value of student|
|SD = Strongly Disagree, D = Disagree, N = Neutral, A = Agree, SA = Strongly Agree|
As can be seen from Table 2, students on the whole perceived themselves to be operating at a reasonably high metacognitive level. Figures 3 to 7 show the extent to which they support each concept of overall metacognition, planning, monitoring, evaluation, and the value of the student contract as a tool to enhance their monitoring processes.
Figure 3: Perceived overall metacognition (Q 1-2)
Figure 4: Perceived planning (Q 3-4)
Figure 5: Perceived monitoring (Q 5-7)
Figure 6: Perceived evaluation (Q 8-9)
The positive overall response suggests that most students perceive themselves to be operating at a high level with regard to their ability to plan, monitor and evaluate their learning processes. The higher reported response to monitoring indicates that most students are comfortable with engaging in strategies that involve monitoring their understandings although as Table 2 (question 7) demonstrates, one student
Figure 7: Perceived value of student contracts (Q 10-13)
disagreed with the concept of note taking as a useful approach. It was also intriguing that two students disagreed with the statement regarding the evaluation of goals (question 9). While comparing what one has achieved with what one has planned to achieve is integral to the process of metacognitive monitoring it would appear that for some students at least, it is not highly valued.
The student contracts, as with the concepts inherent in metacognition also received support overall. Again, some disagreement was received, in particular to the statement regarding the value of the tool as a means of making students better learners (question 13). It would appear that while all subjects can see how the strategy will engage them in planning, monitoring and evaluation, the generalisation of these processes to a general state of metacognitive awareness is not considered an integral outcome of negotiated assessment for all of the students.
Initial findings suggest that students perceive themselves to be operating at a reasonably high metacognitive level. They can also see the value of a tool for negotiated assessment in promoting self monitoring. Nevertheless, the accuracy of student perceptions must be questioned, when at least two students fail to see the value of evaluating performance against goals; an activity integral to the planning, monitoring and evaluation feedback loop that is self monitoring. This is further exacerbated by a tendency for a couple of students to fail to draw the link between the student contracts and the processes that underpin them, with a general concept of metacognition involving enhanced self awareness in the learning process.
Further study will be required to explore the actual patters of self monitoring in which students engage. This may involve analysis of the discourse involved in negotiating assessment to identify the actual nature of self monitoring compared to the students' self perceptions. While this represents the beginning of a project that is necessarily iterative and exploratory over a period of time, initial findings suggest that negotiated assessment through an on line student contract system has the potential to provide a means to expose the processes of assessment that will ensure self monitoring on the part of students, that should lead to outcomes that are fair, representative of the work that students engage in, and above all, lead to the development of students' awareness of the processes that underpin their own learning.
Boekaerts, M. (1997). Self-regulated learning: A new concept embraced by researchers, policy makers, educators, teachers, and students. Learning and Instruction, 7(2), 161-186.
Collis, B. (1997). Supporting project-based collaborative learning via a World Wide Web environment. In B. H. Khan (Ed.), Web-Based Instruction (pp. 213-219). New Jersey: Educational Technology Publications.
English, S. & Yazdani, M. (1999). Computer-supported cooperative learning in a virtual university. Journal of Computer Assisted Learning, 15(2), 2-13.
Haefner, T. (2004). Assessment as a magnification of internal, parallel, and external reflection. Action in Teacher Education, 25(4), 14-19.
Nelson, T. O. & Narens, L. (1994). The role of metacognition in problem solving. In J. Metcalfe & A. Shiminura (Eds.), Metacognition (pp. 207-226). Cambridge: MIT Press.
Jacobson, R. (1998). Teachers improving learning using metacognition with self monitoring learning strategies. Education, 118(4), 579-589.
Klemm, W. R. & Snell, J. R. (1996). Enriching computer-mediated group learning by coupling constructivism with collaborative learning. Journal of Instructional Science and Technology, 1(2). [verified 30 Sep 2004] http://www.usq.edu.au/electpub/e-jist/docs/old/vol1no2/article1.htm
Pressley, M., Van Etten, S., Yokoi, L., Freebern, G. & Van Meter, P. (1998). The metacognition of student scholarship: A grounded theory approach. In D. J. Hacker, J. Dunlosky & A. C. Graesser (Eds), Metacognition in Educational Theory & Practice (pp. 347-366). New Jersey: Lawrence Earlbaum Associates Inc.
Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26, 113-125.
Weinstein, C. E. & Mayer, R. (1986). The teaching of learning strategies. In M. Wittrock (Ed), Handbook of research on teaching (pp. 315-327). New York: Macmillan.
Wilson, J. (1999). Defining metacognition: A step towards recognising metacognition as a worthwhile part of the curriculum. Proceedings AARE Conference, Melbourne. [verified 30 Sep 2004] http://www.aare.edu.au/99pap/wil99527.htm
|Authors: Joe Luca, School of Communications and Multimedia. Edith Cowan University. firstname.lastname@example.org|
Mark McMahon, School of Communications and Multimedia. Edith Cowan University. email@example.com
Please cite as: Luca, J. & McMahon, M. (2004). Promoting metacognition through negotiated assessment. In R. Atkinson, C. McBeath, D. Jonas-Dwyer & R. Phillips (Eds), Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference (pp. 562-570). Perth, 5-8 December. http://www.ascilite.org.au/conferences/perth04/procs/luca.html
© 2004 Joe Luca & Mark McMahon
The authors assign to ASCILITE and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to ASCILITE to publish this document on the ASCILITE web site (including any mirror or archival sites that may be developed) and in printed form within the ASCILITE 2004 Conference Proceedings. Any other usage is prohibited without the express permission of the authors.