| Australasian Journal of Educational Technology 2004, 20(3), 295-315. |
AJET 20 |
This paper reports on the creation of an instrument for use by instructors, students, or researchers to identify, measure or promote critical thinking (CT) in online asynchronous discussions (OADs). Four models of CT were reviewed, synthesised, analysed, and evaluated to clarify the construct. Indicators of specific cognitive processes related to CT were identified, and subsequently retained, rejected or modified to show how the construct might be operationalised in real contexts of use. Subsequent empirical testing of the instrument for the analysis of a transcript of an OAD revealed that while the instrument was valuable in identifying and measuring CT in the OAD, issues of practicality need to be addressed. Implications for research and practice are presented.
Although asynchronous conferencing might afford or support opportunities for engagement in various cognitive processes such as critical thinking, it does not guarantee it. Engagement derives, not from a context of use of a medium of communication, but instead, as a result of numerous and varied factors and conditions, such as the instructional design of the online asynchronous discussion (OAD), the requirements set by the moderator of the discussion, the character of interactions between discussants and, as well, the issue or topic under consideration. Designers, instructors, students and researchers can be supported in their attempts to promote, identify and measure engagement in CT through instruments for the analysis of transcripts of OADs.
A number of models and theoretical perspectives have been proposed in the literature on CT. Newman, Webb and Cochrane (1995) developed an instrument to measure CT in face to face and online discussions. However, their presentation of 46 indicators is cumbersome, and has been criticised for ambiguity and lack of mutual exclusivity between indicators (Bullen, 1998). In Garrison, Anderson and Archer's (2001) development of a tool "to assess the nature and quality of critical discourse" in text based learning (p. 1), the focus is on the interactive nature of critical thinking through the authors' practical inquiry model. While this study is based on a sound conceptual framework, no usable instrument is offered to instructors or students for the assessment and support of CT in practice. In later work, Fahy (2002) used Garrison et al's (2001) practical inquiry model, and Zhu's (1996) Transcript Analysis Tool (TAT). He asserted that Garrison et al's research could be improved on by adopting the sentence as the unit of analysis, as well as the integration of two distinct approaches to analysis of CT in OADs. While Fahy revealed that the two methods of analysis are complementary, analysis using two separate methods is time consuming and impractical for application in educational contexts. What is needed are instruments based on solid conceptual frameworks to ensure validity of the CT construct. These constructs must be usable by designers, instructors, students, and researchers.
The purpose of the study reported on in this paper was to create an instrument that could be used by designers, instructors, students or researchers to identify, measure, or promote CT in the context of an OAD. The primary focus was to derive an instrument that was solidly grounded in the literature on CT in order to ensure validity of the construct. The paper begins with a review of four models of CT. The models were subsequently synthesised in table format to highlight their similarities and differences in their perspectives on CT. A subsequent analysis and evaluation of the models supported identification of key processes and indicators associated with CT. These were then used to design an instrument that can be used for the identification, measurement, or promotion of CT in an OAD. The instrument was then applied to the analysis of an OAD. Results of the application are discussed in relation to the value and practicality of the instrument. Implications for practice and research are presented.
Our starting point for the development of the instrument was a review of existing models of critical thinking. A number of such models have been derived by researchers, including Fahy (2002), Garrison, Anderson, and Archer (2001), Aviv (2001), Bullen (1998, 1997), Newman, Johnson, Webb and Cochrane (1997), Zhu (1996), Henri (1992), Norris and Ennis (1989), and Brookfield (1987). Models chosen for this study include those from Bullen (1998) and Garrison, Anderson and Archer (2001), for their specificity to computer conferencing, while the remaining two models chosen are those from Brookfield (1987), and Norris and Ennis (1989), as they focus on thinking processes in a theoretical context of teaching and learning.
Brookfield's perspective of critical thinking was chosen for its theoretical bases, and its consideration of broad sociological contexts. Bullen's (1998) model draws heavily on the seminal work of Norris and Ennis (1989), and was included for its application of the theoretical model in the context of computer conferencing. Garrison, Anderson and Archer (2001), who draw on their own earlier work (2000) as well as Dewey's (1933) notion of practical inquiry, were chosen as a more recent example, as well as for their prominence in the literature (see, for example, Fahy, 2002; Newman, Webb and Cochrane, 1995 for applications of Garrison et al's model).
Seeking resolution to conflict or uneasiness in one's life, individuals may engage in the next phase of Exploration. According to Brookfield, "during this phase we test out new ways of thinking and acting that seem more congruent with our perceptions of what is happening in our lives" (p. 26). A process of Developing Alternative Perspectives follows exploration, and involves a shift in thinking that allows an individual a sense of comfort, or resolution to the trigger event (whether positive or negative). This may include a complete transition or a modification of old views or behaviours.
Integration results from decisions about alternative perspectives and their value to one's present situation. Brookfield notes Integration may be a particular action that includes visible efforts to change our behaviours in regard to others, our own actions, or in the workplace. Integration may also involve internal reformation involving a change or shift in the way we think about our own thoughts or actions.
Inference is the middle phase, or ability outlined, and requires facilities for making and judging inferences. This includes making and judging deductions through the use of logic and interpretation, making and judging inductions through generalisations, explanations, hypotheses, and investigation, and making and judging value judgements, an ability which requires thinking about the relevance of background information, alternative hypotheses, and consideration of the consequences of a decision.
Advanced Clarification, the penultimate phase or ability in this model, necessitates certain skills as described by the authors: "Advanced clarification abilities are used to provide and evaluate definitions of terms and in identifying assumptions that are left implicit in lines of reasoning" (p. 11). The fifth and ultimate ability concerns the employment of Strategies and Tactics, a process that includes interaction with others, the ability to clearly define the problem, judge solutions, generate alternative solutions, and engage in other cognitive and metacognitive activities, and deciding on an appropriate action.
The second skill of Assessing Evidence involves making decisions about the credibility of sources and observations. Judging credibility requires the application of appropriate criteria such as the reputation of the source, congruence with other sources, and the general ability to correctly assess the source before making one's own decisions or assertions. Making observations, and judging those observations is integral in the demonstration of the skill Assessing Evidence.
The third, and penultimate skill of Making and Judging Inferences is dependent on the previous skill, which allows the assessment of evidence to be used in making inferences. The emphasis of this skill lies in the ability of the learner to make valid inferences, and to make and judge the inferences made by others. Some positive indicators include making and judging deductions through the use of logic, making and judging induction through generalising, hypothesising, and investigating, and making and judging value judgements. Bullen's fourth and ultimate skill, Using Appropriate Strategies and Tactics, may include using strategies to further clarify, or simplify a problem, organising arguments for both sides of an issue, consulting with others as a "reality check" (Bullen, 1997, p. 115), and revisiting conclusions to ensure satisfaction with a final response.
Exploration is the second phase, and is facilitated "in a community of inquiry by iteratively moving between the private and shared worlds - that is, between critical reflection and discourse" (p. 8). Exploration is indicated by activities such as asking questions, generating ideas, and sharing information with others in the community. After exploring the problem, Integration involves making sense of information. As in the previous phase, students reflect, and engage in discursive practices as a means of determining relevance of information in relation to the particular problem or issue. The Integration phase may not be indicated specifically in the contributions of students, so from the perspective of the teacher or researcher, integration must be inferred from an analysis of the student's contributions. Tracking the problem from its inception at the triggering phase may eventually lead to either a vicarious or directly applied solution, which the authors refer to as the Resolution phase. This process of Resolution is not intended to imply finality, and the authors maintain that in testing a solution, students may engage in this process all over again, though the phases are not necessarily followed in order, and some may even be skipped entirely.
| Brookfield (1987) | Norris & Ennis (1989) | Bullen (1998) | Garrison, Anderson & Archer (2001) | |
| Initial Phase | Trigger event | Elementary clarification | Clarification | Triggering event |
| Recognition of, or exposure to an unforeseen event that results in feelings of inner discomfort and perplexity. | Focusing on, and attaining a general level of clarification for the problem through analysing arguments and a question and answer phase. | Appraising and understanding the nature of, and different points of view on the issue, dilemma, or problem. | Identifying or recognising an issue, dilemma or problem from one's experience, or articulated by the instructor or other learners. | |
| Second phase | Appraisal | Basic support | Assessing evidence | Exploration |
| Appraising the situation and engaging in self-scrutiny. Includ-es various ways of dealing with unexpected events, includ-ing identification and clarification of concerns, and seeking others facing similar situations. | Judging sources for credibility, and making and judging one's own observations. Involves using information from others, previously accepted conclusions, and background knowledge. | Judging the credibility of sources, and assessing the evidence in support of inferences. Emphasises the need to establish a strong foundation for making inferences. | Thinking about ideas in both the private and social spheres in order to make preliminary decisions as to what information is relevant to the problem or issue. | |
| Middle phase | Exploration | Inference | ||
| Seeking a means to resolution, or ways of explaining discrepancies that reduce our sense of discomfort. May encourage a person to search for meaning, and explore new ways of thinking and behaving. | Making and judging inductive and deductive inferences, as well as making and judging value-judgements. This includes abilities for making inductions in addition to those required in judging those inferences. | |||
| Penulti- mate Phase | Developing alternative perspectives | Advanced clarification | Making and judging inferences | Integration |
| Developing new ways of thinking and behaving that helps one adapt to the un-expected event. This transition involves an attempt to diminish incongruencies in one's life. | Forming and defining terms, judging and evaluating definitions, taking the context of definitions into consideration, and locating unstated and needed assumptions in reasoning. | Inferring inductively and deductively, and making value judgements. Making decisions with adequate justification, and using evidence to support arguments. | Generating or constructing meaning from the ideas in the Exploration phase, and integrating the relevant information determined in the previous phase. | |
| Ultimate phase | Integration | Strategies and tactics | Using appropriate strategies and tactics | Resolution |
| Negotiating with new perspectives to facilitate integration of change into one's life. Involves integrating conflicting feelings and ideas internally or externally to achieve a resolution. | Interacting with others and deciding on an appropriate action. Problem is defined, possible solutions assessed and alternative solutions generated. Monitoring overall situation and decision-making process. | Using strategies or heuristics to guide thinking in a process of reaching a conclusion, making a decision, or solving a problem effectively, and in an orderly fashion. | Proposing a hyp-othetical solut-ion, or applying a solution directly to the issue, dilemma, or problem. Students at this phase have clear expectations, can test hypotheses and ideas, and view content from a critical perspective. |
We rejected both trigger event and clarification as terms that might be used, and instead adopted the term Recognise as the initial phase. Recognise retains the notion of a trigger or trigger event, or recognition of a problem, issue or dilemma, as articulated in Brookfield (1987), and Garrison, Anderson, and Archer (2001). Because the instrument focuses on the cognitive processes involved in CT, the notion of an event is represented here in terms of the related process of recognition of such an event (or issue, problem, perplexity, etc). In the case of an externally imposed trigger or issue, for example, the delineation of a problem by an instructor or another student is required to recognise the issue.
Brookfield's (1987) Appraisal also includes self scrutiny, and identification and clarification of the trigger event, whereas Norris and Ennis' (1989) Basic Support, and Bullen's (1998) Assessing Evidence focus more on a systematic process of using specific criteria to judge the credibility of a source. Exploration, Garrison, Anderson, and Archer's (2001) second phase, involves an exploration of relevant information through brainstorming, questioning, and exchanging information.
If we accept that recognition of an issue or dilemma must precede clarification, then processes from the initial phase of Norris and Ennis (1989), and Bullen (1998) will be retained for the second phase of the instrument. For the second phase of the instrument, we adopt the term Understand, which retains from the various second phases the following indicators: exploration of relevant issues (Garrison, Anderson & Archer 2001), making observations (see Norris & Ennis 1989), and seeking outside information, external sources, and perspectives of others in order to clarify the nature of the issue or dilemma (see second phase of the four models). In addition, we revisit congruencies between initial and second phases, thereby adding: focusing on or clarifying what is important (see initial phase of Norris & Ennis, 1989; Bullen, 1998; second phase of Brookfield, 1987; and Garrison, Anderson & Archer, 2001), and questioning and exchanging information (see initial phase of Norris & Ennis, 1989).
We adopt the term Analyse as the middle (or third) phase, retaining the concept of exploring new ways of thinking and behaving from Brookfield's (1987) Exploration, and from Norris and Ennis' (1989) Inference, the notion of making and judging inferences and value judgements. Analyse is a period of in depth clarification whereby a learner may employ skills to judge thinking, and to bridge the processes of understanding and evaluating through the judgement of one's own thinking. In addition to the processes from the models, we add to this middle phase indicators such as categorising and classifying evidence, information, knowledge, or perspectives; differentiating between various perspectives; interpreting and explaining the problem, dilemma, or issue; and identifying and filling in gaps in knowledge or information.
These processes serve an important function in critical thinking as a basis for the next phase, Evaluate, through analysis, and sifting through available evidence, information, etc, as a means of engaging in higher level, or new ways of thinking and behaving. Clearly, the additional processes are required after a level of understanding is reached, and before one can attempt to evaluate the evidence, information, etc. Identifying gaps in knowledge or information, and otherwise organising and analysing the evidence, etc, serves to enhance the rigour in the forthcoming higher level process of Evaluate.
The nature of Brookfield's (1987) Developing Alternative Perspectives, while useful as a specific indicator of the behaviours connected with the final phase, is not retained in the penultimate phase. Instead, generating alternative hypotheses and perspectives is included in the final phase, as supported by Norris & Ennis' (1989) final phase. We adopted Evaluate as the penultimate phase, which takes into consideration the abilities to define terms, judge definitions, and identify assumptions (see Norris & Ennis, 1989), and engage in reflection and discourse to determine the relevance of information (Garrison, Anderson & Archer, 2001). Because Bullen's (1998) four-phase model is based on the five phases of Norris and Ennis (1989), we have included the processes of the penultimate phase of Bullen's model in the middle, rather than the penultimate phase, to reflect the organisation of the original model. We include detecting inconsistencies and fallacies, as well as correspondences and congruencies as specific processes related to, but not specified in Norris and Ennis (1989) or Garrison, Anderson and Archer (2001). In addition, retaining or rejecting evidence, information, knowledge, or perspectives is added as an extension of the processes outlined above. Arguably, retaining or rejecting certain evidence, information, knowledge, or perspectives can be achieved through process of judging (Norris & Ennis, 1989), and through the process of iteratively moving between discourse and reflection (Garrison, Anderson & Archer, 2001). However these specific processes are made explicit in the instrument in order to facilitate its use and application by instructors, students, or researchers.
We adopted the term Create to describe the ultimate phase, a phase encompassing the creation of hypothetical as well as actual products or solutions. In other words, the result, or culmination of the processes involved in CT may or may not result in an actual solution or action, rather, as Garrison et al (2001) argue, the resolution may occur "by means of direct or vicarious action" (p. 2). In the context of an OAD, students may be encouraged to propose solutions or resolutions in addition to discussing the application of these solutions in a real life situation. Norris and Ennis' (1989) notion of generating alternative hypotheses is also retained, and in keeping with Garrison et al's (2001) assertion that resolution may be achieved either vicariously or directly, we add the specific processes of constructing, creating, inventing, and devising new knowledge or perspectives, implementing or executing strategies, and acting on a solution, decision, or conclusion.
The processes highlighted in the instrument are represented in Table 2, and have been assigned indicators from the previous discussion of the five phases. For analysis purposes, indicators have been assigned a code containing the first letter of the process along with ordered numbers according to where the indicators appear in the instrument. These numbers have been assigned only to facilitate coding and do not suggest a hierarchical sequence through engagement in CT.
| Process | Descriptor | Code | Specific indicators |
| Recognise | Recognising or identifying an existent issue, dilemma, problem, etc. | R1 | Recognising, identifying, or focusing on an issue, dilemma, problem, inner discomfort, or perplexity requiring further investigation or clarification. |
| Understand | Exploring related evidence, knowledge, research, information, and perspectives. | U1 | Exploring and identifying what is relevant to the issue, dilemma, problem, etc. |
| U2 | Locating background information, knowledge, previously accepted conclusions, or evidence from other sources. | ||
| U3 | Locating alternate perspectives or evidence on the issue, dilemma, problem, etc. | ||
| U4 | Making observations. | ||
| U5 | Clarifying or appraising the nature of the issue, dilemma, problem, etc. | ||
| U6 | Questioning and exchanging information. | ||
| Analyse | Seeking in depth clarification, organising known information, identifying unknown information, and dissecting the issue, dilemma, or problem into its fundamental components. | A1 | Engaging in new ways of thinking and behaving. |
| A2 | Categorising and classifying evidence, information, knowledge, or perspectives. | ||
| A3 | Differentiating similarities and differences in alternate perspectives or evidence on the issue, dilemma, problem, etc. | ||
| A4 | Interpreting and explaining the issue, dilemma, problem, etc. | ||
| A5 | Breaking down the problem, dilemma, issue, etc. into constituent parts. | ||
| A6 | Identifying and filling gaps in knowledge or information, and judging one's own thinking. | ||
| Evaluate | Critiquing and judging information, knowledge, or perspectives. | E1 | Judging the validity, value, applicability, and relevance of information, knowledge, sources. |
| E2 | Critiquing perspectives and assumptions. | ||
| E3 | Detecting inconsistencies, fallacies, as well as correspondences and congruencies. | ||
| E4 | Making and judging definitions. | ||
| E5 | Using evidence to support arguments. | ||
| E6 | Retaining or rejecting evidence, information, knowledge, or perspectives. | ||
| Create | Producing new knowledge, perspectives, or strategies, and implementing them or acting on them. | C1 | Implementing or executing strategies. |
| C2 | Applying actual or hypothetical solutions, decisions, or conclusions. | ||
| C3 | Constructing, creating, inventing, and devising new knowledge or perspectives. | ||
| C4 | Generating alternative hypotheses and perspectives. | ||
| C5 | Acting on a solution, decision, or conclusion. | ||
| C6 | Executing, or implementing change or a plan. |
It is important to consider the instrument and its indicators as a tool or guide in the context of teaching and learning. Not all indicators will necessarily be relevant in all contexts of discussion in an OAD. The indicators are meant to illustrate each of the cognitive processes as a series of actual behaviours described in operationalised terms. These indicators reflect the iterative nature of CT in that they provide various possibilities for engagement in critical thinking at each phase, and allow room for interpretation when identifying, measuring, or promoting CT.
The instrument was applied to the analysis of the OAD using the sentence as the unit of analysis, following Fahy's (2001) observation that, "Sentences are, after all, what conference participants produce to convey their ideas, and are what transcripts consist of" (p. 4). Fahy also offers the caveat that units of analysis beyond the sentence (e.g. Henri's 1992 unit of meaning) can lead to lack of discriminant capability in coding. In this study, the instrument was applied in the measurement of the five processes associated with the construct of CT. One coder analysed the transcript, and the results are presented in Table 3. The total number of messages in the transcript was 79, including messages for which no code was assigned. There was an average of 13 sentences per message.
| Process | Indicator Code | Number of instances |
| Recognise | R1 | 0 |
| Total | 0 | |
| Understand | U1 | 45 |
| U2 | 11 | |
| U3 | 7 | |
| U4 | 11 | |
| U5 | 4 | |
| U6 | 0 | |
| Total | 78 | |
| Analyse | A1 | 16 |
| A2 | 0 | |
| A3 | 0 | |
| A4 | 6 | |
| A5 | 6 | |
| A6 | 3 | |
| Total | 31 | |
| Evaluate | E1 | 48 |
| E2 | 8 | |
| E3 | 1 | |
| E4 | 0 | |
| E5 | 28 | |
| E6 | 9 | |
| Total | 94 | |
| Create | C1 | 0 |
| C2 | 0 | |
| C3 | 0 | |
| C4 | 1 | |
| C5 | 0 | |
| C6 | 0 | |
| Total | 1 |
Coding at the level of the indicator resulted in identification of 45 instances whereby participants explored and identified what was relevant to the issue (U1); 48 instances where discussants engaged in judging information, knowledge, and sources (E1); and 28 instances of using evidence to support arguments (E5). Eleven of the indicators included in the instrument were not coded for in the discussion as no instances were identified.
In relation to the value, in this one case of testing in one context, the instrument revealed the specific behaviours related to CT in which participants engaged. These are represented numerically in Table 3. In this context, we can observe from the table of results that participants engaged little in the last stage of Create. From a designer's or instructor's perspective, such results could serve as a rationale for redesigning the OAD to promote engagement in all processes related to CT. In this regard, more specific or focused structuring activities or moderating might support students' engagement in all processes related to CT, including this final activity.
We can hypothesise as well that the context and type of participants were not suited to engagement in this final process of CT, which required acting on decisions or conclusions. Participants in this context were teachers in training enrolled in a methods course discussing issues that they would possibly face when they became teachers in practice. If the context had been a group of teachers in practice discussing the issue, we might expect that they could have actually implemented in their classes solutions or strategies proposed in the discussion and then reported on their experiences as part of the final process. Similarly, high levels of participation in Understand and Evaluate are representative of participants' focus on exploring known and existing information, judging information, and often drawing on supporting evidence in making claims. The lower representation from Analyse, like the process Create, can serve to inform designers or instructors of the need to modify the OAD, or provide more support for students.
In terms of the instruments' practicality and effectiveness, results revealed that coding at the level of the indicator and sentence is time consuming and cumbersome. The analysis used the sentence as the unit of coding. Given 79 messages with an average of 13 sentences per message, which could each be coded with any of 25 codes, the task was formidable. A more practical approach might be to code using a different unit, for example the message. Another approach would be to code at the level of the process, using the indicator as a guide only. If the approach were to code each message into one process only, the time and labour involved would be significantly reduced. Depending on the detail required in the results, this approach might be feasible. As well, use of Ethnograph or other types of analysis software might facilitate coding at the level of the sentence and indicator.
The practicality of the instrument can also be considered in relation to its capability to discriminate between behaviours. The clustering of codes into three primary indicators exploring and identifying what is relevant to the issue (U1), judging the validity, value, applicability, and relevance of information, knowledge, sources, etc (E1), and using evidence to support arguments (E5), suggests a potential lack of mutual exclusivity between indicators. In addition, coding posed challenges because of ambiguous indicators such as making observations (U4). While useful for a theoretical model, such as Norris and Ennis' (1989), the indicator making observations may be too vague for an instrument that purports to measure CT. The indicator questioning and exchanging information (U6) demonstrates an example of lack of mutual exclusivity within an indicator as it contains two distinct activities, questioning and exchanging information. The nature of the OAD necessarily implies that participants exchange information, and questioning and exchanging represent two separate acts. In addition, some overlap appears to exist between the indicators interpreting and explaining the issue (A4), and breaking down the issue into its constituent parts (A5).
In addition to the first indicator, nine indicators were not coded for in the OAD. This suggests that these indicators might be superfluous or unessential to the CT construct. However use of the instrument in other contexts would be required to make such a determination. Making and judging definitions (E4) was not evident in the OAD, and therefore use of the instrument in other contexts would help determine if this indicator is irrelevant to the construct of CT. However, indicators such as categorising and classifying evidence, information, knowledge or perspectives (A2); and differentiating similarities and differences in alternate perspectives or evidence on the issue, dilemma, problem, etc (A3), which appear relevant to this context, were not found in the OAD either, suggesting that the instrument should be applied with different participants for the purpose of determining its construct relevance.
Instructors interested in promoting CT in their teaching and learning can rely on the instrument to support the design of an OAD for such purposes. In this regard, the instrument can be used to provide them with guidance in the selection of the types of activities, formats of questions, and instructions provided to students. As moderators of OADs, instructors can use the instrument to direct their own interventions to ensure that these actively and directly promote CT. In terms of assessment, instructors can informally rely on the descriptions of processes and their indicators as benchmarks against which to compare students' contributions to the discussion. On a more formal level, instructors can rely on the instrument to develop rubrics for the assessment and measurement of engagement in CT. The creation of these rubrics would require the assignment of a number or percentage scale to the various processes and/ or indicators.
Students as well as instructors can rely on the instrument in order to promote and monitor their own engagement in critical thinking and to enhance their participation in OADs. They can be directed or guided by the instructor or by the course design to be aware of the types of processes in which they can engage while participating in a discussion. The instrument can serve for them as a guide or example of best practices towards which they might aim. In this sense, the instrument provides them with a breakdown and listing of the various types of thinking activities in which they might engage. Providing them with the list of processes and their indicators related to CT could help them to move beyond engagement in lower level thinking processes such as mere description, or identification.
In terms of research, the instrument could benefit from application in a variety of contexts. The instrument was derived from a conceptual framework that supported clarification and validation of the construct. However, empirical testing in a variety of contexts can further validate the construct and its representation in the instrument. In addition, the process followed to develop the instrument could be used to develop instruments for the analysis of transcripts for other cognitive processes, such as knowledge construction. Issues of practicality require further investigation in order to make the instrument easily usable by instructors or students.
Brookfield, S. D. (1987). Developing critical thinkers. San Fransisco: Jossey-Bass.
Bullen, M. (1998). Participation and critical thinking in online university distance education. Journal of Distance Education, 13(2), 1-32.
Bullen, M. (1997). A case study of participation and critical thinking in a university-level course delivered by computer conferencing. Unpublished doctoral dissertation. University of British Columbia, Vancouver, Canada. [viewed 25 Mar 2003, verified 18 June 2004] http://www2.cstudies.ubc.ca/~bullen/Diss/thesis.doc
Dewey, J. (1933). How we think. Boston: Heath.
Fahy, P. J. (2002). Assessing critical thinking processes in a computer conference. [viewed 26 Jan 2004, verified 18 June 2004] http://cde.athabascau.ca/softeval/reports/mag4.pdf
Fahy, P. J. (2001). Addressing some common problems in transcript analysis. The International Review of Research in Open and Distance Learning, 1(2), 2-8. [viewed 26 Jan 2004, verified 18 June 2004] http://www.irrodl.org/content/v1.2/research.html#Fahy
Garrison. D. R., Anderson, T. & Archer, W. (2001). Critical thinking and computer conferencing: A model and tool to assess cognitive presence. American Journal of Distance Education, 15(1), 7-23. [viewed 21 Feb 2003 at http://www.atl.ualberta.ca/cmc/CogPresPaper_June30_.pdf, verified 18 June 2004 at http://communitiesofinquiry.com/documents/CogPres_Final.pdf]
Garrison, D. R., Anderson, T. & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105. [viewed 21 Feb 2003 at http://www.atl.ualberta.ca/cmc/CtinTextEnvFinal.pdf, verified 18 June 2004 at http://communitiesofinquiry.com/documents/Critical_Inquiry_model.pdf]
Gunawardena, C., Lowe, C.A. & Anderson, T. (1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397-431.
Hager, P., Sleet, R., Logan, P. & Hooper, M. (2003). Teaching critical thinking in undergraduate science courses. Science and Education, 12(3), 303-313.
Halpern, D. F. (1989). Thought and knowledge: An introduction to critical thinking, 2nd ed., Hillsdale, NJ: Erlbaum.
Henri, F. (1992). Computer conferencing and computer-mediated communication: Interactive, quasi-interactive, or monologue? In C. O'Malley (Ed), Computer Supported Collaborative Learning, Berlin: Springer-Verlag, 145-164.
Jonassen, D. & Kwon, H. (2001). Communication patterns in computer mediated versus face-to-face group problem solving. Educational Technology Research and Development, 49(1), 35-51.
Kanuka, H. & Anderson, T. (1998). Online social interchange, discord, and knowledge construction. Journal of Distance Education, 13(1), 57-74. [viewed 25 Mar 2002, verified 18 Jun 2004] http://cade.athabascau.ca/vol13.1/kanuka.html
McPeck, J. E. (1992). Thoughts of subject-specificity. In S. P. Norris (Ed), The generalizability of critical thinking: Multiple perspectives on an educational ideal. New York: Teachers College Press, 198-205.
McPeck, J. E. (1981). Critical thinking and Education. Oxford: Martin Robertson.
Murphy, E. (2004). Identifying and measuring problem formulation and resolution in online asynchronous discussions. Canadian Journal of Learning and Technology, 30(1), 5-20. [verified 8 Oct 2004] http://www.cjlt.ca/content/vol30.1/cjlt30-1_art1.html
Newman, D. R., Johnson, C., Webb, B., & Cochrane, C. (1997). Evaluating the quality of learning in computer supported cooperative learning. Journal of the American Society of Information Science, 48, 484-495.
Newman, D. R., Webb, B., & Cochrane, C. (1995). A content analysis method to measure critical thinking in face-to-face and computer supported group learning. Interpersonal Computing and Technology Journal, 3(5), 56-77. [viewed 25 Mar 2002, verified 18 June 2004] http://www.qub.ac.uk/mgt/papers/methods/contpap.html [also at http://www.emoderators.com/ipct-j/1995/n2/newman.html]
Norris, S.P. & Ennis, R. (1989). Evaluating critical thinking. In R. J. Schwartz & D. N. Perkins (Eds), The practitioners' guide to teaching thinking series. Pacific Grove, CA: Midwest Publications.
Oliver, R. (2001). Exploring the development of critical thinking skills through a Web-supported problem-based learning environment. In J. Stephenson (Ed), Teaching and Learning Online: Pedagogies for New Technologies. VA: Kogan Page, 98-111.
Paul, R. W. (1993). Critical thinking: What, why, and how. In C. A. Barnes (Ed), Critical thinking: Educational imperative, 77, 3-24. San Francisco: Jossey-Bass.
Phye, G. D. (1997). Inductive reasoning and problem solving. In G. D. Phye (Ed), Handbook of academic learning. Construction of knowledge. San Diego, CA: Academic Press, 451-471.
Zhu, E. (1996). Meaning negotiation, knowledge construction, and mentoring in a distance learning course. In Proceedings of selected research and development presentations at the 1996 national convention of the Association for Educational Communications and Technology. Indianapolis, USA. (ERIC Document Reproductions Service No. ED397849).
| Author: Dr Elizabeth Murphy is an Assistant Professor of Educational Technology and Second-Language Education in the Faculty of Education, Memorial University of Newfoundland, Canada. Her research interests lie in the use of online discussions in the context of learning and in the design of online learning experiences that reflect a socio-constructivist perspective.
Dr E. Murphy, Faculty of Education, Memorial University, St. John's, NL, Canada A1B 3X8 Tel: +709 737 7634 Fax: +709 737-2345 Email: emurphy@mun.ca Web: http://www.ucs.mun.ca/~emurphy/ Please cite as: Murphy, E. (2004). An instrument to support thinking critically about critical thinking in online asynchronous discussions. Australasian Journal of Educational Technology, 20(3), 295-315. http://www.ascilite.org.au/ajet/ajet20/murphy.html |