AJET 19(3) Oriogun (2003) - understanding online learning levels of engagement using the SQUAD approach to CMC discourse

Australian Journal of Educational Technology
2003, 19(3), 371-387.

Towards understanding online learning levels of engagement using the SQUAD approach to CMC discourse

Peter K. Oriogun
London Metropolitan University, UK

Recent research in content analysis has shown the difficulties of achieving acceptable levels of inter-rater reliability with CMC transcripts. This has lead to the development of semi-structured computer conferencing systems, in which participants choose the type of contribution that they are making from a limited set of alternatives. This article extends previous work with respect to semi-structured approaches to online discourse, suggesting coding categories relevant for problem based learning. The SQUAD approach to online discourse offers definitions for quality with respect to participation, interaction and cognition, when using the message as the unit of CMC transcript analysis, analysing for what the author has termed as 'online learning levels of engagement'. It is argued that the theoretical basis underpinning the SQUAD approach is beneficial for the development of teamwork and cognitive reasoning when learning in small groups, and that it is a relatively straightforward exercise to apply this approach in a different mode of study or subject area.

Introduction

In recent years we have seen the widespread adoption of computer mediated communication (CMC) in education, including extensive interest in using online communications to facilitate asynchronous dialogues, eg. online teamwork. Consequently, recent research, for example on dialogue analysis, has attempted to explore the relationship between online dialogue features (eg. roles, strategies, form and content) and learning (Pilkington, 2001). Such an analysis can provide useful insights into the nature of the learning processes from the perspective of, for example, what a speaker's intention is in a transmitted message and what the receiver perceives has been communicated by the message. However, a problem arises if we wish to investigate specific categories or variables of the learning process, eg. participation, interaction, social, cognitive and metacognitive (Henri, 1992). Specifically, if online interactions are to be transcribed and analysed using some theoretical framework, then the issue of coder interpretation at the time of coding a transcript becomes important. In the research described in this article the three variables of 'participation', 'interaction' and 'cognition' are investigated with the goal of measuring what the author has termed as 'online learning levels of engagement' using the method described as the SQUAD approach to CMC discourse, a semi-structured way of categorising online messages. A full explanation of the meaning of this new approach and its framework is included in this article (see Table 3).

The SQUAD approach to CMC discourse adopts problem based learning (Barrows, 1996; Bridges, 1992; Oriogun et al, 2002) as an instructional method with the goal of solving real problems by:

Creating the atmosphere that will motivate students to learn in a group setting online;
Promoting group interactions and participation over the problem to be solved by the group online;
Helping learners to build up knowledge base of relevant facts about the problem to be solved online;
The newly acquired knowledge is shared by the group online with the aim of solving the given problem collaboratively and collectively;
Delivering various artefacts leading to a solution or a number of solutions to the problem to be solved online.

Research methods

For the study described below, the author employed a combination of grounded theory and case studies to guide this research. Strauss and Corbin (1990) suggest that grounded theory is especially useful for complex subjects or phenomena where little is yet known (as is the case in our study). This is because of the methodology's flexibility, which can cope with complex data and its continual cross-referencing, which allows for grounding of theory in the data thus uncovering previous unknown issues. The SQUAD approach to CMC discourse adopts coding categories relevant to problem based learning (Barrows, 1996; Bridges, 1992; Oriogun et al, 2002) by encouraging students to develop the skills of transferring knowledge into new domains, a skill that they can carry with them throughout their professional lives. This in turn empowers them with responsibilities of managing a largely self directed learning process, as a consequence, they are better equipped and informed to accept the responsibilities of mature professional life (Brine & Shannon, 1994). Foreman and Johnston (1999, p382) suggest that, "case studies can be based on real events in real organisations" (as it is in the case study we present in support of this research). Case studies were "originally devised for use in medicine and law, have long been used in business and management education as a way of encouraging students to develop analytical skills as well as enhancing their practical knowledge" (Foreman and Johnston, 1999, p382). Consequently, the combination of grounded theory and a case study approach allows the extension of theory into an analysis of practice. The research question for this study is as follows:

In what ways can we measure the quality of online learning levels of engagement with respect to 'participation', 'interaction' and 'cognition'?

Literature review

A full review of the different approaches to interaction analysis is beyond the scope of this article. Briefly, the observation and analysis of human communicative interactions are variously called content analysis, conversational analysis, discourse analysis, speech acts, dialogue analysis, and so on. Dialogue analysis is an approach that focuses on examining the patterns to be found in educational interactions. Pilkington's (1999) approach, called DISCOUNT, is based on an attempt to synthesise and extend existing approaches to discourse analysis, including transactional analysis, dialogue game theory and rhetorical structure theory. A number of researchers have developed semi-structured computer conferencing systems, in which participants choose the type of contribution from a limited set of choices. In the issue based discussion forum developed by Duffy, Dueber, and Hawley (1998), students post a message by selecting one of four labels - Hypothesis, Important Point, Evidence, or Learning Issue. Barros and Verdejo (2000) developed a system that includes automatic message analysis features.

Other approaches (eg., Cook, 2001; Baker, 1994) are based on speech act theory, (Austin, 1962; Searle, 1969). In 'classical' speech act theory, only invented, isolated utterances are considered. More recently in speech act theory, dialogue is viewed as a sequence of speech acts, uttered by each party to achieve certain goals. For example, Cook (2001) presents an approach to using empirical data on human teacher-learner interactions to guide the development of a pedagogical agent for supporting musical composition learning. In addition, dialogue analysis is also used to ascertain whether or not a user interacts within a CMC system in a productive manner, in terms of the learning task outcome. This latter approach is the focus of the work described in this article. The SQUAD approach to CMC discourse invites students to post messages based on five given categories, namely, Suggestion, Question, Unclassified, Answer and Delivery.

Commonly measured variables for online discourse

This section explores the literature on the variables used for content analysis of online discourse. Five variables that tend to be investigated in a CMC research context: participation, interaction, social, cognitive and metacognitive elements of online discourse. For example, Henri (1992), identified these five elements as key dimensions for the analysis of online discussion. She used thematic as a unit of analysis. Weiss and Morrison (1998) investigated critical thinking, understanding/correcting, misunderstanding and emotion using thematic and message as units of analysis. McDonald (1998) used thematic as a unit of analysis during the investigation of six variables, namely, participation, interaction, group development, social, cognitive and metacognitive elements.

By contrast, Howell-Richardson and Mellar (1996) used illocutionary act (from speech act theory) as a unit of analysis when they investigated participation, illocutionary properties and focus groups. Hara, Bonk and Angeli (2000) used paragraph as a unit of analysis for the same five variables as Henri (1992). Fahy et al. (2000) investigated interaction, participation and critical thinking, using sentence as a unit of analysis. Oriogun (2003) used message as a unit of analysis when he investigated participation and interaction.

Defining participation indicators for the 'SQUAD' approach

The three variables investigated in this article using the SQUAD approach to online messaging in this study are participation, interaction and cognition. With respect to this study, the author will first define participation, followed by interaction, and, finally, cognition, with a supporting theoretical basis for the three variables. The definition of quality with respect to participation extends the suggestion for grading graduate level student participation in CMC classroom as reported in Hutton and Wiesenberg (2000). The criteria are as follows:

Evidence of completion of readings
Relevance: the student's comment moves the discussion forward
Logic: the points are expressed and elaborated well
Insight: the point reflect a creative or novel approach
Referencing other students' notes in their own comments
Acknowledging the work of others: agree, debate, question, synthesise, or expand
Appropriate etiquette (no 'flaming' or sexist/racist remarks)

Defining interaction indicators for the 'SQUAD' approach

With respect to interaction we define quality along the lines of Fahy (2001), where the meaning of the interaction must be something obvious and constant within the transcripts, and it reflects the interaction of the reader's knowledge and experience with the text in the message. Irrespective of what the writer intends, what the readers understand is based on the interaction between the message and the readers' experience, knowledge, and capability for understanding the topic. We have extended Fahy's definition, for the purpose of this study, by offering the following criteria for grading graduate level student interaction in a CMC discourse:

Low interaction: Resolving conflicts within the group
Medium interaction: Offering alternative solutions to group problems and offering to deliver relevant artefacts for the group's common goal
Active interaction: Delivering relevant artefacts for the group's common goal

Defining cognitive indicators for the 'SQUAD' approach

According to (Ryder, 1994) knowledge is constructed by learners as they engage in dialogue. Furthermore, since the introduction of the "Zone of Proximity Development" (ZPD) continuum by Vygotsky (1962), it has been advocated by a number of authors that social interactions can act as scaffolding in the construction of knowledge. On the basis of this Vygotskian viewpoint, learning can be seen as a social phenomenon and experience.

A number of theories on knowledge building emphasise the socially distributed nature of cognition. Distributed cognition is therefore a process whereby individual cognition is extended to acquire something that an individual would be unable to achieve alone. Knowledge is constructed in associated networks of concepts and nodes. As learning occurs, new information is collected and coupled to existing knowledge networks. New information can then be easily retrieved to solve problems, and to apply in context. Students are expected to learn about the world based on their own research and study. Students determine their "knowns" and "unknowns". They seek knowledge to address their "unknowns". They engage in collaborative learning in their small groups to work on the problems (Wee, Kek & Sim, 2001, p159).

Bruer (1993) argues that learning is quicker when students possess self motivating skills generally referred to as metacognitive skills. Learning in PBL encourages metacognitive skills. In line with our usage of PBL in this article, we have adopted the adaptation of Henri's (1992) descriptors for "Reasoning Skills" as suggested by (Hara, Bonk & Angeli, 2000) in support of cognitive indicators for the SQUAD approach. See Table 1 for these descriptors.

Table 1: Cognitive Indicators Descriptors
(adapted from Hara, Bonk & Angeli, 2000)

Reasoning skills Definitions

Elementary clarification Observing or studying a problem, identifying its elements, and observing their linkages in order to come to a basic understanding.

In depth clarification Analysing and understanding a problem to come to an understanding which sheds light on the values, beliefs, and assumptions which underlie the statement of the problem.

Inferencing Induction and deduction, admitting or proposing an idea on the basis of its link with propositions already admitted as true.

Judgement Making decisions, statements, appreciations, evaluations and criticisms. Sizing up.

Application of strategies Proposing coordinated actions for the application of a solution, or following through on a choice or a decision.

Online learning matrix for the SQUAD approach

In this section the author describes his proposed Online Learning Matrix (OLM) for the SQUAD approach to CMC discourse. The OLM shown in Table 2 grouped the messages posted by the students in terms of the levels of learning engagement achieved by each participant as thus:

Very Low
Low
Nominal
High
Very High

The grouping of messages in terms of what the author considers to be the hierarchical ordering of the type of learning associated with the postings made by participants are in line with the theoretical basis that underpins the SQUAD approach as explained earlier, see also Table 3 for the cognitive indicators.

In line with the cognitive indicators underpinning the SQUAD approach (see Tables 1 and 3 respectively) the OLM was constructed. The OLM details the levels of online levels of engagements when using this new approach to categorising CMC discourse. Figure 1 is the consolidation of Table 2 in terms of �Low�, �Nominal� and �High� levels of online engagements. Consequently, messages posted with the title Question or Q, and Unclassified or U are deemed to be generally of "Low Level of Online Engagement"; messages posted with title Suggestion or S, and Delivery or D are deemed to be generally of "High Level of Online Engagement"; finally, messages posted with the title Answer or A remains as "Nominal". Therefore, depending on the level of granularity required, Figure 1 could also have been drawn showing all the five hierarchical ordering in Table 2.

Table 2: The online learning matrix (OLM) for the SQUAD approach to CMC messaging

Message Category Very Low Low Nominal High Very High

S - Suggestion

X

Q -Question
X

U -Unclassified X

A -Answer

X

D -Delivery

X

The SQUAD approach to online messaging - theoretical framework

Table 3 below shows the theoretical framework for the SQUAD approach to CMC discourse within a PBL environment, together with the descriptors for 'participation', 'interaction' and 'cognition'. Sections 5-7 above elaborate further on the meaning of these descriptors with respect to the theoretical basis suggested in this article in support of this new approach to online discourse.

The study

The case study presented in this article is from a module titled Software Engineering that the author teaches at the London Metropolitan University. This module is of advanced standing, and, compulsory for the MSc in Computing offering at the University. It typically attracts between 30-80 students per semester. In the 2002-03 academic year, 38 students completed the coursework element of the module in the first semester (two semester per year). The 38 students were split randomly into 7 tutorial groups, 4 groups consisting of 6 members each and 2 groups consisting of 7 members each.

Table 3: The SQUAD approach to CMC discourse:
Descriptors for participation, interaction and cognition

Message
category Description Example Cognitive
indicators Participation
indicators Interaction
indicators

S
Suggestion The process whereby the mere presentation of an idea to a receptive individual leads to the acceptance of the idea. Students engage with other students within their coursework groups by offering advice, a viewpoint, or an alternative viewpoint to a current one. -Elementary classification
-In depth classification
-Inferencing
-Judgement
-Application of strategies Students engaging other students actively by taking a lead role in online discourse by posting meaningful and relevant messages to the group. The message will be accessed and processed by other members of the group for the cycle of communication to complete.

Q
Question A form of word address to a person in order to elicit information or evoke a response. Students may seek clarification from the tutor or other students in order to make appropriate decisions relating to the group coursework. -Elementary classification
-In-depth classification The message is posed in such a way that some or all the group members will engage in the ongoing discussion. The message will be accessed and processed by other members of the group for the cycle of communication to complete.

U
Unclassified Not in the list of categories of messages stipulated by the instigator of the task at hand. This tends to happen at the start of the online postings. Students may be unsure of what the message is suppose to convey. In most cases, it falls within one of the four classified categories. -Elementary classification This type of message may or may not engage other students. In most cases, the message could be re-aligned to fall within the four classified categories by the coder of the final transcript at the end of semester. This type of message may or may not engage other students. In most cases, the message could be re-aligned to fall within the four classified categories by the coder of the final transcript at the end of semester.

A
Answer Reply, either spoken or written, as to a question, request, letter or article. Students are expected to respond to this type of message with a range of possible solutions / alternatives. -Elementary classification
-In depth classification
-Inferencing
-Judgement Responding to a query or question will inevitably involve most, if not all the group members, especially if the response is not in line with other group members' opinions. All group members are expected to deliver parts of the final product by working collaboratively.

D
Delivery The act of distribution of goods, mail etc. Students are expected to produce a piece of software at the end of the semester. They all have to participate in delivering aspects of the artefacts making up the software. -Elementary classification
-In-depth classification
-Inferencing
-Judgement
-Application of strategies Each member of the group is expected to play active role in delivering parts of the artefact making up the final software product. This is also expressed in the marking scheme for the module. All group members are expected to deliver parts of the final product by working together collaboratively.

Each group was assigned a designated tutorial assistant. The author was the sole lecturer on the module. The tutorial assistants assisted the students during the laboratory sessions dealing with the practical aspects of the coursework for the module. There were two assessments for the course: group coursework (50%) and a final examination (50%). Group coursework had an individual element attracting 17% of the marks (this was the CMC discourse of the module). The SQUAD approach was adopted for the CMC discourse of the module throughout the semester. The author directed his students to use a previously developed, enhanced Problem Based Learning Grid (Oriogun et al., 2002) and its predecessor (Oriogun & Georgiadou, 2000) to help them to promote learning spaces as environments that provide the learner with the responsibility for their own development and allow the learner to participate and be incumbent in a new social dynamics (Oriogun et al., 2002 p497).

The five categories for the SQUAD approach were fully explained to the students at the beginning of the semester, namely that, when a student wishes to make a suggestion to the rest of the group regarding their ongoing assignment, the posting title must be Suggestion (S); in the case asking the group a specific or general question, the posting title must be Question (Q); if a student is answering a question the posting title must be Answer (A); where a student is delivering aspects of the artefacts for the group assignment the posting title must be Delivery (D); finally, the most demanding category to rate is the unclassified category. Postings leading to or with the potential of being categorised as Unclassified (U) are as follows:

where a student forgets to put a title or category for their posting;
where a student posts two or more kinds of categories in a single message;
during the data analysis stage, when some of the messages could be re-categorised by the final rater of the message transcript - in this case the author.

The author also explained the rationale and the theoretical basis for the SQUAD at the start of the semester. Students were also asked to use a publicly available bulletin board system to facilitate their CMC discourse. The reasons for this are as follows:

to afford students the opportunity to familiarise themselves with various commercially available bulletin board systems before selecting one;
not to depend solely on the bulletin board system supported by the university;
to adopt the win-win process model using an alternative discussion forum to the win-win router developed at the University of Southern California (Boehm et al. 1995).

All of the student groups decided to use the Yahoo groups system to conduct the CMC discourse. The group chosen for this study posted 237 messages among its six members over a period of 64 days. The author also participated in the group's CMC discourse. Messages 36-40 and 66 were missing. It took a total of 3hours for the author to re-categorise some of the messages posted. It is useful to read the first 20-40 messages to check that where students have titled a posting with Unclassified (U), that it is correct, otherwise the message must be re-categorised by the final rater - in this case the author. However, as this was the first time the exercise was conducted as a case study, the author checked the content of all the messages to validate that the posting titles were in the correct category. Otherwise, the posting was re-categorised accordingly.

The whole idea of the SQUAD approach is to reduce the inter-rater reliability measure of CMC discourse. It is therefore expected that once the categorisation has been explained to students fully as documented above, the only category that may be problematic is the Unclassified or U titled postings. The author re-categorised 13% of the messages posted under the category Unclassified (see more details under the results section of this article). Six of the actual messages posted by the students who participated in the study are shown in the Appendix. These were randomly selected to represent all of the categories in the SQUAD. Figure 1 below shows the contributions of the students (numbered S1 to S6) and the author during the semester.

Figure 1: Participant's online learning levels of engagement (see Table 2)
using the SQUAD approach to CMC discourse

Results

Student 1 sent 12.1% of all the messages posted throughout the semester. Out of the total of 28 messages sent by Student 1, 68% were of High level, 14% were of Nominal level, and 18% were of Low level. Student 2 sent 14.2% of all the messages posted throughout the semester. Out of the total of 33 messages sent by Student 2, 70% were of High level, 6% were of Nominal level, and, 24% were of Low level. Student 3 sent 15.6% of all the messages posted throughout the semester. Out of the total of 36 messages sent by Student 3, 67% were of High level, 6% were of Nominal level, and, 27% were of Low level. Student 4 sent 14.3% of all the messages posted throughout the semester. Out of the total of 33 messages sent by Student 4, 73% were of High level, 12% were of Nominal level, and 15% were of Low level. Student 5 sent 9.1% of all the messages posted throughout the semester. Out of the total of 21 messages sent by Student 5, 63% were of High level, 14% were of Nominal level, and, 23% were of Low level. Student 6 sent 15.2% of all the messages posted throughout the semester. Out of the total of 35 messages sent by Student 6, 76% were of High level, 9% were of Nominal level, and 15% were of Low level.

The author sent 19.5% of all the messages posted throughout the semester. Out of the total of 45 messages sent by the author, 20% were of High level, 44% were of Nominal level, and 36% were of Low level. The author contributed most of the category 'S', 'U', and 'A' messages. The tutor asked very few questions, and he did not post any messages under 'D' for delivery of an artefact towards the students' coursework. Figure 1 above shows a graphical representation of the students' online learning levels of engagement.

Discussion

On the whole, all the students engaged with the learning experience by contributing high levels of 'Learning Levels of Engagement' throughout the semester. All of the participants actively engaged in the learning experience as more than 60% of their CMC contributions were deemed to be of appropriate standards with respect to cognition, participation and interaction, as defined with the theoretical framework that underpins the SQUAD approach. The most interesting observation made by the author, when he adopted the SQUAD approach to online messaging, was that students were able to concentrate on the group coursework, and were able to assist one another in terms of being informative with respect to the assigned tasks. Some group members also felt that they were being watched by the author, as a consequence, they were very careful as to the tone of their messages. The group whose work is the basis of the case study in this article, pointed out in their final report that:

Part of the project was to set up and use a yahoo groups account in order to record communication between members of the group, which was both assessed by and overseen by one of the course tutors. It was felt that the inclusion of one of the tutors created an artificial environment, partly because this meant that the developers were unable to express their feelings openly and also that some of the messages that were being posted to the site could be interpreted as seeking to impress the supervising member of staff rather than enhance the overall level of communication throughout the group.

This coursework brought the students together as they had not worked together previously within a Problem Based Learning environment at the University. The group of students further reported on their experiences as part-time students undertaking a software engineering project as quoted below:

...we encountered a number of problems because we are part-time evening students. One of these problems was to work on files outside of the University computing facilities. We were effectively prevented from using FTP and Telnet, which meant any testing could only be done during opening hours of the computing building at the university. These hours were very limited. Eventually we were able to overcome the problems and connect to the department of computing server, but this resulted in the loss of a week's work at least... It is a testament to the dedication of the group that we were able to meet up as frequently as was achieved, and would probably go some way to explain the dependency on using other forms of communication, including the very large number of messages sent through the yahoo groups site

The SQUAD approach is a semi-structured computer conferencing system, it is however, advisable for anyone interested in adopting the SQUAD for capturing what the author has termed as "online learning levels of engagement" to first experiment with either an inter-rater reliability measure (Cohen 1960; Henri 1992; Hara, Bonk & Angeli 2000; Fahy 2001) or the Transcript Reliability Cleaning Percentage TRCP as proposed by Oriogun (2003), in order to generate appropriate acronym from the initial categorisation of CMC messages within their subject discipline. The SQUAD approach is simple enough for academics with limited mathematical background and can provide insights into the nature of the learning process from the prospective of categorisation of online discourse.

It has been suggested by Mason & Romiszowski (1996) that

The most glaring omission in CMC research continues to be lack of analytical techniques applied to the content of the conference transcripts. Given that the educational value of computer conferencing is much touted by enthusiasts, it is remarkable that so few evaluators are willing to tackle this research area. (p.443).

The SQUAD approach to CMC discourse is a way of addressing this suggestion by offering a theoretical framework that facilitates what the author is calling the "online learning levels of engagement".

Conclusion

The SQUAD approach adds to the significant body of literature, which argues the need for learning scaffolding when using small group learning. It is claimed that the SQUAD approach appears to have a high degree of generality in that it appears to be a relatively straightforward exercise to apply in a different mode of study or subject area. It is argued in this article that the cognitive indicators suggested for each category of message within the SQUAD approach to CMC discourse together with the participation and interaction indicators provides a quick method for addressing aspects of quality of what the author has termed as "online learning levels of engagement".

Future extension to this research will be to develop the theoretical framework to include social and metacognitive elements within a CMC message, and to develop a dedicated tool for students to use when adopting the SQUAD approach to CMC discourse.

References

Austin, J. L. (1962). How to do things with words. New York: Oxford University Press.

Baker, M. J. (1994). A model for negotiation in teaching-learning dialogues. Journal of Artificial Intelligence in Education, 5(2), 199-254.

Barrows, H. (1996). Problem-based learning in medicine and beyond: A brief overview. In L. Wilkerson and W. Gijselaers (Eds), Bringing Problem-Based Learning to Higher Education: Theory and Practice. New Directions for Teaching and Learning, 68, 3-11. San Francisco: Jossey-bass Publishers.

Barros, B. & Verdejo, F. (2000). Analyzing student interaction process in order to improve collaboration. The DEGREE approach. International Journal of Artificial Intelligence in Education, 11, , 221-241. [verified 9 Nov 2003] http://computing.unn.ac.uk/staff/cgpb4/ijaied/members00/archive/vol_11/barros/full.html

Boehm, B., Bose, P., Horowitz, E. & Lee M. J. (1995). Software requirements negotiation and renegotiation aids: A theory-W based spiral approach. IEEE Proceedings for the 17th ICSE Conference, 1995.

Bridges, E. M. (1992). Problem-based learning for administrators. ERIC Clearing House, University of Oregon.

Brine, J. & Shannon, S. (1994). Consolidating professional skills and developing the confidence of graduating students. In S. E. Chen, R. M. Cowdry, A. Kingsland & M. J. Ostwald (Eds), Reflections on Problem Based Learning. Sydney, Australia: Wild & Wooley Pty Ltd.

Bruer, H. S. (1993). Schools for Thought: A Science of Learning in the Classroom. Cambridge, MA: MIT Press.

Cook, J. (2001). Bridging the gap between empirical data on open-ended tutorial interactions and computational models. International Journal of Artificial Intelligence in Education, 12, 85-99. [verified 27 Oct 2003] http://computing.unn.ac.uk/staff/cgpb4/ijaied/members01/archive/vol_12/cook/full.html

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurements, 20, 37-46.

Duffy, T., Dueber, B. & Hawley, C. (1998). Critical thinking in a distributed environment: A pedagogical base for the design of conferencing systems. In C. Bonk & K. King (Eds). Electronic contributors: Learner-centered technologies for literacy, apprenticeship, and discourse (pp. 51-78). New Jersey: Erlbaum.

Fahy, P. J., Crawford, G., Ally, M., Cookson, P., Keller, V. & Prosser, F. (2000). The development and testing of a tool for analysis of computer mediated conferencing transcripts. Alberta Journal of Education Research, 46(1), 85-88.

Fahy, P. J. (2001). Addressing some common problems in transcript analysis, International Review of Research in Open and Distance Learning, 1(2). http://www.irrodl.org/content/v1.2/research.html#Fahy [viewed 24 Mar 2003, verified 18 Sep 2003]

Foreman, J. & T. Johnston. 1999. Key aspects of teaching and learning in business and management studies. In H. Fry, S. Ketteridge, and S. Marshall (Eds), A Handbook for Teaching and Learning in Higher Education, 372-390. London: Kogan Page.

Hara, N., Bonk, C. & Angeli, C. (2000). Content analysis of online discussion in an applied educational psychology course. Instructional Science, 28(2), 115-152.

Henri, F. (1992). Computer conferencing and content analysis. In A. Kaye (Ed), Collaborative learning through computer conferencing: The Najaden papers, pp 117-136. London: Springer-Verlag.

Howell-Richardson, C. & Mellar, H. (1996). A methodology for the analysis of patterns of participation within computer mediated courses. Instructional Science, 24, 47-69.

Hutton & Wiesenberg, (2000). Quality online participation: Learning in CMC classroom. RCVET Working Knowledge Conference Papers. Research Centre for Vocational Education and Training, University of Technology, Sydney, Australia, 10-13 Dec 2000. [viewed Mar 2003, verified 18 Sep 2003] http://www.rcvet.uts.edu.au/wkconference/working%20knowledge64.pdf

Mason, R. & Romiszowski, A. (1996). Analysing computer conferencing interactions. Computers in Adult Education and Training, 2(3), 161-173.

McDonald, J. (1998). Interpersonal group dynamics and development in computer conferencing: The rest of the story. In Proceedings of 14th Annual Conference on Distance Teaching and Learning, pp. 243-48. Madison, WI: University of Wisconsin-Madison [ERIC Document ED422864]

Oriogun, P. K. & Georgiadou, E. (2000). Towards making learning agents think of their roles in the learning process through a problem-based learning grid. Proceedings, 2nd Asia Pacific conference on PBL, Singapore, 4-7 Dec 2000. [verified 18 Sep 2003] http://www.tp.edu.sg/pblconference/full/OriogunGeorgiadouPaper.pdf

Oriogun, P. K., French, F. & Haynes, R. (2002). Using the enhanced Problem-Based Learning Grid: Three multimedia case studies. In A. Williamson, C. Gunn, A. Young & T. Clear (Eds), Winds of Change in the Sea of Learning: Proceedings of the ASCILITE Conference. Auckland, New Zealand: UNITEC Institute of Technology, 8-11 December 2002, pp495-504. http://www.ascilite.org.au/conferences/auckland02/proceedings/papers/040.pdf

Oriogun, P. K. (2003). Content analysis of online inter-rater reliability using the transcript reliability cleaning percentage: A software engineering case study. Presented at the ICEIS 2003 Conference, Angers, France, 23-26 April 2003, pp.296-307, ISBN 972-98816-1-8.

Pilkington, R. (2001). Analysing educational dialogue interaction: Towards models that support learning (Introduction to the IJAIED Special Issue on Analysing Educational Dialogue). International Journal of Artificial Intelligence in Education, 12, 1-7. [verified 9 Nov 2003] http://computing.unn.ac.uk/staff/cgpb4/ijaied/members01/archive/vol_12/pilkington/full.html

Pilkington, R. M. (1999). Analysing Educational Discourse: The DISCOUNT Scheme (CBLU Technical Report 99/2). Leeds: Computer Based Learning Unit, The University of Leeds, Leeds, LS2 9JT.

Ryder, M. (1994). Augmentation of the intellect: Network instruments, environments and strategies for learning. [viewed 28 Jan 2003, verified 18 Sep 2003] http://carbon.cudenver.edu/~mryder/augment.html

Searle, J. R. (1969). Speech Acts: An essay in the philosophy of language. Cambridge: Cambridge University Press.

Strauss, A. and Corbin, J. (1990). Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Sage: London.

Vygotsky, L. S. (1962). Thoughts and Language. Cambridge, Mass: MIT.

Wee, K. N., Kek, Y.C., & Sim H. C. (2001). Crafting effective problems for problem based learning. In P. Little & P. Kandlbinder (Eds), The Power of Problem Based Learning, 157-168. ISBN 0 7259 11387. Australia: Australian Problem Based Learning Network.

Weiss, R. & Morrison, G. (1998). Evaluation of a graduate seminar conducted by listserv. [ERIC Document Reproduction Service, ED 423868]

Appendix: Messages sent by students

MESSAGE 14
From: "student 2" <student2_email>
Date: Wed, 02 Oct 2002 07:26:15 +0000
Subject: S -Re: [im54p21] Re: U - Meeting

He is the customer so he would need to answer questions that you
would ask any customer. It is a fairly vague specification so it is a
very good idea to go back to him with a list of options.

MESSAGE 173
From: "student 1" <student1 _email>
Date: Tue Nov 19, 2002 11:15 am
Subject: Q - Formatting messages

Dear all,
This is a petty question, but...
When we paste emails into the document - should we include the
original message, if a message is a reply?
I would say not, but we need to be consistent. Any thoughts?
student 1

MESSAGE 3
From: "student 3" <student 3_email>
Date: Wed, 25 Sep 2002 22:10:11 -0000
Subject: U - [im54p21] Re: Help!

Was having similar problems earlier on, but everything seems to be OK now.
Regards
Student 3

MESSAGE 141
From: student 4 <student 4_email>
Date: Tue Nov 12, 2002 1:15 pm
Subject: A - web pages

Student 2 / student 5,
Both login and registration will have one page. The site will allow
access and give message post/reply functions to both message streams
(undergrad and postgrad) for any registered user.
Hope that answers it. Any chance of emailing draft html pages so
I can link some code to it? Thanks
Student 4

MESSAGE 76
From: student 5 <student 5_email>
Date: Tue Oct 15, 2002 7:52 am
Subject: D - an explanation as to life cycle plans

Hi all
continuing my very busy morning, I have now put a file on with my
explanation for the life cycle plan I hope that it is readable, for this time
of the morning - I hope that this busy morning is taken into consideration
when I am completely incoherent at this evenings meeting
can't we do genetic engineering instead ?
student 5

Author: Peter K. Oriogun
Learning Technology Research Institute
Department of Computing, Communications Technology and Mathematics
London Metropolitan University
166-220 Holloway Road, London N7 8DB
Email: p.oriogun@londonmet.ac.uk

Please cite as: Oriogun, P. K. (2003). Towards understanding online learning levels of engagement using the SQUAD approach to CMC discourse. Australian Journal of Educational Technology, 19(3), 371-387. http://www.ascilite.org.au/ajet/ajet19/oriogun.html

[ AJET 19 ] [ AJET home ]
HTML Editor: Roger Atkinson [rjatkinson@bigpond.com]
This URL: http://www.ascilite.org.au/ajet/ajet19/oriogun.html
Created 14 Nov 2003. Last revised 14 Nov 2003.
Page access count

Reasoning skills	Definitions
Elementary clarification	Observing or studying a problem, identifying its elements, and observing their linkages in order to come to a basic understanding.
In depth clarification	Analysing and understanding a problem to come to an understanding which sheds light on the values, beliefs, and assumptions which underlie the statement of the problem.
Inferencing	Induction and deduction, admitting or proposing an idea on the basis of its link with propositions already admitted as true.
Judgement	Making decisions, statements, appreciations, evaluations and criticisms. Sizing up.
Application of strategies	Proposing coordinated actions for the application of a solution, or following through on a choice or a decision.

Message Category	Very Low	Low	Nominal	High	Very High
S - Suggestion				X
Q -Question		X
U -Unclassified	X
A -Answer			X
D -Delivery					X

Message category	Description	Example	Cognitive indicators	Participation indicators	Interaction indicators
S Suggestion	The process whereby the mere presentation of an idea to a receptive individual leads to the acceptance of the idea.	Students engage with other students within their coursework groups by offering advice, a viewpoint, or an alternative viewpoint to a current one.	-Elementary classification -In depth classification -Inferencing -Judgement -Application of strategies	Students engaging other students actively by taking a lead role in online discourse by posting meaningful and relevant messages to the group.	The message will be accessed and processed by other members of the group for the cycle of communication to complete.
Q Question	A form of word address to a person in order to elicit information or evoke a response.	Students may seek clarification from the tutor or other students in order to make appropriate decisions relating to the group coursework.	-Elementary classification -In-depth classification	The message is posed in such a way that some or all the group members will engage in the ongoing discussion.	The message will be accessed and processed by other members of the group for the cycle of communication to complete.
U Unclassified	Not in the list of categories of messages stipulated by the instigator of the task at hand.	This tends to happen at the start of the online postings. Students may be unsure of what the message is suppose to convey. In most cases, it falls within one of the four classified categories.	-Elementary classification	This type of message may or may not engage other students. In most cases, the message could be re-aligned to fall within the four classified categories by the coder of the final transcript at the end of semester.	This type of message may or may not engage other students. In most cases, the message could be re-aligned to fall within the four classified categories by the coder of the final transcript at the end of semester.
A Answer	Reply, either spoken or written, as to a question, request, letter or article.	Students are expected to respond to this type of message with a range of possible solutions / alternatives.	-Elementary classification -In depth classification -Inferencing -Judgement	Responding to a query or question will inevitably involve most, if not all the group members, especially if the response is not in line with other group members' opinions.	All group members are expected to deliver parts of the final product by working collaboratively.
D Delivery	The act of distribution of goods, mail etc.	Students are expected to produce a piece of software at the end of the semester. They all have to participate in delivering aspects of the artefacts making up the software.	-Elementary classification -In-depth classification -Inferencing -Judgement -Application of strategies	Each member of the group is expected to play active role in delivering parts of the artefact making up the final software product. This is also expressed in the marking scheme for the module.	All group members are expected to deliver parts of the final product by working together collaboratively.