Australasian Journal of Educational Technology
2011, 27(4), 581-599. 		AJET 27

Evaluation and improvement of student's question-posing ability in a web-based learning environment

Yu-Feng Lan and Pin-Chuan Lin
National Formosa University

With the rapid development of web-based learning environments, question-posing activities have become an important teaching and learning mode. However, this learning approach has some difficulties, one being the lack of a practical approach to assist teachers in evaluating the question-posing ability of all learners. To remedy this problem, the present study developed a system of integrating a reward mechanism into assessment activities, to measure the question-posing ability of individual students in a web-based learning environment. The system has been applied to a learning activity in a programming design course and 100 first-year college students were invited to participate in this research. After the experiment, the results showed the proposed system can serve as both a learning and an assessment tool in higher education, by encouraging students to carry out active learning, constructive criticism and knowledge sharing. A positive satisfaction is derived from the learning activities with the system under two dimensions of perceived usefulness and perceived ease of use. Most importantly, students' learning performance improved significantly.

Introduction

With the rapid development of e-learning, web-based learning activities are gaining popularity as a promising learning strategy (Potelle & Rouet, 2003; Wilson & Stacey, 2004). Several studies have reported the benefits of such learning activities (Dewiyanti, Brand-Gruwel, Jochems & Broers, 2007; Gray, Thompson, Sheard, Clerehan & Hamilton, 2010; Kilic-Cakmak, 2010), including the ability to improve communication and collaborative interaction, provide more learning opportunities for geographically dispersed users, encourage active learning, and enhance the learner's feedback process. Particularly, one of the most important web-based learning activities is question-posing and this method provides an instruction mode for both teaching and learning (Yu, Liu & Chan, 2005).

Essentially, question-posing as a learning activity has long been recognised as a useful strategy to improve knowledge creation and sharing (Barak & Rafaeli, 2004; Yu et al., 2005; Denny, Hamer, Luxton-Reilly & Purchase, 2008). Prior studies suggested engaging in the process of question-posing can help students' cognitive growth (Topping & Ehly, 2001; English, 1998). Additionally, from theoretical perspectives such as information processing theory (Gagne, 1985; Shiffrin & Atkinson, 1969), social construction of knowledge (King, 1989; Piaget, 1962) and social modeling theory (Bandura, 1986), researchers have provided evidence that can help explain why question-posing learning activity would be beneficial and conducive to learning. Most importantly, challenging students to assume an active role in posing questions can promote independence in learning (Bruner, 1990; Marbach-Ad & Sokolove, 2000).

Based on question-posing learning activities, several researchers have developed various support systems to improve the quality of teaching and learning in such learning environments, including QSIA (Rafaeli, Barak, Dan-Gur & Toch, 2004), QPPA (Yu et al., 2005), PeerWise (Denny et al., 2008), and Concerto II (Hazeyama & Hirai, 2009), and have examined practical issues that were faced when integrating the learning activity into an instructional method such as who is the main question poser (e.g., instructors or learners), who will be involved in assessment and what are the differences of scoring between assessors (e.g., self, peer and teacher), what modes of posing content are allowed (e.g., text, figure and table), and how to provide communication support (e.g., group review, asynchronous discussion forum or BBS) (Hirai & Hazeyama, 2009; Denny et al., 2009; Purchase, Hamer, Denny & Luxton-Reilly, 2010).

Generally, most of the developed systems emphasised that these question-posing activities could be seen as social collaborative learning, which involves instructors and students creating and sharing learning resources/experiences, and encourages participants to both contribute to the learning of their peers and to value their peers' contributions. As a result, several benefits were reported according to the findings of the above research. These include the ability to: (1) develop communication, teamwork and peer and self assessment skills, (2) improve higher order cognitive processes such as evaluation, reflection and critical thinking, (3) encourage active learning, (4) foster deep learning, and (5) enhance learner's feedback process (Barak & Rafaeli, 2004; Fellenz, 2004; Rafaeli et al., 2004; Yu et al., 2005).

Nevertheless, implementation of the learning activity is not totally without problems. For example, efforts to improve assessment processes focused mostly on self and peer assessments. However, the issue of considering the role of the teacher/expert to improve the process has attracted relatively little attention, and many students doubt their ability to provide constructive feedback and grade appropriately (Ballantyne et al., 2002). Besides, the processes of evaluating question-posing ability are over-generalised and simplified in many studies (Barak & Rafaeli, 2004; Yu et al., 2005; Hazeyama & Hirai, 2009). Measuring and understanding the learner's question-posing ability is important because it can help curriculum developers and designers provide adaptive instructions to enhance the quality of the learning process, and facilitate instructors in understanding the learning status of individual students. Most importantly, whether a correlation exists between students' question-posing ability and their cognitive ability is unclear (Dori & Herscovitz, 1999; Denny et al., 2008). Therefore, finding ways to evaluate and improve students' question-posing ability is worth examining (Yu et al., 2005).

A novel method of merging multi-modes of assessment and reward mechanism in a web-based learning system was examined in this study. It involved a knowledge sharing assignment where students were required to contribute question items for reflective learning and rank their peers' contributions in a way that each learner's question-posing ability can be automatically calculated. In this study, the question-posing ability is defined as the ability of students to obtain information on a learning topic by themselves using their own cognitive strategies, collaborative learning, and ideas exchanged with peers and instructor. The value of a posed question is evaluated according to the preferences of oneself, peers, and the instructor.

Purpose of the study

The aims of the present study are to develop and implement a web-based learning system, called the Question-Posing Indicators Service (QPIS) System, designed to assist teachers in evaluating the question-posing ability of individual students, and enhance the reflective thinking and quality of learning activities among the instructor, student and peers. In view of the above, this research tries to answer the following research questions: In the following sections, an overview of the proposed system is presented. Second, the research methodologies are described. Third, the findings for the research questions are presented. Finally, we discuss the implications of our findings for future research.

Overview of the proposed system

System architecture

The design of the QPIS system is presented in Figure 1. The system server uses Windows Server 2003 as its operating system and Microsoft SQL Server 2005 as its database server. As can be seen, the system comprises three modules: question-posing module, tool module, and assessment module. With respect to the teacher interface, an instructor can use the assessment module (i.e., expert assessment) to evaluate students' question-posing contents.

Figure 1

Figure 1: The design of the QPIS system

For the student interface, students can use the question-posing module to pose a question, use the tool module to search self or peers' question-posing contents, and evaluate his/her and peers' posed contents. Finally, for the peer interface, peers can use the tool module to give critical comments and evaluate the posed questions value (i.e., peer assessment). More specifically, each module is described as follows.

Modules of the QPIS system

Question-posing module
In traditional teaching, questions are privately owned and displayed by the teachers. Dillon (1990) suggested that questions should come from both teacher and students. Therefore, in this study, the function of the question-posing module is to require students who participate in the learning activity to develop questions related to the course learning contents according to their learning and reflective thinking. The idea of this function is to encourage learners to participate actively in knowledge construction from different aspects. Additionally, from a knowledge sharing perspective, students first have to develop their own knowledge and then contribute it for the benefit of others.

As shown in Figure 2, the students were asked to select a type of question and then provide a question and its corresponding solution in the construction area. After finishing the question-posing, students were required to assess the question they had constructed (i.e., self assessment).

Figure 2

Figure 2: Screenshot of the question-posing function

Tool module
The QPIS system provides a tool module for students to search the posed question and give comments to their peers. The purpose of this function is to assist students in improving their problem-solving ability. During the learning activities, students can fully refer to the posed questions of peers.

With the search function, students can search the related authoring of question items they want from the internal database by entering some keywords (see Figure 3). In addition, students can also browse their peers' question of interest through selecting a specific student's ID and then the posed questions titles would be shown. When they clicked on the specific question title, the corresponding contents were presented (see Figure 3).

Assessment module
Online assessment is one of the key elements of the teaching and learning process (Byrnes & Ellis, 2006). The purposes of providing the assessment module are to assist the instructor in evaluating the question-posing ability of individual students, enhance meaningful learning by implementing a question-posing learning activity, encourage interactive learning by distributing the students' posed questions and answers on the web, and facilitate self and peer assessments.

In regard to how to evaluate the knowledge value for a specific posed question, the content usefulness and content richness instruments developed by Davis (1989) and Daft, Lengel & Trevino (1987) were adapted to measure this construct and support self, peer and expert assessments accordingly. Particularly, the content richness has been argued to play an important role in shared meaning and understanding (Sun & Cheng, 2007). Generally, multimedia content could provide diverse interactive capabilities and presentations of media types (such as picture, animation and audio). Prior research suggested that multimedia content can influence learners' levels of understanding (Chen, Ghinea & Macredie, 2006). Thus, it is expected that posed content may provide less meaning when it is only presented in text based mode, compared with multimedia content.

Regarding how to objectively evaluate the usefulness and content richness of a posed question for assessors, in this study each question created by learners is scored on a five-point Likert scale with six items (one is strongly disagree and five is strongly agree), which is represented by explicit descriptions for easy understanding instead of just a number score. The contents of the six items are: (1) "The posed question makes it easier to learn and reflect", (2) "The posed question increases my understanding", (3) "The posed question enhances the effectiveness of my learning", (4) "The posed question is only presented in text-based mode", (5) "The posed question consists of image/annotation types", and (6) "Overall, I find the posed question contains various media types (e.g., text, image, or annotation)" as shown in the bottom of Figure 3. Note the result of item 4 needs to be reversed (i.e., ans = 6 - ans) according to the content richness, as mentioned above. An average of these items is calculated to determine the knowledge value of a specific question-posing.

Figure 3

Figure 3: Example of using the QPIS system to search for posed questions

Further, to measure and understand the level of a learner's question-posing ability, the present study develops four sub-modules to evaluate the indicators of question-posing ability. They comprised a question type analysis agent, and self, peer and expert assessment modules, which are described as follows.

Question type analysis agent
Regarding the relationship between thinking skills and the type of question-posing, Barak and Rafaeli (2004) modified the Bloom taxonomy (Bloom, Engelhart, Furst, Hill & Krathwohl, 1956) as three categories: lower order thinking skills (i.e., true/false questions), intermediate order thinking skills (i.e., multiple choice questions), and higher order thinking skills (i.e., matching and short answer questions). The proposed assessment of the question type analysis agent was adopted above for evaluating students' question-posing abilities. In this study, students were required to identify which type of question-posing they created. Additionally, the instructor could encourage students to develop specific type of questions by giving a higher score in the assessment activities.

Self and peer assessment modules
Larisey (1994) indicated higher education students should be given opportunities for self-directed learning and critical reflection to mirror the world of learning beyond formal education. According to the above view, the self assessment could enable students to find their learning strengths and weaknesses (Peat, Franklin, Devlin & Charles, 2005; Oliver & Omari, 1999). In contrast to self assessment, Williams (1992) suggested that peer assessment seems to motivate deeper learning and produces better learning outcomes. Moreover, some researchers indicated students positively value the peer assessment and it is possible to resolve a substantial number of their questions (Rosmalen et al., 2008). In short, peer assessment intends to enhance learning through critically evaluating others' works and synthesising comments given by their peers (Yu, et al., 2005). According to the above view, in this study the instructor encourages students to participate in both assessment activities.

Expert assessment module
The expert assessment module enables teachers to evaluate the quality of question-posing by students and browse the results analysis regarding the evaluation of the question-posing ability of individual students. Additionally, Chen et al., (2009) pointed out that experts could help students improve their thinking by constructively commenting on their learning outcomes. Therefore, in this study, the instructor plays the role of expert, and also participates in evaluating learners' question-posing ability, which further improves their learning performance.

Indicators of question-posing ability

In this study, question-posing ability was calculated by a scoring mechanism according to the above assessment modules. The aim of the scoring mechanism is to assist the teacher in evaluating the question-posing ability of individual students. With reference to indicators of question-posing ability perspective, the teacher can understand precisely the posed question value of individual students. Both definitions and formulations of the proposed indicators involve various rules of evaluating posed questions in the QPIS system according to the learner's learning status, interaction, and performance (see Appendix).

In general, regarding the assigning of an item-weighted ratio of assessment modules (i.e., tw, sw and pw) and the type of question-posing (i.e., qw), the instructor can encourage students to pose specific types of question or prompt them to participate in a specific assessment activity by setting a higher corresponding item-weighted ratio.

Apart from the amount of question-posing, the scoring mechanism primarily depends on the weights assigned to each type of question-posing and expert, self, and peer assessments. The weights assigned can be viewed as a kind of reward mechanism. More details concerning definitions and formulations of the indicators of question-posing ability are presented in the Appendix, where the study also provides an illustrative example to explain the entire calculation process in greater detail.

Methods

Sample

The participants were 100 first year undergraduates, majoring in the Department of Information Management and taking a Basic Programming Design course. The participants were randomly divided into two groups, namely the experimental group and control group. The control group students (N=50) received a traditional teaching approach. That is, all the study sessions were conducted in a classroom equipped with personal computers and an LCD projector. The experimental group's treatment was similar to the control group's, but included the QPIS system for teaching and learning.

Experimental tools and materials

To increase the reliability of the experiment, all participants learned the same course materials, which were given by the instructor. Experimental group students were required to pose questions for reflective thinking according to their learning during and after class. The instruction tools included J2SE SDK 5.0 and UltraEdit v13.10 for students to learn and create their work. After finishing their work, learners were asked to pose questions based on their reflection in the QPIS system.

Procedures

Regarding the use of the QPIS for web-based learning, Figure 4 shows the learning activity procedures in this study. The experiment was conducted over an eight week period and executed once every two weeks for an online examination as cognitive ability grades. Actually, the online examination tested student's cognitive ability, and the final cognitive ability grade was the average of the four online examination grades on a 0-100 scale. All online examinations used 40 randomly extracted questions (i.e., true/false 25%; multiple-choice 25%; matching 25%; and short-answer 25%), constructed by the students and all stored in the QPIS system.

During the eight weeks, the experimental group students used the QPIS system to develop questions related to the learning topics. In addition, the participants could assess and comment on their own and peers' questions in the QPIS system. Essentially, this process attempted to enhance the reflective and critical thinking of individual students.

Besides, in this study, we aimed to measure whether assigning a higher score for specific type of question-posing could stimulate the reflective thinking of individual students. Students were constantly encouraged by their instructor to develop higher order thinking questions (i.e., matching and short answer questions) in the third week.

Concerning the assessment activities, all questions posed by each learner were evaluated by at least five randomly assigned peers. That is, when entering the QPIS system, learners will find the system randomly assigning peer IDs for them to evaluate. For self assessment, learners began by conducting a self assessment when they posed a question. By evaluating learners' posed questions from the assessment modules, the QPIS system will automatically calculate the indicators of question-posing ability of the individual students.

After eight weeks, the experimental group students were asked to fill out a feedback questionnaire for measuring their attitudes towards the use of the QPIS system. A post-test of learning performance and knowledge of the course was conducted for both the experimental and control groups. System logs and question-posing content were also extracted for analysis.

Figure 4

Figure 4: Learners' procedures for using the QPIS system in the experiment

Data analysis and results

This section consists of four parts. Each part relates to a certain research question and presents data for its answer.

The changes in the students' pre- and post-test scores between groups

In this research, the pre-test scores represent the entry behaviour of learning, and the post-test scores represent the learning effects. Before the experiment, to measure if there was a difference between the groups in terms of learners' prior knowledge of Basic Programming Design concepts, an independent sample t-test was conducted for the pre-test scores. The result indicated no significant difference was found between the groups (t = 0.32, p = 0.75 > 0.05), implying the participants of both groups did not differ in their initial prior knowledge.

After learning for an eight week period, the study administrated a post-test to examine whether there was difference for each group. As shown in Table 1 and Table 2, both groups showed significant differences in paired t-test. These imply both teaching strategies assisted students in learning.

Table 1: Results of the paired t-test for the pre- and post-test of the experimental group

MeasureNPre-test meanSDPost-test meanSDt
Knowledge of the course5034.0014.1465.9010.62-18.11*
*p<0.05

Table 2: Results of the paired t-test for the pre- and post-test of the control group

MeasureNPre-test meanSDPost-test meanSDt
Knowledge of the course5033.1013.8056.8011.19-16.60*
*p<0.05

Moreover, the study attempted to further examine whether the experimental group could really enhance the students' learning performance more than the control group. The analysis results revealed there was a significant difference in post-test performance between the two groups (t = 4.169, p < 0.05). In other words, this result indicated the QPIS system would benefit students more.

Analysis of different types of question-posing

As shown in Figure 5, this study analysed the system logs of learners' type of question-posing for eight weeks. The results showed the types of posed questions were generally "true/false" and multiple choice in the first to third week. This trend may imply, in the beginning, learners were not familiar with the course content and the initial reflective thinking was insufficient. To measure whether assigning a higher score for specific types of question-posing could stimulate the reflective thinking of individual students, the instructor tried to provide higher scores in encouraging students to develop more higher order thinking questions (i.e., matching and short answer). Interestingly, the result indicated an increasing amount of matching and short answer questions after the third week. This change may indicate instructors can enhance students' reflective thinking by providing more encouragement and rewards.

Figure 5

Figure 5: The percentage of different types of question-posing

Additionally, by using the QPIS system, students have more opportunities to interact with peers and the instructor. After more reflective thinking and peer viewing, students had more foundational knowledge to support the question-posing learning activities. On the whole, there was increased question-posing for higher order thinking among learners after using the QPIS system.

The relationship between question-posing ability and cognitive ability

The third research question was: What is the relationship between question-posing ability and cognitive ability? Table 3 presents a descriptive analysis of the indicators of question-posing ability and cognitive ability grades.

Table 3: Descriptive analysis of the indicators of question-posing ability and cognitive ability grades

Research instrumentNMinimumMaximumMeanSD
Indicators of question-posing ability5012.4560.5034.8712.96
Cognitive ability grades5040.0095.0070.9612.84

When using the QPIS system, repeated self, peer and expert assessments ultimately determined the students' indicators of question-posing ability. In this study, the relationship between the students' indicators of question-posing ability and their cognitive ability grades is of central interest. A linear regression analysis was performed to examine whether the indicators of question-posing ability predicted the cognitive ability grades. The indicators of question-posing ability were defined as the independent variable and the cognitive ability grades were defined as the dependent variable. The statistical analysis showed a significant relationship (beta = 0.68, p < 0.01). This result may imply with the QPIS system (1) the students who performed well on their question creation and sharing, had greater success in their cognitive ability grades compared to their counter peers, and (2) the students could reflect on comprehensive thinking by themselves using their own cognitive strategies, collaborative learning, and ideas exchanged with peers and the instructor.

Questionnaire analysis

Reliability and validity of the questionnaire
To analyse the students' attitudes toward using the QPIS system, a Likert style five-point (Strongly agree, Agree, Undecided, Disagree, Strongly disagree) questionnaire was conducted. The perceived usefulness and ease of use were measured by seven and five items, respectively, adopted following the recommendation of Davis (1989). Our study utilised Cronbach's alpha to evaluate the internal consistency of each dimension of the questionnaire. The Cronbach's alpha values for two attitudes were 0.89 and 0.91, respectively. These values show the high reliability of the measurement. Regarding the validity of the questionnaire, prior studies (Sun & Cheng, 2007; Lee, 2010; Wu, et al., 2010) have demonstrated the validity of the adopted instruments of this study.

Results of questionnaire analysis
For the perceived usefulness listed in Table 4, on average, students reported moderate learning attitudes toward the usefulness of QPIS system. They indicated the use of QPIS saved time and allowed tasks to be completed more quickly. When asked about the learning aspect of QPIS system, students agreed with the statement that using QPIS enhanced their effectiveness in question-posing; they also agreed that using QPIS improved the quality of question-posing. Regarding the perception of peer assessments, students strongly agreed that peers evaluated fairly the posed questions.

Table 4: Perceived usefulness of the QPIS system

No.QuestionStrongly
agree		AgreeUndecidedDisagreeStrongly
disagree		Mean
score		Std.
dev.
1Using the QPIS system saved me time.7
(14.00%)		19
(38.00%)		16
(32.00%)		8
(16.00%)		0
(0.00%)		3.500.93
2The QPIS system enabled me to accomplish tasks more quickly.7
(14.00%)		17
(34.00%)		17
(34.00%)		8
(16.00%)		1
(2.00%)		3.420.99
3Using the QPIS system enhanced my effectiveness in question-posing.7
(14.00%)		25
(50.00%)		18
(36.00%)		0
(0.00%)		0
(0.00%)		3.780.68
4Using the QPIS system improved the quality of question-posing.8
(16.00%)		24
(48.00%)		18
(36.00%)		0
(0.00%)		0
(0.00%)		3.800.70
5Using the QPIS system made it easier to learn.6
(12.00%)		20
(40.00%)		19
(38.00%)		5
(10.00%)		0
(0.00%)		3.540.84
6I think the peer assessments were objective.27
(54.00%)		18
(36.00%)		5
(10.00%)		0
(0.00%)		0
(0.00%)		4.440.67
7Overall, I found the QPIS system was useful for my learning.15
(30.00%)		22
(44.00%)		8
(16.00%)		5
(10.00%)		0
(0.00%)		3.940.93

Regarding the perceived ease of use (see Table 5), students noted they were rarely confused and rarely made errors when using the QPIS system. Overall, they thought the QPIS system was easy to use.

Table 5: Perceived ease of use of the QPIS system

No.QuestionStrongly
agree		AgreeUndecidedDisagreeStrongly
disagree		Mean
score		Std.
dev.
1I was rarely confused when using the QPIS system.22
(44.00%)		20
(40.00%)		7
(14.00%)		1
(2.00%)		0
(0.00%)		4.260.78
2I rarely made errors when using the QPIS system.23
(46.00%)		20
(40.00%)		5
(10.00%)		2
(4.00%)		0
(0.00%)		4.280.81
3I found it easy to get the QPIS system to do what I wanted it to do.20
(40.00%)		21
(42.00%)		8
(16.00%)		1
(2.00%)		0
(0.00%)		4.200.78
4It was easy for me to understand how to perform tasks while using the QPIS system.10
(20.00%)		22
(44.00%)		18
(36.00%)		0
(0.00%)		0
(0.00%)		3.840.74
5Overall, I found the QPIS system easy to use.17
(34.00%)		22
(44.00%)		11
(22.00%)		0
(0.00%)		0
(0.00%)		4.120.75

Discussion

With the advent of networking and multimedia technologies, the emphasis on web-based education for supporting student learning has been a focus of contemporary education (Yu, et al., 2005). The present study developed the QPIS system to be applied in a web-based learning environment and to assist teachers in evaluating and understanding the individual's question-posing ability.

Particularly, the proposed system appeared to cultivate a motivating learning environment by including various aspects of motivation, such as knowledge sharing, multi-modes of assessment activities, and providing various rewards. Also, this study provided evidence consistent with prior research (Barak & Rafaeli, 2004), that question-posing and assessment modules can serve as both learning and assessment tools in higher education, by encouraging students to carry out active learning, constructive criticism and knowledge sharing.

To obtain a fair assessment and assist the instructors to understand learners' question-posing ability, in contrast to previous research (Barak & Rafaeli, 2004; Yu et al., 2005; Denny et al., 2008), in this study self, peer and expert assessments were considered simultaneously. Besides, this study integrated a reward mechanism into assessment activities, not only to arouse students' learning motivation, but also to develop a method to evaluate learner's question-posing ability. By scoring the learner's contribution of question-posing from the assessment modules, the QPIS system can automatically calculate the indicators of question-posing of individual students. These indicators can assist teachers to understand learner's learning status, interaction, and performance.

Research on types of question has traditionally distinguished between general or high-level questions and specific or low-level questions (Hamilton, 1985). In addition, several studies have found high level questions are more beneficial than low level questions for students' knowledge development (Andre, 1979; Barak & Rafaeli, 2004). Generally, the composed true/false question was easier for most students. In contrast, the matching and short-answer questions could be seen as higher-order thinking level questions (Barak & Rafaeli, 2004). Following the experiment, the results indicated a growing trend of posed questions in terms of higher-order thinking would gradually increase while using the QPIS system. In other words, students could compose more matching and short-answer questions than other types of question.

This study aimed to discover whether a learner's question-posing ability could be used effectively as an alternative evaluation tool for assessing learning outcomes. Based on our findings, students can improve their learning performance while using the QPIS system. Further, regarding the relationship between the learner's question-posing ability and cognitive ability, the result showed a greater question-posing ability can predict a higher cognitive ability of individual students. This finding is useful for both instructor and students in understanding learning status instantly.

Finally, the result of the questionnaire survey showed there are positive attitudes toward using the QPIS system as a question-posing learning environment. Although most students agreed on the perceived usefulness and ease of use of the proposed system, it still left a few students dissatisfied. A few students noted that using the QPIS system with question-posing took time and was not an easy task. The problem might be improved by providing a more user friendly interface and more rewards as incentives. Compared to Yu's, et al. (2005) result, indicating younger learners felt the question-posing learning activity was difficult, our study found undergraduate students enjoyed the learning activity and thought it can help their learning. One possible reason could be that the good qualities of question-posing require more reflective thinking and mature ideals, but younger learners do not have fully developed knowledge and ideals.

Conclusion

There are several important implications for practice. First, a better understanding of the indicators of learners' question-posing ability can enable instructors to promptly adapt their teaching strategies and materials, and provide appropriate support when needed. Second, the instructor might encourage students to develop specific type of question by setting a higher corresponding item-weighted ratio for the assessment activities. Third, students can follow the steps including prompting questions, answering questions, and viewing references to enhance knowledge and reflect on their learning. Further, we believe this research can provide valuable insights for teachers and researchers in terms of the strategy of question-posing in a web-based learning.

This study has some limitations. First, the experiment results were only from one course, more different courses could be considered. Second, learners' learning style was not measured. However, this limitation may lead to different degrees of learners' learning outcomes in the question-posing learning activity. Whether different learning styles will affect question-posing ability is unclear. Finally, it would be very interesting to explore deeper the affective impact of different reward strategies and its influence on the learners' motivation to participate. This issue is unclear and it might be another direction for future work.

References

Andre, T. (1979). Does answering higher level questions while reading facilitate productive reading? Review of Educational Research, 49(2), 280-318. http://dx.doi.org/10.3102/00346543049002280

Shiffrin, R. M. & Atkinson, R. L. (1969). Storage and retrieval processes in long-term memory. Psychological Review, 76(2), 179-193. http://dx.doi.org/10.1037/h0027277

Ballantyne, R., Hughes, K. & Mylonas, A. (2002). Developing procedures for implementing peer assessment in large classes using an action research process. Assessment & Evaluation in Higher Education, 27(5), 427-441. http://dx.doi.org/10.1080/0260293022000009302

Bandura, A. (1986). Social cognitive theory. New York: Holt, Rinehart & Winston.

Barak, M. & Rafaeli, S. (2004). On-line question-posing and peer-assessment as means for web-based knowledge sharing in learning. International Journal of Human-Computer Studies, 61(1), 84-103. http://dx.doi.org/10.1016/j.ijhcs.2003.12.005

Bloom, B. S., Engelhart, M. B., Furst, E. J., Hill, W.H. & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals, Handbook 1: The cognitive domain. Longmans Green, New York.

Bruner, J. S. (1990). Acts of meaning. Harvard University Press, Cambridge.

Byrnes, R. & Ellis, A. (2006). The prevalence and characteristics of online assessment in Australian universities. Australasian Journal of Educational Technology, 22(1), 104-125. http://www.ascilite.org.au/ajet/ajet22/byrnes.html

Chen, N. S., Wei, C. W., Wu, K. T. & Uden, L. (2009). Effects of high level prompts and peer assessment on online learners' reflection levels. Computers & Education, 52(2), 283-291. http://dx.doi.org/10.1016/10.1016/j.compedu.2008.08.007

Chen, S. Y., Ghinea, G. & Macredie, R. D. (2006). A cognitive approach to user perception of multimedia quality: An empirical investigation. International Journal of Human-Computer Studies, 64(12), 1200-1213. http://dx.doi.org/10.1016/j.ijhcs.2006.08.010

Daft, R. L., Lengel, R. H. & Trevino, L. K. (1987). Message equivocality, media selection, and manager performance: Implications for information systems. MIS Quarterly, 11(3), 355-366. [verified 8 July 2011] http://misq.org/message-equivocality-media-selection-and-manager-performance-implications-for-information-systems-1.html?SID=1sqqo71l97f95traavinsq2ap2

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and end user acceptance of information technology. MIS Quarterly, 13(3), 319-340. [verified 8 July] http://misq.org/perceived-usefulness-perceived-ease-of-use-and-user-acceptance-of-information-technology.html?SID=1sqqo71l97f95traavinsq2ap2

Denny, P., Hamer, J., Luxton-Reilly, A. & Purchase, H. (2008). PeerWise: Students sharing their multiple choice questions. Proceedings of the Fourth International Workshop on Computing Education Research (ICER 2008), Sydney, Australia, pp. 51-58. http://dx.doi.org/10.1145/1404520.1404526

Denny, P., Luxton-Reilly, A. & Simon, B. (2009). Quality of student contributed questions using PeerWise. Proceedings of the Eleventh Australasian Conference on Computing Education, Vol. 95 of CRPIT, Wellington, New Zealand, pp. 45-53. http://portal.acm.org/citation.cfm?id=1862724

Dewiyanti, S., Brand-Gruwel, S., Jochems, W. & Broers, N. J. (2007). Students' experience with collaborative learning in asynchronous computer-supported collaborative learning environments. Computers in Human Behavior, 23(1), 496-514. http://dx.doi.org/10.1016/j.chb.2004.10.021

Dillon, J. T. (1990). The practice of questioning. Routledge, London.

Dori, Y. J. & Herscovitz, O. (1999). Question-posing capability as an alternative evaluation method: Analysis of an environmental case study. Journal of Research in Science Teaching, 36(4), 411-430. http://dx.doi.org/10.1002/(SICI)1098-2736(199904)36:4<411::AID-TEA2>3.0.CO;2-E

English, L. D. (1998). Children's problem posing within formal and informal contexts. Journal for Research in Mathematics Education, 29(1), 83-106. http://www.nctm.org/eresources/article_summary.asp?URI=JRME1998-01-83a&from=B

Fellenz, M. R. (2004). Using assessment to support higher level learning: The multiple choice item development assignment. Assessment & Evaluation in Higher Education, 29(6), 703-719. http://dx.doi.org/10.1080/0260293042000227245

Gagne, R. M. (1985). The conditions of learning and theory of instruction (4th ed). New York: Holt, Rinehart and Winston.

Gray, K., Thompson, C., Sheard, J., Clerehan, R. & Hamilton, M. (2010). Students as Web 2.0 authors: Implications for assessment design and conduct. Australasian Journal of Educational Technology, 26 (1), 105-122. http://www.ascilite.org.au/ajet/ajet26/gray.html

Hamilton, R. J. (1985). A framework for the evaluation of the effectiveness of adjunct questions and objectives. Review of Educational Research, 55, 47-85. http://dx.doi.org/10.3102/00346543055001047

Hazeyama, A. & Hirai, Y. (2009). Concerto II: A collaborative learning support system based on question posing. In Hijon Neira, R. (Ed), Advanced learning. InTechOpen. http://www.intechopen.com/source/pdfs/8590/InTech-Concerto_ii_a_collaborative_learning_support_system_based_on_question_posing.pdf

Kilic-Cakmak, E. (2010). Learning strategies and motivational factors predicting information literacy self-efficacy of e-learners. Australasian Journal of Educational Technology, 26(2), 192-208. http://www.ascilite.org.au/ajet/ajet26/kilik-cakmak.html

Hirai, Y. & Hazeyama, A. (2007). A learning support system based on question-posing and its evaluation. Fifth International Conference on Creating, Connecting and Collaborating through Computing, pp. 180-185, Kyoto, Japan, January 2007, IEEE Computer Society, Los Alamitos, CA, USA. http://doi.ieeecomputersociety.org/10.1109/C5.2007.2

King, A. (1989). Verbal interaction and problem-solving within computer-assisted cooperative learning groups. Journal of Educational Computing Research, 5(1), 1-15. http://baywood.metapress.com/openurl.asp?genre=article&id=doi:10.2190/YNV2-QRB2-HUCN-DGJK

Larisey, M. M. (1994). Student self assessment: A tool for learning. Adult Learning, 5(6), 9-10.

Lee, M. C. (2010). Explaining and predicting users' continuance intention toward e-learning: An extension of the expectation-confirmation model. Computers & Education, 54(2), 506-516. http://dx.doi.org/10.1016/j.compedu.2009.09.002

Marbach-Ad, G. & Sokolove, P. G. (2000). Can undergraduate biology students learn to ask higher questions? Journal of Research in Science Teaching, 37(8), 854-870. http://dx.doi.org/10.1002/1098-2736(200010)37:8<854::AID-TEA6>3.0.CO;2-5

Oliver, R. & Omari, A. (1999). Using online technologies to support problem based learning: Learners' responses and perceptions. Australian Journal of Educational Technology, 15(1), 58-79. http://www.ascilite.org.au/ajet/ajet15/oliver.html

Piaget, J. (1926). The language and thought of the child. New York: Harcourt Brace.

Peat, M., Franklin, S., Devlin, M. & Charles, M. (2005). Revisiting the impact of formative assessment opportunities on student learning. Australasian Journal of Educational Technology, 21(1), 102-117. http://www.ascilite.org.au/ajet/ajet21/peat.html

Potelle, H. & Rouet, J. F. (2003). Effects of content representation and readers' prior knowledge on the comprehension of hypertext. International Journal of Human-Computer Studies, 58(3), 327-345. http://dx.doi.org/10.1016/S1071-5819(03)00016-8

Purchase, H., Hamer, J., Denny, P. & Luxton-Reilly, A. (2010). The quality of a PeerWise MCQ repository. Proceedings of the Twelfth Australasian Conference on Computing Education, Brisbane, Australia, pp. 137-146. http://portal.acm.org/citation.cfm?id=1862219.1862238

Rafaeli, S., Barak, M., Dan-Gur, Y., & Toch, E. (2004). QSIA - a web-based environment for learning, assessing and knowledge sharing in communities. Computers & Education, 43(3), 273-289. http://dx.doi.org/10.1016/j.compedu.2003.10.008

Sun, P. C. & Cheng, H. K. (2007). The design of instructional multimedia in e-Learning: A media richness theory-based approach. Computers & Education, 49(3), 662-676. http://dx.doi.org/10.1016/j.compedu.2005.11.016

Sung, Y. T., Chang, K. E., Chiou, S. K. & Hou, H. T. (2005). The design and application of a web-based self- and peer-assessment system. Computers & Education, 45(2), 187-202. http://dx.doi.org/10.1016/j.compedu.2004.07.002

Sung, Y. T., Lin, C. S., Lee, C. L. & Chang, K. E. (2003). Evaluating proposals for experiments: An application of web-based self-assessment and peer assessment. Teaching of Psychology, 30(4), 331-334. [verified 8 July 2011] http://www.psychology.heacademy.ac.uk/s.php?p=168&db=615

Topping, K. J. & Ehly, S. W. (2001). Peer assisted learning: A framework for consultation. Journal of Educational and Psychological Consultation, 12(2), 113-132. http://dx.doi.org/10.1207/S1532768XJEPC1202_03

Van Rosmalen, P., Sloep, P. B., Brouns, F., Kester, L., Berlanga, A., Bitter, M. & Koper, R. (2008). A model for online learner support based on selecting appropriate peer tutors. Journal of Computer Assisted Learning, 24(6), 483-493. http://dx.doi.org/10.1111/j.1365-2729.2008.00283.x

Williams, E. (1992). Student attitudes towards approaches to learning and assessment. Assessment & Evaluation in Higher Education, 17(1), 45-58. http://dx.doi.org/10.1080/0260293920170105

Wilson, G. & Stacey, E. (2004). Online interaction impacts on learning: Teaching the teachers to teach online. Australasian Journal of Educational Technology, 20(1), 33-48. http://www.ascilite.org.au/ajet/ajet20/wilson.html

Wu, J. H., Tennyson, R. D. & Hsia, T. L. (2010). A study of student satisfaction in a blended e-learning system environment. Computers & Education, 55(1), 155-164. http://dx.doi.org/10.1016/j.compedu.2009.12.012

Yu, F. Y., Liu, Y. H. & Chan, T. W. (2005). A web-based learning system for question-posing and peer assessment. Innovations in Education and Teaching International, 42(4), 337-348. [verified 8 July 2011] http://chan.lst.ncu.edu.tw/publications/2005-Yu-awl.pdf

Appendix

Appendix - part 1

Appendix - part 2

Authors: Dr Yu-Feng Lan (corresponding author)
Department of Information Management, National Formosa University
No. 64, Wunhua Road, Huwei Township, Yunlin County 632, Taiwan
Email: yflan@nfu.edu.tw Web: http://sparc.nfu.edu.tw/~yflan

Pin-Chuan Lin
Department of Information Management, National Formosa University, Taiwan

Please cite as: Lan, Y. F. & Lin, P. C. (2011). Evaluation and improvement of student's question-posing ability in a web-based learning environment. Australasian Journal of Educational Technology, 27(4), 581-599. http://www.ascilite.org.au/ajet/ajet27/lan.html

[ PDF version of this article ] [ AJET 27 ] [ AJET home ]
HTML Editor: Roger Atkinson [rjatkinson@bigpond.com]
This URL: http://www.ascilite.org.au/ajet/ajet27/lan.html
Created 10 Aug 2011. Last revised 10 Aug 2011.
Page access count