Australasian Journal of Educational Technology
2012, 28(5), 912-930. 		AJET 28

College students' intention to continue using a personal response system: Deriving a model from four theoretical perspectives

C. Rosa Yeh
National Taiwan Normal University
Yu-Hui Tao
National University of Kaohsiung

The use of personal response systems (PRS) in classrooms is gaining popularity in the higher education institutes of Taiwan. However, past research rarely adopts theories from the information system domains, and their focus was primarily on the UK and US context. Therefore, this study adopted a theory-based approach to explore the perceptions of Taiwanese college students on PRS continuance usage, incorporating a collection of related theories, including expectation-confirmation theory, information systems success model, motivation theory, and agency theory. As an initial foray into PRS adoption theories, this study aims to provide findings and implications that will enable future researchers to extend studies on PRS usage with a wider base of theoretical support.

Introduction

A personal response system (PRS), which is intended to improve classroom interaction, is a recent development in instructional technology. PRS is named differently in the literature; it is sometimes referred to as an audience response system, student response system, electronic response system, electronic voting system, or clickers (Freeman et al., 2007; Laxman, 2011; Lin et al., 2011). Since the commercialisation of PRS applications in 1999, several PRS review papers have been published. The most recent one is that by Kay and LaSage (2009), in which four earlier major review papers (Judson & Sawada, 2002; Fies & Marshall, 2006; Simpson & Oliver, 2007; Caldwell, 2007) were analysed and compared, leading Kay and LaSage to determine three categories of PRS benefits, three categories of PRS challenges, and four future research directions.

Even with the abundance of PRS review studies, two issues remain. First, several PRS studies involve theories in the education domain; it is hard to find studies that truly adopted the technology-based theories to explain student usage of such educational technology in the classroom. MacGeorge et al. (2008) is a close example. Through a data-driven approach, some relevant constructs of technology-based theories such as ease of use and enjoyment were derived. Meanwhile, both MacGeorge et al. (2008) and Kay and LeSage (2009a) pointed out a lack of systematic PRS research, as well as the observation that "data collection instruments are noticeably lacking in reliability and validity analysis." Second, regardless of an increasing trend of PRS application in the higher education system in other parts of the world, few related studies can be found outside Europe, America, and Australia. Notable exceptions are Laxman (2011) and Lin et al. (2011) who studied the systems in Singapore and Taiwan respectively. Extant literature had revealed significant differences among Western and Eastern students as shown in the meta-analysis of Yamazaki (2005). Learning styles differ among samples from different countries based on the Kolb Learning Styles Inventory. Specifically, Taiwanese students are similar to Hong Kong students, but very different from students in Canada, France, Australia, Germany, and USA. Therefore, students in Taiwan and Hong Kong are more likely to perceive PRS usage differently from their Western counterparts.

Based on these arguments, we believe there is a need to explore PRS with technology-based theories and relate them to educational theories via a more systematic approach. We believe that implementation strategies or practices should be customised to the specific circumstances of Taiwan to avoid possible cultural biases as a result of basing only upon the Western literature. The main research objective is to propose a theory-based research model fusing technology adoption with education theories for a more insightful understanding of PRS usage. To achieve this objective, an empirical study was conducted using university students in three courses in Taiwan as subjects.

Literature review

This study attempts to integrate theoretical frameworks from two different fields: the technologies and education. The theories used in this study include expectation-confirmation theory (ECT) and the information systems (IS) success model, which are often discussed in technology adoption studies, as well as motivation theory and agency theory, which are related to educational perspectives. Before introducing these four theories, a brief review on the current use of PRS in Taiwan is provided as a background for the study.

Development and use of PRS in Taiwan

In Taiwan, the earliest PRS with the name of EduClick I, was developed by the Learning Technology Laboratory at the National Central University in 1999 (Liu et al., 2003a). In 2002, an enhanced EduClick II was created to be used with the virtual city, called EduCities, to provide a network-supported environment. This new integrated environment allowed teachers to share and exchange instructional materials with each other, gave parents access to view their children's performance at school, and facilitated students' response to teacher's questioning and feedback activities during class. A further enhanced version, EduClick III, was developed in 2003 to support six teaching activities, including conducting tests, competing for right answers, playing elimination games, playing instant question and answer, and choosing the respondents. Its integrated software provides basic learning management functions, such as editing learning materials, managing the class, recording reports and organising reports.

Because the EduClick systems can greatly enhance the class interactions between teachers and students, the Taipei City Government in Taiwan installed PRS as an experiment in its elementary schools. Each school received two to nine sets of PRS; each set included a standard PRS software, one infrared receiver, one teacher remote control, and 35 student remote controls. Although the initial evaluations and later reports available from the Internet all revealed positive outcomes, an in-depth investigation by Chen and Tan (2008) indicated that the actual utilisation rate was not as expected, due to many implementation challenges faced by the adopting teachers who did not receive adequate training and support.

The commercialisation of EduClick was conducted by a collaborative partner, HABOOK Information Technology Inc. (http://www.habook.com.tw/english/), which has promoted PRS successfully to higher educational institutes in recent years. According to HABOOK's website, they have over 320 clients from elementary schools to colleges in Taiwan. Another local provider, Jector Digital System, Inc. (http://www.jector.com.tw/index.html), also developed a similar PRS system to share the educational market in Taiwan. It is difficult to calculate the exact number of colleges that have adopted PRS in Taiwan for two reasons: 1) some colleges purchased PRS systems but the systems are not in use; 2) some colleges, such as National University of Kaohsiung, purchased PRS systems from both providers in Taiwan. However, based on our observations during the data collection period of this study, it is clear that many colleges in Taiwan have purchased the PRS technology, though unfortunately only a very small number of professors in the majority of those colleges have actually incorporated PRS activities in their classes.

For example, National University of Kaohsiung has only three professors who have utilised PRS in more than one of their classes. Another example, Feng-Chia University, which received the most funding from the Teaching Excellence Program of the Ministry of Education in recent years, only listed two professors who have actively used PRS in multiple courses (http://www.itd.fcu.edu.tw/wSite/ct?xItem=100238&ctNode=25145&mp=287101). Most of those adopting professors encountered the same challenges of inadequate support as seen in the elementary schools (Chen & Tan, 2008). In contrast, Taipei Medical University (TMU) has the largest scale of implementation (http://www.wretch.cc/blog/habook/9560532) because it has carried out successfully the "one remote control per student" policy that approximates the ideal, large scale implementation approach of universities in the West with proactive and thoughtful support from the school administration (http://excellence.tmu.edu.tw/~TMU_TEACH/pro/super_pages.php?ID=pro1&Sn=20).

Literature can be found on the experimental or research oriented use of PRS in elementary schools to colleges in Taiwan. In addition to the previous mentioned research studies on the development of EduClick systems (Liu et al., 2003a), the investigation of the usage in elementary schools (Chen and Tan, 2008), and a proof of concept test on a proposed clicker-assisted concept change model (Lin et al., 2011), there are also reports on teaching strategy for area calculation in fourth grade (Tan & Huang, 2009), and learning improvement on concepts using the combination of dynamic representations and PRS (Lin and Wu, 2011). These literatures would not be possible if the PRS implementation has not reached different levels of the educational institutes in Taiwan. One major reason for the quick adoption by schools in Taiwan is the affordable price for obtaining a locally made PRS system which can be as low as US$1,000 for 10 client licenses. Thus for a regular class size of 35 in elementary schools, it costs only US$4,000, and for a regular class size of 60 in colleges in Taiwan, it costs only US$6,000 to set up. Nevertheless, the support from the school to the adopting teachers remains an issue for PRS to be widely used in those schools with PRS facilities.

Expectation-confirmation theory and information systems success model

ECT and the IS success model are two IS theories commonly used in technology adoption literature. ECT was proposed by Oliver (1980) for use in marketing research (Figure 1). This theory suggests that the initial tendency to purchase commodities affects consumer behaviour for repurchase (Churchill & Surprenant, 1982). Bhattacherjee (2001) proposed an IS continuance model that relates satisfaction and perceived usefulness to the degree to which user expectations about an IS are confirmed. Since then, ECT has been used to discuss and understand the continued use of information technology.

Figure 1

Figure 1: Expectation confirmation theory (Oliver, 1980)

DeLone and McLean (1992) compiled over 100 research papers over a period of seven years beginning in 1981. They proposed six constructs, namely, system quality, information quality, use, user satisfaction, individual impacts, and organisational impacts, for measuring the successful factors of information systems. They believe that system quality and information quality significantly affect usage and user satisfaction, whereas usage and user satisfaction mutually influence each other. Because DeLone and McLean (1992) emphasised the technical view and did not consider the service role played by the IS departments, Pitt et al. (1995) added service quality to this model (Figure 2). After a decade, DeLone and McLean (2003) conducted an analysis using 144 journal articles, in which their 1992 model over a 10-year period was cited, and with service quality included in the model.

Figure 2

Figure 2: Revised IS success model (Pitt et al., 1995)

Motivation theory in instructional design

Keller (1983) defines motivation as "the magnitude and direction of behaviour. In other words, it refers to the choices people make to what experiences they will approach or avoid, and the degree of effort they will exert in that respect." Motivation theories in educational psychology can be divided into three groups: behaviour theories that include the use of reward, punishment, and reinforcement to motivate; cognitive theories that focus on attributional theories and equilibrium-seeking motivation; and humanistic theories that stress the need for achievement and gratification as motivators (Goh & Hooper, 2007). Among related theories, Keller's (1987) ARCS model for motivation and Wlodkowski's (1985) time continuum model are two holistic approaches to motivational design, as indicated by Goh and Hooper (2007). These two models are two of the most cited references on motivational design.

Because motivation factors and learning strategies are often linked in theoretical, as well empirical discussions, Pintrich, Smith, Garcia and Mckeachie (1991) developed a Motivated Strategies for Learning Questionnaire (MSLQ) "to assess college students' motivational orientations and their use of different learning strategies for a college course" (p.3). MSLQ combines measurements for two broad categories, namely, motivation and learning strategies. The motivation category consists of two sections, a value and task section, each having three subscales. The value section consists of the subscales of intrinsic goal orientation, extrinsic goal orientation, and task value. The task section includes the subscales of control of learning beliefs, self-efficacy for learning and performance, and test anxiety. The learning strategies category is also divided into two sections. A cognitive and metacognitive strategies section includes subscales labeled rehearsal, elaboration, organisation, critical thinking, and metacognitive self-regulation. The resource management strategies section, on the other hand, includes the subscales of time and study environment management, effort regulation, peer learning, and help seeking (Pintrich et al., 1991, 1993). Most important, Duncan and McKeachie (2005) concluded that MSLQ has been widely adopted in many studies and thus proven to be an efficient, practical, and ecologically valid measurement of student motivation and learning strategies in higher education.

Agency theory

Agency theory goes back to the 1960s, as economists attempted to apply the risk-sharing concept to explore the relationships between individuals and organisations, as well as those among individuals. Jensen and Meckling (1976) first described an agency relationship as beginning when a principal assigns responsibilities to an agency by according the agent a certain authority. This contractual relationship is thus called agency relationship. Eisenhardt (1989) later defined it as the relationship between a principal and an agent who is appointed to accomplish assigned tasks.

Agency theory assumes that human beings are boundedly rational, risk averse, and motivated by self-interest, and that organisations are characterised by goal incongruence, information asymmetry, and efficiency norms (Eisenhardt, 1989). Goal incongruence happens when the agent's goals differ from those of the principal. Information asymmetry problems occur when the principal cannot accurately observe the agent's behaviour or private information used in such behaviour. The principal and agent may also have differential risk preferences resulting in different attitudes toward risky behaviour. Owing to problems within a contractual relationship, cooperative effort within organisations is often plagued by opportunistic behaviour on the part of organisational members. Agency theory argues that incentive systems and control structures may help mitigate problems associated with such behaviour (Bhattacherjee, 1998).

Figure 3

Figure 3: Agency theory (Bhattacherjee, 1998)

Agency theory is often applied in studies to explore how an organisation - the principal - can use incentives and control mechanisms to inspire or motivate the employees - the agents - to fulfill the goals and objectives set by the organisation (Bhattacherjee, 1998; Eisenhardt, 1989). Figure 3 shows an agency theory-based model of IT usage behaviour, which is influenced by outcome-based or behaviour-based incentives (Bhattacherjee, 1998). The relationship between behaviour-based incentives and IT usage behaviour is moderated by various control structures, such as monitoring. Both relationships between outcome-based or behaviour-based incentives and IT usage behaviour are moderated by risk aversion and goal conflict behaviours.

The generalisation of agency theory has gone beyond traditional management studies. Any parties involved in a relationship with potential conflicts of interest may be characterised as having an agency relationship. However, agency theory-based studies are more frequently applied to information outsourcing (Moynihan, 2002), project management (Mahaney & Lederer, 2003; Shin, 2004), and online consumer behaviour (Pavlou, Liang & Xue, 2007). Agency theory has not been applied prominently in the education domain.

Research methods

As an initial research initiative, this study was conducted with a small sample in Taiwan. The following sections briefly introduce the derivation of the integrated model and the research design.

Model derivation

A PRS is an IS for educational purposes; thus, it is a good candidate for testing variables from cross-disciplinary theories. The research model is an integration of four theories linked by ECT because the primary objective is to understand the continuance usage of a PRS as perceived by the students. Two changes are made to ECT in this study. First, perceived performance is transformed into learning performance to reflect the educational setting in this study. Second, to reduce the number of constructs, the expectation construct is excluded because it can be inferred through the difference between the counterpart of cognitive performance and confirmation. The changes are feasible because each measurement item of confirmation repeats the same question of the paired measurement items of expectation and cognitive performance, except for their differences (Premkumar & Bhattacherjee, 2008).

Three expansions of the ECT-based model are adopted in this study. The first expansion comes from the IS success model shown in Figure 2. In theory, the three independent variables of the IS success model, namely, system quality, information quality, and service quality, exert potential influence on the two outcome variables, user satisfaction and usage. In particular, user satisfaction provides a natural linkage between ECT and the IS success model. Furthermore, in the original MSLQ model, the six motivation subscales are believed to influence the nine learning strategy subscales (Duncan & McKeachie, 2005). Therefore, motivation plays an important role in determining the use of a PRS as a learning strategy in this study. Finally, continuance usage is an equivalent expression of IT usage behaviour in Bhattacherjee's (1998) agency theory model (Figure 3). We may assume that the outcome-based incentives as defined in agency theory have a certain impact on continuance usage in this educational setting, as well.

The integrated model derived from the theoretical rationales above can be seen in Figure 4, in which satisfaction and intention to continue usage are the key factors linking the IS success model, MSLQ, and agency theory in the proposed integrated model. The integrated model encompasses four theories, and 15 corresponding research hypotheses can be automatically derived as follows:

Figure 4

Figure 4: Proposed integrated model

Research design

This PRS study was conducted on students enrolled in three required classes on two subjects, Management Information Systems for juniors and Introduction to Information Management for freshmen, offered by the Department of Information Management at National University of Kaohsiung during three consecutive semesters from Fall 2010 to Fall 2011 in Taiwan. These three courses were taught in English. The FingerClick PRS in use was installed by Jector Digital System in a 120-seat classroom with an infrared-based clickers-like system that came with a teacher remote control, two receivers, 60 student remote controls, a class management system, and a learning profile database.

The PRS was used primarily to improve some of the benefits summarised in the classroom environment and learning categories by Kay and LeSage (2009a, 2009b), which include enabling students to come to class more, be more focused in class, participate anonymously, become more engaged in class, discuss misconceptions among peers, as well as enabling the instructors to modify teaching based on student feedback to improve their learning performance and to promote qualitative differences in learning. The common PRS questions for confirming students' understanding of the lecture materials can be a yes-or-no question such as "An airline reservation system is a typical example of a management information system?"or a multiple-choice question such as "Poor business processes and unsupportive culture are examples of which dimension of business problems? a. management; b. people; c. technology; or d. infrastructure." Sometimes the PRS is also used to survey certain ad-hoc questions, such as whether students have previewed certain lecture materials, recall certain concepts from other courses, or feel that PRS activities help their learning compared to other classes without PRS.

Students were encouraged to use PRS for the following reasons. First, when students answered a question using PRS, it contributed to their class participation score regardless of whether the answers were correct. Second, about two-thirds of the students responded in the first meeting that they favoured knowing their peer's answers and opinions. Third, over half of the students responded in the first few meetings that PRS use in class enabled them to keep up with their reading assignments at home, which has been a common challenge among college students in Taiwan. Finally, some students who had the habit of coming to class late or skipping morning classes responded that the roll calls forced them to become more punctual for classes or at least cut down on their chances to miss the whole three-hour session.

The students in each class were divided into 10 groups. Over the 18-week course schedule, in each of the three-hour sessions, the PRS was used to carry out the roll call at least twice and perform at least three Q&A activities regarding the materials. From the sixth week on, students in the 10 groups were required to prepare and present book chapters or problem-solving cases as course assignments. The course design allowed students to experience and familiarise themselves with the use of the PRS during the first five meetings, facilitated by the instructor before they were asked to design PRS applications on their own. There were 163 students registered in these three courses, but one dropped out after the mid-term examination. Among the remaining 162 students, 138 students filled out an online questionnaire at the end of the course.

The questionnaire was divided into five sections. The first section contained items measuring the ECT constructs (Premkumar & Bhattacharjee, 2008; Tao et al., 2009). The second contained the items measuring the IS success model constructs (Yen, Li, & Niehoff, 2008; Kettinger & Lee, 1994). The third contained items measuring the motivation theory constructs (Duncan & McKeachie, 2005), and the fourth section contained items measuring agency theory constructs (Bhattacherjee, 1998; Cheong & Park, 2005; Venkatesh & Davis, 2000; Norton & Moore, 2006). The collected data in these four sections were used in the model-based analysis. The Appendix provides a complete list of these items according to research constructs. Students were asked to respond to each questionnaire item using a 5-point Likert scale, with 1 representing "extremely disagree" and 5 representing "extremely agree." The questionnaire was modified from the original English sources and translated to Chinese by the authors with simple validation on the accuracy of translation. The analytical method used in this study is the structural equation model (SEM) based on partial least squares (PLS) for model-based analysis.

Model-based analysis and discussion

The sample size of 139 exceeds the minimum number of 100 for the PLS as suggested by Gefen et al. (2000). Thus, SmartPLS (Ringle et al., 2005) was used to pursue the path model analysis. To make the analysis concise, most of the validity data are summarised in Table 1. The second row in Table 1 displays the means and standard deviations (SD) of the variables. The majority of the variables had an average between 3.28 and 3.86, indicating moderately positive perceptions. Conversely, the "intention to continue usage" reflects a neutral perception with the average of 3.13. Combining the highest averaged variable "learning performance," the descriptive statistics may indicate the dilemma the students face; that is, although they may not be in favor of using the PRS again in class, their overall perceptions on the PRS are positive, particularly reflected by their high self-reported learning performance (3.86). This implies a gap between what they have perceived positively and their intention to continue using such an educational technology in future classes.

The SEM path model analysis includes the measurement model and model fitting. The PLS measurement model includes item reliability (factor loading), composite reliability, and average variance extract and its square root. After excluding items with factor loadings below 0.7, a minimum hurdle suggested by Fornell & Larcker (1981), the new factor loadings were derived and shown in the fourth column. LP3, CF3, TV3 and GC1 were retained only because at least two items are required in each latent variable. All values, except for the construct composite reliability of "confirmation" in the fifth column, exceeded the value 0.7 suggested by Nunnally (1978). This implies the acceptability of the overall reliability despite the fact that "confirmation" deserves a better treatment in its items. The convergent validity can be further examined to determine whether the average variance extracted (AVE) is greater than 0.5 (Fornell & Larcker, 1981). The sixth column shows that all AVE values are greater than 0.5, indicating good convergent validity. Discriminant validity can be determined by the square root of AVE; as shown in the last column of Table 1, the square root of AVE is higher than other construct pairs in the correlation table that is omitted. The square roots of AVE are all higher than the construct-pair correlations, which indicate adequate discriminant validity in our variables. The evidence suggests that the overall reliability and validity are adequate.

The final empirically supported model is presented in Figure 5. According to Cohen (1977), R-square is considered high if the value is greater than 0.5, medium if greater than 0.3, and low if greater than 0.1; these values explain the model variations. Among the three dependent variables, the R-square values of "confirmation" (0.23) can be classified as medium-low level, whereas the "intention to continue use" (0.50) and "satisfaction" (0.48) border high level.

Model-wise, hypotheses H1 and H6 in ECT were firmly sustained, but H2 was weakly supported at p=0.1 level in the no-interaction model when H14 and H15 were not considered. In other words, this research partially substantiated the ECT model in PRS usage. Specifically, learning performance influences the confirmation that learning performance is better than the previous expectation, which in turn, influences the satisfaction of students. As a result, student satisfaction leads to the intention to continue using the PRS in the future.

Table 1: Descriptive statistics and validity data

VariablesMean
SD		ItemsFactor loading
(>0.7)		Composite
reliability
(>0.7)		Average variance
extracted (AVE)
(>0.5)		Square root of AVE
Learning performance3.86
0.53		LP10.890.740.590.77
LP2deleted
LP30.63
Confirmation3.45
0.54		CF10.930.660.530.73
CF2deleted
CF30.42
Satisfaction3.46
0.54		SA1deleted0.870.770.88
SA20.86
SA30.89
Intention to continue usage3.13
0.64		ICU10.760.870.700.84
ICU20.87
ICU30.88
Service quality3.53
0.50		SEQ1deleted0.900.840.92
SEQ20.82
SEQ30.84
SEQ40.75
Information quality3.40
0.55		IQ10.850.840.730.85
IQ20.85
IQ3deleted
System quality3.45
0.57		SQ1deleted0.900.840.92
SQ20.91
SQ30.92
Intrinsic goal orientation3.49
0.54		IGO1deleted0.800.670.82
IGO20.80
IGO30.88
Extrinsic goal orientation3.40
0.50		EGO1deleted0.760.610.78
EGO20.79
EGO30.72
Task value3.64
0.57		TV10.880.760.620.79
TV2deleted
TV30.68
Control of learning beliefs3.52
0.5		CLB1deleted0.820.700.84
CLB20.87
CLB30.92
Self-efficacy for learning and performance3.51
0.43		SELP10.790.860.750.87
SELP2deleted
SELP3deleted
SELP40.88
Test anxiety3.37
0.47		TA1deleted0.840.730.85
TA20.85
TA30.86
Incentives3.36
0.48		INC1deleted0.770.640.80
INC20.79
INC30.89
Risk aversion3.28
0.55		RA1deleted0.860.680.82
RA20.92
RA30.92
Goal conflict3.42
0.63		GC10.570.800.660.81
GC2deleted
GC30.95

Figure 5

Figure 5: Empirically supported research model

All three hypotheses, H3 to H5, related to the IS success model in PRS usage were validated. Specifically, the data supported the hypotheses that service quality is positively related to student satisfaction, whereas system quality and information quality were weakly supported at p=0.1 level. Only one hypothesis of the motivation theory, H7, was weakly supported in the interaction model when H14 and H15 were considered. In other words, only the intrinsic goal orientation of the students contributes to their motivation to continue using the PRS in future classes.

Finally, among the three agency theory hypotheses (H13 to H15), none was supported. However, both the t statistics of the interactions in terms of risk aversion (1.54) and goal conflict (1.38) were close to 1.645 at the p=0.1 level. Thus, the business-oriented agency theory may still have significant potential as an effective theory in explaining continuing usage of the PRS in educational settings.

In summary, all hypotheses in the IS-oriented models, ECT, and IS success model, were positively validated, whereas only the intrinsic goal orientation in motivation theory and none of the hypotheses in agency theory were sustained.

We surveyed the students at the end of each semester after using PRS. In general, despite more than 60 percent of students responding that PRS did help them improve class learning, attendance and participation, only about 40 percent of them were willing to continue taking classes with PRS in the future. When asked about the reasons behind this 20 percent gap, some students replied in class that they were accustomed to and thus feel more comfortable with the traditional course design with term tests and take-home assignments. In more private conversations with a few students who were recognised for good academic performance and learning attitude, they responded that the use of PRS required their full attention during the entire class sessions, leaving them less flexibility in classes, which was exhausting or even stressful at times. This echoes some early surveys such as D'Arcy et al (2007) that students prefer not to be monitored by PRS for attendance or accumulate points for participation. However, students also admitted that had the records of participation in PRS not been included in the final grade, attendance and attention in class would have significantly decreased. The feedback above may explain why all factors but intrinsic goal orientation in motivational theory failed to influence the continuing use of PRS in our model. Meanwhile, the incentives provided in this study were not appropriate to motivate the students to continue using PRS in the future. This could have been attributed to their perceived goal conflict or level of risk aversion when taking courses using PRS, as explained by the agency theory.

The flexibility students preferred in class was confirmed by a national survey to the 2005 junior college students in Taiwan as summarised by Tao et al. (2009). The survey showed 20.3% of the juniors never or hardly participated in class activities; 29.9% often and sometimes avoided attending classes; and 49.7% often or sometimes did irrelevant things in class. The reasons behind these statistics may relate to the fact that over 30% of students were busy with part-time jobs and other activities such as online chatting instead of studying. Another national survey conducted in 2003 showed over 50% of the junior students admitted that they spent less than an hour per day on schoolwork in their freshmen year. In other words, a high percentage of college students in Taiwan preferred flexibility in attending the classes over the use of the PRS pedagogical tool which was perceived to improve their attendance, participation and attention in class. This poses a great challenge to college teachers in Taiwan.

Conclusions and implications

In this initial, theory-based exploration of PRS continuance usage, ECT is fully sustained. However, only service quality in IS success model and intrinsic goal orientation in MSLQ motivation theory are supported, while all agency theory hypotheses fail.

Accordingly, two groups of implications may be derived by referencing the empirically validated model as seen in Figure 5. The first group of implications comes from the positive results. In particular, the fully sustained ECT implies that if the instructor can appropriately manage students' expectations and stimulate adequate learning performance, then confirmation will be influenced, which will influence student satisfaction and thus the intention to continue using the PRS in future classes. According to empirical results related to IS success model, the instructor can focus on service quality of PRS sessions to effectively increase student satisfaction, which then leads to an intention for future use of PRS. In other words, any issues related to information quality and system quality may be compensated by having good service quality in PRS setting. Finally, students with intrinsic goal orientation are well motivated by PRS usage in the classroom. The instructor only needs to make sure no mistake is made to discourage this type of students for their future usage of PRS in other classes. The reason could be that PRS allows intrinsic goal-oriented students to compare their answers with peers in anonymity, which helps students realise misconceptions instantly and motivate students to focus more and work harder to be competitive.

The second group of implications comes from the negative results. The unsatisfactory empirical results of the hypotheses in motivation theory might have been constrained by the data collected in this educational setting, which may be altered or improved with more effective strategies. The intention to continue using the PRS may be subject to different motivational beliefs, which may be evoked via appropriately customised strategies. For example, the incentives awarded to students in this study are the participation and attendance scores recorded by the PRS. Therefore, awarding better or more suitable incentives to students as suggested by agency theory, such as awarding more valuable extra points, praising good student performance, or providing summative assessment feedback (Hancock, 2010), may motivate students with extrinsic goal orientations in their intention to continue using PRS. Another example is that if the instructors can appropriately associate the PRS questions with core learning concepts, common misconceptions, and simulated test questions, students with task value beliefs may be more motivated to continue using PRS. Similarly, control of learning beliefs, self-efficacy for learning and performance, and test anxiety may be addressed by customised strategies, as suggested by the perceived advantages and disadvantages of PRS by students (Barnett, 2006), in different educational settings.

Contributions, limitations, and future work

We contribute to PRS research by adding the Taiwanese perspective to extant Western literature. Furthermore, an understanding of PRS usage is explored through a theory-based integrated model in a systematic approach, as suggested by Kay and LeSage (2009a). This case study has demonstrated its potential academic and practical contributions in furthering knowledge on PRS usage, particularly in a Taiwanese context.

On account of the nature of the case study used, some limitations should be clarified before applying the results of this study. First, in most constructs in this study, partial measurement was used to reduce the total number of items. This was implemented to enhance the convenience of filling out the questionnaire within a reasonable time. For agency theory in particular, this study did not distinguish between behaviour-based and outcome-based incentives, and the moderating variables of monitoring, evaluation, and multiple-period contract were excluded. Furthermore, this case study only collected data from one university, which limits generalisability.

To make the research more robust in terms of reliability and validity, as well as generalisability, future work should conduct data collection with complete measurements of corresponding theories and in a representative region or country, such as the entirety of Taiwan. Furthermore, future research may involve more research constructs that are directly associated with the PRS context, such as the benefits and challenges summarised in Kay and LeSage (2009a).

References

Barnett, J. (2006). Implementation of personal response units in very large lecture classes: Student perceptions. Australasian Journal of Educational Technology, 22(4), 474-494. http://www.ascilite.org.au/ajet/ajet22/barnett.html

Bhattacherjee, A. (1998). Managerial influences on intraorganizational information technology use: A principal-agent model, Decision Sciences, 29(1), 139-162. http://dx.doi.org/10.1111/j.1540-5915.1998.tb01347.x

Bhattacherjee, A. (2001). Understanding information systems continuance. An expectation - confirmation model. MIS Quarterly, 25(3), 351-370. http://dx.doi.org/10.2307/3250921

Brewer, C. A. (2004). Near real-time assessment of student learning and understanding in biology courses. BioScience, 54(11), 1034-1039. http://dx.doi.org/10.1641/0006-3568(2004)054[1034:NRAOSL]2.0.CO;2

Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best-practice tips. Life Sciences Education, 6(1), 9-20. http://w.lifescied.org/content/6/1/9.short

Chen, C.H. & Tan, N. C. (2008). An action research: The impact of IRS in second grade mathematics class. Providence Studies on Humanities and Social Sciences, 2(2), 1-34. [in Chinese] http://readopac2.ncl.edu.tw/nclJournal/search/detail.jsp?sysId=0006590775&dtdId=000040&search_type=detail&la=ch

Cheong, J. H. & Park, M. C. (2005). Mobile internet acceptance in Korea. Internet Research, 15(2), 125-140. http://dx.doi.org/10.1108/10662240510590324

Churchill, G. A. & Surprenant, C. (1982). An investigation into the determinants of consumer satisfaction. Journal of Marketing Research, 19(4), 491-504. http://dx.doi.org/10.2307/3151722

Cohen, J. (1997). Statistical power analysis for the behavioral sciences. (Revised Edition). New York: Academic Press.

D'Arcy, C. J., Eastburn, D. M. & Mullally, K. (2007). Effective use of a personal response system in a general education plant pathology class. The Plant Health Instructor, http://dx.doi.org/10.1094/PHI-T-2007-0315-07

DeLone, W. H. & McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information Systems Research, 3(1), 60-95. http://dx.doi.org/10.1287/isre.3.1.60

DeLone, W. H. & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30. http://www.jmis-web.org/articles/v19_n4_p9/index.html

Duncan, T. G. & McKeachie, W. J. (2005). The making of the motivated strategies for learning questionnaire. Educational Psychologist, 40(2), 117-128. http://dx.doi.org/10.1207/s15326985ep4002_6

Eisenhardt, K. M. (1989). Agency theory: An assessment and review. Academy of Management Review, 14(1), 57-74. http://dx.doi.org/10.2307/258191

Fies, C. & Marshall, J. (2006). Classroom response systems: A review of the literature. Journal of Science Education and Technology, 15(1), 101-109. http://dx.doi.org/10.1007/s10956-006-0360-1

Fornell, C. & Larcker D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error: Algebra and statistics. Journal of Marketing Research, 18(3), 382-388. http://dx.doi.org/10.2307/3150980

Freeman, M., Bell, A., Comerton-Forde, C., Pickering, J. & Blayney, P. (2007). Factors affecting educational innovation with in class electronic response systems. Australasian Journal of Educational Technology, 23(2), 149-170. http://www.ascilite.org.au/ajet/ajet23/freeman.html

Gefen, D., Straub, D. W. & Boudreau, M.-C. (2000). Structural equation modeling and regression: Guidelines for research practice. Communications of the Association for Information Systems, 4(7), 1-79. http://aisel.aisnet.org/cais/vol4/iss1/7/

Goh, T.-T. & Hooper, V. (2007). To txt or not to txt: That's the puzzle. Journal of Information Technology Education, 6, 441-453. http://www.jite.org/documents/Vol6/JITEv6p441-453Goh283.pdf

Hancock, T. M. (2010). Use of audience response systems for summative assessment in large classes. Australasian Journal of Educational Technology, 26(2), 226-237. http://www.ascilite.org.au/ajet/ajet26/hancock.html

Jensen, M. C. & Meckling, W. H. (1976). The theory of the firm: Managerial behavior, agency costs and ownership structure. Journal of Financial Economics, 3(4), 305-360. http://dx.doi.org/10.2139/ssrn.94043

Judson. E. & Sawada, D. (2002). Learning from past and present: Electronic response systems in college lecture. Journal of Computers in Mathematics and Science Teaching, 21(2), 167-181. http://www.editlib.org/p/15113

Kay, R. H. & LeSage, A. (2009a). A strategic assessment of audience response systems used in higher education. Australasian Journal of Educational Technology, 25(2), 235-249. http://www.ascilite.org.au/ajet/ajet25/kay.html

Kay, R. H. & LeSage, A. (2009b). Examining the benefits and challenges of using audience response systems: A review of the literature. Computers & Education, 53(3), 819-827. http://dx.doi.org/10.1016/j.compedu.2009.05.001

Keller, J. M. (1983). Instructional design theories and models: An overview of their current status. Hillsdale, NJ: Lawrence Erlbaum.

Keller, J. M. (1987). Development and use of the ARCS model of instructional design. Journal of Instructional Development, 10(3), 2-10. http://www.springerlink.com/content/72828k22416p4156

Kettinger, W. J. & Lee, C. C. (1994), Perceived service quality and user satisfaction with the information service function. Decision Sciences, 25(5/6), 737-766. http://dx.doi.org/10.1111/j.1540-5915.1994.tb01868.x

Laxman, K. (2011). A study on the adoption of clickers in higher education. In Hong, K. S. & Lai, K. W. (Eds), ICT for accessible, effective and efficient higher education: Experiences of Southeast Asia. Australasian Journal of Educational Technology, 27(Special issue, 8), 1291-1303. http://www.ascilite.org.au/ajet/ajet27/laxman.html

Lin, Y.-C., Liu, T.-C. & Chu, C.-C. (2011). Implementing clickers to assist learning in science lectures: The Clicker-Assisted Conceptual Change model. Australasian Journal of Educational Technology, 27(6), 979-996. http://www.ascilite.org.au/ajet/ajet27/lin.html

Liu, T.-C., Liang, J.-K., Wang, H.-Y. & Chan, T.-W. (2003a). The features and potential of interactive response system. In Proceedings of the International Conference on Computers in Education, Hong Kong, 315-322. http://scholar.google.com.tw/scholar_url?hl=zh-TW&q=http://citeseerx.ist.psu.edu/viewdoc/download
%3Fdoi%3D10.1.1.105.9585%26rep%3Drep1%26type%3Dpdf&sa=X&scisig=AAGBfm0Sq7Gl0citB
7316_yfznq-2lLeoQ&oi=scholarr&ei=eAGiT5KbHdGImQWIs83oBw&ved=0CCAQgAMoADAA

Liu, T. C., Wang, H. Y., Liang, J. K., Chan, T. W., Yang, J. C. & Ko, H. W. (2003b). Wireless and mobile technologies to enhance teaching and learning. Journal of Computer Assisted Learning, 19(3), 371-382. http://jcal.info/abstracts/2003/v193/index.htm#liu

Lin, J.-W. & Wu. Y.-L. (2011). Investigating the impact of dynamic presentations with interactive response system on diagnosing students' alternative conceptions of simple and series connections. Journal of Education & Psychology, 34(1), 79-107. [in Chinese] http://www.joep.nccu.edu.tw/paper/paper.php?action=show_content&Sn=5

MacGeorge, R. L., Homan, S. R., Dunning, J. B., Elmore, D., Bodie, G. D., Evans, E., Khichadia, S., Lichti, S. M., Feng, A. B. & Geddes, A. B. (2008). Student evaluation of audience response technology in large lecture classes. Educational Technology Research and Development, 56(2), 125-145. http://dx.doi.org/10.1007/s11423-007-9053-6

Mahaney, R. C. & Lederer, A. L. (2003). Information systems project management: An agency theory interpretation. The Journal of Systems and Software, 68(1), 1-9. http://www.scribd.com/doc/57188584/6596504-Information-Systems-Project-Management

Moynihan, T. (2002). Coping with client-based 'people-problems': The theories-of-action of experienced IS/software project managers. Information & Management, 39(5), 377-390. http://dx.doi.org/10.1016/S0378-7206(01)00104-5

Norton, W. & Moore, W. (2006). The influence of entrepreneurial risk assessment on venture launch or growth decisions. Small Business Economics, 26(3), 215-226. http://dx.doi.org/10.1007/s11187-004-5612-y

Nunnally, J. (1978). Psychometric theory. McGraw-Hill, New York.

Oliver, R. L. (1980). A cognitive model for the antecedents and consequences of satisfaction decisions. Journal of Marketing Research, 17, 460-469. http://dx.doi.org/10.2307/3150499

Pavlou, P. A., Liang, H. & Xue, Y. (2007). Understanding and mitigating uncertainty in online exchange relationships: A principal-agent perspective. MIS Quarterly, 31(1), 105-136. http://misq.org/understanding-and-mitigating-uncertainty-in-online-exchange-relationships-a-principal-agent-perspective.html

Pintrich, P. R., Smith, D. A. F., Garcia, T. & McKeachie, W. J. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor: University of Michigan, National Center for Research to Improve Postsecondary Teaching and Learning. http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED338122&ERICExtSearch_SearchType_0=no&accno=ED338122

Pintrich, P. R., Smith, D. A. F., Garcia, T. & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53(3), 801-813. http://dx.doi.org/10.1177/0013164493053003024

Pitt, L. F., Watson, R. T. & Kavan, C. B. (1995). Service quality: A measure of information systems effectiveness. MIS Quarterly, 19(2), 173-187. http://misq.org/service-quality-a-measure-of-information-systems-effectiveness.html

Premkumar, G. & Bhattacherjee, A. (2008). Explaining information technology usage: A test of competing models. Omega, 36(1), 64-75. http://dx.doi.org/10.1016/j.omega.2005.12.002

Ringle, C. M., Wende, S. & Will, A. (2005). SmartPLS 2.0 (beta). http://www.smartpls.de/

Shin, M. (2004). A framework for evaluating economics of knowledge management systems. Information and Management, 42(1), 179-196. http://dx.doi.org/10.1016/j.im.2003.06.006

Simpson, V. & Oliver, M. (2007). Electronic voting systems for lectures then and now: A comparison of research and practice. Australasian Journal of Educational Technology, 23(2), 187-208. http://www.ascilite.org.au/ajet/ajet23/simpson.html

Tan, N.-C. & Huang, S.-Y. (2009). The study by using TAIRS teaching strategy about area in fourth grade. Research and Development in Science Education Quarterly, 6, 43-69. [in Chinese] http://sce.tmue.edu.tw/cat/news.php?Sn=218

Tao, Y.-H., Cheng, C.-J. & Sun, S.-Y. (2009). What influences college students to continue using business simulation games? The Taiwan experience. Computers & Education, 53(3), 929-939. http://dx.doi.org/10.1016/j.compedu.2009.05.009

Venkatesh, V. & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186-204. http://www.jstor.org/stable/2634758

Wlodkowski, R. J. (1985). Enhancing adult motivation to learn. San Francisco: Jossey-Bass.

Yamazaki, Y. (2005). Learning styles and typologies of cultural differences: A theoretical and empirical comparison. International Journal of Intercultural Relations, 29(5), 521-548. http://dx.doi.org/10.1016/j.ijintrel.2005.07.006

Yen, H. R., Li, E. Y. & Niehoff, B. P. (2008). Do organizational citizenship behaviors lead to information system success?: Testing the mediation effects of integration climate and project management. Information & Management, 45(6), 394-402. http://dx.doi.org/10.1016/j.im.2008.04.004

Appendix

ConstructItem
System qualityThe PRS functions reliably.
The PRS operates efficiently.
The PRS design satisfies my needs.
Information qualityThe PRS provides accurate information.
The PRS provides timely information.
The PRS provides useful information.
Service qualityThe PRS is dependable.
The PRS provides prompt services to users.
Users feel safe in their interactions with the PRS.
The operation hours of the PRS are convenient to its users.
Learning performanceI can learn new skills if I use the PRS.
The PRS helps me improve my results.
The learning periods are more flexible if I use the PRS.
ConfirmationThe PRS, with the ability to help me learn new skills, has exceeded my previous expectations.
The PRS, with the ability to improve my results, has exceeded my previous expectations.
The PRS provides the flexibility to learn in my own time.
Intrinsic goal-orientedIn a class such as this, I prefer a course material that really challenges me so I can learn new things.
The most satisfying aspect for me in this course is trying to understand the content as thoroughly as possible.
When I have the opportunity in this class, I choose course assignments that I can learn from even if they do not guarantee a good grade.
Extrinsic goal-orientedGetting a good course grade is the most satisfying achievement for me.
If I can, I want to acquire better grades than most of the other students in this class.
I want to do well in this class because it is important to show my abilities to my family, friends, employer, or others.
Task valueI think I will be able to use what I learned in this course in other courses.
I think learning the course material is useful for me.
Understanding the subject matter of this course is very important to me.
Control of learning beliefsIf I study in an appropriate manner, I will be able to learn the material in this course.
It is my own fault if I do not learn the material in this course.
If I try hard enough, I will understand the course material in the class using a PRS.
Self efficacy for learning and performanceI believe I will receive an excellent grade in this class.
I am confident I can learn the basic concepts taught in this course.
I am confident I can do an excellent job on the assignments and tests in this course.
I am certain I can master the skills being taught in this class.
Test anxietyWhen I take a test, I think about how poorly I am doing compared with other students.
When I take a test, I think about items on other parts of the test that I cannot answer.
I have an uneasy, upset feeling when I take an exam.
IncentivesThe instructor offers incentives for using the PRS.
I get rewarded when I achieve goals in class.
The instructor generally gives us enough incentives for learning.
Goal conflictsI intend to use the PRS in class.
I intend to use the PRS as much as possible.
Assuming students had access to the PRS, they intend to use it.
Risk aversionI am not willing to take risks when choosing a work environment.
I prefer a low risk/high-security work environment with predictable income over a high-risk and high-reward environment.
I prefer to remain in an environment that has problems that I know about rather than take the risks associated with a new environment with unknown problems, even if the new environment offers greater rewards.
I view job-related risks as a situation to be avoided at all costs.
SatisfactionIt is enjoyable for me to participate in a course using the PRS.
I like to learn new skills through the instructional method using the PRS.
I hope all courses can integrate the PRS into the instructional method to facilitate practice.
Intention to continue usageI intend to continue using the PRS to learn new skills.
I intend to increase my use of the PRS in the future.
The PRS will be my first tool of choice when learning new skills.

Authors: Dr Chu-Chen Rosa Yeh, Assistant Professor
Graduate Institute of International Human Resource Development
National Taiwan Normal University, 162, Section 1, Heping East Road, Taipei, Taiwan
Email: rosayeh@ntnu.edu.tw

Dr Yu-Hui Tao, Professor (corresponding author)
Department of Information Management, National University of Kaohsiung
700 Kaohsiung University Road, Kaohsiung, Taiwan
Email: ytao@nuk.edu.tw

Please cite as: Yeh, C.-C. R. & Tao, Y.-H. (2012). College students' intention to continue using a personal response system: Deriving a model from four theoretical perspectives. Australasian Journal of Educational Technology, 28(5), 912-930. http://www.ascilite.org.au/ajet/ajet28/yeh.html

[ PDF version of this article ] [ AJET 28 ] [ AJET home ]
HTML Editor: Roger Atkinson [rjatkinson@bigpond.com]
This URL: http://www.ascilite.org.au/ajet/ajet28/yeh.html
Created 26 July 2012. Last revised 26 July 2012.
Page access count