| Australasian Journal of Educational Technology 2011, 27(Special issue, 8), 1291-1303. |
AJET 27 |
A study on the adoption of clickers in higher education
Kumar Laxman
Nanyang Technological University
Audience response systems or 'clickers' are being used widely in both large and small educational settings. Clickers leverage upon a number of technological affordances to allow for adaptive and flexible learning to be accomplished. To promote active learning in its classrooms, Nanyang Technological University (NTU) in Singapore rolled out a campus-wide initiative called "Learning that Clicks!" on using the interactive technology of clickers. This initiative will enable the necessary transformation of pedagogy and learning design to support the epistemological paradigm shift of becoming more student-centric in nature. This preliminary study attempted to investigate the experiences of undergraduate students in learning in clicker-supported instructional environments. A survey consisting of nine items was administered to 640 students from 12 classes in the Engineering, Humanities and Sciences schools to solicit their views on the effectiveness of clicker technology as an instructional device. Generally, students felt that the use of clickers has improved the quality of their learning experiences. Overall, this study reveals that clicker technology offers great promise in promoting more collaborative and engaging learning environments and innovating instructional delivery, provided lecturers apply sound pedagogical principles in their teaching.
One of the campus-wide initiatives that was rolled out was "Learning that Clicks!" [http://clickers.ntu.edu.sg/] - the innovative use of the interactive instructional technology of 'clickers' (or 'audience response systems', or 'personal response systems', or 'electronic voting systems'). This initiative was designed to enable the necessary transformation of pedagogy to reflect the student-centred, digitally driven learning styles of current and subsequent generation/s of learners. More importantly, this initiative will assist in the promotion of active learning experiences of students without radically altering existing physical classroom facilities (Gan, 2011). This project will also enable faculty at NTU to achieve the desired goal of "teach less and learn more" where lecturers are encouraged to teach less expository content information and instead facilitate their students' active participation in the learning process.
As a result of this initiative, 95% of lecture theatres, tutorial rooms and seminar rooms in NTU's different schools have been equipped progressively with clicker technology [https://compass.ntu.edu.sg/resources/Pages/Clickers.aspx#benefits]. The lecture theatres were of particular interest as it was envisaged that this technology has the potential to transform the current one-way, teacher-directed information transmission model dominating NTU's lecture experiences. About 150 undergraduate courses have been reported to have used clickers in classroom teaching. The size of a credit-card, clickers provide students with a 'cool' factor in handling the devices as well as support the creation of interactive engagement in large lecture theatres and smaller classrooms. Initial training was provided on a large scale to faculty to familiarise them with the technical features of clickers and procedures for managing the technology in the teaching facilities. This has been made easier by the fact that the technology is user friendly and simple to operate. About 300 faculty (about 10% of total faculty strength) have attended these training sessions thus far. All undergraduate students were issued clickers free of charge but they have to purchase replacements at subsidised rates if they lose or damage them.
However, as mentioned earlier, much less is known about the lived experiences of students in using the clicker technology in the classrooms (MacArthur, 2010). Penuel, Roschelle, Crawford, Shechtman & Abrahamson (2004) mentioned that the effects of interactive pedagogies and technologies need to be tested and measured through the implementation of appropriate research design studies - something which they argue is currently lacking in practice. Besides detailing the aspects of the research study that was carried out in NTU to ascertain students' experiences of the utility of clickers in increasing learning, this paper will also contribute to the limited existing research corpus on the Asian perspectives of the use of clickers in institutions of higher learning.
Modern clickers leverage upon a number of technological affordances to allow for adaptive and flexible learning. The first is that clickers are wireless handsets with each unit possessing a unique signal to enable responses from each individual student to be identified and recorded (Caldwell, 2007). The next key component embedded in clicker technology is the linking to data manipulation and a projection display. Responses from students can be displayed on the projection screen instantly as bar charts or in other appropriate formats. The ability to tabulate and display the collective data to the entire class is an important feature. This provision of immediate display of results enables the lecturer to easily assess students' understanding of material covered in class and offer remedial instructions to correct student misunderstandings, if any. Students also obtain prompt feedback on how well they are learning. The third and final component is a personal computer loaded with the software that facilitates the collection, processing, display and storage of response data. This software is usually the component most staff have to use to operate clicker technology (Caldwell, 2007). Overall, Parsons (2005) notes that though clicker technological hardware may sound complex, most clicker systems are relatively easy to use with only an "intermediate" level of computer skills needed, thereby freeing the instructor from technical operations and and allowing concentration on pedagogical delivery.
Instructors have reported the use of clickers in classes ranging from 15 students (Draper, 2002) to more than 200 students (Cue, 1998). These findings buttress the argument that clicker technology can effectively support both small and large groups teaching. Clicker technology has been incorporated in a wide ranging number of courses spanning across different disciplinary fields of expertise such as nursing (Halloran, 1995), communication (Jackson & Trees, 2003), computer science (Draper, 2002), business (Cue, 1998) and economics (Simpson & Oliver, 2006). Clickers have also been leveraged upon in the curricular implementation of different types of instructional formats, ranging from optional tutorials to formal standard lectures and cooperative learning through peer collaboration (Nichol & Boyle, 2003).
Clickers-supported questioning could also help reveal student misunderstandings and allow for prompt remediation to be done by lecturers (Wood, 2004). Besides correcting student misconceptions as part of formative assessment, by gauging student responses to posed questions, lecturers are also able to responsively modify subsequent directions of instruction to best align them with current levels of student conceptual understanding (Caldwell, 2007). In short, clickers can offer constructive feedback to both students and teachers and enable students to become more active participants rather than be peripheral observers in the classroom (Beekes, 2006).
Validity refers to whether the items in the survey instrument measure what they had set out to measure (Burns, 2000). A 'construct' is a quality which has been suggested to explain aspects of human behavior. Construct validity can be ascertained through a variety of kinds of indirect evidence - for example, the items in a measuring instrument must be internally consistent in showing agreement that they are measuring the same qualities (Burns, 2000). In the context of the study described in this paper, a high alpha coefficient value was obtained for internal consistency, thus evidencing good construct validity.
Content or face validity is an indication as to whether the items measure what the researcher wishes to measure. It is the representativeness or sampling adequacy of the content of a measuring instrument (Burns, 2000). To ensure content validity of the survey administered in this study, the researcher developed the items in the survey in close cooperation with the project-leader of NTU's clicker initiative, who has been involved in this project with the university for a few years and had thus gained extensive experience in clicker-supported pedagogy. His inputs and advice were regularly sought throughout the period of implementation of this study to ensure that the study set out to achieve its intended goals. In addition, the items in the survey were carefully and simply worded to ensure that there would be little ambiguity in participants' understanding of the information each item was attempting to elicit from them - this also helped to ensure a high degree of content validity.
| Survey items | Responses | |||
| Frequency | Percentage | |||
| 1. | The clickers are used in my | Lectures | 176 | 28.2 |
| Tutorials/ Seminars | 122 | 19.6 | ||
| Both | 17 | 2.7 | ||
| Neither | 309 | 49.5 | ||
| 2. | In this course, on average, clickers are used ____ times within each class. | 0-2 | 238 | 37.8 |
| 3-5 | 226 | 35.9 | ||
| 6-8 | 48 | 7.6 | ||
| >8 | 118 | 18.7 | ||
| 3. | The use of clickers promotes opportunities for peer interaction/s with my fellow students. | Strongly disagree | 65 | 11.4 |
| Disagree | 112 | 19.6 | ||
| Agree | 250 | 43.9 | ||
| Strongly agree | 143 | 25.1 | ||
| 4. | The use of clickers has increased my opportunities to interact with my lecturer during lessons. | Strongly disagree | 60 | 10.0 |
| Disagree | 131 | 21.9 | ||
| Agree | 318 | 53.2 | ||
| Strongly agree | 89 | 14.9 | ||
| 5. | Clickers have been used to check my understanding of content. | Yes | 517 | 84.6 |
| No | 94 | 15.4 | ||
| 6. | The use of clickers has increased my motivation to attend class. | Strongly disagree | 125 | 20.1 |
| Disagree | 177 | 28.5 | ||
| Agree | 203 | 32.7 | ||
| Strongly agree | 116 | 18.7 | ||
| 7. | The use of clickers has resulted in my feeling that I am more engaged with my learning. | Strongly disagree | 52 | 8.7 |
| Disagree | 132 | 22.2 | ||
| Agree | 348 | 58.4 | ||
| Strongly agree | 64 | 10.7 | ||
| 8. | Clickers help me learn better because of the immediate feedback I get during lessons. | Strongly disagree | 31 | 5.1 |
| Disagree | 68 | 11.2 | ||
| Agree | 378 | 62.1 | ||
| Strongly agree | 132 | 21.6 | ||
| 9. | Overall, the use of clickers has improved the quality of my learning experience in this course. | Strongly disagree | 38 | 6.5 |
| Disagree | 89 | 15.2 | ||
| Agree | 376 | 64.2 | ||
| Strongly agree | 83 | 14.1 | ||
External validity asks the question as to how well the results of the study can be generalised to the population or settings (Burns, 2000). In this study, the subjects involved came from a broad spread across the different schools (humanities, sciences and engineering) of NTU to ensure a good representativeness of NTU's large student population.
The survey results for each of the survey items are presented in Table 1 (the complete listing of survey items can be found in the Appendix).
Nearly 97% of student respondents provided the feedback that clickers have been used in their lectures and tutorials or both (item 1 in the survey). On average, the frequency of usage of clickers for each lesson (item 2) was reasonably high. Given the large scale of the rollout of the clicker initiative throughout the university within a limited span of time and the provision of, thus far, only basic technical training to faculty on the functional mechanisms of clickers, the extent of utility of clickers in teaching is definitely encouraging. It is hoped that with more structured training being planned for offering in the future, on effective pedagogical techniques associated with the use of clickers in the classroom, faculty will be oriented to a wider repertoire of effective strategies to better enable them to successfully incorporate clicker technology in their teaching. Exploring ways of embedding clickers in their teaching would encourage faculty to rethink their traditional teaching methods and adopt instructional practices that deliver curricular content in technology-supported innovative ways to meet the needs of NTU's 'digitally' driven learners. An anecdotal observation of some of the classes utilising clicker technology found students being actively involved in their learning and an increase in 'buzz' level of the classes.
About 70% of student respondents either agreed or strongly agreed with the statements in items 3 and 4 of the survey, noting the use of clickers in fostering a learning environment in their classrooms that is more interactive and collaborative in nature. Clickers increased opportunities for faculty-student and student-student interactions, particularly during lecture times which have traditionally been passive learning experiences and lecturer-directed. Application of clicker technology during class makes learning more personalised, as students' responses to posed questions provide cues to faculty on how to proceed with their teaching. Clickers offer a significant opportunity for faculty to capture shy voices and voices from the back of the class. Students who might otherwise not contribute to open class discussions, or who prefer to conform to popular opinions, are prompted to participate by providing their opinions on issues being discussed (Dangel & Wang, 2008). Hake (1998) noted that in practice, interactive learning engagement methods have been shown to be twice as effective as traditional teacher- centric lectures. The benefits of clickers in engendering peer learning are significant since peer instructions have been found to be less threatening and enable learners, by virtue of their similar ages, language and commonly shared experiences to clear up each other's confusions, misunderstandings and misconceptions (Wood, 2004).
Overall, the opportunities offered by clickers in implementing interactive pedagogy during class time appear to have made students enjoy and participate in the lessons. During informal talks with some of the students who had responded negatively to the impact of application of clickers in promoting socially interactive learning environments, it was found that clicker devices of themselves did not influence the quality of interactions. It was the quality of teaching that mattered. This finding is in accord with a conclusion from the clicker literature: clicker technology, like most other technology, is not a panacea in and of itself in solving pedagogical conundrums (Jackson & Trees, 2003; Wood, 2004; Parsons, 2005). In fact, Jackson and Trees (2003) concluded that there are few (if any) collections of good clicker questions available for most fields of educational study. Thus, workshops on designing good clicker questions and applying effective questioning/facilitation strategies were identified as important areas of training that need to be offered for faculty development to promote more effective use of clickers in teaching and learning.
About 84% averred that clickers have been used appropriately by faculty to check their understanding of content covered during classes (item 5 in the survey). This finding could be better understood by linking it to the survey item which solicited students' views on whether clickers enabled them to learn better due to the provision of immediate diagnostic feedback (item 8). About 84% of student participants responded with a strongly agree or agree to item 8 of the survey on the ability of clickers to help them to learn better, due to the immediate feedback they get when data statistics is instantaneously generated and displayed on screen. This promptness in feedback facilitated better reinforcement of learning and allowed faculty to provide immediate remediation, where necessary, when misconceptions were identified. Being able to regularly monitor students' understanding during lesson time meant that faculty can now responsively manage instructional pathways - enabling teaching to be flexible, adaptive and customised. Anonymously gathering and projecting students' responses to questions via the clicker system allowed students to compare their understanding with that of their peers and gain mastery of relevant subject content matter. However, faculty need to take note that besides being prompt, feedback also has to be directive and specific to provide useful guidance for students to organise their own learning (Benson, Mattson & Adler, 1995).
Analysing clickers' impact on the affective domain of learning, it was found that students were somewhat ambivalent about clickers' ability as a motivator to encourage them to attend classes (item 6 in survey). Though some studies such as Hanson's (2007) observed that students and faculty expressed a positive view of clicker systems in relation to perceived improvement in levels of attendance and motivation, we are unable to draw a similar conclusion based upon our data. This reinforces the notion that clickers, or for that matter, any type of technology is not of itself a 'magical bullet' to solve educational problems. The pedagogical ways in which technologies are embedded in the learning processes actually determine the success of an educational initiative. It is important for educators to be aware that the benefits of clickers in enhancing the quality of teaching do not happen automatically or immediately with the introduction of clickers within classroom contexts. Educators need to properly formulate their instructional goals and carefully plan for clicker questions in class discussions to attain those goals (Dangel & Wang, 2008). About 69% of the students responded to item 7 of the survey either strongly agreeing or agreeing that they felt they are better engaged in their learning with the use of clickers. This is an encouraging sign since students have acknowledged the positive impact of clickers in increasing engagement with their learning during class time.
Overall, about 74% of students either agreed or strongly agreed that clickers as an educational tool have the capacity to bring about improved learning (item 9 in survey). One student casually commented thus: "Clickers have kept us on our toes. We have to be attentive in class and be involved during discussions to be able to respond to questions when our lecturers pose them. This becomes particularly relevant when the lecturer decides to have an unannounced quiz in class to test our understanding on what we have learnt thus far."
Some of these principles include asking the right types of formative and summative questions at appropriate junctures of the lessons, to check for content understanding, resolve misconceptions and generate new ideas (Pritchard, 2006). More research studies are being planned as a follow-up to this preliminary study, to explore best practices and case studies of effective pedagogical uses of clickers in NTU. Such exemplars could then help to promote the widespread diffusion and integration of clicker technology within curricular implementation. The instructional contexts, learning styles and questioning techniques that are best suited to the use of clickers also need to be investigated. In short, the findings of this study do encourage a scaling up of the application of clicker technology in instructional approaches in support of NTU's paradigm shift: teach less and learn more.
Beekes, W. (2006). The "millionaire" method for encouraging participation. Active Learning in Higher Education, 7(1), 25-36. http://dx.doi.org/10.1177/1469787406061143
Benson, D., Mattson, L. & Adler, L. (1995). Prompt feedback. In S. R. Hatfield (Ed), The seven principles in action: Improving undergraduate education (pp. 55-66). Bolton, MA: Anker Publishing Co.
Boyle, J. T. & Nicol, D. J. (2003). Using classroom communication systems to support interaction and discussion in large class settings. Association for Learning Technology Journal, 11(3), 43-57. http://repository.alt.ac.uk/413/
Burns, R. B. (2000). Introduction to research methods (4th ed.). London : SAGE Publication.
Burnstein, R. A. & Lederman, L. (2001). Using wireless keypads in lecture classes. Physics Teacher, 29, 8-11. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.112.3323&rep=rep1&type=pdf
Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best-practice tips. CBE-Life Sciences Education, 6(1), 9-20. http://lifescied.org/content/6/1/9.full.pdf+html
Campbell, J. & Mayer, R. E. (2009). Questioning as an instructional method: Does it affect learning from lectures? Applied Cognitive Psychology, 23(6), 747-759. http://dx.doi.org/10.1002/acp.1513
Cue, N. (1998). A universal learning tool for classrooms? Paper presented in The First Quality in Teaching and Learning Conference (Hong Kong, SAR, China, December 10-12, 1998). http://celt.ust.hk/files/public/nelsoncue.pdf
Dangel, H. L. & Wang, C. X. (2008). Student response systems in higher education: Moving beyond linear teaching and surface learning. Journal of Educational Technology Development and Exchange, 1(1), 93-104. http://www.sicet.org/journals/jetde/jetde08/paper08.pdf
Draper, S. W. (2002). Evaluating effective use of PRS: Results of the evaluation of the use of PRS in Glasgow University. Glasgow University, Scotland. [viewed 14 Mar 2011] http://www.psy.gla.ac.uk/~steve/ilig/papers/eval.pdf
Elliot, C. (2003). Using a personal response system in economics teaching. International Review of Economics Education, 1(1), 80-86. http://www.economicsnetwork.ac.uk/iree/i1/elliott.htm
Gan, C. L. (2011). Effective learning in classrooms of tomorrow at NTU. Singapore. Paper presented at the E-learning Asia Forum 2011 (Singapore, June 8, 2011). http://elfasia.org/elfa2011/download/presentation/Session%201C_Gan%20Chee%20Lip.pdf
Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64-74. http://web.mit.edu/rsi/www/2005/misc/minipaper/papers/Hake.pdf
Hall, R. H., Collier, H. L., Thomas, M. L. & Hilgers, M. G. (2005). A student response system for increasing engagement, motivation, and learning in high enrolment lectures. In Proceedings from the Eleventh Americas Conference on Information Systems (pp. 621-626). Omaha, NE: AMCIS.
Halloran, L. (1995). A comparison of two methods of teaching: Computer managed instruction and keypad questions versus traditional classroom lecture. Computers in Nursing, 13(6), 285-288.
Hanson, C. R. (2007). An evaluation of a student response system used at Brigham Young University. Unpublished master's thesis. Brigham Young University, Utah. http://contentdm.lib.byu.edu/ETD/image/etd2127.pdf
Jackson, M. & Trees, A. (2003). Clicker implementation and assessment. Boulder, CO: Information and Technology Services and Faculty Teaching Excellence Program, University of Colorado. http://comm.colorado.edu/mjackson/clickerreport.htm [viewed 24 Jan 2011, not found 4 Dec 2011]
Kay, R. H. & LeSage, A. (2009). Examining the benefits and challenges of using audience response systems: A review of the literature. Computers & Education, 53(3), 819-827. http://dx.doi.org/10.1016/j.compedu.2009.05.001
Kennedy, G. E. & Cutts, Q. I. (2005). The association between students' use of an electronic voting system and their learning outcomes. Journal of Computer Assisted Learning, 21(4), 260-268. http://dx.doi.org/10.1111/j.1365-2729.2005.00133.x
Kuder, G. & Richardson, M. (1937). The theory of the estimation of test reliability. Psychometrika, 2(3), 151-156. http://dx.doi.org/10.1007/BF02288391
Lin, Y.-C., Liu, T.-C. & Chu, C.-C. (2011). Implementing clickers to assist learning in science lectures: The Clicker-Assisted Conceptual Change model. Australasian Journal of Educational Technology, 27(6), 979-996. http://www.ascilite.org.au/ajet/ajet27/lin.html
MacArthur, J. R. (2010). Factors that promote success in large enrolment general chemistry courses taught with clickers. Unpublished doctoral dissertation, University of Northern Colorado, Greeley, Colorado. http://adr.coalliance.org/cogru/fez/eserv/cogru:348/MacArthur_unco_0161N_10039.pdf
Mazur, E. (1997). Peer instruction: A user's manual. Upper Saddle River, NJ: Prentice Hall.
Meeker, W. Q. & Escobar, L. A. (1998). Statistical methods for reliability data. Hoboken, New Jersey: Wiley.
Middendorf, J. & Kalish, A. (1996). The "change-up" in lectures. National Teaching and Learning Forum, 5(2), 1-5. http://www.ntlf.com/html/pi/9601/article1.htm
Moreau, N. (2009). Do clickers open minds? Use of a questioning strategy in developmental mathematics. Unpublished doctoral dissertation, Capella University, Minneapolis, Minnesota. http://gradworks.umi.com/33/89/3389211.html
Nichol, D. J. & Boyle, J. T. (2003). Peer instruction versus class-wide discussion in large classes: A comparison of two interaction methods in the wired classroom. Studies in Higher Education, 28(4), 457-473. http://dx.doi.org/10.1080/0307507032000122297
Parsons, C. V. (2005). Decision making in the process of differentiation. Learning & Leading with Technology, 33(1), 8-10. http://business.highbeam.com/4240/article-1G1-136162812/decision-making-process-differentiation
Paschal, C. (2002). Formative assessment in physiology teaching using a wireless classroom communication system. Advances in Physiology Education, 26(4), 299-308. http://advan.physiology.org/content/26/4/299.full
Penuel, W. R., Roschelle, J., Crawford, V., Shechtman, N. & Abrahamson, L. (2004). Workshop report: Advancing research on the transformative potential of interactive pedagogies and classroom networks. Menlo Park: SRI International.
Poulis, J., Massen, C., Robens, E. & Gilbert, M. (1998). Physics lecturing with audience paced feedback. American Journal of Physics, 66(5), 439-441. http://dx.doi.org/10.1119/1.18883
Pritchard, D. (2006). The use of "clicker" technology to enhance the teaching/learning experience. C.A.L.T. E-learning enhancement grant 2006, University of Otago, New Zealand. http://hedc.otago.ac.nz/hedc/asd/Digital-Resources-for-your-Teaching/mainParagraphs/012/document/ClickersReport.pdf [viewed 13 Jan 2011]
Reay, N. W., Bao, L., Li, P., Warnakulasooriya, R. & Baugh, G. (2005). Toward the effective use of voting machines in physics lectures. American Journal of Physics, 73, 554-558. http://dx.doi.org/10.1119/1.1862638
Robson, C. (1993). Real-world research: A resource for social scientists and practitioner-researchers. Malden: Blackwell Publishing.
Simpson, V. & Oliver, M. (2006). Using electronic voting systems for lectures then and now: A comparison of research and practice. Australasian Journal of Educational Technology, 23(2), 187-208. http://www.ascilite.org.au/ajet/ajet23/simpson.html
Yourstone, S. A., Kray, H. S. & Albaum, G. (2008). Classroom questioning with immediate electronic response: Do clickers improve learning? Decision Sciences Journal of Innovative Education, 6(1), 75-88. http://dx.doi.org/10.1111/j.1540-4609.2007.00166.x
Wood, W. B. (2004). Clickers: A teaching gimmick that works. Developmental Cell, 7(6), 796-798. http://www.colorado.edu/MCDB/MCDB6440/ClickersDevCellC.pdf