|Australasian Journal of Educational Technology
2011, 27(1), 105-121.
Information skills and critical literacy: Where are our digikids at with online searching and are their teachers helping?
Judine Ladbrook and Elizabeth Probert
University of Auckland
International studies and theorists have posited that digital technologies play an important role in students' lives and that students display a broad range of literacy skills when using them. The New Zealand Curriculum (2007) states that students should be literate, critical thinkers who actively seek, use and create knowledge. This paper reports on findings from a New Zealand investigation of the extent to which students exhibited and teachers promoted critical information technology literacy skills. Using survey, diary and focus groups, the investigation explored teachers' beliefs about students' online information literacy and students' self-reported research strategies. Results from the investigation show students possess limited online information and critical evaluation skills and teacher pedagogical practice is not addressing this. The paper makes a case for teachers to develop both familiarity and confidence with online text types, alongside professional learning in online and offline information literacy pedagogical strategies.
We begin by defining information literacy and exploring previous literature and research to which this study is linked. We then position the research by briefly describing the New Zealand context. Next we outline the methodology used and give background information on the participants. The main part of the paper is concerned with the results of the investigation and we conclude with recommendations for teacher learning. These recommendations centre particularly on developing teacher information literacy pedagogical knowledge and their experience and confidence with online text types, which will enable students to be critical and responsible users of texts while researching online.
For the terms "Internet text" or "online text" in this paper we use the definition of Coiro and Dobler (2007) of "information (hypertext and otherwise) found within the open networked system of the Internet" (p. 12).
To be information literate, a person must be able to recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information ... information literate people are those who have learned how to learn.Within this definition there are skills such as question formation, brainstorming, categorising, skimming and scanning, evaluating printed material, using contents pages and indexes, note making, synthesising information and presenting findings, all of which can be used across many levels and areas of school. With the advent of the world wide web, students have also been searching online texts, so the skills of using search engines and databases, and evaluating online material can be added to the above.
Given this background, our research investigated the extent to which students exhibited and teachers promoted critical online information literacy skills, and we explored teachers' beliefs about students' online information literacy and students' self-reported research strategies. We found that, despite teachers often being critical of students' information literacy online, paradoxically, they still maintained that their students knew more than they did when operating in online environments and displayed a reluctance to use online text. The students in our research possessed limited online critical evaluation and information skills, yet their teachers were not addressing this in their classroom text choices.
The New Zealand National Education Monitoring Project (NEMP), which reports on the achievement of year four (8 years of age) and year eight (12 years of age) students, showed that there was little evidence of an increase in information skills between 1997 and 2005 (Crooks & Flockton, 1998; Flockton, Crooks & Baker, 2002; Flockton, Crooks & White, 2006). Few year 4 and year 8 students could describe a coherent process or strategy for finding and using information for a research or study topic, and less than 50% were able to ask two or three 'strong' questions for an inquiry, even when working with others. Students also lacked evaluative skills and were indiscriminate in their use of Internet information.
Furthermore, an Education Review Office evaluation (Education Review Office, 2005) of almost 400 New Zealand schools revealed that information literacy was not well developed in most schools and particularly not in secondary schools. There was little evidence that schools were explicitly and systematically implementing an information process model across the curriculum, and few schools collected achievement data to show improvement in student information literacy skills. Similar conclusions were reached by Hipkins (2005), whose research into the inquiry process used in internally assessed aspects of high stakes assessment for qualifications in years 11-13 (15-18 years of age), noted that many students were not being taught generic information literacy skills. Students also understood research to be no more than "information retrieval and repackaging" (p. 27).
Our research enquired into whether there had been a change in students' skills in locating, evaluating, synthesising and communicating information (Mokhtari Kymes, & Edwards, 2009) since 2005, which had also seen the implementation of the 2007 curriculum.
Year 10 English teachers in the three schools were invited to participate and the researcher administered the survey during English Department meetings. From a possible 33 teachers, 24 participated. Of these, 17 (70.8%) were female and 7 (29.2%) males. Most teachers were under the age of 46 (75%), with those in the youngest age range having been adolescents at the advent of the Internet; that is, at least 37.5% of the teachers were under the age of 16 when the world wide web began. Twenty-two teachers (91.7%) had majored in English at university, with 25% having a postgraduate qualification in English. The majority (91.7%) completed their teacher training in New Zealand and over 70% of them had 10 years' or less experience teaching English, with 45.8% having 5 years or less. Therefore, most respondents were academically well qualified in their teaching subject, trained in the New Zealand curriculum and school qualifications, young and in the early stages of their career (see Table 1).
|Teacher age ranges||University academic|
qualifications in English
|Number of years experience|
|9 (37.5%) in 21-35 range
9 (37.5%) in 36-45 range
4 (16.7%) in 46-55 range
2 8.3%) in over 55 range
|2 (8.3%) had year 2 undergraduate
16 (66.7%) had BA majors
3 (12.5%) had BA (Hons)
3 (12.5%) had MA or higher
|11 (45.8%) had 1-5 years|
6 (25%) had 6-10 years
2 (8.3%) had 11-15 years
1 (4.2%) had 16-20 years
1 (4.2%) had 21-25 years
2 (8.3%) had 26-30 years
1 (4.2%) had over 30 years
|Survey questions used to select|
diary and focus group participants
Scale: never, almost never, about once a month, about once a week, almost every day, every day
|Have you used the Internet in the past week?||Yes|
|Have you used the Internet for research in the past week?||Yes|
|How frequently do you use the Internet for your own interest?||Almost every day or more|
|If you want to find out about something you enjoy, do you read a book, magazine or find information on a website?||Website|
|Do you use a search engine when finding information?||Yes|
|Do you have a page on a networking site?||Yes|
|How frequently do you visit your networking site page?||Almost every day or more|
|How frequently do you check email?||About once a week or more|
From the 22 invited participants, who represented over 10% of survey respondents in each school, 20 diaries were completed by 8 males and 12 females and three after-school focus groups were conducted with 16 students, (two groups of four and one of eight), comprising 7 males and 9 females. Focus group sizes came within the range that Kruegar and Casey (2000) recommend as being "easier to recruit and host" and "more comfortable for participants" (p. 74).
Below is a list of page and screen texts that some students access out-of-school. Rate their
usefulness, (in your opinion), as text for enhancing year 10 literacy practices in the classroom.
|Games on screen||Play scripts||Manuals|
('how to' books)
|Text messages||Graphic novels||Film|
Students were texted reminders every night and over the 14 days a text relationship developed between the researcher and many of the students. After the first evening students were texting her back, wishing her a "gr8" weekend and so on. This had positive spin offs not just for the completion of the diaries, but also for a more relaxed start to the focus groups. Students and researcher were not strangers. The one student who did not complete the diary or attend a focus group, was the only student who was unavailable by text or phone.
Diaries asked students to fill in six columns. For computer reading this meant:
The function of the focus groups was to explore diary responses in depth to ascertain the "how" and the "why" that lay behind them. It was also to capture the student voices and allow them to explain, illustrate and expand answers in an environment with people who had similar levels of Internet activity. Diaries were completed before the focus group phase. This meant that the main researcher had a perspective on the students' habits and this informed the discussion guide. The diaries were also used in the focus groups to facilitate initial discussion.
Questions in the discussion guide were deliberately open ended, for example "tell me about a time when a teacher helped you with your research" or "what are the first steps you do when ...?" Question roots on how students accessed online information and used it for research, and what teachers had taught them about using online resources, were used to give insight into the processes and skills students used. Clarification was sought if answers were ambiguous or unclear, and students were often asked to amplify their responses.
The focus groups
Because of the different sizes of the three focus groups, responses were analysed together rather than each group being the unit for analysis. This gave more significance to the findings; it was the shared experiences of year 10 students rather than unique experiences or differences between the groups that was sought (Fern, 2001, p. 173). Initially, student comments were colour coded under two main steps in the research process, "accessing information" and "using information". As the iterative process of analysis progressed, these two research steps were further divided into categories. Student comments around "accessing information" fell into three categories: using search engines, choosing sites and judging site trustworthiness. Comments around "using information" were categorised under: reading information, judging accuracy of information and reporting research findings. This then left further student comments that fell into a third aspect, "how teachers help with research". At this point, interesting patterns emerged that we examine in our findings.
There were 10 comments from 8 different teachers (61.5%) about student preference for operating in a digital environment. All believed that students preferred to do research activities on the Internet rather than any other way and that any research was done using Google. They said that students enjoyed working with the Internet and found it an "easy option." They also believed that students frequently communicated digitally, especially via social networking sites.
Twelve teachers (92.3%) commented on student skill level on the Internet and these centred around students being well informed and confident when using the Internet, that they could access information and navigate between pages and sites, and that in all of these things they knew more than their teachers.
However, when it came to students' information literacy and critical literacy, teachers were critical of student skill levels. Twenty comments from 11 of the 13 teachers (84.6%) centred around students not being able to discriminate between information, their inability to see what was important to their research questions and their lack of skill in determining the authority of what they read. An inability to extract, analyse and synthesise information, a lack of close reading skills and not using skimming and scanning strategies were at the core of teacher comments. There were also recurring comments (92.3% of respondents) about students using 'cut and paste' as their main research strategy rather than critical information skills.
Teacher ratings of text as useful for enhancing year 10 literacy practices
The top six text types seen as extremely useful for extending students' literacy practices were the novel (58.3% of teachers), short stories and play scripts second equal (41.7%), poetry (33.3%) and fifth equal, non-fiction books and film (29.2%). When added to those rated as very useful a similar hierarchy emerged with the most popular being the novel (91.6% of teachers), followed by film (79.2%), short stories (66.7%), poetry (66.6%), biographies (54.1%) with play scripts, non-fiction books and newspapers sixth equal (50.0%). The most popular digital text was informational web pages, with 12.5% seeing these as extremely useful and 37.5% seeing them as either extremely useful or very useful. These results echoed previous New Zealand research (Ladbrook, 2009).
Teachers' reasons for not accessing hyperfiction
As well as particular attitudes to hyperfiction, this question explored teacher perceived constraints around professional development in digital text types and the use of digital technology, the availability of online access in the classroom and moving to another location for access, and personal interest in digital text and screen reading.
Table 4 demonstrates that the most important constraint for the teachers was online access in their classrooms. Couple this with the need to move a class of teenagers to another location, and access becomes a major constraint. Knowledge of digital text types and confident use of the technology also emerged as major constraints. Personal constraints such as interest and liking to read on screen were less important issues for this group of teachers.
|Reason||Extremely important||Very important||Important||Total|
|Classroom online access||54.5%|
|Knowledge of the technology||27.3%|
|On screen reading||27.3%|
|Knowledge of digital text||4.8%|
The focus groups (n = 16)
Focus group discussions centred on how students accessed and used information from the Internet and how teachers helped them with the skills involved. All 16 students (7 males and 9 females) across the three focus groups used Google as their first research step. They mentioned three strategies at this point. They all usually entered a key word, but three (18.7%) said that they sometimes asked a question and one would sometimes enter a phrase. None were using key words connected by Boolean symbols and none used a search engine other than Google.
After using the search engine, there was agreement that the strategy for choosing a site was to enter the first listed. Student reasons for operating this way were expediency and "I think it's the best one," or "It's usually, like, the first one that's the most viewed and most reliable." However, as discussion progressed there was more elaboration including "I tend to look more at what it says in the caption below...," "If I think it's something that I think relates to the topic and has some of the keywords, like lots of them" and "if it's an extremely long name I probably won't go for it because it's probably fake."
While reading more about the site and looking for key words, judgments were also made based on whether they had long names or were in bold letters. If they judged the site as not very useful, students looked at the second on the list, with some (18.7%) going to the third. None went further. It was also clear that if the students did not at first understand what they read, they did not persist but moved rapidly to another site.
After accessing a site many of the students (75%) made judgments as to trustworthiness based on what it looked like - in their words, whether it looked "dodgy." Explanations of this were based mainly on layout, graphics and colour. It was untrustworthy if "it just looks cheap" or "it looks confusing," and was trustworthy if "it's nicely presented, it's not just white background, black writing" because that "shows a professionalism" and "it doesn't have the adverts or pop ups." Also popular were sites where the layout was divided into sections, "'cause if it's, like, in one big chunk you probably wouldn't want to read it" and "if it's the subheading you're looking for then you can actually read what you're looking for, so you don't have to read the whole thing." As for the reliability of the information, most students (93.7%) were trusting after having made their initial layout judgments.
When asked by the researcher how they knew the information on the site was accurate, student responses included "You don't", "I don't know, I just go on it", and "You trust it". One boy summed up when he said "if it sounds convincing it probably is true." If there was information on what students were looking for, then they used it without any critical analysis, although one of the 16 students did say he gave thought to "what I know about the topic" showing that there was some prior knowledge check. This student was also the only one who checked several sites to ensure that they gave similar information.
The strategy students had for using the information was to cut and paste it into an MS Word document and work with it from there. This did not entail interpretation, analysis or synthesis. Their approach was simpler. All agreed that they copied those sentences needed for their research or copied and pasted the whole page and then, as one student said, "I choose, like, what I need for my research and the rest I just delete." Students also lacked confidence in their own writing ability, "you always paste it in first, then get rid of the stuff you don't want and then you can kind of write in your own words ... but sometimes it's really hard to like make it in your own words when it's written so nicely." Another noted that he did not want to "dumb it down" by making too many changes.
When writing up findings, rather than using their own report structure, students stayed with the order in which information was reported in the text they were using. If any information did not seemed to fit after their initial deletions (which they called "editing"), then that was also deleted. All students knew the need to write references for their work, but with so many only using one site for research (75%) it was interesting to see how this was achieved. Strategies ranged from listing the sources that their chosen site listed, putting in sites that they hadn't used but which had come up on their initial Google search, and making them up. As they said, "You go to Wikipedia and you look up all the references and sources and you write them down from there" and "It's not like they're going to check everyone's assignment and go onto the sites and see if there's any information there."
The students did suggest that if teachers gave them several URLs and a couple of articles or chapters in books from which to choose information, and also built their prior knowledge, then they would be unable to plagiarise because teachers would know their information sources and also it would stop them from adding false references. They suggested that the prior knowledge work would enable them to judge information more accurately.
When asked to describe a time a teacher had helped them with research it became apparent that there was little help given, although this varied from school to school. Students from one school (8 of the 16 students) noted that "They kind of expect you to know because you're already at high school." One stated that she couldn't remember a time when she had had some help. Those from the second school (4 students) reported that their teachers gave minimal help and they reiterated that there seemed to be an assumption that many research skills had already been taught before students reached secondary school. Students from the third school (4 students) discussed how their teachers sometimes gave them a list of helpful sites but when asked if their teachers had taught them how to research on the Internet, they responded in a similar way to students from the other schools.
No students reported receiving any help at secondary school when using books for research, apart from teachers taking them to the library. Again students noted that teachers tended to take it for granted that they had learned how to research when at primary school.
The most teachers were doing to help these students with online research was to sometimes recommend sites. According to the students, they were not taught how to search the Internet using a variety of search engines and Boolean terms, how to check a site's currency, relevance and the credibility of its information, how to close read and critique that information and then to responsibly synthesise information from multiple sources into a structured research report.
The many student comments about the lack of teacher help when researching and using Internet resources are also connected with their discussions that teachers assume that many of the necessary skills have been covered at lower curriculum levels. This suggests that how to diagnostically assess students' online research skills as well as how to address the results of such assessment is a critical area for teacher learning.
Teacher professional learning about information literacy and strategies to guide students in the five areas outlined by the New Literacies Research Lab (Mokhtari et al., 2009, p. 354) is necessary, if our students are to become information literate in the sense advocated by the American Library Association (1989). The type of program we recommend would also include using diagnostic assessment and knowing what and how to assess research strategies, the interpretation of results, and strategies for acting on these. Such a program should be across all curriculum areas and therefore schools would need to decide which curriculum subjects would take responsibility for the teaching and monitoring of the taught skills and strategies. The use of a school-wide model for information processing would also be beneficial as would common methods for teaching various skills.
We recommend a mix of intensive and consistent use of the technology coupled with a sector-wide initiative that includes preservice teachers (Rockman, 2005; Curzon, 2004). Recently Owston (2009) has posited that "effective professional learning ... should be long term, school based, focused on the learning of all students, and linked to the curricula that teachers have to teach" (p. 271). Owston stresses the advantages of 'blended learning' for teachers, a combination of online and face to face approaches over an extended time, and discusses several current designs for this. Such professional learning would also supply teachers with a common language with which to discuss research processes across the curriculum.
Easy classroom access to Internet resources is an issue for many teachers and one that school administrators need to rectify. But access alone will not solve the current situation. Our research has also revealed that there is a gap in subject English teachers' knowledge about the range, characteristics and pedagogical applications of digital text types that could be used to enhance students' literacy gains and critical information literacy skills. Furthermore, there is a need for all subject teachers to take responsibility for the teaching of the structure and language of research report writing in their particular subject area.
Finally, it is our contention that academic performance will be hindered if students do not have critical information literacy skills and do not know how to carry out and report research investigations in an adept, critical and responsible manner. In a knowledge-based world, these critical skills will also impact positively on their work and leisure lives. The concept of information literacy is, as Doyle (1994) points out, "central to all successful learning and, by extension, to all successful living" (p. 44) and Bruce (2002) refers to information literacy as the "catalyst" needed to transform "the information society of today into the learning society of tomorrow . . . bringing information practices that are effective in professional, civic and personal life into curriculum" (p. 4). Thus, the importance of addressing students' information literacy skills on and offline, by developing the knowledge and skills of their teachers, cannot be understated.
American Library Association (1989). Presidential committee on information literacy: Final report. Chicago: American Library Association. http://www.ala.org/ala/mgrps/divs/acrl/publications/whitepapers/presidential.cfm
Bruce, C. (2002). Information literacy as a catalyst for educational change: A background paper. In Lifelong learning: Whose responsibility and what is your contribution? 3rd International Lifelong Learning Conference, 13-16 June 2004, Yeppoon, Queensland. http://eprints.qut.edu.au/4977/
Bryce, C. & Withers, G. (2003). Engaging secondary school students in lifelong learning. Melbourne: Australian Council for Educational Research Ltd.
Coiro, J. & Dobler, E. (2007). Exploring the online reading comprehension strategies used by sixth-grade skilled readers to search for and locate information on the Internet. Reading Research Quarterly, 42(2), 214-257.
Combes, B. (2009a). Digital natives or digital refugees? Why we have failed Gen Y? Paper presented at the 38th Annual Conference of the International Association of School Librarianship: Preparing students for the future. Padua, Italy.
Combes, B. (2009b). Generation Y: Are they really digital natives or more like digital refugees? Synergy, 7(1), 9. http://www.slav.schools.net.au/synergy/vol7num1/coombes.pdf
Crooks, T. & Flockton, L. (1998). Information skills: Assessment results 1997. Dunedin: Educational Assessment Research Unit, University of Otago.
Curzon, S. (2004). Developing faculty-librarian partnerships in information literacy. In I. Rockman (Ed.), Integrating information literacy into the higher education curriculum: Practical models for transformation. San Francisco: Jossey Bass.
Doyle, C. (1999). Information literacy in an information society. In K. Haycock (Ed.), Foundations for effective school library media programs. Englewood, CO: Libraries Unlimited.
Doyle, C. (1994). Information literacy in an information society: A concept for the information age. [verified 7 Feb 2010; 3.0 MB] http://www.eric.ed.gov:80/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED372763
Education Review Office (2005). Student learning in the information landscape. Wellington: New Zealand Education Review Office.
Fern, E. F. (2001). Advanced focus group research. London: Sage.
Flockton, L., Crooks, T. & Baker, L. (2002). Information skills: Assessment results 2001. Dunedin: Educational Assessment Research Unit: University of Otago.
Flockton, L., Crooks, T. & White, J. (2006). Information skills: Assessment results 2005. Dunedin: Educational Assessment Research Unit: University of Otago.
Hargreaves, A. (2003). Teaching in the knowledge society: Education in an age of insecurity. New York: Teachers College Press.
Hipkins, R. (2005). Information literacy and student research. Set: Research Information For Teachers, 2, 27-31.
Honan, E. (2008). Barriers to teachers using digital texts in literacy classrooms. Literacy, 42(1), 36-43.
Kruegar, R. & Casey, M. A. (2000). Focus groups: A practical guide for applied research. Thousand Oaks: Sage Publications.
Labbo, L. D. (2006). Literacy pedagogy and computer technologies: Towards solving the puzzle of current and future classroom practices. Australian Journal of Language and Literacy, 29(3), 199-210. [verified 7 Feb 2011] http://www.alea.edu.au/site-content/publications/documents/ajll/Labbo.pdf
Ladbrook, J. (2009). Teachers of digikids: Do they navigate the divide? Australian Journal of Language and Literacy, 32(1), 68-82. [verified 7 Feb 2011] http://alea.edu.au/site-content/publications/Ladbrook.pdf
Lankshear, C. & Knobel, M. (2003). New literacies: Changing knowledge and classroom learning. Buckingham: Open University Press.
Leu, D. J., Zawilinski, L., Castek, J., Banerjee, M., Housand, B. C., Liu, Y., et al. (2007). What is new about the new literacies of online reading comprehension? In L. S. Rush, A. J. Eakle & A. Berger (Eds.), Secondary school literacy: What research reveals for classroom practice. (pp. 37-68). Urbana, Ill: National Council of Teachers of English.
Lonsdale, R. & Armstrong, C. (2004). Crossing the educational divide. Paper presented at the International Association of School Librarianship (IASL) conference: From Aesop to e-book: The story goes on. Dublin, Ireland.
Ministry of Education (2003a). Digital horizons: Learning through ICT. A strategy for schools, 2002 - 2004 (Revised ed.). Wellington: Learning Media.
Ministry of Education (2003b). Education Priorities for New Zealand. [viewed 18 Aug 2004, not found 7 Feb 2011] http://www.beehive.govt.nz/mallard/priorities
Ministry of Education (2004). Making a bigger difference for all students: A schooling strategy discussion document. Wellington: Ministry of Education.
Ministry of Education (2007). The New Zealand Curriculum. Wellington: Learning Media.
Moje, E., Young, J., Readance, J. & Moore, D. (2000). Reinventing adolescent literacy for new times: Perennial and millennial issues. Journal of Adolescent and Adult Literacy, 43(5), 400-410.
Mokhtari, K., Kymes, A. & Edwards, P. (2009). Assessing the new literacies of online reading comprehension: An informative interview with W. Ian O'Byrne, Lisa Zawilinski, J. Greg McVerry, and Donald J. Leu at the University of Connecticut. The Reading Teacher, 62(4), 354-357.
Oblinger, D. G. & Oblinger, J. L. (Eds.) (2005). Educating the net generation. An EDUCAUSE e-book. http://www.educause.edu/educatingthenetgen
Owston, R. D. (2009). Comments on Greenhow, Robelia, and Hughes: Digital immersion, teacher learning, and games. Educational Researcher, 38, 270-273.
Prensky, M. (2001). Digital natives, digital immigrants. Parts 1 & 2. On the Horizon, 9(5/6). http://www.marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part2.pdf
Probert, E. (2009). Information literacy skills: Teacher understandings and practice. Computers & Education, 53, 24-33.
Rockman, I. (2005). Successful strategies for integrating information literacy into the curriculum. In I. Rockman (Ed.), Integrating information literacy into the higher education curriculum: Practical models for transformation. San Francisco: Jossey Bass.
United Nations (2008). The global information society: A statistical view. [viewed 14 Apr 2009, verified 7 Feb 2011; 1.9 MB]. http://www.itu.int/ITU-D/ict/material/LCW190_en.pdf
Van Zijl, P., Bennett, D., Darling, V., Shields, G. & Bennett, E. (2006). Curriculum alignment project: Information literacy survey report. Dunedin: Otago Polytech.
Veen, W. & Ben, V. (2006). Homo zappiens: Growing up in the digital age. London: Network Continuum.
Walraven, A., Brand-Gruwel, S. & Boshuizen, H. (2008). Information-problem solving: A review of problems students encounter and instructional solutions. Computers in Human Behavior, 24(3), 623-648.
|Authors: Judine Ladbrook (corresponding author)|
Principal Lecturer, School of Arts, Languages and Literacies
Faculty of Education, University of Auckland
Private Bag 92601, Symonds Street, Auckland, New Zealand
Elizabeth Probert, Senior Lecturer, School of Arts, Languages and Literacies
Faculty of Education, University of Auckland
Private Bag 92601, Symonds Street, Auckland, New Zealand
Please cite as: Ladbrook, J. & Probert, E. (2011). Information skills and critical literacy: Where are our digikids at with online searching and are their teachers helping?. Australasian Journal of Educational Technology, 27(1), 105-121. http://www.ascilite.org.au/ajet/ajet27/ladbrook.html