Figure 1: Changes of students' perception of eBooks about being an effective learning tool
| Australasian Journal of Educational Technology 2009, 25(1), 30-44. |
AJET 25 |
Usability and usefulness of eBooks on PPCs:
How students' opinions vary over time
Paul Lam, Shun Leung Lam, John Lam and Carmel McNaught
The Chinese University of Hong Kong
An Outstanding Paper Award recipient, ascilite Melbourne 2008 ConferenceThe number of academic eBooks in university libraries is increasing. Reading digital materials such as newspapers is valued in Hong Kong; however, students' perceptions of the value of reading academic eMaterials are not well known. In this study we investigated 12 students' use of academic eBooks on mobile readers in two ways. Six students had brief introduction to eBooks and they reported their first impressions of the technology. The other six students had a 12-week eBook reading period that was closely monitored by the research team. The experiences from the two groups of students generally affirmed that the technology has potential to enhance teaching and learning in a university setting. However, the experiences (especially of the long-term users) highlight a number of challenges that need to be addressed.
EBooks are "text in digital form, or digital reading material, or a book in a computer file format, or an electronic file of words and images" (Rao, 2003, pp. 86-87). EBooks are downloadable and are portable if they are stored in light portable devices such as pocket personal computers (PPCs) or smartphones.
The technology has also made possible a growth in the number of publications with shorter publishing times. The rising costs of paper books and the potential to link multimedia resources to eBooks have major impacts on the strategies used by university libraries in their support of scholarly communication (Ching, Poon & McNaught, 2006). There are 247 entries in a list of available eBook libraries (Cavanaugh & Cavanaugh, 2008). One of the largest collections of academic eBooks collections is NetLibrary, which at present has over 20,000 volumes of academic eBooks.
The Library of The Chinese University of Hong Kong (CUHK) has made a significant investment in licences to many leading eBook collections in the world (including NetLibrary). At CUHK, the use of eBooks is still not high. Since the NetLibrary system was launched in February 2006, there have been about 4,000 to 7,000 visits a month (cf. at present there are over 20,000 students and 1,100 teachers at the University). The present study is designed to provide the information needed for successful eBook take up at CUHK. The S-curve of development that is the characteristic of diffusion of innovations (Rogers, 2003) suggests that acceptance of a new technology does not undergo a stable and linear development. The level of adoption is very low at the beginning. The greatest challenge is achieving the 'growth' stage, but this is seldom achieved without careful planning and focused strategies.
The phenomenon of eBooks is quite different from the now widespread use of eJournals. The length and structure of a book, and our traditional mode of reading books, make the experience of reading a book quite different from reading a focused and relatively short article or report. It is therefore important to consider how academics and students will (or won't) adapt to reading whole books onscreen.
However, there are also negative reports of the use of the technology in the academic setting. Not all students can feel comfortable with the technology. For example, some students find reading long text on a small computer screen tiring, less efficient and even 'painful' (Wilson, 2003, p. 11) compared with reading paper based text. Wilson (2003) also reported complaints about the ineffective navigational controls on eBook readers, as being "awkward, difficult or time-consuming to use" (p. 14).
The success of the eBook technology seems to rely heavily on the ease of the reading process. Malama, Landoni and Wilson (2004) found that users regard the layout of the eBook on screen as a strong motivating force; also important are the ease of navigation through a clear user interface, and a clear and logical structure to support readers' sense of place in the eBook. The optimal use of screen real estate is even more important with mobile eBooks than with desktop computers. Wilson, Landoni and Gibb (2002) used the book metaphor as a crucial design feature, including having a cover, a table of contents, a consistent and tidy typographical style, simple classification of book sections, and cues for reading progress.
Features available on eBook readers can be extensive. Apart from page turning, scrolling of text, and searching, more advanced eBook-related functionalities are constantly being developed. Wilson (2003), through collecting user feedback on using eBooks, found that users appreciate a quick and accurate search function, an annotation function, and the inclusion of hypertexts and dictionaries on the reader. Lam, Lam and McNaught (in press) evaluated the perceived usefulness of the functionalities on four popular eBook readers on the market which can be used on PCs and PPCs and found them, in general, to be acceptable.
The first phase focused on students' first impressions of reading eBooks on a PPC. Six students were invited to participate in a number of sessions: introduction to the technology, guided hands on training with the hardware and software, and a guided reading session. Care has been taken to make sure the students:
In the second phase, another six students (referred to as Students 1 to 6 below) were invited to use eBooks in a more naturalistic fashion for an extended period of time (12 weeks). Our interest in this phase was to investigate the actual use of the technology as a learning tool. The criteria for student selection in this phase were the same as in the first phase (the six students were from these academic disciplines: psychology, medicine, accounting, information engineering, economics and chemistry). Each of them received on loan from the project a PPC for the reading tasks. Adobe Reader was the suggested PPC eBook reader to use. The PPCs involved were Mio DigiWalker P550, HP iPAQ hx2400 and HP iPAQ hx2700. The training was similar to that in the first phase. However, the post-training reading extended for about 50 hours over 12 weeks. The reading period was spread over four months and the students were advised to spread their reading load evenly over the period. The students were free to choose what they would like to read as long as they were academic books related to their own disciplines.
Through this two phase research design, we looked at two main issues concerning eBooks: usability and usefulness.
The usability issue relates to the practicality of the various procedures required to use the software and hardware, and the ease of use of the technology. It relates to issues such as interface design that facilitates effective human-computer interaction (Wilson et al., 2002). In this study, there are three sub-questions:
| Main evaluation questions | Useable? | Useful? | ||||
| 1. Easy to set up | 2. Easy to learn with | 3. Easy to use | 4. Enjoy | 5. Under- stand | 6. Willing to use | |
| Phase 1 pre-reading | Yes | |||||
| Phase 1 function learning and reading | Yes | Yes | Yes | Yes | Yes | |
| Phase 1 home reading | Yes | Yes | Yes | |||
| Phase 2 function learning | Yes | Yes | ||||
| Phase 2 extended reading | Yes | Yes | Yes | Yes | ||
Below are the procedures involved in the pre-reading setup:
In the training sessions of phase 2, the students could learn the procedures to use NetLibrary, download eBooks and transfer them to the PPCs. No significant problems were found. Minor problems were resolved on the spot. Students also seemed to have positive opinions of the technology at this stage as revealed in the pre-reading questionnaire they filled in right after the training. More problems arose, however, during the following weeks when the students were actually using the eBooks on their own and were using them more extensively. For example, the students mentioned the following challenges during the use in weblogs and/or in interviews.
| Operation | Easy to learn? (No. of students needing help during training) | Easy to use? (No. of ongoing challenges) | ||
| Phase 1 | Phase 2 | Phase 1 | Phase 2 | |
| Problem concerning changing the display format | 7 | 5 | 5 | |
| Read in the view such that no matter what the magnification is, the text will be automatically rearranged to fit the width of the document pane | 2 | 3 | 2 | |
| Read in full screen mode | 3 | 1 | 3 | |
| Rotate 90 degrees clockwise to read | 1 | |||
| Reflow the passage whenever calling on the Default Zoom function | 1 | |||
| Exit the full screen mode | 1 | |||
| Problem concerning navigation | 6 | 4 | ||
| Jump to the next page | 1 | |||
| Back to the last two pages you visited without typing the page numbers | 2 | |||
| Jump to the first page that contains a picture (except the title page) | 1 | |||
| Tap the bottom of the screen, instead of dragging and scrolling, to proceed reading | 2 | 1 | ||
| Problem concerning jump to page | 1 | |||
| Problem concerning the 'Back' function | 2 | |||
| Don't know how to add toolbar button | 4 | 2 | ||
| Don't know how to add toolbar button | 2 | 1 | ||
| Don't know that they should add a button | 2 | 1 | ||
| Problem concerning searching | 2 | 4 | 2 | 2 |
| Search the words "They are", where capital "T" in the word "They" is important, starting from the first page | 1 | 2 | ||
| Search the word "an", starting from the first page | 1 | |||
| Don't know how to find the next searching result | 1 | 1 | ||
| Misunderstand and misuse of searching functions | 1 | 1 | ||
| Don't know the function of 'auto-scroll' | 1 | |||
| Don't know the function of 'bookmark' | 1 | |||
| Problem concerning basic operations of PPC | 8 | |||
| Check the free memory in PPC | 2 | |||
| Close unused program on PPC | 4 | |||
| Display the viewing menu (similar to the right click of mouse on PC) | 2 | |||
| Totals | 19 | 17 | 13 | 2 |
It can be seen that students in phase 1 seemed to have more problems in using the functions than those in phase 2. Taking into consideration that phase 1 students used these functions right after they had learnt about them while the phase 2 students were using them in a more extended timeframe, the difference between the two groups is easy to understand. The difference actually also suggests that many of these functions can be effectively self-learnt, given enough time, and so students in phase 2 did not need to resort to the researchers for help.
In phase 1, most students said the eBook reading process was enjoyable because it is convenient to read portable eBooks. Some commented that the novelty factor made it interesting. There were, however, some negative comments. Two students said their eyes were tired when reading eBooks, while two students mentioned limitations related to the hardware. One of them said that the eBook could only be read indoors because of lighting issues. A number of students told us that whether eBook reading is enjoyable depends on the content of the eBook. This suggests that success of eBooks rely on multiple factors: hardware, software and as well as the materials.
In phase 2, more negative comments were heard. Based on our conversations with the students in the interviews and the students' blog entries, we drew up a list of 19 existing characteristics (Table 3) that influence the perception of how useful the technology is for learning. At the end of reading questionnaire, these characteristics were listed for the students to rate: firstly, their opinion of how important the feature was for learning, and secondly, how satisfied they were about how these features worked. It may not be very meaningful to study features that were regarded as less important to the learning process. Thus, 10 features were identified (italics in Table 3). They were features that received an average score of above 4 (out of 5) in importance by the six students. The discussion below focuses mostly on these 10 features.
Many of these features that were regarded by the students as highly important were found by the students to be less satisfactory. Many of them had an average score of below '3'. Students 1, 2 and 5 were particularly negative. They only complimented features d> (the PPC operates and reacts quickly (i.e. fast to open the files and to use the functions)) and e (there are no restrictions of opening hours in NetLibrary). However, on the other hand, they rated a low score to many (five to seven) of the ten most important features.
| Features | Importance (scale of 5) | Satisfaction (scale of 5) |
| a. The usable time after charging up the PPC is long enough. | 4.5 | 2.7 |
| b. The PPC (hardware) always works well. | 4.3 | 2.5 |
| c. NetLibrary has a large collection of academic eBooks that seem to be useful to me. | 4.3 | 2.2 |
| d. The PPC operates and reacts quickly (i.e. fast to open the files and to use the functions). | 4.3 | 3.8 |
| e. There are no restrictions of opening hours in NetLibrary. | 4.3 | 4.3 |
| f. The functions of Adobe Reader (PPC version) can always be used smoothly. | 4.2 | 3.5 |
| g. The eBooks in the NetLibrary are of interest to me. | 4.2 | 2.0 |
| h. NetLibrary is a useful place to go for finding information related to my study. | 4.2 | 2.2 |
| i. The PPC screen is big enough for easy reading. | 4.2 | 2.5 |
| j. The PPC is light in weight. | 4.2 | 4.0 |
| k. The time required to charge up the PPC is appropriate. | 4.0 | 2.8 |
| l. No traveling to the library is required if I borrow a book in NetLibrary. | 4.0 | 4.2 |
| m. It's convenient to renew the eBooks. | 4.0 | 2.5 |
| n. I can comfortably use a PPC to read eBooks during travel. | 4.0 | 3.5 |
| o. It is easy to self-explore how to use the functions of Adobe Reader (PPC version). | 3.8 | 3.2 |
| p. The functions provided by the Adobe Reader (PPC version) are plentiful. | 3.8 | 2.7 |
| q. The expiry date of an eBook in NetLibrary is long enough. | 3.8 | 2.2 |
| r. I can change the display format easily (e.g. rotate screen, change to full screen mode, and reflow the page, etc). | 3.8 | 3.3 |
| s. Navigation is easy (e.g. jump to different pages, search for specific information, and go to next and previous page, etc). | 3.2 | 3.0 |
Their feedback from interviews and weblogs also echo the similar comments:
When we compared the survey scores concerning enjoyment and comfort in using eBooks between the two groups of students, we found a large contrast (Table 4). Care is needed in our interpretation of the data, particularly in relation to the small number of students who were involved in the study. Nevertheless, the data served to indicate interesting patterns which can be the subject of future research. It seems that the more experience the students had with eBooks, the less they found the technology enjoyable and comfortable to use.
| Concepts measured | Phase 1 | Phase 2 | ||
| Mean | SD | Mean | SD | |
| Overall easiness of the eBook software functions | 4.8 | 0.3 | 3.3 | 0.8 |
| Enjoyment of the eBook reading process | 4.0 | 0 | 2.7 | 0.5 |
| Comfort in reading on the PPC screen | 4.3 | 0.6 | 3.4 | 0.8 |
| Assistance of eBook Reader features on reading | 4.0 | 0 | 3 | 0.7 |
| Assistance of eBook Reader features on understanding | 4.0 | 0 | 3.2 | 0.7 |
| Learnability of eBooks | 3.9 | 0.2 | 2.7 | 1.0 |
| Future use of eBooks | 4.0 | 0 | 2.8 | 1.2 |
In the interviews held after the phase 1 home reading sessions, the students noted that this inefficiency could be because they were not familiar with reading eBooks and concentrated less when reading eBooks. Besides, more time was spent processing the eBook content. A number of students said that they read eBooks more slowly than they read paper books. They said they were not familiar with eBook reading; their eyes would get tired easily when reading eBooks, and each reading session was quite short.
In phase 2, the students were asked to rate in the end of reading questionnaire whether they understand the content from reading text on a PPC versus reading text on paper. The mean score for reading on a PPC was 2.7 and for reading a paper book was 3.8.
On the contrary, we have reason to believe that these students did not form a habit of using eBooks and would not be likely to continue using eBooks after this research study. For example, all the students were found to read only the minimum amount of reading as required by the project on the PPCs. If the students found the technology helpful, we would expect them to use it to read other non-required books as well. That just did not happen.
Negative opinions on using eBooks as a whole were also collected in the end of reading questionnaire and in the interviews/ weblogs:
Figure 1: Changes of students' perception of eBooks about being an effective learning tool
In interviews, Student 4 said that an eBook is good if the paper version of the book is too thick. Student 6 thought he would choose eBook if it is a reference book and it needed to be carried around all the time. It is interesting to note that Student 4 is a student of Information Engineering and Student 6 is a Chemistry student. Judged by their disciplines, they may be more capable and comfortable with technology.
Many of the features and functions are available in the eBook software seem to be easy to learn and use. However, there are concerns with the various other aspects of the technology (such as the NetLibrary, the processes of finding and downloading the eBooks, the hardware, and the reading process itself), and these in turn had negatively impacted on students' views about its usability, usefulness and the likelihood they would actively use the technology. The lesson is that the success of the technology as an effective learning tool relies on every aspect of the technology working well.
The setup procedures are cumbersome and difficult. They put unnecessary stress on students who use the technology often and who are not particularly comfortable with computers. The process of downloading and then transferring the eBooks to the PPCs is also not straightforward. The Adobe Reader and PPC are not stable and sometimes lead to unexplained failures (for example, what happened to Student 1 in phase 2). The authentication process that is required before eBooks from a PC can be transferred to a PPC is also problematic.
The short loan period decreases the overall practicality of the technology. A more comprehensive set of book titles is also necessary. Although not mentioned by the students in this study, as each of the six students in phase 2 read a different book, we realised that there are only a certain number of copies available for loan by students in a given institution, limiting the value of eBooks as required texts. All these points may well be addressed as the system matures.
The functions and features of the software can be further improved too. The students in phase 2 mentioned some new features they thought would be very useful to their eBook experience. Additional new functions that suggested were the functions for jotting down notes and/or highlighting text; these are very important reading strategies for our students (for whom English is a second language), particularly when they are reading academic books. Other areas such as navigation of lengthy text and viewing large images need improvement. Also, students wish to download the eBook directly from NetLibrary to the PPC, rather than via a PC.
The current PPC hardware has limitations which inhibit reading. The small screen restricted students from scanning through the whole page, and thus slowed down their reading speeds. More importantly, viewing diagrams and images is difficult. The power supply also depletes quite rapidly. Students needed to charge the battery very frequently and this process also takes a long time. The PPC can be unstable, as in the case of Student 1 who had a malfunctioning of the device's backlighting.
Despite the negative opinions of the students about eBook technology, we should not be too pessimistic about the strategy yet. Advantages of eBooks are also obvious from our study of the students' reading habits. Portability is certainly a very important strength for eBooks. The students told us that much of the eBook reading was done during their travel to and from the University. If they were reading a large paper book, they would find it too heavy to carry around and too cumbersome to open for reading in public transport. One student commented that she could carry dozens of eBooks with no additional weight.
We are aware of the limitation of this study in that the number of subjects was small. Future investigations should include more students in authentic learning environments - that is, within the context of actual courses. At this stage at CUHK, teachers give eBooks as references in courses but not as primary references, for the reasons explained above. Overall, we consider eBooks to be a technology still in a developmental phase.
Ching, H. S., Poon, P. W. T. & McNaught, C. (Eds.) (2006). eLearning and digital publishing. Dordrecht: Springer.
Hernon, P., Hopper, R., Leach, M. R., Saunders, L. L. & Zhang, J. (2006). E-book use by students: Undergraduates in economics, literature and nursing. The Journal of Academic Librarianship, 33(1), 3-13.
Lam, P., Lam, J. & McNaught, C. (in press). How usable are eBooks in an mLearning environment? International Journal of Web Based Communities (IJWBC).
Lemken, B. (1999). Ebook: The missing link between paper and screen. Designing Electronic Books Workshop. In Proceedings of the ACM CHI99 Conference on Human Factors in Computing Systems. Pittsburgh, PA.
Leung, C. H., Chan, Y. Y. & Chan, C. S. C. (2003). Analysis of mobile commerce market in Hong Kong. ACM International Conference Proceeding Series, 50, 408-412. Pittsburgh, Pennsylvania.
Malama, C., Landoni, M. & Wilson, R. (2004). Fiction electronic books: A usability study. Eighth European Conference on Research and Advanced Technology for Digital Libraries, Bath, 12-17 Sep. http://www.springerlink.com/content/nehrdgnk1b8wj9vq/fulltext.pdf [viewed 29 Jul 2008].
Morton, D. A., Foreman, K. B., Goede, P. A., Bezzant, J. L. & Albertine, K. H. (2007). TK3 eBook software to author, distribute, and use electronic course content for medical education. Advances in Physiological Education, 31, 55-61.
NetLibrary. http://www.netlibrary.com/ [viewed 29 Jul 2008].
Rao, S. S. (2003). Electronic books: A review and evaluation. Library Hi Tech, 27(1), 85-93.
Rogers, E. M. (2003). Diffusion of innovations (5th ed.), New York: Free Press.
Shiratuddin, N., Landoni, M., Gibb, F. & Hassan, S. (2003). E-book technology and its potential applications in distance education. Journal of Digital Information, 3(4). [viewed 29 Jul 2008] http://jodi.tamu.edu/Articles/v03/i04/Shiratuddin/
Simon, E. J. (2002). An experiment using electronic books in the classroom. Journal of Computers in Mathematics and Science Teaching, 21(1), 53-66.
Wilson, R. (2003). Ebook readers in higher education. Educational Technology & Society, 6(4), 8-17. http://www.ifets.info/journals/6_4/3.pdf
Wilson, R., Landoni, M. & Gibb, F. (2002). Guidelines for designing electronic books. Sixth European Conference on Research and Advanced Technology for Digital Libraries (ECDL 2002), Rome, 16-18 September. [viewed 29 Jul 2008] http://www.cis.strath.ac.uk/research/publications/papers/strath_cis_publication_27.pdf
This article received an Outstanding Paper Award at ascilite Melbourne 2008 Conference, gaining the additional recognition of republication with minor corrections in AJET. The reference for the Conference version is:
Lam, P., Lam, S. L., Lam, J. & McNaught, C. (2008). Usability and usefulness of eBooks on PPCs: How students' opinions vary over time. In Hello! Where are you in the landscape of educational technology? Proceedings ascilite Melbourne 2008. http://www.ascilite.org.au/conferences/melbourne08/procs/lam.pdfAuthors: Paul Lam, Shun Leung Lam, John Lam & Carmel McNaught Centre for Learning Enhancement And Research (CLEAR) The Chinese University of Hong Kong, Shatin, Hong Kong Further details at http://www.cuhk.edu.hk/clear/ Contact author: paul.lam@cuhk.edu.hk Please cite as: Lam, P., Lam, S. L., Lam, J. & McNaught, C. (2009). Usability and usefulness of eBooks on PPCs: How students' opinions vary over time. Australasian Journal of Educational Technology, 25(1), 30-44. http://www.ascilite.org.au/ajet/ajet25/lam.html |