A = Agree; N = Neither agree nor disagree; D = Disagree. Where numbers do not add up it is because student did not respond.
x = the question was not asked about that tool.
| Australasian Journal of Educational Technology 2010, 26(7), 966-979. |
AJET 26 |
Students' perception of supplementary, online activities for Japanese language learning: Groupwork, quiz and discussion tools
Keri Moore and Sumiko Iida
University of New South Wales
This paper discusses the frequency of use and the reasons for non-use of the online component of a course developed in Blackboard 9, one popular learning management system (LMS) in tertiary education. The study particularly focuses on the three tools in the LMS: Groupwork tool, Downloadable quiz and Discussion forum. These tools were used in a Japanese upper beginners-level course in 2009. At the end of the semester, a student survey was conducted. The survey result indicates that although use of the online component of the course and of the individual tools differed, the frequency of the student use of online supplementary course materials and tools has an effect on their positive perception of the educational value, the development of scholarly skills, that this type of teaching strategy makes. The frequency of student use is determined by their attitude towards online learning, their ability to use the different functions, their peer's use of the same tools, and their access to computers.
The current trend in the higher education sector is to progressively introduce more online course modules to courses within all manner of programs. Whether students fully utilise online components of courses and individual tools, and whether they view these strategies as adding value to their educational experience is not known. Because students use one kind of technology in their personal lives it cannot be assumed automatically that they will be immediately competent in the effective use educational technology. If students are not comfortable with and have no interest in using technology, then any intended educational value will not be realised. Thus it should not be automatically assumed that students like to learn through technology.
The course used for this study was an upper beginners level of Japanese communication course, taught in 2009 by one of the authors of this paper. Since the introduction of LMS at the University of New South Wales, this course and its prerequisite courses were taught using WebCT Vista. Prior to the whole university shifting its use of LMS product to Blackboard 9 from 2010, the course tried out Blackboard 9 as a pilot with 17 other courses across the university. Using Blackboard 9 was therefore the first time for both the instructors and the students. In this iteration the following tools were used:
The questions were:
By doing so, the aim of the study is to expand on the current scholarly discourse concerning the use of technology in tertiary language education. Previous research on the use of technology in second/foreign language teaching focused on: the effectiveness of the computer-mediated communication as a learning and teaching resource (Ramzan & Saito, 1998); the use of a telecollaborative learning environment (Belz & Kinginger, 2002); audiographic conferencing (Hampel, 2003); electronic discussion (Hanna & de Nooy, 2003); synchronous chat systems used to explain technical concepts to student (Day, Wood, Scutter & Astachnowicz, 2003); online role plays ( Milton, 2004); students use of electronic course supplements, such as film to improve their levels of proficiency (Bueno, 2006); the development of a tool to evaluate oral tests (Laborda, 2006); the use of computer technology in the assessment (Hunt, Neill & Barnes, 2007); problems and strategies in oral communication tasks (Lam, 2007); online interaction with native speakers (Lee, 2004; Mahfouz & Ihmeideh, 2009); Learners' scaffolding using social network service (Christensen, 2010); and video presentation on students' blogs to enhance public speaking skills (Shih, 2010).
It was determined from the above review that there is a scarcity of information about the students' frequency of use of online tools and the reasons for their use or non use in a second/foreign language learning situation. Further, there is limited literature focusing on the students' perception of the educational value of online supplementary course work and tools.
With these theories in mind, the Groupwork tool was chosen so that students could develop one of their assignments: public speech in Japanese. It was envisaged that through the use of the Groupwork tool, students would draft the videos presentation of their speeches and share them with classmates so that they could receive and give feedback. Using the Groupwork Tool was assumed to develop the students' understanding of Japanese, ability to share and to work in a group, to communicate, their ability to reflect, critically evaluate, and to study independently.
In contrast, the Online downloadable quiz was developed with the notion that students wishing to achieve their greatest potential in the course would engage in practice exercises to develop their knowledge and skills. The quiz was not an assessable task but for the revision of the course content. The quiz took a variety of formats such as ordering phrases, gap-fill, match-up, and rewriting sentences. The aim of using this tool was to develop the students' understanding of Japanese, skills in reflection, and capacity for independent study.
Using a Course discussion tool was expected to enable students to continue the topics of discussion they held from classes in every week. Students were also encouraged to use the tool for community collaborations, to exchange information and ideas about Japanese culture. It was envisaged that by using this tool, students would enhance one another's understanding of Japanese, sharing of knowledge, communication, and skills in reflection and in using information technology.
Due to the nature of the course, different types of assessment tasks including quizzes, tests, presentations, homework and an exam, were set to assess student's reading, writing, listening and speaking skills. In addition, the students were advised that they were required to access Blackboard 9 at least twice a week throughout the semester. The expectation was that by regular access to the supplementary course materials and learning strategies, students would develop the necessary skills to be successful in the course. It was also hoped that in doing so, students would experience a stronger sense of community and a friendlier, more productive learning environment.
The aims of this evaluation then, was to contribute to the literature in e-teaching and e-learning by exploring the success of our endeavours, as far as the students perceived them to be.
At the beginning of the semester, students were informed about the study and introduced to the use of Blackboard 9, particularly these tools during the course.
For the group work, students were firstly asked to sign up to a group, then to select applications (available under the Group work tool) they wished to use prior to the actual group activity. Once they settled in each group, they were advised to commence their work by 1) video record their self introduction in Japanese, and embed the video via YouTube to get to know each other, 2) exchange the idea of the speech topic, 3) upload speech drafts (up to the second draft uploading was mandatory), 4) video record the presentation practice. All of these semester-long tasks encouraged students to comment upon posts by others in the group.
For the quiz, out of 5 chapters taught during the semester, 3 were made available online, including PDF files of similar quizzes for the students to download. Quizzes for the last two chapters were not made online due to a variety of formats that were difficult to fit online. Each quiz was released in the week the chapter was completed for students to self-test their knowledge and skills.
As for the discussion tool, three forums were created: weekly course related discussion, Japanese popular culture discussion and Japanese traditional culture discussion. These forums were open to students to share their questions, answers, opinions, requests and experience of the course and Japanese culture.
Of these three tools, the discussion tool did not require any instruction as it had been commonly used often by students in their earlier courses. Also, instruction for the quiz tool was not necessary, but students were advised every time a new quiz was released. However, the group tool needed a clear technical instruction, particularly on video embedding. Thus, an information sheet was created and uploaded to the Blackboard 9 site, and a demonstration of the group work was given in the lecture. Students who experienced technical problems during the semester contacted the instructor for individual support.
Data was collected by a survey presented as hard copy and conducted in the last class of the semester. Students completed the survey anonymously.
The validity of the survey content and its construction were strengthened by the preparation process which included the extensive literature search and also a review by several groups of academics. The majority of questions/statements were quantitative in that they required students to respond on a Likert scale, either: 'Strongly agree'; 'Agree'; 'Neither agree nor disagree': 'Disagree' or 'Strongly disagree'. In addition, there were some qualitative, open ended questions in which students were asked to give examples or further explain their responses to the quantitative questions. Plus, the students were asked to indicate if they accessed the tool frequently, moderately, less than expected or hardly ever. Where applicable, the students were asked to select from a number of choices, the reasons for their non-use of the course materials.
All data was entered into SPSS and in the first stage of analysis the students' responses were analysed for frequency. During that phase of data analysis, deductive analysis showed that even with visual examination, patterns of responses were clearly different based on student's answers to the question about how often they used a tool. Therefore, the responses to questions about the three tools were divided into two groups for analysis.
Secondly, a Hotelling trace coefficient was utilised, which presents a statistic for a multivariate test of mean differences between two groups. Thirdly, Levene's test of equality of error variance was used. It tests the null hypothesis that the variances of the group are the same. If this is greater than a selected percentage (usually 5%) then it is considered too great to usefully apply parametric tests.
The qualitative comments underwent inductive analysis in that they were analysed manually which allowed categories of themes to be developed (Charles & Mertler, 2002). The results are presented below.
The online resources helped the development of scholarly skills such as the:
A = Agree; N = Neither agree nor disagree; D = Disagree. Where numbers do not add up it is because student did not respond. x = the question was not asked about that tool. |
Hotelling's Trace found F(16,184)=2.616 and (p<0.001). This means that students perceived the Groupwork tool differently depending on their frequency of use, their level of involvement overall. There was one exception: the analysis of the question regarding the statement 'the Groupwork tool improved my ability to share my knowledge and opinion with others', showed that the error variance was not equal according to Levene's Test, which turned out to be significant. So the results of the students' response to that statement will be interpreted with caution. Table 1, shows the spread of students' responses to questions about the Groupwork tool.
The qualitative data showed the following themes emerged from the students' comments about the lack of use of the Groupwork Tool. They were:
Suggestions for improvements included requests for more flexibility and more information and to be able to retake the test; to allow the tests to be retaken several times and for answers to be provided. Also, some students wanted more precise rewards for effort; they wanted the results of the quiz to be included in their grades.
The aim of selecting the quiz tool was to develop the students' skills understanding of Japanese, reflection and independent study, and this seems to be true with the exception that improving their understanding was not as clearly established.
With regard to the qualitative data, only a handful of students gave reasons why they did not access the discussion tool. They were as follows:
It was anticipated that the discussion tool would enhance student mutually understanding of Japanese, sharing of knowledge, communication, skills in reflection, and in using information technology, all seemed to be realised except there are some questions over the development of communication skills that have not been resolved.
The qualitative comments suggest that some students seemed to struggle to adjust themselves to a new LMS environment. This may be due to the fact that the use of Blackboard 9 in the course was limited to only 18 courses amongst over a few thousands of courses where WebCT Vista was mainly used. As a result, the students had to use two different LMS applications for the courses they studied in the semester.
Even so, the majority of the students accessed Blackboard 9 a few times a week as required in the course. Well over half the respondents thought that it helped them to develop scholarly skills such as preparation for class and assessment, and that it integrated their course work, that it helped them feel part of part of what? and to develop a scholarly community, to express their ideas, interact and seek and receive feedback, and it added value to their face to face interactions. The students also said the LMS had a positive impact on their progress in the course. The only area in which the students had mixed views was in relation to whether or not the online module helped them explore the subject matter more deeply. This means that the students' low access to the course module does not directly infer that they do not value online learning as an effective education medium.
Closer inspection of the (n=57) students' responses outlining reasons for their non-use of the online supplementary course activities found that they did not like Blackboard 9 in particular, (n=31), they do not like online modules in general (n=13), and the home environment was not amenable (n=7). Only two students entered their negative views on learning with technology. These numbers are significant and cannot be ignored. For the students who are not keen on technology, a hands on exercise is needed. Also, it would be advantageous if such students learned that the online course modules are not as difficult as they believe at the beginning of the semester. For students who do not have broadband access at home, it is a difficulty, but this is a reality and we should not expect that it is always the case that students can afford broadband access. Of course, the university has a number of computers for students to use on campus, and perhaps such students can use them to do the tasks or submit assignments, but students recount that these computers are not convenient for updating course related information at the pace that is required by the instructor. This home computer network problem seems to be always the case for some students (e.g. Shih, 2010). In order for students to catch up with the course related information it may be useful to trial a link between the course module and their mobile phones or portable electronic device, which almost everyone uses nowadays.
The following are reflections upon the results with each of the three tools used in the course.
Secondly, a major technical problem was encountered in the uploading of the self-introduction videos students had recorded using a webcam. It was one of the required activities in the group work for the purpose of getting know each other quickly. During the Blackboard 9 training session for teachers, it was advised that the video upload could be easily done by copying and pasting embedded code in the HTML window of the text editor. However, it was not until the students started to work on the task that we realised that the function was available only on the teacher window with the 'editor' button, despite the text editor in the student window showing a tool icon to embed video streaming. Students who followed the video embedding process thus ended up failing. After a number of consultations and discussions, a way to upload a video from the student window was found, but it required a complex editing of the original embedded code. As a result, many students could not handle the technology and the instructor had to intervene. Students seemed to have been discouraged from uploading videos since then and no student posted a video of presentation practice.
Thirdly, as one student commented, there were students who attempted to use the function actively, but they were discouraged by the inactivity of other students who did not post anything, apart from a required initial video introduction, and thus gave up the group communication on the tool.
Fourthly, although the public presentation itself is an assessable task, no clear mark was allocated to its preparation using this group work tool. Thus, it seems that many students ignored it.
These points suggest that the Groupwork tool did not work as effectively as had been expected. Which one of these factors most directly affected students' inactive attitude in the use of the Groupwork tool is unknown. In fact, many students wrote positive comments about the tool. Given these responses it is assumed that solving of the technical problems would enhance the students' use of the tool in future classes. In order to achieve this goal, it would be ideal for students to give feedback to each other on their work in progress. It would also be useful if they could rehearse their own presentations in terms of verbal and nonverbal style, prior to the actual presentation.
A few students pointed out the technical inflexibility of the online quiz. It was indeed difficult to create a quiz for a character based language course. The match-up, and multiple answer type of questions were not the problem, but when making quizzes that ask students to write in such as a gap filling quiz, some difficulties were encountered. The computer simply marked as wrong some answers that should have been marked as correct had they been completed on paper and marked manually. Thus, unfortunately, we must admit that this tool (particularly online quizzes) or the way it was used in the course was premature, and did not work effectively as had been expected. Finally, a few students suggested including the quizzes as a part of an assessment task. The reason why it was not included in the formal grade was due to concerns regarding academic misconduct. This is a task that can be carried out by anyone, which means if students wish to strive for a higher mark, they might easily ask a native speaker of Japanese or a senior student of Japanese to do it.
Anecdotal evidence suggests that both teachers and students felt uneasy using this tool compared with the one on WebCT Vista. Unlike WebCT Vista in which only one discussion can have multiple categories where multiple topics can be set and viewed without leaving the top page, Blackboard 9 allows only access to one forum at a time. Thus for example, after reading a post under a weekly course related discussion forum, in order to see a post in Japanese culture discussion, the student must go back to the top page. Students who had been used to discussions on WebCT Vista became very frustrated by this, and as a result, the number of total posts in the discussion tool on Blackboard 9 was only 168, which was a large decrease compared to 398 in the previous semester when WebCT Vista was used.
The results show that the majority of students used the Discussion tool less than expected (n=43), yet the majority agreed the tool helped them to share information, improved understanding and reflection, but not apparently, their communication skills. Although most of the students' comments were what one would typically expect, such as they felt they had nothing to contribute, others liked the idea of instant discussions. Three students made particular comments that they preferred WebCT Vista for this particular tool.
When the findings are considered against the principles of adult learning theory (Knowles, Elwood & Swanson, 1978) it seems that students may bring their preconceived ideas about learning via technology and it may not be possible in every instance to overcome their preconceptions. As far as social learning theory (Bandura, 1986) is concerned, the lack of use of the tools in this iteration of the course limited the possibility for students to learn from each other.
It is clear from this study that the frequency with which students use online supplementary course materials has an effect on their positive perception of the educational value of the tools. However their frequency of use is determined by course requirements, student attitudes to online learning, their ability to use the different functions, their classmate's use of the same, and their access to computers. What cannot be determined is, whether at the outset students held preconceptions about online learning activities being a negative experience. Alternatively, they may have had a positive expectation and tried the online activities in this course and did not like them, or they may not have wanted to try.
To answer our first question: "what is the students' frequency of use, and what are the reasons for the non-use of the online coursework and tools and how they use or do not use them?": of the 57 students (52%) who gave reasons for their non-use of the online supplementary course materials, (n=31, 29%) were unsatisfied with Blackboard 9 itself and (n=13, 12%) do not like online learning. Given this result, the main reason for student non use of the online course module is due more to the application itself rather than LMS per se. However, it is necessary to point out that the majority of the students had been using WebCT Vista for more than a year since they entered the university and were used to using it. Thus, it seems that the problem they encountered more in the transition to a totally new application with a completely different structure than in using the Blackboard 9 itself.
To answer our second question: "what is the students' perception of the educational value of the tools and is that associated with their frequency of use of the tools?": the answer seems to be 'yes', there seems to be a significant difference in perception of usefulness of the tools depending on the frequency of use.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.
Belz, J. & Kinginger, C. (2002). The cross-linguistic development of address form use in telecollaborative language learning: Two case studies. The Canadian Modern language Review, 59(2), 189-214.
Bueno, K. (2006). Stepping out of the comfort zone: Profiles of third-year Spanish students' attempts to develop their speaking skills. Foreign Language Annals, 39(3), 451.
Charles, C. & Mertler, C. (2002). Introduction to educational research (4th ed.). Boston, MA: Allyn and Bacon.
Christensen, M. (2010). Impediments to linguistics scaffolding in computer mediated language learning. Paper presented at the 35th Applied Linguistics Association of Australia, University of Queensland, Brisbane, 4-7 July 2010.
Day, I., Wood, D., Scutter, S. & Astachnowicz, S. (2003). Interdisciplinary trials of synchronous, voice-based communication systems. In Interact: Integrate: Impact. Proceedings ASCILITE Adelaide 2003. http://www.ascilite.org.au/conferences/adelaide03/docs/pdf/152.pdf
Hampel, R. (2003). Theoretical perspectives and new practices in audio-graphic conferencing for language learning. ReCALL, 15(1), 21-36.
Hanna, B. E. & de Nooy, J. (2003). A funny thing happened on the way to the forum: Electronic discussion and foreign language learning. Language, Learning and Technology, 7(1), 71-85. http://llt.msu.edu/vol7num1/hanna/default.html
Hunt, M., Neill, S. & Barnes, A. (2007). The use of ICT in the assessment of modern languages: The English context and European viewpoints. Educational Review, 59(2), 95-213.
Knowles, M., Elwood, F. & Swanson, R. (1978). The adult learner: The definitive classic adult education and human resource development. (6th ed.). London: Elsevier.
Laborda, J. (2006). Designing an Internet based tool for oral evaluation in the University Access Examination in Spain. In CALL and monitoring the learner. Proceedings of the International CALL conference, Antwerp, Universiteit Antwerpen, 20-22 August 2006. http://www.eric.ed.gov:80/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED504819
Lam, W. (2007). Tapping ESL learners' problems and strategies in oral communication tasks: Insight from stimulated recall. Prospect, 22(1), 56-70.
Lee, L. (2004). Learners' perspectives on networked collaborative interaction with native speakers of Spanish in the US. Language, Learning and Technology, 8(1), 83-100. http://llt.msu.edu/vol8num1/lee/default.html
Mahfouz, S. & Ihmeideh, F. (2009). Attitudes of Jordanian university students towards using online chat discourse with native speakers of English for improving their language proficiency. Computer Assisted Language Learning, 22(3), 207-227.
Milton, J. C. P. (2004). From parrots to puppet masters: Fostering creative and authentic language. In B. Holmberg, M. Shelley & C. White (Eds.), Distance education and languages: Evolution and change (pp. 242-257). [verified 15 Dec 2010] http://hdl.handle.net/1783.1/2363
Ramzan, Y. & Saito, R. (1998). Computer-mediated communication in foreign language learning: A case of the students of Japanese. In Flexibility the next wave? Proceedings ASCILITE Wollongong 1998. http://www.ascilite.org.au/conferences/wollongong98/asc98-pdf/ramzansaito-0066.pdf
Shih, R. (2010). Blended learning using video-based blogs: Public speaking for English as a second language students. Australasian Journal of Educational Technology, 26(6), 883-897. http://www.ascilite.org.au/ajet/ajet26/shih.html
| Authors: Keri Moore, Learning and Teaching @ University of New South Wales UNSW Sydney NSW 2052. Email: keri.moore@unsw.edu.au Dr Sumiko Iida, Lecturer and Japanese Studies Program Coordinator School of Languages and Linguistics, University of New South Wales UNSW Sydney NSW 2052. Email: s.iida@unsw.edu.au Please cite as: Moore, K. & Iida, S. (2010). Students' perception of supplementary, online activities for Japanese language learning: Groupwork, quiz and discussion tools. Australasian Journal of Educational Technology, 26(7), 966-979. http://www.ascilite.org.au/ajet/ajet26/moore.html |