|Australasian Journal of Educational Technology
2007, 23(2), 227-247.
This paper reports on a project involving software selection in the context of a curriculum redesign of a university level Japanese language program. The project aimed to improve learning outcomes, increase flexibility in student access, and increase flexibility in approaches to teaching and learning, through the use of a variety of software packages and digital resources. In doing so, an imperative was to ensure the solutions adopted were manageable within the existing organisational arrangements of the Department and the University. The selection process has led to the development of three instruments which form the CICTO Framework for Software Selection.
Prior to the project, the Japanese Studies staff had acquired considerable experience with teaching in an e-learning environment, which had led to an understanding of the potential of ICTs to provide multimedia content, communication opportunities and interactive learning experiences. They were also aware that the recent generation of social and communication software solutions and the availability of high quality interactive language resources on the Internet had the potential to enhance their existing ICT based programs, and provide more sophisticated verbal interactions, collaboration and feedback relevant to the language learning context. Thus, the need to review existing approaches and evaluate new ones was a major impetus for the project.
Providing a more flexible learning environment for students was another driver. As well as balancing work, family and lifestyle commitments, an increasing number of students are enrolling in the language program concurrently with studies in other disciplines, or as part of a double major program. These students could benefit greatly from more flexible timetabling and more flexible access to resources and learning experiences. Changing financial arrangements within the Department, combined with increasing demands on time, space and resources, also contributed to a need to explore ways of increasing the effectiveness and efficiency of the existing teaching model based on a traditional campus based experience. A reduced dependency on face to face teaching through the effective use of technology was seen as a way to provide more flexibility in teaching and learning, greater access to resources, and a reduction in demands on classroom spaces.
Preliminary explorations of possible solutions, prior to the onset of the project, led to the realisation that one software solution would not fit all, due to differences in approaches to teaching, learning processes and learning outcomes. Hence, a suite of solutions from which teachers could choose was desirable. There was also the realisation that the effectiveness of any ICT based solution could be completely undermined by a lack of fit with the organisational arrangements surrounding its use, for example, a lack of interoperability with existing technical infrastructure and administrative procedures, or inadequate training and support for users. Collectively, this pointed to the need for a software selection procedure that focused primarily on curriculum imperatives and the needs of the learner, but at the same time addressed the technical capability of the software and its fit within the organisational environment in which it was to be used. Following is an overview of the procedure adopted, the instruments of analysis that were developed and the experiences encountered in the quest for sustainable solutions.
After reviewing the extensive body of literature on software selection, we found three frameworks that had elements suiting our needs. In the context of educational change, we agreed with the proponents of activity theory who suggest that successful educational change does not happen merely by introducing technology. It is dependent on several factors: the psychology of teachers, technical support, ease of access to technology, and institutional support (Ringstaff, Kelley & Dwyer, 1993, cited in Bellamy, 1996).
The same concerns were echoed by Collis and Moonen (2001), who proposed a 4-E Model to describe the factors that influence adoption of technologies for teaching and learning. The successful uptake of technologies by an individual is a function of the four Es - the environment (organisational context), educational effectiveness, ease of use, and engagement (personal engagement of teachers).
Working from the teacher's perspective, Bates and Poole (2003) developed their SECTION model for software selection which advocated the following for consideration:
1. The learning environment: Philosophical and theoretical underpinnings
The first category addresses the learning environment. Bates and Poole (2003) note that "... the choice and use of technology are absolutely dependent on beliefs and assumptions we have about the nature of knowledge, how our subject discipline should be taught, and how students learn" (p. 25). Different philosophical perspectives engender particular pedagogical approaches which, in turn, influence the roles and expectations of teachers and students (Bates & Poole, 2003; Gillani, 2003; Reeves, 1997). Software, for example, that is designed for highly structured, instructor led environments with well defined pathways through activities, may not be suited to an environment which aims to support co-constructed experiences where students can determine pathways and activities, and actively communicate and collaborate in pursuit of knowledge and understanding. Hence, it is important to have a clear understanding of philosophical and theoretical underpinnings to ensure that there is compatibility with the chosen software.
The second category explores the nature of the student cohort. Students come with great diversity in their background and experiences. Different levels of prior knowledge and experience in a particular subject will influence the design of resources and activities, and the type of scaffolding needed to support learning. Cultural background, IT skills, access arrangements and modes of study will all influence the choice of technology as well as the support required to ensure the effective and efficient use of the technologies for learning. In addition, age and experience in the digital world will influence students' attitudes towards and expectations of the use of technologies for teaching and learning (Oblinger & Oblinger, 2005). We therefore need to understand the characteristics of the student cohort to ensure that the technology chosen is appropriate and will not impede their learning.
In categories 3 and 4, we clearly define aims, outcomes, teaching and learning processes, and the supporting activities and resources. It is of fundamental importance that these elements are aligned (Biggs, 2003). Teaching strategies, learning processes, activities and resources are invariably different for different teaching aims and learning outcomes (McKeague & Di Vesta, 1996). In the Japanese language context, for example, the processes underpinning the aims and outcomes associated with the development of vocabulary are quite different to those for the development of communication skills and increasing cultural awareness.
Category 5 addresses assessment and feedback. As we know, assessment has a pivotal role in motivating students to learn, in providing feedback on progress and in providing measures of achievement. Technological solutions can be employed in various ways to support the assessment process, for example through quizzes, simulations and modeling exercises. Timely and rich feedback can be generated through discussion forums and the use of rubrics. In addition, software packages often come with in-built monitoring tools to review student's progress (Macdonald, J. 2002; Phillips & Lowe, 2003). The choice of software will depend on how each of the defined outcomes is assessed, and how feedback is to be provided for both formative and summative tasks.
When applying the curriculum needs analysis, we shared the same experience as Burston (2003), who found that the most difficult aspect of software selection was establishing the "teacher fit", because teachers find it hard to articulate the theoretical underpinnings of their practice. In view of this, and to assist the process, the analysis was facilitated by an academic developer, with expertise in curriculum design and development. The facilitator guided a series of discussions that enabled the Japanese Studies team to analyse their teaching and learning context using their discipline specific terminology.
The Japanese Studies team had previously developed a completely online Japanese beginners program named Gengoro which had been integrated into various other ICT language teaching projects. They had already collected a substantial amount of relevant information and knowledge of ICTs and had a sound understanding of the technology issues relating to the diversity in their student cohort. For them, the more challenging aspect of the needs analysis was articulating their philosophy and unpacking the contributions technologies could make to the curriculum. Nonetheless, on reflection, the team noted that one of the key lessons learnt was that articulating the teaching philosophy was central to understanding educational needs, for it is educational need that determines the criteria for selecting the software.
Embedded in their philosophy was the primacy of the role of the teacher in providing the best possible environment for students to learn. This involves providing high quality learning materials, motivating learning experiences where students are comfortable to explore and take risks, and providing opportunities to build on their prior understandings. It also involves providing students with feedback as well as opportunities to assess their own progress. While the learning process is seen as a joint endeavour by the teachers and students, it is the responsibility of students to take advantage of all of the opportunities provided. Good two way communication between teachers and students and collaboration amongst participants is important to achieving successful outcomes.
Technology to support this philosophy needed to include effective individual and group communication tools, access to authentic (online) learning materials, and facilities interacting with content and exploring ideas. Technologies that allowed self paced learning and catered well for student feedback, and automatic or delayed teacher responses were also important.
To explore issues relating to curriculum, we began by identifying the 'key aspects of language learning', namely writing, speaking, listening, and cultural awareness. Although described as separate entities, a holistic approach to teaching was adopted to reflect the implicit interrelationships evident in proficient Japanese speakers. Addressing these aspects involved recognition of vocabulary, reproduction and imitation of vocabulary, knowledge and application of grammar, deconstruction of words and sentences, intercultural awareness and understanding, discrimination in language use, as well as spoken and written production of language. These were the elements for which we were seeking support through technical solutions. In addition, provision of effective and efficient feedback to students as they were developing their written and oral communication skills was another area for solutions.
|Instructional processes and learning experiences||Instructional|
|Possible software solutions|
|Recognition of characters and scripts (Hiragana Katakana and Kanji)||Being proficient in reading and understanding simple passages using Japanese scripts||Memorising Japanese characters|
Decoding words and their meaning both in and out of context
Comprehending vocabulary and grammar
|Classroom instruction using examples of scripts and practicing for recall|
Use of dictionary to make meaning of the words
|Flash cards, computer based exercises|
Written scripts and readers
Online quizzes with feedback
|Development of intercultural awareness and understanding||Being able to communicate in written and spoken forms in a way that shows an awareness of norms, cultural/ social differences.||Perceiving different contexts|
Analysing different behaviours
Discriminating between different behaviours in different contexts
Understanding cause and effect in behaviour
Developing self reflective strategies
|Use of exemplars|
Exposure to authentic experiences
Modeling by teachers
Communicating with people from different cultures
Broadcast TV Newspapers
|Online discussions, email and chat to enable interaction with authentic speakers|
Student presentations (e.g., PPT )
Use of wikis for group projects
Use of blogs for personal reflections
Online role plays
To ensure the technologies proposed were closely aligned with the intent of the curriculum, a simple alignment grid was used to articulate the relationships between teaching aims, learning outcomes, learning processes, instructional processes, learning experiences and resources. Two examples showing the alignment process in action are presented in Table 1. The examples are for aims associated with recognising characters, and developing cultural awareness and understanding. Use of the grid also had the added advantage of assisting teachers to articulate a rationale for their practice. An extension to the grid, not shown in Table 1, is a column for assessment tasks to ensure their alignment with all the elements of the curriculum.
Arising from the curriculum needs analysis, several software solutions were identified as being suitable. For example, Wimba, software to enable classroom collaboration and interactive communications, with voice facilities to support speaking and listening exercises, was particularly suitable for recognition of vocabulary, reproduction and imitation of vocabulary, and spoken production of language. The use of videos, broadcast TV, newspapers, and web resources that exemplified authentic contexts were particularly suited to the development of intercultural awareness and discrimination in language use. Text based discussion tools and quizzes available in WebCT (a learning management system) and LAMS (a system for sequencing learning activities) could be employed to develop activities for the deconstruction of words and sentences and written production of language. Each had strengths and weaknesses in relation to educational needs. To develop a more comprehensive understanding of the impact of these technologies on staff and students from educational and operational perspectives, each of the solutions was subjected to further analyses using the ICT requirements and capability analysis, and the organisational support and interoperability analysis.
As a result we realised we required an instrument that could raise awareness of the underlying technical issues related to integrating the software into existing technical infrastructure. The instrument also needed to be focused less on the technical minutia - IT experts could do this - and more on the capability of the software to accommodate the administrative, teaching and learning functions and workflow patterns arising from the requirements identified in the curriculum needs analysis. Too often the educational limitations of the software only become evident after substantial commitment has been made in terms of dollars and time. Complicated, time consuming and costly adaptations to overcome the limitations can render the software unsuitable, despite a sound educational fit. Therefore, the instrument needed to guide teachers in evaluating the potential limitations and benefits of software solutions in light of the identified teaching and learning processes.
As a result of our explorations, we adapted a checklist, developed by Foshay and Almed (2000), of the technical requirements for a range of educational software solutions. The outcome, the ICT requirements and capability analysis, is a questionnaire suited to teasing out the complexities present in a large networked environment. Throughout its development we were conscious that the functionality required of a piece of software is context specific. Furthermore, although we have attempted to define some of the functionality that may be required, we are aware that a definitive description is not possible given the dynamic nature of the software environment and the needs arising from different educational contexts. For our purposes, we found six broad categories of issues to be of particular relevance. The categories along with the issues to be explored are shown in Figure 2.
The first category, teaching and learning process management, serves the purpose of ascertaining the strengths and weaknesses the software in question has in supporting the processes identified in the needs analysis. In reality this was an iterative process - explorations of the capability of the software led to new insights and ideas about teaching which, in turn, led to changes to the original conceptions of the desired teaching and learning processes and interactions.
The second category looks at issues arising from assessment requirements. One of the requirements for Japanese studies was the provision of efficient and effective feedback to students when developing vocabulary and related declarative knowledge. The use of computer based applications for this purpose has been well recognised (Fletcher-Flinn & Gravatt, 1995; Kulik & Kulik, 1991). The oral capability of Wimba offered potential solutions for the development and assessment of written and spoken vocabulary and speaking skills. However, associated with the use of technologies for assessment purposes were issues related to security, authentication and data management which placed additional demands on the functionality required of the software. These and similar issues needed to be identified and resolved for the software to be efficient and effective.
The management of content, as addressed in the third category, has great potential to reduce the workload of teachers, especially in relation to reusing content from semester to semester, or transferring from one program to another. The ability for teaching staff to be able to easily develop materials themselves was of particular importance to the Japanese Studies team, not only for maintaining currency, but also to reduce on-going specialist development costs.
1. Teaching and learning process management
In the curriculum needs analysis you identified the educational functionality required of this piece of software in relation to teaching aims and learning outcomes, and more specifically:
If the software is to be used for assessment purposes, consider the strengths and weaknesses of the software package in regard to the following features:
The content creation and management features required of the software will be dependent on the defined educational requirements specified in the needs analysis. In relation to the educational requirements:
User management features may include the provision of tools or functionalities. In relation to the educational requirements, which of the following features are required? If so, are the features currently supported by the software?
There are standard technical and usability features that must be operating at an acceptable level before software can be considered for educational use. IT staff will have their own checklist to ensure the software package in question meets institutional requirements. Teaching staff need to consider their satisfaction with the following features:
Does the software package have potential to meet a range of current and future teaching and learning needs?
Another aspect for consideration was in relation to content creation. First, different content presentation modes (visual, audio, multimedia) may desirable but not all software programs support multimedia content. Second, the role of students in relation to content creation has to be considered. In the past, much of the content being used for teaching has been provided for students to access (Miodusa & Nachmias, 2002). However, as we move from a teaching paradigm of acquisition of information to co-construction of learning (de Boer & Collis, 2002), there is the need to enable students to create content individually and collaboratively. This was a requirement for the Japanese Studies team when developing intercultural awareness. These and other considerations in relation to content creation and management are listed in category 3.
Managing users is the fourth category. Providing access, creating and managing student and staff accounts and passwords can be time consuming. Some software packages may be interoperable with the institution's student systems to allow direct feed of student data and hence saving administrative work. There were also a number of user management features that were desirable for the Japanese Studies context, for example, grouping and defining access for student groups, providing individualised pathways and options and enabling individuals to save work, exit and resume at a later point. In some cases there was the need to track performance and monitor student progress. Some of these features may be built into the software; others may not, therefore it is important to identify need from the outset.
Collis and Moonen (2001) found that efficiency and ease of use were important conditions for the successful adoption of software. Reliability of performance, security of data, easy access, fast download speeds, intuitive navigation, and availability of help functions are some of the recognised usability features (Bates & Poole, 2003; Burston, 2003; Foshay & Ahmed, 2000; Krauthamer, 2000) that needed to be considered for the smooth operation and the acceptance of the software by all its users. The technical usability issues of this nature relevant to the Japanese Studies context are listed in category 5.
Finally, because of the cost, time and effort associated with integrating new technologies into the curriculum, it was desirable that the software met a range of teaching and learning needs rather than having limited applications. Therefore, the potential of the software to meet future need was an important consideration.
When undertaking the ICT requirements and capability analysis, the Japanese Studies team was not always able to answer the technical questions posed and assistance was sought from the relevant support staff on campus. The ensuing discussions were informative but also exhausting and there were moments when the team felt intimidated by the technical terms used in the discussion. Too much detail in some instances also proved to be confusing and some of the questions could only be answered after the software had been trialed. Nonetheless, the overall process of considering the usage of the software in depth was necessary for building a sophisticated understanding of the requirements and the software's capability in meeting those requirements.
In practice, many decisions required a careful weighing up of strengths and weaknesses. Wimba, for example, was proposed as a possible solution to providing interactive speaking and listening opportunities for the students. Unfortunately a current limitation of Wimba was the version in use did not allow Japanese script input. This meant students could not experience dual channel input from reading Japanese script and listening to a voice over. Despite this limitation, Wimba could still be used to provide voice over combined with English language script. In addition it was established that the software worked well for on campus access and for most students at home. It was therefore decided that the advantages of the package far outweighed the disadvantages.
Six areas of particular relevance to the Japanese Studies context were identified for inclusion in the organisational support and interoperability analysis: software administration and interoperability with enterprise systems; student and staff training and support; financial and asset management arrangements; compliance with legislative requirements; alignment with academic practice, policy and procedural frameworks; and provision of risk management. These are outlined in Figure 3, along with the issues for exploration under each of the categories.
In relation to these issues, adopting software that was supported by the University's Central Services had major advantages for Japaneses Studies staff and their Department. When the software is supported, as in the case of the University's learning management system, WebCT, teaching staff are given access to the software and its tools; technical support is provided through a Help Desk; hands on training and documentation is provided for students and staff in how to use the software; user accounts are automatically created and managed. Staff do not have to concern themselves with the back-end technical infrastructure arrangements related to hosting arrangements on servers, backups, authentication of users, administration of licenses, software upgrades and patches, risk management strategies, etc. Professional development opportunities and educational development services are available to assist with integrating the software into their curricula and developing resources. Central Services provide guidelines and advice to ensure adherence to the regulatory frameworks that the University has to comply with including those related to accessibility, confidentiality and privacy, intellectual property and copyright, and records management. Financial arrangements are in place to ensure the costs for licenses and upgrades are covered and the financial outlays for students are kept to a minimum.
1. Software administration and interoperability with enterprise systems
Taking into account arrangements for the above, adopting software that was not centrally supported would have implications for both academic and general staff arising from:
The organisational interoperability analysis was complex and input was required from a number of organisational units across the university. Nevertheless there were benefits in undertaking the analysis. Technical staff were able to appreciate the full complexity of teaching and learning needs; likewise academics were able to better understand the limitations of the software and the complexity of the infrastructure arrangements necessary to achieve seamless delivery. There was also the realisation from all parties that selecting software was not an individual act. It required a team effort in uncovering all the educational, technical and organisational ramifications.
Wimba scored well in all analyses for its 'Voice Board' feature and will also be included as a tool across the Department. It proved to be easy to use by both students and staff. The software enabled students to listen and prepare spoken responses; practise, edit and review their responses; record responses; listen to other students' responses and communicate with other group members. For teachers, the software was particularly useful for modeling spoken language, providing spoken feedback in English and Japanese, assessing spoken Japanese and monitoring the progress of individual students. For students, practising language skills using Wimba was often less intimidating and less confrontational than in a face to face environment, and it allowed them to more easily concentrate on their areas of weakness. Overall, the inclusion of Wimba in the suite of software solutions greatly enhanced the teaching and learning of the spoken component in the curriculum.
iLecture, a university wide, web based lecture recording technology, enables the automatic recording of lectures which are then made available to students through a web interface in WebCT. This tool is used to support lesson review and to assist students with special needs. Like WebCT, Wimba and LAMS, iLecture is centrally supported within the University and hence scored high on the organisational analysis. It also scored high on the technical capability analysis and high in some aspects of the curriculum needs analysis in that it caters for students with special needs. It will continue to be used as part of the strategy to deal with classes with a wide range of abilities.
LAMS is to be used principally as a management tool to organise multi-level tasks associated with research projects, groupwork and student presentations. Its capacity to support group interaction, link to external web resources and share files is useful for scaffolding project development. From a teaching perspective, the ability to provide structure through the provision of a predefined sequence of tasks and to monitor a student's progress along the sequence could be useful for supporting the staged development of knowledge and skills. Although it has potential, its strength as defined by the curriculum needs analysis and the technical capability analysis was limited to teaching support and the facilitation of project development processes for students, therefore it will be included as an optional choice for staff.
The three instruments that have been developed for this project are an important step in ensuring a comprehensive and balanced approach to software selection. Collectively they form the CICTO Framework (curriculum, ICT and organisational) for integrating software solutions into educational environments. It must be recognised that the Framework is a living entity and will change over time as the educational context, technologies and organisational arrangements mature. Each context is different, therefore the categories and issues will vary from one context to another. Nevertheless the value of such a framework is that it places the focus of decision making at the heart of the matter - the educational experience.
Bellamy, R. K. E. (1996). Designing educational technology: Computer-mediated change. In B. A. Nardi (Ed.), Context and consciousness (pp. 123-146). London: MIT Press.
Biggs, J. (2003). Teaching for quality learning at university. Philadelphia, SRHE and Open University Press.
Burston, J. (2003). Software selection: A primer on sources and evaluation. CALICO Journal, 21(1), 29-44.
de Boer, W. & Collis, B. (2002). A changing pedagogy in e-learning: From acquisition to contribution. Journal of Computing in Higher Education, 13(2), 87-101.
Collis, B. & Moonen, J. (2001). Will they use it? In B. Collis & J. Moonen (Eds.), Flexible learning in a digital world: Experiences and expectations (pp. 44-66). London: Kogan Page.
Fletcher-Flinn, C. M. & Gravatt, B. (1995). The efficacy of computer assisted instruction (CAI): A meta-analysis. Journal of Computing Research, 12(3), 219-242.
Foshay, R. & Ahmed, M. I. (2000). A practical process for reviewing and selecting educational software. PLATO Learning. Inc.. Bloomington. MN. (Report No. BBB37097). (ERIC Document Production Service No. ED464608) [verified 15 Apr 2007] http://eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED464608
Gengoro. http://www.gengoro.com.au/ [viewed 7 Aug 2006]
Gillani, B. (2003). Theories and the design of e-learning environments. Lanham, Maryland: University Press of America.
iLecture - Lectopia. http://ilectures.uwa.edu.au/ [viewed 7 Aug 2006]
Kulik, C. C. & Kulik, J. (1991). Effectiveness of computer-based instruction: An updated analysis. Computers in Human Behaviour, 7, 75-94.
LAMS. http://www.lamsinternational.com/ [viewed 7 Aug 2006]
McKeague, C. A. & Di Vesta, F. J. (1996). Strategy orientations, learner control and learning outcomes: Implications for instructional support of learning. Educational Technology Research and Development, 44(2), 29-42.
Macdonald, J. (2002). Developing competent e-learners: The role of assessment. Paper presented at the Learning Communities and Assessment Cultures Conference organised by the EARLI Special Interest Group on Assessment and Evaluation, University of Northumbria , 28-30 August 2002 [viewed 7 Aug 2006] http://www.leeds.ac.uk/educol/documents/00002251.htm
Mioduser, D. & Nachmias, R. (2002). WWW in education: An overview. In H. Adelsberger, B. Collis & M. Pawlowsky (Eds.), Handbook on information technologies for education & training. Berlin/Heidelberg/New York: Springer.
Oblinger, D. & J. Oblinger (2005). Is it age or IT: First steps toward understanding the net generation. In D. Oblinger and J. Oblinger (Eds), Educating the net generation. Educause. [viewed 7 Aug 2006] http://www.educause.edu/educatingthenetgen
Phillips, R. & Lowe, K. (2003). Issues associated with the equivalence of traditional and online assessment. In G. Crisp, D. Thiele, I. Scholten, S. Barker and J. Baron (Eds), Interact, Integrate, Impact: Proceedings 20th ASCILITE Conference. Adelaide, 7-10 Dec 2003. http://www.ascilite.org.au/conferences/adelaide03/docs/pdf/419.pdf
Reeves, T. (1997). Evaluating what really matters in computer-based education. [viewed 31 May 2006] http://educationau.edu.au/jahia/Jahia/pid/179 [for figures, see http://www.educationau.edu.au/jahia/webdav/site/myjahiasite/users/root/public/]
Tergan, S. (1998). Checklists for the evaluation of educational software: Critical review and prospects. Innovations in Education and Training International, 35(1), 9-20.
Wimba. http://www.horizonwimba.com/ [viewed 7 Aug 2006]
WebCT. http://www.webct.com/ [viewed 7 Aug 2006]
|Authors: Dr Maree Gosper, Karen Woo, Institute of Higher Education Research and Development, Macquarie University, NSW 2109 Australia. |
Email: firstname.lastname@example.org, email@example.com
Helen Muir, Christine Dudley, Kayo Nakazawa, Department of Asian Languages, Division of Humanities, Macquarie University, NSW 2109 Australia.
Email: firstname.lastname@example.org, email@example.com, firstname.lastname@example.org
Please cite as: Gosper, M., Woo, K., Muir, H., Dudley, C. and Nakazawa, K. (2007). Selecting ICT based solutions for quality learning and sustainable practice. Australasian Journal of Educational Technology, 23(2), 227-247. http://www.ascilite.org.au/ajet/ajet23/gosper.html