Australasian Journal of Educational Technology
2011, 27(Special issue, 8), 1369-1387. 		AJET 27

Evaluation of user acceptance of mixed reality technology

Rasimah Che Mohd Yusoff
Universiti Teknologi Malaysia
Halimah Badioze Zaman and Azlina Ahmad
Universiti Kebangsaan Malaysia

This study investigates users' perception and acceptance of mixed reality (MR) technology. Acceptance of new information technologies has been important research area since 1990s. It is important to understand the reasons why people accept information technologies, as this can help to improve design, evaluation and prediction how users will respond to a new technology. MR is one of the potential technologies that has gained attention in recent time, offering a unique environment as it combines real and virtual objects, interactive in real time and registered in three dimensions. This paper discusses a study into users' acceptance of a mixed reality prototype, named Mixed Reality Regenerative Concept (MRRC). MRRC was developed using mixed reality technology to provide Biomedical Science students with exposure to regenerative concepts and tissue engineering processes. MRRC integrates situated learning as the model of instruction, emphasising authentic context and activities. Volunteer sampling was used in this study to obtain 63 participants comprising 2nd, 3rd and 4th year Biomedical Science students in two public universities in Malaysia, who had not previously experienced mixed reality technology. In this study, the constructs used to determine acceptance of mixed reality technology were personal innovativeness (PI), perceived enjoyment (PE), perceived ease of use (PEOU), perceived usefulness (PU), and intention to use (ITU). Results from simple correlation analyses showed positive linear correlations between the constructs. However, findings from regression analysis suggested that perceived usefulness was the most important factor determining users' intention to use this technology in the future. Findings from this study also suggested that tertiary level science students showed a high willingness to use mixed reality technology in the future.

Introduction

Acceptance of new information technology (IT) is one area of studies that many researchers and practitioners have investigated since the 1990s (Venkatesh, 2000). User acceptance can be defined as the demonstrable willingness within a user group to employ IT for the tasks it is designed to support (Dillon, 2001). Previous research claimed that lack of user acceptance has been an impediment to the success of newly implemented IT systems (Davis, 1993). It is important to understand the reasons why people accept IT as this can help in improving the design, evaluation, and prediction of how users will respond to the new technology (Dillon & Morris, 1996).

Augmented reality (AR) and mixed reality (MR) are amongst the potential technologies that have attracted attention in recent times. The MR environment is unique since it combines real and virtual objects, interactive in real time and registered in three dimensions. It can support a seamless interaction between the real and virtual environments. Because of these characteristics, the experiences offered by MR are realistic, authentic, engaging, and fun (Kirkley & Kirkley, 2005). Dede (2005) claimed that properly designed virtual environments with AR or MR interface can foster neomillenial learning styles through physical and sensory immersion. The implementation of this new technology in teaching and learning can increase students' motivation (Hanson & Shelton, 2008).

Although MR was invented almost 50 years ago, use of MR applications tends to be related only to laboratory research. However, it is now becoming more widely used in various settings such as entertainment, education and training (Goldiez, Livingston, Brown & Hancock, 2004). Azuma, Yohan, Reinhold, Steven, Simon and Blair (2001) remarked that although many research systems have been developed, only a few have matured beyond laboratory based prototypes, and social acceptance issues need to be addressed before they can be accepted more widely. Even though proponents of MR put forward many advantages for MR, further implementation of MR will require improved understanding of users' perceptions of this technology. Feedback on end users' views of MR system can guide developers of MR systems and administrators who are considering implementing MR technology in the future.

Purposes of the study

This study investigated users' perceptions and acceptance of MR technology. An MR prototype named Mixed Reality Regenerative Concept (MRRC) was designed and developed for the purposes of this study. The following research questions were investigated:

Review of relevant literature

Mixed reality technology

The first AR interface was developed by Sutherland in the 1960s (Sutherland, 1965). However, it was not until the early 1990s that the term "augmented reality" was coined by Caudell and Mizell (1992). Caudell and Mizell were researchers at Boeing Corporation, who dedicated their time towards developing experimental AR systems to train workers installing wiring harnesses in planes. Later, Milgram and Kishino (1994) described a reality-virtuality continuum that spans from the real environment to a pure virtual environment as shown in Figure 1. AR lies near the real world end of the line, with the predominate perception being the real world augmented by computer generated data. Augmented virtuality (AV) is a term created by Milgram (Milgram & Kishino, 1994) to identify systems which are mostly synthetic, with some real world imagery added, such as texture mapping video onto virtual objects. MR consists of AR and AV. However, the definition by Azuma (1997) is widely accepted as a "benchmark" to describe AR and MR. Azuma described the three characteristics of an AR environment: it combines real and virtual objects, is interactive in real-time, and is registered in three dimensions. This definition of an AR system is extended by the definition of an MR system as an interactive system that combines digital and physical entities. Kondo (2006) assumed that MR is similar to virtual reality technology that combines interactive three-dimensional computer graphics with the real world. According to Dubois, Gray and Nigay (2009) there is one important difference between AR and MR - while an AR system deals primarily with the output (e.g., visual, 2-D, 3-D) of a system, an MR system deals also with inputs from the real world to the system. Figure 2 shows an example of MR scene in which a user is holding a starfish. The user can press a key to view how a starfish may regenerate a lost arm.

Figure 1

Figure 1: Milgram reality-virtuality continuum

Figure 2

Figure 2: Mixed reality scene

With the characteristics discussed previously, the experiences offered by MR are different from other technologies. MR can support seamless interaction between real and virtual environments, allowing the use of a tangible interface metaphor for object manipulation and the ability to change smoothly between reality and virtuality. Users of MR applications in the present time tend to be heterogeneous. The users are changing from developers and researchers to a wide variety of consumers as applications are becoming increasingly common in many fields such as education, medicine, military, entertainment, and industry (Azuma et al., 2001). However, to be successful in introducing MR applications to the users, more attention needs to be paid to the usability of the applications. Thus, performance, ergonomics and ease of use studies are important issues for future research in MR (Träskbäack & Haller, 2004).

Technology acceptance

Louho, Kallioja and Oittinen (2006) defined technology acceptance as the way people perceive, accept, and adopt technology use. Technology acceptance models are used to explain how users come to use or accept a specific technology. These models have their origins in the disciplines of psychology, information systems and sociology. The Theory of Reasoned Action (TRA) (Ajzen & Fishbein, 1980) and the Theory of Planned Behaviour (TPB) (Ajzen, 1985) have been widely used to predict and understand behaviour and attitudes. Many models to measure end users' acceptance of a new technology have been suggested. such as Innovation Diffusion Theory (IDT) (Rogers, 1995), Technology Acceptance Model (TAM) (Davis, 1989) and Unified Theory of Acceptance and Use of Technology (UTAUT) (Venkatesh, Morris, Davis & Davis, 2003).

The Technology Acceptance Model (TAM) has become one of the most widely used and empirically validated models within information systems research (King & He, 2006). Drawing on the belief-attitude-behaviour models as exemplified by the TRA, Davis, Bagozzi and Warshaw (1989) suggested that the major factors influencing intention to use any technology are predicted by perceived usefulness and perceived ease of use. TAM has been applied to different technologies and has been tested in different contexts. According to Davis (1989), the goal of TAM is "to provide an explanation of the determinants of computer acceptance that is generally capable of explaining user behaviour across a broad range of end-user computing technologies and user populations, while at the same time being both parsimonious and theoretically justified" (p. 985). One of the assumptions of research performed using TAM is that usage of the technology is voluntary (Seymour, Makanya & Berrange, 2007). Figure 3 illustrates the technology acceptance model by Davis (1993).

Figure 3

Figure 3: Technology acceptance model (TAM)

Since user acceptance research involves different technologies, to determine the acceptance of a specific technology, researchers usually merge the basic TAM model with other constructs that are deemed appropriate for the technology system being tested (Legris, Ingham & Collerette, 2003). Determining the constructs associated with user acceptance of new systems is an important research area in the field of information systems (Chesney, 2006). Constructs are weighted factors used to explain the acceptance and use of the technology. Table 1 shows some of the previous research that used the TAM model and its links with some related constructs.

Table 1: Extended TAM

ContextExtended constructs
Games in classroom (Bourgonjon, Valcke, Soetaert & Schellens, 2009)Gender, gaming experience, learning opportunity
Virtual reality in clinical (Bertrand & Bouchard, 2008)External control, computer anxiety, intrinsic motivation
Virtual learning environment (Van Raaij & Schepers, 2006)Personal innovativeness, computer anxiety
Program LEGO robot (Chesney, 2006)Perceived enjoyment
Online games (Hsu & Lu, 2004)Flow, social, critical mass
Email notification interface agents (Serenko, 2008)Personal innovativeness, perceived enjoyment

Technology acceptance of mixed reality technology

MR is a relatively new technology, considering the limited commercially and publicly available applications. However, as a research field, it has existed for almost five decades with applications in diverse domains, such as military applications, entertainment, technical support and industry applications (Nilsson & Johansson, 2008). Furthermore, MR systems are expensive to design and develop. History has shown the need to carry out user studies to understand their perceptions of usability and usefulness of new technologies before a technology-based system matures and succeeds in being accepted in our daily lives. Knowing the acceptance levels of and perceptions toward new technologies early helps to avoid having to do expensive remedial work (Theng, Lim, Liu & Cheok, 2007).

A new technology is considered to have been integrated successfully into an organisation or workplace when it is used by the people for the tasks it is intended for. There are many instances where technology has been introduced in organisations and then not been used for a number of reasons. One major contributor to lack of usage may be the usability of the product or the system itself. Another issue is how well the system operates in tandem with the users in a social context. Sometimes, the users are not interested in using the system because they do not see the same potential in the system as the management who decided to introduce it into the organisation. Davis (1993) described two important factors that influence the acceptance of new technology in organisations: the perceived usefulness of a system and perceived ease of use. Users will accept a system that they consider as useful even though it may be perceived as harder to use. Thus, it is important to understand users' acceptance because the success or failure of a system depends on how well people like the system, how easy it is to use, and the system's effectiveness. If a system is not liked by users, it will not be used, and the money spent on its development will be wasted. For an MR system, this means that even though the system may be awkward or bulky (e.g. head-mounted), if the applications are good and useful, users will accept it. Equally, if an MR system is not perceived as useful, then the system will not be used, even though it may be easy to use or people enjoy using it (Nilsson & Johansson, 2008).

A few studies have been conducted to determine acceptance of MR applications. Theng et al. (2007) investigated participants' perceptions of usefulness and usability of Plant Mixed Reality System (PMRS), designed for primary school children (11-12 years old). Preliminary results indicated that the participants' strong intention to use PMRS for learning, and this intention was influenced directly by perceived usefulness and indirectly through perceived usability and social influence. System quality, personal innovativeness, and compatibility were found to be important external factors. Nilsson and Johansson (2008) also designed and developed an MR instructional tool on operating and assembling medical devices, called TROCAR. In their study, the majority of the participants found the MR system fun to use and work with, and several of them also wanted to use it as a part of their work.

Situated learning

Social constructivist is a theory where effective learning occurs in authentic settings and involves social interactions. One of the well known social constructivist approaches is situated learning. Situated learning has made a significant impact on educational thinking since it was first expounded by Brown, Collins and Duguid (1989). In contrast with most classrooms learning activities that involve abstract knowledge which is out of context, Lave and Wenger (1990) argued that learning is situated; that is, learning is embedded within activity, context and culture and called this a process of "legitimate peripheral participation". Among others, Albert Bandura is considered a leading proponent of the social learning theory, elaborating that learning is a social process, that human learn vicariously before learning it directly because that is the way one can "acquire large, integrated patterns of behaviour without having to form them tediously by trial and error" (Bandura, 1997, p. 12). Dede (2005), further, mentioned that the harder the task to be learned, the more one must learn it through observation first. At its simplest, situated learning is learning that takes place in the same context in which it is applied.

Technology can play an important role in integrating 21st century skills and mediating authentic experiences in the classroom (Roschelle, Pea, Hoadley, Gordin & Means, 2001). Simulations and virtual reality provide the basis for one form of situated learning by modelling specific aspects of real world complex systems. Users can experiment with the system either by manipulating parameters or participating inside the system and observing the outcomes of their manipulation and participation. Simulations situated in rich, realistic 3D virtual worlds might be described as "heavily" virtual but less authentic since they depart so much from the actual world. Learning environments that are "light" on virtual information, by contrast, provide less simulated sensory input, but remain closer to the actual world and can take advantage of its affordances for authenticity (Rosenbaum, Klopfer & Perry, 2007). MR technology can be used to balance the strengths and weaknesses of virtual media in creating authentic learning environments, by combining real world scenes with virtual objects together with live social interactions with other participants. More intuitive ways of interaction with virtual objects using MR helps users to have authentic experiences.

Research methodology

The mixed reality system

The Mixed Reality Regenerative Concept (MRRC) was developed using MR technology to provide Biomedical Science students with exposure on regenerative concepts and in-vitro processes of tissue engineering. The MRRC integrates some components of situated learning as a model of instruction: users, authentic context and authentic activities (Rasimah, Halimah & Azlina, 2010). In MRRC, learning takes place in a virtual laboratory as shown in Figure 4(a). The users act as new researchers and they have to perform a task in the laboratory. As the users explore and navigate the virtual lab, they interact with the apparatus in the laboratory and experience a few MR environments. They read the virtual memo to learn about the task which will require them to perform skin tissue engineering (Figure 4(b)). To complete the tasks, the users have to interact with the system, for example holding the marker in front of a webcam to view virtual objects then rotate, tilt and shake the marker to view objects at different angles. Users can also zoom into and out from the virtual object by moving the marker closer or further away from the camera as shown in Figure 4(c) and 4(d). The MRRC used markers printed on a hard paper as a tangible interface that serve as a physical handler in interacting with a virtual object in the MR environments. Using MR technology, users can interact with the virtual objects as they would with an object in the real world. This provides a natural and intuitive mechanism to view and understand potentially complex three dimensional objects.

Figure 4(a)Figure 4(b)
Figure 4(a): Navigate virtual laboratoryFigure 4(b): Read an MR memo
Figure 4(c)Figure 4(d)
Figure 4(c): Holding a virtual skinFigure 4(d): Skin structure

Figure 4: Illustrating virtual laboratory activities

System configuration

The MRRC could be considered as a mirror metaphor. A web camera captures the image of an object in front of the screen and projects the image on the screen. The mirrored image of the physical image is then augmented with the virtual environment. Figure 5 shows the system configuration of MRRC.

Figure 5

Figure 5: System configuration

Hardware

The main hardware components for MR are the display device, tracking device, input device, and computer.

Software

The software used in the development of MRRC included offline and real-time software technologies (Liarokapis, 2007). The offline software technologies included a number of tools used to prepare the content in the augmentation (i.e. virtual information) as well as the MR environment. The offline software used comprised FLARToolKit, PaperVision3D, 3D Studio Max, Adobe Photoshop, SoundForge, Unifi 3D, and Visual Basic. The real-time software technology used was FLARManager consisting of FLARToolKit and PaperVision3D, and also Adobe Flash Player 10.

Participants

This study was conducted at two public universities in Malaysia using volunteer sampling. The participants consisted of 63 volunteers who were 2nd, 3rd and 4th year Biomedical Science students. Twenty six of the respondents were male (41%) and thirty seven were female (59%).

Constructs of the study

This study used the constructs in the generic acceptance model consisting of perceived usefulness, perceived ease of use, and the intention to use. In addition to these basic constructs of TAM, the study also investigated how the participants' personal innovativeness and level of enjoyment while interacting with MR influenced their intention to use the MR application.

Many researchers have investigated how users' personalities could influence their behaviours. One of the pioneer theories in this area of research is Rogers' Innovation Diffusion Theory (IDT) (Rogers, 1995). IDT suggests that users' personality differences can potentially influence how users form their intention to perform behaviours. Through exploring technology adoption stages, Rogers revealed that: (i) users with higher levels of personal innovativeness are more likely to have a more favourable attitude towards new technologies; and (ii) highly innovative users are more willing to embrace new technologies into their daily routine by coping with the uncertainty of innovative technologies. Lee, Qu and Kim (2007) in their study on online travel shopping indicated that innovativeness influenced traveller's online shopping behaviour. Similarly, Parveen and Sulaiman (2008) and O'Cass and Fenech (2003) suggested that highly innovative web users were more likely to develop more positive attitudes towards new technologies. Since personal innovativeness may influence the adoption of new technologies to a large extent, it is considered as one of the most important constructs that affect users' acceptance of MR technology. Thus, it was appropriate for this study, to determine the relationship between personal innovativeness in IT and perceived ease of using MR.

MRRC was developed to provide Biomedical Science students with experience in regenerative concepts and tissue engineering processes. The use of MR technology will give the users a different experience since it is engaging, exciting, and fun. Thus, the prototype of MRRC can be considered as partly utilitarian and partly recreational (dual system) in that it can be used to learn in a pleasurable way. Enjoyment is one of the constructs that is commonly used to determine the acceptance of the dual system (Chesney, 2006; Theng et al, 2007; Serenko, 2008). For this study, enjoyment was one of the important constructs that might affect acceptance of MR technology. Previous studies claimed that perceived enjoyment has no direct influence on intention to use, but that it could influence the ease of use and usefulness (Heerink, Krose, Wielinga & Evers, 2008; Sun & Zhang, 2005; Venkatesh et al., 2003). Therefore, this study also investigated whether enjoyment was related to intention to use the MR system.

Prior research also indicated that perceived usefulness is an important indicator for technology acceptance (Davis, 1989; Venkatesh et al., 2003). When a person feels that using a technology system will enhance their study, he or she might intend to adopt MR technology in the future. So, it is appropriate to test perceived usefulness with intention to adopt MR technology. According to Davis (1989) perceived ease of use is one of the major behavioural beliefs influencing user intention towards technology acceptance. This study also sought to determine relationships between perceived ease of use, perceived usefulness of MRRC, and intention to use MRRC in the future. Additionally, this study examined whether ease of use of MRRC affects the participants' enjoyment. Table 2 defines the constructs used to determine users' perception and acceptance towards MR technology.

Table 2: Construct definition

ContextExtended constructs
Personal innovativeness
(PI)		The willingness of an individual to try out any new information technology (Agarwal & Prasad, 1998).
Personal innovativeness is used to measure the individual traits.
Perceived enjoyment
(PE)		The extent to which the activity of using a specific system is perceived to be enjoyable in its own right, aside from any performance consequences resulting from system use (Venkatesh et al., 2003). The feeling of pleasure or contentment during the VE experience (Lin & Parker, 2007).
Enjoyment measures the respondents' pleasure feeling while interacting with the system.
Perceived usefulness
(PU)		The extent of which the person thinks that using the system will enhance his or her job performance (Davis, 1989).
Perceived usefulness measures the desired system functionality.
Perceived ease of use
(PEOU)		The extent to which the person perceives that using the system will be free of effort (Davis, 1989).
Perceived ease of use measures their satisfaction on the MR system.
Intention to use
(ITU)		The degree to which a person has formulated conscious plans to perform or not to perform some specified future behaviours (Davis, 1989).

Research instrument

The researcher prepared a questionnaire consisting of 22 items to measure the participants' perception and acceptance of the prototype MRRC system. The questionnaire consisted of five constructs: personal innovativeness (PI), perceived enjoyment (PE), perceived usefulness (PU), perceived ease of use (PEOU), and intention to use (ITU). Items in each construct were measured in five-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree). The items for personal innovativeness were adopted from Agarwal and Prasad (1998) while the items for perceived enjoyment were based the research instrument used in Venkatesh et al.'s (2003) study. The items for perceived ease of use, perceived usefulness, and intention to use were adopted from the study by Davis (1989) that has been validated in numerous reports.

Procedures

The participants had never been exposed to the MRRC system. This study followed the data collection technique suggested by Nov and Ye (2008), Theng et al. (2007) and Davis (1989). One of limitations of previous technology acceptance studies was the reliance on retrospective surveys conducted long after the systems were introduced, and long after the users' acceptance decision was made (Venkatesh et al., 2003). To overcome this limitation, in the present research, data was collected immediately after the introduction of the MRRC system and before the users used it on a regular basis.

The study was divided into three sessions comprising a demonstration, brief hands on, and task-oriented sessions. In the demonstration session, the participants were briefed on the MR concepts and the MRRC system. Then, they were given the opportunity to interact and get used to the system approximately for 30 minutes. In the task-oriented interaction session, the participants were provided with a list of tasks they had to complete. One of these tasks was to identify animals that have been subjected to an engineered tissue process. To obtain the answer, participants needed to interact with the system. They had to hold the marker in order to view the virtual 3D objects, rotate the marker to view different sides of the objects, tilt the marker to view the objects clearer, and zooming in to get a closer view or zooming out to get a whole picture of an object. After they had completed the task-oriented interaction session, the participants completed the questionnaires.

Results

Constructs reliability

A pilot test for the questionnaire was conducted with 17 participants, from 2nd and 3rd year students majoring in Biomedical Science who were not involved in the actual study. The participants in the pilot study also completed the task-oriented interactions session before completing the questionnaires. The data was analysed using SPSS 17.0. Cronbach's alphas were used to determine the reliability of the 22 item, five constructs questionnaire, as shown in Table 3.

Based on Table 3, the five constructs showed high reliability levels. The Cronbach's alpha values were also almost the same as those reported in previous studies. Van Raaij and Schepers (2008) reported reliability coefficients for personal innovativeness at 0.83 while the reliability coefficient obtained in this study was 0.70, which was slightly lower. For perceived enjoyment, the value of 0.876 was similar to that reported by Chesney (2006). King and He (2006) reported Cronbach's alpha values of 0.89, 0.87, 0.86 for perceived usefulness, perceived ease of use, and intention to use constructs respectively, while the pilot study reported values of 0.779, 0.814, and 0.891.

Table 3: Constructs reliability

ConstructsNo. of itemsCronbach's alpha
Personal innovativeness (PI)40.701
Perceived enjoyment (PE)60.876
Perceived usefulness(PU)40.779
Perceived ease of use (PEOU)40.814
Intention to use (ITU)40.891

Overall, the internal consistency of reliability coefficients for the constructs in this study measured using the questionnaire based on the pilot study was acceptable, as the values were between 0.65 and 0.95 (Piaw, 2006). However, as the sample size for the pilot study was small (less than 100), factor analysis procedures were not used, as small sample size may affect the factor analysis by making the solution unstable (Guadagnoli &Velicer, 1988).

Demographics of the participants

The actual study was conducted using 63 volunteer participants who were 2nd, 3rd and 4th year Biomedical Science students in two public universities in Malaysia. Forty-one percent of the participants were male (n=26) and fifty-nine percent were female (n=37). Table 4 shows the respondents' demographic profiles. The majority of the participants used computers intensively in their daily activities (mean value of 3.76 on a 4-point Likert scale). A majority did not play computer games frequently (mean value 2.46). Most of the participants were not familiar with MR (mean value 1.57).

Table 4: Demographic profile of respondents
(N = 63; 4 point scale)

MeasureMeanStd. dev.
Frequency of using computers3.760.440
Frequency of using games2.461.370
Knowledge about MR1.570.603

Personal innovativeness

Results of the participants' personal innovativeness in IT are shown in Table 5. A majority of the participants (73%) always looked forward to experimenting with new technologies. However, only 33% were always the "first person to try new technologies". All participants, except one, either agreed or strongly agreed that they were not hesitant to try new technologies. The majority of the participants expressed a strong agreement to experiment with new technologies.

Perceived enjoyment

Table 6 shows the results of participants' enjoyment while interacting with MRRC. All participants, except three, expressed their agreement that the system was fun. A majority (93%) either strongly agreed or agreed that the system was pleasant to use. All the participants except one, either agreed or strongly agreed that they enjoyed using MRRC. Even though 60% of the respondents were unhappy when the session was over, the remaining 40% of the respondents were not sure. 91% were willing to repeat the same experience while 8% of the respondents were not sure and one person disagreed. However, the majority (97%) either agreed or strongly agreed that learning using MRRC was an interesting experience.

Table 5: Personal innovativeness (PI)
(1 = Strongly disagree to 5 = Strongly agree. N = 63)

No.StatementStrongly
agree		AgreeNot sureDisagreeStrongly
disagree		MeanStd.
dev.
1Look forward to experimenting with new technologies73.0%25.4%1.6%0.0%0.0%4.710.490
2The first person to try new technologies33.3%55.6%11.1%0.0%0.0%3.220.634
3Not hesitant to try new technologies57.1%41.3%1.6%0.0%0.0%4.560.532
4Like to experiment with new technologies63.5%34.9%1.6%0.0%0.0%4.620.521

Table 6: Perceived enjoyment (PE)
(1 = Strongly disagree to 5 = Strongly agree. N = 63)

No.StatementStrongly
agree		AgreeNot sureDisagreeStrongly
disagree		MeanStd.
dev.
1System is fun to use41.3%54.0%4.8%0.0%0.0%4.370.576
2System is pleasant34.9%58.7%6.3%0.0%0.0%4.290.580
3Feel enjoyment52.4%48.0%1.6%0.0%0.0%4.510.535
4Unhappy the session over4.8%52.4%42.9%0.0%0.0%3.620.580
5Willing to repeat the same experience42.9%47.6%7.9%1.6%0.0%4.320.692
6Interesting experience61.9%34.9%3.2%0.0%0.0%4.590.557

Perceived ease of use

Table 7 shows the participants' perceived ease of use with regards to MRRC. A majority of the participants either strongly agreed or agreed that it was easy for them to learn how to use MRRC. For the statement of "whether it was easy for MRRC to do what they want", all of the respondents except five expressed their agreement. Seventy-eight percent of the participants thought that it was easy for them to become skilful in using MRRC while 22% of them were not sure. However, the majority of the participants (90%) reacted positively that MRRC was easy to use.

Table 7: Perceived ease of use (PEOU)
(1 = Strongly disagree to 5 = Strongly agree. N = 63)

No.StatementStrongly
agree		AgreeNot sureDisagreeStrongly
disagree		MeanStd.
dev.
1Learning to use MRRC would be easy34.9%61.9%3.2%0.0%0.0%4.320.534
2I would find it easy to get MRRC to do what I want it to do41.3%50.8%7.9%0.0%0.0%4.330.622
3It would be easy for me to become skilful at using MRRC42.9%34.9%22.2%0.0%0.0%4.210.786
4Overall, I would find MRRC easy to use30.2%58.7%11.1%0.0%0.0%4.190.618

Perceived usefulness

Table 8 shows findings for the participants' views on perceived usefulness of MRRC. The majority (94%) agreed that MRRC helped them to quickly understand the regenerative concept. All respondents either agreed or strongly agreed that MRRC could enhance their understanding of tissue engineering processes. Almost all respondents except for one, either agreed or strongly agreed that MRRC would make their study easier and useful in teaching and learning.

Table 8: Perceived usefulness (PU)
(1 = Strongly disagree to 5 = Strongly agree. N = 63)

No.StatementStrongly
agree		AgreeNot sureDisagreeStrongly
disagree		MeanStd.
dev.
1Using MRRC would enable me to understand regenerative concepts quickly.41.3%52.4%6.3%0.0%0.0%4.350.600
2Using MRRC would enhance my understanding on tissue engineering process.46.0%54.0%0.0%0.0%0.0%4.460.502
3MR technology would make it easier to do my study.54.0%44.4%1.6%0.0%0.0%4.520.535
4I find MR technology useful in teaching and learning.57.1%41.3%1.6%0.0%0.0%4.560.532

Intention to use

Table 9 shows the participants' views on their intention to use MRRC. In general responses toward the intention to use MRRC were positive. Almost all the participants either agreed or strongly agreed to use an MR-based system when it is made available in the university and applied in other subjects. If the participants can have access to an MR-based system, 87% of them would use it frequently while 13% were not sure. However, if the participants can access the MR-based system, all of them except for two, have the intention to use it.

Table 9: Intention to use (ITU)
(1 = Strongly disagree to 5 = Strongly agree. N = 63)

No.StatementStrongly
agree		AgreeNot sureDisagreeStrongly
disagree		MeanStd.
dev.
1I intend to use any system using MR technology when it becomes available in my university.55.6%38.1% 6.3%0.0%0.0%4.490.619
2I intend to use other applications like MRRC in other subjects.55.6%38.1%6.3%0.0%0.0%4.490.619
3Given that I had access to the system, I predict that I would use it frequently.41.3%46.0%12.7%0.0%0.0%4.290.682
4Assuming I had access to the system, I intend to use it.47.6%49.2%3.2%0.0%0.0%4.440.562

Table 10 shows a descriptive analysis for the five constructs. Overall, the users responded positively to the acceptance of MR technology (means above 4.0 out of 5 with small values of standard deviation). They are considered as innovative in IT. The respondents perceived the technology as easy to use and useful for their study. They also perceived enjoyment and willingness to use this technology in the future.

Table 10: Descriptive analysis of constructs
(1 = Strongly disagree to 5 = Strongly agree. N = 63)

ConstructsMeanStd. dev.
Personal innovativeness (PI)4.2770.544
Perceived enjoyment (PU)4.2830.586
Perceived ease of use (PEOU)4.2630.640
Perceived usefulness (PU)4.4730.542
Intention to use (ITU)4.4270.621

Spearman's rank correlation coefficients were used to determine the strengths and directions of the relationships between the constructs. The results in Table 11 showed that the constructs were positively correlated. There were significant strong relationships between perceived ease of use (PEOU) and perceived usefulness (PU) (r=0.745, p<0.001), perceived ease of use (PEOU) and perceived enjoyment (PE) (r=0.710, p<0.001), and perceived usefulness (PU) and intention to use (ITU) (r=0.703, p<0.001). The coefficients showed a moderately strong significant relationships between personal innovativeness (PI) and perceived ease of use (PEOU) (r=0.612, p<0.001), and perceived ease of use (PEOU) and intention to use (ITU) (r=.564, p<0.001). The correlation between perceived enjoyment (PE) and intention to use (ITU) (r=.492, p<0.01) was weak.

Table 11: Correlations between the constructs

Relationshipp-value
(p)		Correlation
coefficient (r)
PI and PEOU0.0000.612***
PEOU and PU0.0000.745***
PEOU and PE0.0000.710***
PEOU and ITU0.0000.564***
PE and ITU0.0020.492**
PU and ITU0.0000.703***
Note: *** p<0.001, ** p< 0.01

A regression analysis was performed to examine the effects of PEOU, PU, and PE on ITU (Table 12). The results showed that the constructs of PEOU and PE did not significantly impact on intention to use MR (p=.443, p=.093 respectively), while perceived usefulness (PU) remained significant [F(1, 35) = 51.074, p<0.05)] in determining intention to use (ITU) MR. There was a strong correlation between dependent construct (ITU) and independent construct (PU) (r=0.77). The R2 value of 0.593 indicated that almost 60% of the changes in ITU were contributed by changes in PU. Independent t-tests results showed the other independent constructs of PEOU (r=.443) and PE (r=.93) did not impact on the dependent construct. Furthermore, the co-linearity tolerance values for PEOU and PE were 0.395 and 0.284 respectively, which were more than 0.10, indicating that the data did not have problems of co-linearity. In addition, as the residual value was 1.387 which was between ±3.3, the data did not have any extreme values (outliers).

Table 12: Regression analysis finding

Variables/modelRR2Adjusted R2Std. error of the estimate
Perceived usefulness (PU).770.593.5821.306

Discussion and conclusions

The main purpose of this study was to investigate the acceptance of MR technology in education. The Mixed Reality Regenerative Concept (MRRC) was developed using MR technology to provide Biomedical Science students with exposure to regenerative concepts and in-vitro processes of tissue engineering. Volunteer sampling was adopted in this study, involving 63 participants comprising of 2nd, 3rd and 4th year Biomedical Science students who had no previous experience with MR technology. A majority of the participants used computers extensively in their daily activities and all played computer games. The participants considered themselves as innovative (mean value of 4.27 on a 5-point Likert scale) indicating they were willing to try new technology. To determine the users' acceptance of MRRC, the study used five constructs that combined the basic TAM model (perceived usefulness, perceived ease of use, and intention to use), personal traits (personal innovativeness) and experience with using MR (perceived enjoyment). Even though the participants had never experienced an MR system, they showed a strong willingness to use an MR application in the future.

The results from simple correlation analyses showed positive relationships between the constructs. There was a strong positive correlation between perceived ease of use (PEOU) and perceived usefulness (PU) of the MR system. This indicated that the participants found the system easy to use and less effort was needed to operate it. They perceived the easy to use system as useful, as it can contribute to their academic performance. This result was similar to the findings reported in previous research by Davis (1989) and Venkatesh et al. (2003). The construct of perceived ease of use (PEOU) also had a strong positive influence on perceived enjoyment (PE). This meant that the MR system was easy to use and this could increase the participants' enjoyment during the interaction session. This result was supported by findings from previous research by Chesney (2006) and Venkatesh et al. (2003). The study's findings also indicated that perceived usefulness (PU) was strongly and positively related to intention to use (ITU). As the participants perceived the system to be useful, they developed stronger behavioural intentions towards the use of same technology in the future.

There was a moderately strong positive correlation between personal innovativeness (PI) and perceived ease of use (PEOU). It seems logical to suggest that if people tend to frequently explore new information technologies by experimenting with them, they become more proficient at learning the design and functionality of the new systems, including the MRRC even though they had not experienced it before. This result is supported by findings from prior research by Parveen and Sulaiman (2008) and Lee, Qu and Kim (2007). Perceived ease of use (PEOU) had a moderately positive influence on intention to use MR system. This finding suggested that as this was their first experience in interacting with an MR environment that required them to manipulate markers and camera, most of them had difficulties becoming familiar with the system. Enjoyment had a weak positive correlation to intention to use. Since the design of the MRRC system was simple and might be lacking in enjoyment features, this could affect their intention to use the system in the future. Overall, it is concluded that, user behavioural intentions to use MR technology, were affected mostly by perceptions of usefulness, moderately impacted by perceptions of ease of use, and relatively weakly influenced by perceptions of enjoyment of the MRRC.

However, the results from regression analysis suggest that perceived usefulness (PU) is the most important construct affecting users' intention to use the MR application in the future. Thus, the participants were not influenced by the ease of use or enjoyment features of new technology or educational tool but put more importance on its usefulness, as the main reason to use the system in the future to help in their studies. In order to ensure that educational applications using MR technology are widely accepted, it is important for MR applications developers to work closely with subject matter experts to produce a tool that is useful for teaching and learning. MR technology can be used to create real life experiences and make the learning environment more enjoyable, and hopefully motivate students to learn.

There are a number of limitations in this study. Firstly, it was conducted using only 63 participants. Future research could replicate the study with more participants and greater representativeness in terms of gender and ethnicity. Secondly, the results involved only descriptive statistics, simple correlation and simple linear regression analyses. More advanced regression analyses could be implemented to better assess the nature of the relationships between the constructs.

References

Agarwal, R. & Prasad J. (1998). A conceptual and operational definition of personal innovativeness in the domain of information technology. Information Systems Research, 9(2), 204-215. http://dx.doi.org/10.1287/isre.9.2.204

Ajzen, I. (1985). From intentions to actions: A theory of planned behaviour. In J. Kuhi & J. Beckmann (Eds.), Action-control: From cognition to behaviour (pp. 11-39). Heidelberg: Springer.

Ajzen, I. & Fishbein, M. (1980). Understanding attitudes and predicting social behaviour. Englewood Cliffs, NJ: Prentice-Hall.

Azuma, R. (1997). A survey of augmented reality. Presence: Teleoperators and Virtual Environments, 6(4), 355-385.

Azuma, R., Yohan, B., Reinhold, B., Steven, F., Simon, J. & Blair, M. (2001). Recent advances in augmented reality. Computer Graphics and Applications, 21(6), 34-47. http://dx.doi.org/10.1109/38.963459

Bandura, A. (1997). Social learning theory. Englewood Cliffs, N.J.: Prentice-Hall.

Bertrand, M. & Bouchard, S. (2008). Applying the technology acceptance model to VR with people who are favourable to its use. Journal of CyberTherapy & Rehabilitation, 1(2), 200-207. http://journalofcybertherapy.webs.com/volume1issue22008.htm

Bourgonjon, J., Valcke, M., Soetaert, R. & Schellens, T. (2009). Students' perceptions about the use of video games in the classroom. Computers & Education, 54(4), 1145-1156. http://users.ugent.be/~mvalcke/CV/games.pdf; http://dx.doi.org/10.1016/j.compedu.2009.10.022

Brown, J. S., Collins, A. & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-42. http://dx.doi.org/10.3102/0013189X018001032

Caudell, T. & Mizell, D. (1992). Augmented reality: An application of heads-up display technology to manual manufacturing processes. In the Proceedings of the 25th Hawaii International Conference on System Sciences Vol 2 (pp. 659-669). Hawaii: IEEE. http://dx.doi.org/10.1109/HICSS.1992.183317

Chesney, T. (2006). An acceptance model for useful and fun information systems. Human Technology, 2(2), 225-235. http://www.humantechnology.jyu.fi/articles/volume2/2006/chesney.pdf

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340. http://www.jstor.org/stable/249008

Davis, F. D. (1993). User acceptance of information technology: System characteristics, user perceptions and behavioral impacts. International Journal Man-Machine Studies, 38(3), 475-487. http://dx.doi.org/10.1006/imms.1993.1022

Davis, F. D., Bagozzi, R. P. & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982-1003. http://www.jstor.org/stable/2632151

Dede, C. (2005). Planning for neomillennial learning styles: Implications for investments in technology and faculty. In D. G. Oblinger & J. L. Oblinger (Eds), Educating the Net Generation (pp. 15.1-15.22). Washington, DC: EDUCAUSE. http://www.educause.edu/educatingthenetgen

Dillon, A. (2001). User acceptance of information technology. In W. Karwowski (Ed), Encyclopedia of human factors and ergonomics. London: Taylor and Francis. [preprint; viewed 21 Oct 2011]. http://www.ischool.utexas.edu/~adillon/BookChapters/Dillon-acceptance.htm

Dillon, A. & Morris, M. G. (1996). User acceptance of information technology: Theories and models. In M. Williams (Ed.), Annual Review of Information Science and Technology (Vol 31), (pp. 3-32). Medford NJ: Information Today. [verified 11 Dec 2011; preprint] http://www.ischool.utexas.edu/~adillon/BookChapters/User%20acceptance.htm

Dubois, E., Gray, P. & Nigay, L (2009). The engineering of mixed reality system. London: Springer-Verlag.

Goldiez, B., Livingston, M. A., Dawson, J., Brown, D., Hancock, P., Baillot, Y. & Julier, S. J. (2004). Advancing human centered augmented reality research. In the Proceedings for the Army Science Conference. Orlando, FL. http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDoc.pdf&AD=ADA433480

Guadagnoli, E. & Velicer, W. F. (1988). Relation of sample size to the stability of component patterns. Psychological Bulletin, 103(2), 265-275. http://www.sciencedirect.com/science/article/pii/S0033290902008067

Heerink, M., Krose, B., Wielinga, B. & Evers, V. (2008). Enjoyment, intention to use and actual use of a conversational robot by elderly people. In Proceedings of HRI'08 (pp. 113-120). Amsterdam: IEEE/ACM. http://dx.doi.org/10.1145/1349822.1349838

Hanson, K. & Shelton, B. E. (2008). Design and development of virtual reality: Analysis of challenges faced by educators. Educational Technology & Society, 11(1), 118-131. http://www.ifets.info/journals/11_1/9.pdf

Hsu, C. L. & Lu, H. P. (2004). Why do people play on-line games? An extended TAM with social influences and flow experience. Information & Management, 41(7), 853-868. http://dx.doi.org/10.1016/j.im.2003.08.014

Kondo, T. (2006). Augmented learning environment using mixed reality technology. In the Proceedings of the World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (ELEARN 2006) (pp. 83-87). Chesapeake, VA: AACE. http://www.editlib.org/p/23664 [viewed 10 Sep 2011].

King, W. R. & He, J. (2006). A meta-analysis of the technology acceptance model. Information & Management, 43(6), 740-755. http://dx.doi.org/10.1016/j.im.2006.05.003

Kirkley, S. E. & Kirkley, J. R. (2005). Creating next generation blended learning environments using mixed reality, video games and simulations. TechTrends, 49(3), 42-53. http://dx.doi.org/10.1007/BF02763646

Lave, J. & Wenger, E. (1990). Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.

Lee, H. Y., Qu, H. & Kim, Y. S. (2007). A study of the impact of personal innovativeness on online travel shopping behavior - A case study of Korean travellers. Tourism Management, 28(3), 886-897. http://dx.doi.org/10.1016/j.tourman.2006.04.013

Legris, P., Ingham, J. & Collerette, P. (2003). Why do people use information technology? A critical review of the technology acceptance model. Information and Management, 40(3), 191-204. http://dx.doi.org/10.1016/S0378-7206(01)00143-4

Liarokapis, F. (2007). An augmented reality interface for visualizing and interacting with virtual content. Virtual Reality, 11(1), 23-43. http://dx.doi.org/10.1007/s10055-006-0055-1

Lin, J. J. W. & Parker, D. E. (2007). User experience modeling and enhancement for virtual environments that employ wide-field displays. In the Proceedings of the 1st International Conference on Digital Human Modeling (pp. 423-433). Berlin: Springer Verlag. http://dx.doi.org/10.1007/978-3-540-73321-8_49

Louho, R., Kallioja, M. & Oittinen, P. (2006). Factors affecting the use of hybrid media applications. Graphic Arts in Finland, 35(3), 11-21. http://media.tkk.fi/GTTS/GAiF/GAiF_PDF/GAiF2006_3-2.pdf

Milgram, P. & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IECE Trans. on Information and Systems (Special Issue on Networked Reality), E77-D(12), 1321-1329. [verified 11 Dec 2011] http://www.alice.id.tue.nl/references/milgram-kishino-1994.pdf

Nilsson, S. & Johansson, B. (2008). Acceptance of augmented reality instructions in a real work setting. In the Proceedings of Conference on Human Factors in Computing Systems (pp. 2025-2032). New York: ACM Press. http://www.ida.liu.se/~susni/papers/chi952-Nilsson.pdf; http://dx.doi.org/10.1145/1358628.1358633

Nov, O. & Ye, C. (2008). Personality and technology acceptance: Personal innovativeness in IT, openness and resistance to change. In Proceedings of the 41st Hawaii International Conference on System Sciences (pp. 448). Hawaii: IEEE. http://dx.doi.org/10.1109/HICSS.2008.348

O'Cass, A. & Fenech, T. (2003). Web retailing adoption: Exploring the nature of Internet user's web retailing behaviour. Journal of Retailing and Consumer Services, 10(2), 81-94. http://dx.doi.org/10.1016/S0969-6989(02)00004-8

Parveen, F. & Sulaiman, A. (2008). Technology complexity, personal innovativeness and intention to use wireless Internet using mobile devices in Malaysia. International Review of Business Research Papers, 4(5), 1-10. http://www.bizresearchpapers.com/1[1].%20Ainin.pdf

Piaw, C. Y. (2006). Asas statistik penyelidikan (Buku 2) [Fundamentals of research statistics (Book 2)]. Kuala Lumpur, Malaysia: McGraw Hill Malaysia.

Rasimah, C. M., Halimah, B. Z. & Azlina, A. (2010). Design a situated learning environment using mixed reality technology - A case study. World Academy of Science, Engineering and Technology (WASET), 71, 887-892. [viewed 8 Sep 2011]. http://www.waset.org/journals/waset/v71/v71-156.pdf

Rogers, E. M. (1995). Diffusion of innovations (4th ed). New York: The Free Press.

Roschelle, J., Pea, R., Hoadley, C., Gordin, D. & Means, B. (2001). Changing how and what children learn in school with computer-based technologies. The Future of Children, 10(2), 76-101. http://ctl.sri.com/publications/downloads/PackardChangingLearning.pdf

Rosenbaum, E., Klopfer, E. & Perry, J. (2007). On location learning: Authentic appllied science with networked augmented realities. Journal of Science Education and Technology, 16(1), 31-45.

Serenko, A. (2008). A model of user adoption of interface agents for email notification. Interacting with Computers, 20(4-5), 461-472. http://dx.doi.org/10.1016/j.intcom.2008.04.004; http://foba.lakeheadu.ca/serenko/papers/Serenko_IwC_model_of_user_adoption.pdf

Seymour, L., Makanya, W. & Berrange, S. (2007). End-users' acceptance of enterprise resource planning systems: An investigation of antecedents. In Proceedings of the 6th Annual ISOnEworld Conference, (pp. 26-1 - 26-22). Las Vegas, USA: Information Institute. http://www.commerce.uct.ac.za/informationsystems/staff/personalpages/lseymour/2009/2007_Seymour_ISONEWORLD-ERP.pdf; http://dx.doi.org/10.1.1.105.9129

Sutherland, I. E. (1965). The ultimate display. In the Proceedings of IFIP Congress (pp. 506-508). Arlington, VA: Federation of Information Processing Societies. http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.136.3720

Sun, H. & Zhang, P. (2005). Causal relationships between perceived enjoyment and perceived ease of use: An alternative approach. Journal of Association for Information Systems, 7(9), 618-645. http://aisel.aisnet.org/jais/vol7/iss9/24/

Theng, Y. L., Lim, C. M. L., Liu, W. & Cheok, A. D. (2007). Mixed reality systems for learning. A pilot study understanding user perceptions and acceptance. In Proceedings of HCI (14) (pp. 728-737). Heidelberg: Springer Berlin. http://dx.doi.org/10.1007/978-3-540-73335-5_79

Träskbäack, M. & Haller, M. (2004). Mixed reality training application for an oil refinery: User requirements. In Proceedings of Virtual Reality Continuum and its Applications in Industry (VRCAI04) (pp. 324-327). Singapore: ACM. http://dx.doi.org/10.1145/1044588.1044658

Van Raaij, E. M. & Schepers, J. J. L. (2008). The acceptance and use of a virtual learning environment in China. Computers & Education, 50(3), 838-852. http://dx.doi.org/10.1016/j.compedu.2006.09.001

Venkatesh, V., Morris, M. G., Davis, G. B. & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478. http://www.jstor.org/stable/30036540

Venkatesh, V. (2000). Determinants of perceived ease of use: Integrating control, intrinsic motivation, and emotion into the technology acceptance model. Information Systems Research, 11(4), 342-365. http://dx.doi.org/10.1287/isre.11.4.342.11872

Authors: Rasimah Che Mohd Yusoff
Advanced Informatics School, Universiti Teknologi Malaysia
Email: rasimah.cc@utm.my Web: http://www.staff.utm.my/portalv3/portal/rasimah.cc

Azlina Ahmad, Associate Professor
Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia
Web: http://www.ukm.my/

Halimah Badioze Zaman, Professor
Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia
Web: http://www.ukm.my/

Please cite as: Rasimah, C. M. Y., Ahmad, A. & Zaman, H. B. (2011). Evaluation of user acceptance of mixed reality technology. In Hong, K. S. & Lai, K. W. (Eds), ICT for accessible, effective and efficient higher education: Experiences of Southeast Asia. Australasian Journal of Educational Technology, 27(Special issue, 8), 1369-1387. http://www.ascilite.org.au/ajet/ajet27/rasimah.html

[ PDF version of this article ] [ AJET 27 ] [ AJET home ]
HTML Editor: Roger Atkinson [rjatkinson@bigpond.com]
This URL: http://www.ascilite.org.au/ajet/ajet27/rasimah.html
Created 23 Dec 2011. Last correction 11 Jan 2012.
Page access count