Blended learning and curriculum renewal across three medical schools: The rheumatology module at the University of Otago

Simon Stebbings, Nasser Bagheri, Kellie Perrie, Phil Blyth and Jenny McDonald
University of Otago

In response to the challenges created by the implementation of a new medical school curriculum at the University of Otago in 2008, we aimed to develop a blended learning course for teaching rheumatology within the existing musculoskeletal course. We developed a multimedia online learning resource structured to support class based problem-based learning (PBL) sessions, and enhance student engagement and promote clinical reasoning. We also aimed to align teaching over three geographically separate campuses, promote more student-centred approaches to learning and meet the challenge of the limited teaching time available for undergraduate learning in the field of rheumatology. Our redesigned course was evaluated longitudinally over eighteen months through student focus groups, Blackboard and Moodle electronic access data, and course evaluation questionnaires. The data collected indicated an overwhelmingly positive response to the changes in teaching methods. Online materials integrated into the new curriculum and combined with in-class PBL and clinical sessions, proved popular with students. Students accessed the newly developed online materials far more frequently than the previously available unstructured content, which they felt to be of limited value or relevance to their studies. Furthermore the blended learning approach allowed delivery of common content across three separate campuses.

Introduction and background

Implementation of a new curriculum for undergraduates is frequently challenging. The trend across medical education worldwide is to teach rheumatology within the framework of a musculoskeletal medicine module (Glynne, 2000). The amount of time devoted to this aspect of the curriculum is frequently disproportionately low in terms of the burden of disease in society (Bilderback et al., 2008).

Two further challenges were apparent at the University of Otago. Firstly, in common with many other medical schools (Ozuah, 2002), undergraduates at the University are divided between three distant campuses, located in Dunedin, Wellington and Christchurch for their clinical attachments. This provides a challenge in maintaining a shared curriculum and consistent approach to teaching across these sites. Secondly, the University of Otago has instituted major changes to the medical school curriculum in line with other Australasian Universities, in part determined by recommendations from the Australian Medical Council (AMC 2009). This had resulted in a move to a learner centred rather than teacher centred approach. Students are expected to bring more critical and analytic thinking to interactive sessions.

In order to meet these challenges, it became apparent that a radical approach was necessary to develop rheumatology teaching and learning for future students undertaking their fifth year musculoskeletal attachment. Previously students rated clinical bedside teaching highly, but didactic elements of the course were not well received. In line with the medical school's new curriculum it was decided to replace didactic teaching where core information had been previously been conveyed, with problem based learning (PBL) scenarios undertaken in small groups (Wilson et al., 2006). In the newly designed course, students were to have access to core content, through the online materials, which were explicitly integrated with the face to face PBL sessions in a blended learning format.

Blended learning is an increasingly prevalent term often used to describe the integration of web-based technologies and face to face teaching, when used as an alternative to more traditional course structures (Bonk & Graham, 2006). However, blended learning is arguably an ill-defined concept (Oliver & Trigwell, 2005) and may also refer to the 'blending' learning theories, or even blending different technologies. Probably the most common meaning of blended learning in the literature is some combination of face to face instruction with online teaching (Bliac, Goodyear & Ellis, 2007). According to Garrison (2004):

... the real test of blended learning is the effective integration of the two main components (face-to-face and Internet technology) such that we are not just adding on to the existing dominant approach or method.

'Blended learning' describes learning activities that involve a systematic combination of co-present (face-to-face) interactions and technologically mediated interactions between students, teachers and learning resources.

Whilst the University has used Blackboard as its learning management system (LMS) since 2001, the rheumatology content made available to students via Blackboard was limited to PowerPoint presentations and Word documents summarising didactic lectures. We were aware, from tracking statistics available from within Blackboard, that students were rarely accessing this information. Quality and presentation of materials, their relevance to student needs and flexible access, have all been identified as characteristics of successful online or web-based learning programs (Sun, Tsai, Finger, Chen & Yeh, 2006). Developing a new blended learning program would also be an opportunity for us to critically review our course materials in relation to these features.

Aims

Methods

The schema for this approach was as follows:

1. An initial evaluation of current teaching methods and unstructured online resources was undertaken at the three clinical schools. The opinions of students were canvassed using evaluation questionnaires and an initial focus group discussion undertaken, informed by free text comments made in the evaluation questionnaires. In particular, views were sought relating to the student's attitudes to online learning. As part of this process, access data relating to existing online resources was collected.

2. The structure and design of a new rheumatology course was decided based upon:
   1. The requirements of the University's new curriculum;
   2. The international recommendations for an undergraduate curriculum in rheumatology;
   3. Student attitudes, comments and suggestions;
   4. Advice from educationalists at the University with reference to educational theory.

3. Implementation of the new blended learning course:
   1. Problem based learning scenarios were developed for use in group sessions;
   2. Online materials in appropriate multimedia formats;

4. Course evaluation was undertaken to assess the pedagogic value of individual activities. The following were used:
   1. Course evaluation questionnaires;
   2. A post-intervention focus group;
   3. Informal feedback;
   4. LMS electronic access data.

5. Fine tuning of course content and delivery. Revision of course materials and design was based upon feedback from students and teachers.

1. Initial case study and evaluation of current teaching methods

Students were approached and written consent was obtained from volunteers who wished to take part in a focus group evaluation. Two focus groups were convened, one with students prior to development of the new curriculum and one 18 months after its implementation. Both sessions were semi-structured and were recorded and transcribed. The focus group facilitator posed questions conceptualised from comments and responses to student feedback questionnaires delivered on two occasions over the preceding 12 months. Initial analysis of usage data of existing online materials, together with anecdotal comments from students suggested dissatisfaction with current materials.

Access data relating to the use of Blackboard and different areas of the online course were recorded over the evaluation period. Access data for the new course was analysed and compared with access data to materials, which had previously been available on Blackboard (PowerPoint and Word documents derived from lectures).

Course evaluation questionnaires consisted of a question followed by a 5 point Likert scale. The number and percentages of each response were recorded. An option of free text written feedback was given at the end of the questionnaire. Questionnaires were distributed to students in the final tutorial at the end of each seven-week musculoskeletal course. Questionnaires were voluntary and returned in a sealed envelope by a student representative.

2. Structure and design of the new rheumatology course

The design of the course and its modes of delivery were developed in conjunction with a specialist in higher education and from review of educational literature. The views of students gained from focus group interviews were incorporated into the design, particularly with respect to modes of delivery.

The course was redesigned to reflect our key aims, namely that students would:

• Develop background knowledge via resources provided in the blended learning environment;
• Develop their ability to recognise background knowledge in relation to clinical problems through attending PBL sessions;
• Learn to apply this knowledge to practical contexts that reflected medical practice, through attending the PBL sessions.

3. Content and delivery of the new blended learning course

The new blended learning course was carefully structured. Online learning materials were made relevant to examinable clinical problems and were designed specifically to ensure students entered the classroom with the background knowledge required to interpret signs, symptoms and investigations in patients with rheumatic disease. The 18 students were split up into four groups of 4-5 to work together on PBL topics, with a facilitator circulating and summarising the discussion at the end of a one-hour session, which consolidated learning and promoted critical thinking and clinical reasoning.

The new blended learning course was developed to provide a complete course. There were clear learning objectives and a clear examinable curriculum.

Blended learning course design
The following areas were developed and provided within Blackboard for students to access on their own:

• Seminar e-learning - this area was designed as a series of interactive web pages. Learning modules covered essential core knowledge designed to assist with the interpretation of problem based learning scenarios. The interactive pages consisted of navigable content interspersed with self-assessment case based questions with immediate feedback. The interactive web pages were designed so that they could be readily integrated with a variety of learning management systems (e.g. Blackboard, Moodle, etc), as we felt it was important not to constrain the materials to a single LMS format.

  Each seminar was designed to be engaging, with photographs, tables, graphs and interactive questions. Each seminar was designed so that it could be readily updated. Seminars were developed as a complete course of study which would obviate the need for student to use textbooks, although suggestions for further inquiry were included for students who wished to explore topics in more depth.

• Ten-minute topics - these were audiovisual presentations - created from PowerPoint presentations with a narration by the teacher. Each topic was restricted to a maximum of ten minutes duration and designed for viewing not only on computers but also on mobile devices- such as iPod, iPhone and tablet PCs. Presentations were developed for podcasting in with reference to suggestions made by Sandars (Sandars, 2009). It has been suggested that podcasts provide support for students with a variety of learning needs, including those where English is not a first language (a significant proportion of our medical students) (Pearce & Scutter, 2010). The ability to replay podcasts and to view them at different locations and whilst 'on the move' may suit the learning styles of some students. Audio materials can be a powerful learning medium and long-term memories are retained to a greater extent when materials are heard than read. Selective replay of podcasts may also prove useful for exam revision (Sandars, 2009).

• PBL - this section contained PDF files each consisting of three cases based on a theme, which were to be discussed in groups during the seminars and were available for pre-reading. This allowed students to preview the material prior to the seminars if they wished and tailor their reading to the cases to be discussed. Subsequent PBL sessions activate prior knowledge and develop existing conceptual frameworks developed by the student (Wood, 2003)

• Summary PDF - these were brief summaries of topics based on a format similar to that used in the Oxford Handbook series (Longmore et al., 2007) and which is popular with students. The topics chosen overlap with the 'ten-minute topics' and may suit different learning styles or act as revision aids.

• Weblinks - carefully selected links to other sites for appropriate learning resources.

• 'Fun in MSK' - lighthearted quiz and Internet derived humorous materials with an educational aspect relevant to rheumatology. These were designed as links that had educational content but could be used as 'time-out' from specific examinable material.

• Online video - a locally and internationally produced video showing methods of musculoskeletal examination. The videos provide a resource for reviewing joint examination in a personal setting.

• Self-assessments - a radiology quiz with immediate feedback. As radiology is an important aspect of rheumatological diagnosis, this quiz helps to reinforce key radiological features for students in a non-threatening environment.

Peer to peer presentations
In the last week of the course, students were required to present a ten-minute topic of their own in groups of 4 or 5 to their peers. They were encouraged to choose topics they felt had not been covered sufficiently in the course. The sessions were conducted by the facilitator in an informal manner, with certificates and a prize for the best presentation. Some of the best presentations were selected for adaptation and upload to the LMS. The format encouraged students to explore some areas in more depth, develop their presentation skills and work as a group. There were positive reinforcements from their peers and the facilitator.

4. Course evaluation and data sources

Evaluation was performed using the modalities previously described. Summative assessments before and after the intervention were evaluated. The main purpose of this was to ascertain that students were developing knowledge in the key areas we wished to examine.

5. Revision of course materials based upon feedback from students and teachers

Results

Student's evaluation of pre-existing course in 2007 (prior to new design)

Could be made more stimulating and interesting.

It would be good if we had more patient examples.

Focus group discussion
A theme of the focus group discussion was that students wanted a structured learning program and desired clarity with regard to learning objectives.

... if we'd had written down a list of what we go through ... at the start, that (sic) might have cleared up some, of what we're meant to be doing.

... a formal handout, you know, just something that people can refer back to.

The only time I used Blackboard was to find last years OSCE [Oral Structured Short Case Examination] questions and to find useful online web addresses.

There was nothing useful on there.

Yeah, I hate doing stuff on the computer.

You can't show that you knew 90 percent of it.

Student's evaluation of new course in during development in 2008

Has the course had helped them to achieve a deeper understanding of the subject?
Has the course helped you to think through clinical problems yourself?
Have the PBL seminars contributed to your understanding of the subject?

Will be good to see how it develops.

More material would be better.

Very informative and enjoyable.

Very helpful, especially 10 min (topics) and quizzes.

Excellent great links and very well put together.

Evaluation of new course following full implementation 2009

Once Blackboard is fully developed it will probably be the main learning tool.

It could replace textbooks as it's easier to follow and has immediate up to date information.

Blackboard could be the way of the future by meeting all media needs : e-learning, podcasting. It will definitely optimise learning opportunities.

Easy to read format and it is user friendly.

Great alternative to clinical work.

Students felt challenged and motivated to learn (100% rating scoring 1or 2 on the Likert scale). They also found the course gave them a deeper understanding of the clinical area. Students commented that there was significant preparation required outside tutorial time and this was reflected in their evaluations (with 44% rating this as onerous - scoring 1 or 2).

Blackboard access data

In 2009 four cohorts ('runs') of students completed the course over the academic year. Each attachment was of seven weeks duration (runs 1-4). In 2009 Run 2 (April-May) and 4 (September-October) used the Blackboard materials as an integral part of their learning, whilst students in Run1 (January-March) and Run 3 (June-August) were taught using the old course structure, without specific reference to the Blackboard materials, thus acting as a control group.

Figure 1 demonstrates that since students in Run 3 were accessing the new online materials at high rates as their run progressed despite being a 'control' group. On questioning, this group cited 'word of mouth' from fellow students, as being an incentive to access these materials.

When specific areas of the online course were analysed, the interactive e-learning seminars were most popular, accounting for nearly half of all hits. This implies students were returning to these e-learning seminars on several occasions. Next most popular were the '10 minute topic' audiovisual podcasts, accounting for 17% of access.

Figure 1: Hits on the Blackboard Rheumatology site for 2007-2009

Not all students accessed the online learning resources equally. Reviewing access data in one group of 18 students, one student accessed the site on only two occasions, seven students on 50-100 occasions, and two on more than 100 occasions.

Course convenors decided to change the LMS hosting the course content from Blackboard to Moodle in 2010. The perceived benefits of this move were a greater autonomy in design, and the accessibility of Moodle. It was also easier to update materials on Moodle. The Blackboard LMS was restricted by University IT protocols, which removed much autonomy from the developers of the course. This move proved popular with students, who found Moodle easier to access, as it was separate from the Blackboard course which was structured as a hierarchy for the entire curriculum. An evaluation performed after the move to Moodle confirmed this as a popular (100% rating the course as 1 or 2 on the Likert scale).

Student end-of-course performance

In total, MCQ assessment results were available during 2011 for 63 students from the Christchurch campus and 52 from the Dunedin campus. Mean results and standard deviations respectively were 37.35 (±4.55) and 37.54 (±4.65), p = 0.53. The results showed a striking similarity between the two centres.

Figure 2: Hits on the Moodle Rheumatology site by students and mean score in end-of-run MCQ assessment in 2011

Discussion

The opportunity of a new curriculum allowed us to meet this challenge and explore innovative teaching methods for the teaching of the rheumatology component of the musculoskeletal medicine course. The combination of online teaching resources and PBL allowed much more time for classroom discussion. We have also found time within the course to institute additional clinical teaching as a result of re-prioritising contact time with students. Feedback from students with respect to these changes has been overwhelmingly positive.

Students were initially wary of blended learning and showed little enthusiasm for the proposed developments. It is recognised that some students are not comfortable with using computers as a primary source of learning (Sun et al., 2006). Furthermore, negative experiences of existing course material posted on Blackboard, in a poorly organised, unstructured, file repository format may well have influenced students in their attitude to the proposed developments. A previous study assessing attitudes of students to blended learning in rheumatology noted that only one third of students felt that this would be a useful learning tool (Wilson et al., 2006). Poor quality instructional design for online learning is recognised as having a negative impact on the use made of such materials by students, but this is issue not limited to web-based learning (Cook, 2007).

Despite these initial negative attitudes to blended learning, as the course was developed and new materials were integrated into the course, students rapidly adopted the course into their learning structures. This was demonstrated by far greater access to the new materials when compared with previously available material on Blackboard. Enthusiastic feedback was also elicited in focus groups. An interesting and unexpected development was that word of mouth resulted in students who had completed the course prior to the development of the new module accessing the course repeatedly and in large numbers prior to their end of year exams.

Small-group, problem-based learning sessions were received initially with little enthusiasm by the students, though most medical schools in Australasia have implemented PBL based courses for undergraduates. PBL is not without controversy, with most knowledge based assessments showing little advantage in PBL. In addition the underlying educational principles and theory behind PBL have been questioned (Colliver, 2000). However, motivation of students and the generic analytical skills of future doctors are felt to be fostered better by the PBL approach (Wood, 2003). Once instituted as part of the integrated course, our students found PBL an enjoyable and relevant experience, particularly after exposure to key topics from the blended learning materials.

One suggested advantage of blended learning is that it allows flexible learning, where students can choose when and where they access the course materials. Interestingly, although our students appreciated opportunities for self-paced learning, some felt that learning done outside class hours added to their perceived workload, and this was highlighted by responses to the evaluation questionnaires. Not all students are comfortable with using online resources and this was demonstrated by the widely variable access to materials. Blended learning should be seen as part of a blended educational approach and not a 'one size fits' all resource.

We chose not to compare the results of summative assessments before and after the developing the new course. Previous studies have shown that PBL does not necessarily improve performance in summative assessments (Colliver, 2000). It has been suggested that evaluation of blended learning is best undertaken through a peer review process and assessments of learner satisfaction, content and usability. We did evaluate student performance in their end of attachment MCQ at two campuses (Figure 2). These results were strikingly similar. We feel that this provides evidence that our goal of achieving delivery of common core knowledge at distant sites was achieved. Furthermore, students commented that they felt well prepared for their end of attachment assessments, which were devised to assess knowledge of the core content we had included on Moodle.

Conclusion

Our case study demonstrates that a blended educational approach can prove a popular and worthwhile teaching method. We hope to continue to evaluate and further develop the rheumatology course during coming years using a design-research framework. Our future aims are to identify specific elements of the course which may help medical students to recognise, appreciate and appropriately manage a range of rheumatological conditions. In this way we hope to prepare students for future medical practice where rheumatological conditions will commonly be encountered.

Acknowledgments

References

Bilderback, K., et al. (2008). Design and implementation of a system-based course in musculoskeletal medicine for medical students. The Journal of Bone & Joint Surgery, 90(10), 2292-2300. http://dx.doi.org/10.2106/jbjs.g.01676

Bliac, A., Goodyear, P. & Ellis, R. (2007). Research focus and methodological choices in studies into students' experiences of blended learning in higher education. The Internet and Higher Education, 10(4), 231-244. http://dx.doi.org/10.1016/j.iheduc.2007.08.001

Bonk, C. J. & Graham, C. R. (Eds.) (2006). Handbook of blended learning: Global Perspectives, local designs. San Francisco: Wiley.

Colliver, J. A. (2000). Effectiveness of problem-based learning curricula: Research and theory. Academic Medicine, 75(3), 259-266. http://journals.lww.com/academicmedicine/Abstract/2000/03000/Effectiveness_of_Problem_based_Learning_Curricula_.17.aspx

Cook, D. A. (2007). Web-based learning: Pros, cons and controversies. Clinical Medicine, 7(1), 37-42. http://www.ingentaconnect.com/content/rcop/cm/2007/00000007/00000001/art00011

Garrison, D. R. & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education, 7(2), 95-105. http://dx.doi.org/10.1016/j.iheduc.2004.02.001

Garrison, D. R. & Anderson, T. (2003). E-learning in the 21st century: A framework for research and practice. London: Routledge/Falmer.

Glynne J. A., Herrick, A. & Marsh, D. (2000). Musculoskeletal medicine and surgery. Churchill Livingstone.

Laurillard, D. (2002). Rethinking university teaching: A conversational framework for the effective use of learning technologies (2nd ed.). p.99. Routledge Falmer, London and New York.

Longmore, M., Wilkinson, I., Turemezei, T., et al (2007). Oxford handbook of clinical medicine, 7th Ed. Oxford University Press.

Oliver, M. & Trigwell, K. (2005). Can blended learning be redeemed? E-Learning and Digital Media, 2(1), 17-26. http://dx.doi.org/10.2304/elea.2005.2.1.17

Ozuah, P. O. (2002). Undergraduate medical education: Thoughts on future challenges. BMC Medical Education, 2, 8. http://www.biomedcentral.com/1472-6920/2/8

Pearce, K. & Scutter, S. (2010). Podcasting of health sciences lectures: Benefits for students from a non-English speaking background. Australasian Journal of Educational Technology, 26(7), 1028-1041. http://www.ascilite.org.au/ajet/ajet26/pearce.html

Reeves, T., McKenny, S. & Herrington, J. (2011). Publishing and perishing: The critical importance of educational design research. Australasian Journal of Educational Technology, 27(1), 55-65. http://www.ascilite.org.au/ajet/ajet27/reeves.html

Ruiz, J. G., Mintzer, M. J. & Leipzig, R. M. (2006). The impact of e-learning in medical education. Academic Medicine, 81(3), 207-212. http://journals.lww.com/academicmedicine/fulltext/2006/03000/the_impact_of_e_learning_in_medical_education.2.aspx

Sandars, J. (2009). Twelve tips for using podcasts in medical education. Medical Teacher, 31(5), 387-389. http://dx.doi.org/10.1080/01421590802227958

Sun, P.-C., Tsai, R. J., Finger, G., Chen Y. & Yeh, D. (2006). What drives a successful e-learning? An empirical investigation of critical factors influencing learner satisfaction. Computers & Education, 50(4), 1183-1202. http://dx.doi.org/10.1016/j.compedu.2006.11.007

Wilson, A. S., et al. (2006). Development of an interactive learning tool for teaching rheumatology -- a simulated clinical case studies program. Rheumatology, 45(9), 1158-1161. http://rheumatology.oxfordjournals.org/content/45/9/1158.full.pdf

Wood, D. F. (2003). Problem based learning. BMJ, 326, 328. http://dx.doi.org/10.1136/bmj.326.7384.328

Woolf, A. D., Walsh, N. E. & Akesson, K. (2004). Global core recommendations for a musculoskeletal undergraduate curriculum. Annals of the Rheumatic Diseases, 63(5), 517-524. http://dx.doi.org/10.1136/ard.2003.016071

Authors: Simon Stebbings, Rheumatologist and Senior Lecturer Department of Medicine, Dunedin School of Medicine University of Otago, PO Box 913, Dunedin 9054, New Zealand Email: simon.stebbings@otago.ac.nz Nasser Bagheri, Research Fellow, Department of Public Health School of Medicine and Health Sciences, University of Otago, Wellington PO Box 7343, Wellington South, New Zealand Email: Nasser.bagheri@otago.ac.nz Kellie Perrie, Registrar in Rehabilitation Medicine Southern District Health Board, Dunedin Hospital Dunedin 9054, New Zealand. Email: kellie.perrie@southerndhb.govt.nz Phil Blyth, Senior Lecturer, eLearning in Medicine Department of the Dean, Otago School of Medical Sciences University of Otago, Dunedin, New Zealand. Email: phil.blyth@otago.ac.nz Jenny McDonald, Lecturer, Higher Education Development Centre University of Otago, Dunedin, New Zealand. Email: jenny.mcdonald@otago.ac.nz Please cite as: Stebbings, S., Bagheri, N., Perrie, K., Blyth, P. & McDonald, J. (2012). Blended learning and curriculum renewal across three medical schools: The rheumatology module at the University of Otago. Australasian Journal of Educational Technology, 28(7), 1176-1189. http://www.ascilite.org.au/ajet/ajet28/stebbings.html